Show HN: Real-time voice chat with AI, no transcription Hi HN -- voice chat with AI is very popular these days, especially with YC startups ( https://twitter.com/k7agar/status/1769078697661804795 ). The current approaches all do a cascaded approach, with audio -> transcription -> language model -> text synthesis. This approach is easy to get started with, but requires lots of complexity and has a few glaring limitations. Most notably, transcription is slow, is lossy and any error propagates to the rest of the system, cannot capture emotional affect, is often not robust to code-switching/accents, and more. Instead, what if we fed audio directly to the LLM - LLM's are really smart, can they figure it out? This approach is faster (we skip transcription decoding) and less lossy/more robust because the big language model should be smarter than a smaller transcription decoder. I've trained a model in just that fashion. For more architectural information and some training details, see this first post: https://tincans.ai/slm . For details about this model and some ideas for how to prompt it, see this post: https://tincans.ai/slm3 . We trained this on a very limited budget but the model is able to do some things that even GPT-4, Gemini, and Claude cannot, eg reasoning about long-context audio directly, without transcription. We also believe that this is the first model in the world to conduct adversarial attacks and apply preference modeling in the speech domain. The demo is unoptimized (unquantized bf16 weights, default Huggingface inference, serverless speed bumps) but achieves 120ms time to first token with audio. You can basically think of it as Mistral 7B, so it'll be very fast and can also run basically anywhere. I am especially optimistic about embedded usage -- not needing the transcription step means that the resulting model is smaller and cheaper to use on the edge. Would love to hear your thoughts and how you would use it! Weights are Apache-2 and on Hugging Face: https://ift.tt/A1wbIFJ... https://ift.tt/QyDmgMK March 20, 2024 at 12:37AM
Show HN: Real-time voice chat with AI, no transcription https://ift.tt/MdiEB7l
Related Articles
Show HN: I Am Not Helpful AI – An AI roasts your idea instead of validating them https://ift.tt/E5WsAkPShow HN: I Am Not Helpful AI – An AI roasts your idea instead of valid… Read More
Show HN: Lazycontainer: A Terminal UI for Apple Containers https://ift.tt/WHbn8eRShow HN: Lazycontainer: A Terminal UI for Apple Containers Apple final… Read More
Show HN: I built a tool to create App Screenshots https://ift.tt/VP0Z6soShow HN: I built a tool to create App Screenshots I built a tool to cr… Read More
Show HN: TNX API – Natural Language Interactions with Your Database https://ift.tt/3E1lC9DShow HN: TNX API – Natural Language Interactions with Your Database He… Read More
Show HN: Pickaxe – a TypeScript library for building AI agents https://ift.tt/9oacGSNShow HN: Pickaxe – a TypeScript library for building AI agents Hey HN,… Read More
Show HN: Logcat.ai:AI-powered observability for Operating Systems(Android+Linux) https://ift.tt/HtgJkXYShow HN: Logcat.ai:AI-powered observability for Operating Systems(Andr… Read More
Show HN: I made a fun quiz that reviews last week's top posts on r/programming https://ift.tt/icKGMk2Show HN: I made a fun quiz that reviews last week's top posts on r/pro… Read More
Show HN: Comparator - I built a free, open-source app to compare job offers https://ift.tt/089VpGuShow HN: Comparator - I built a free, open-source app to compare job o… Read More
0 Comments: