Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU I spent the last few days building out a nicer ChatGPT-like interface to use Mistral 7B and Llama 3 fully within a browser (no deps and installs). I’ve used the WebLLM project by MLC AI for a while to interact with LLMs in the browser when handling sensitive data but I found their UI quite lacking for serious use so I built a much better interface around WebLLM. I’ve been using it as a therapist and coach. And it’s wonderful knowing that my personal information never leaves my local computer. Should work on Desktop with Chrome or Edge. Other browsers are adding WebGPU support as well - see the Github for details on how you can get it to work on other browsers. Note: after you send the first message, the model will be downloaded to your browser cache. That can take a while depending on the model and your internet connection. But on subsequent page loads, the model should be loaded from the IndexedDB cache so it should be much faster. The project is open source (Apache 2.0) on Github. If you like it, I’d love contributions, particularly around making the first load faster. Github: https://ift.tt/CZIlF7G Demo: https://secretllama.com https://ift.tt/CZIlF7G May 4, 2024 at 02:56AM
Show HN: I built a free in-browser Llama 3 chatbot powered by WebGPU https://ift.tt/c8CEYe5
Related Articles
Show HN: Sqlauthz: Declarative permissions management for PostgreSQL https://ift.tt/T37FdrhShow HN: Sqlauthz: Declarative permissions management for PostgreSQL I… Read More
Show HN: Sigkill, utility for decrypting and exporting signal chats https://ift.tt/vCj6W03Show HN: Sigkill, utility for decrypting and exporting signal chats ht… Read More
Show HN: SpongeBob delivered straight from your terminal https://ift.tt/grNFheIShow HN: SpongeBob delivered straight from your terminal https://ift.t… Read More
Show HN: FlashFlashCards- create Anki flashcards from screenshots https://ift.tt/zT8o5IXShow HN: FlashFlashCards- create Anki flashcards from screenshots http… Read More
Show HN: DALL-E 3 and GPT-4 Vision Recursive Image Generator https://ift.tt/JFwaz93Show HN: DALL-E 3 and GPT-4 Vision Recursive Image Generator It's like… Read More
Show HN: Visualising Rising Temperatures for the Netherlands https://ift.tt/Dzblk49Show HN: Visualising Rising Temperatures for the Netherlands Something… Read More
Show HN: Get any piece of Google Earth as a single normalized glTF 3D model https://ift.tt/L8hRO6nShow HN: Get any piece of Google Earth as a single normalized glTF 3D … Read More
Show HN: Modern C++ implementations of a words counter with benchmarks https://ift.tt/TWu7JZwShow HN: Modern C++ implementations of a words counter with benchmarks… Read More
0 Comments: