Show HN: I replicated Anthropic's monosemanticity research using just my MacBook Hi everyone, I've been working on an open-source implementation of Anthropic's research on monosemanticity ("Towards Monosemanticity"). The problem Anthropic is trying to solve is that language models are hard to interpret because individual neurons can be responsible for multiple different things. The research finds that training a small autoencoder on neuron activations can result in "features" which are much easier to interpret. When I was reading the original research, I got really excited when I realized that the models they used were really small, and I could probably train them from scratch with just my M3 MBP. My models are somewhat undertrained compared to what Anthropic produced, but I think my results are still very compelling. Let me know what you think! https://ift.tt/Ddfgl2K April 30, 2024 at 10:56PM
Show HN: I replicated Anthropic's monosemanticity research using just my MacBook https://ift.tt/aopBX16
Related Articles
Show HN: Salty, a minimalist DevOps tool inspired by Saltstack (and Ansible) https://ift.tt/3hPhoX7Show HN: Salty, a minimalist DevOps tool inspired by Saltstack (and An… Read More
Show HN: Docstring – AI-generated code documentation and hosting https://ift.tt/3ze1HyxShow HN: Docstring – AI-generated code documentation and hosting https… Read More
Show HN: Classified.html, encryption solution that is just a file, web/terminal https://ift.tt/2XxMwTBShow HN: Classified.html, encryption solution that is just a file, web… Read More
Show HN: Open Sukkah – 'Airbnb' for Public Sukkahs https://ift.tt/3zqSov8Show HN: Open Sukkah – 'Airbnb' for Public Sukkahs https://opensukkah.… Read More
Show HN: Changelogs.gallery – Discover the best changelogs on the internet https://ift.tt/39mECzcShow HN: Changelogs.gallery – Discover the best changelogs on the inte… Read More
Show HN: DALL·E mini – Generate images from text https://ift.tt/3nKGY3bShow HN: DALL·E mini – Generate images from text https://ift.tt/3C1CHN… Read More
Show HN: Nussknacker https://ift.tt/2XQogMTShow HN: Nussknacker https://ift.tt/2XGmgqN September 22, 2021 at 03:5… Read More
Show HN: I Built Four Eight-Foot-Long Handwriting Robots https://ift.tt/3kkHh2GShow HN: I Built Four Eight-Foot-Long Handwriting Robots https://twitt… Read More
0 Comments: