Show HN: I replicated Anthropic's monosemanticity research using just my MacBook Hi everyone, I've been working on an open-source implementation of Anthropic's research on monosemanticity ("Towards Monosemanticity"). The problem Anthropic is trying to solve is that language models are hard to interpret because individual neurons can be responsible for multiple different things. The research finds that training a small autoencoder on neuron activations can result in "features" which are much easier to interpret. When I was reading the original research, I got really excited when I realized that the models they used were really small, and I could probably train them from scratch with just my M3 MBP. My models are somewhat undertrained compared to what Anthropic produced, but I think my results are still very compelling. Let me know what you think! https://ift.tt/Ddfgl2K April 30, 2024 at 10:56PM
Show HN: I replicated Anthropic's monosemanticity research using just my MacBook https://ift.tt/aopBX16
Related Articles
Show HN: Semantic Grep – A Word2Vec-powered search tool https://ift.tt/u7Sn4ZpShow HN: Semantic Grep – A Word2Vec-powered search tool Much improved … Read More
Show HN: News-Research Aggregation https://ift.tt/1GPofDCShow HN: News-Research Aggregation Have made a previous submission abo… Read More
Show HN: How I wrote a LaTeX paper without writing any LaTeX https://ift.tt/ntSoJRXShow HN: How I wrote a LaTeX paper without writing any LaTeX Stempad i… Read More
Show HN: ThinkPost – split-panel note taking & brainstorming app for devs https://ift.tt/d3fUbDzShow HN: ThinkPost – split-panel note taking & brainstorming app f… Read More
Show HN: Chrome Extension to Open Google Maps Locations in Apple Maps https://ift.tt/Khl6TiZShow HN: Chrome Extension to Open Google Maps Locations in Apple Maps … Read More
Show HN: Run Llama 3.1 8B in the browser https://ift.tt/6PER93UShow HN: Run Llama 3.1 8B in the browser https://app.wiz.chat July 29,… Read More
Show HN: Preprocessor I've been working 4 years now https://ift.tt/OoLQPIgShow HN: Preprocessor I've been working 4 years now Hey there, I'm her… Read More
Show HN: Symbols > We are building Figma for developers https://ift.tt/GoRE8n4Show HN: Symbols > We are building Figma for developers What is Sym… Read More
0 Comments: