Show HN: Running LLMs in one line of Python without Docker Hello Hacker News! We're Yangqing, Xiang and JJ from lepton.ai. We are building a platform to run any AI models as easy as writing local code, and to get your favorite models in minutes. It's like container for AI, but without the hassle of actually building a docker image. We built and contributed to some of the world's most popular AI software - PyTorch 1.0, ONNX, Caffe, etcd, Kubernetes, etc. We also managed hundreds of thousands of computers in our previous jobs. And we found that the AI software stack is usually unnecessarily complex - and we want to change that. Imagine if you are a developer who sees a good model on github, or HuggingFace. To make it a production ready service, the current solution usually requires you to build a docker image. But think about it - I have a few python code and a few python dependencies. That sounds like a huge overhead, right? lepton.ai is really a pythonic way to free you from such difficulties. You write a simple python scaffold around your PyTorch / TensorFlow code, and lepton launches it as a full-fledged service callable via python, javascript, or any language that understands OpenAPI. We use containers under the hood, but you don't need to worry about all the infrastructure nuts and bolts. One of the biggest challenge in AI is that it's really "all-stack": in addition to a plethora of models, AI applications usually involves GPUs, cloud infra, web services, DevOps, and SysOps. But we want you to focus on your job - and we take care of the rest "boring but essential" work. We're really excited we get to show this to you all! Please let us know your thoughts and questions in the comments. https://www.lepton.ai/ October 4, 2023 at 10:07PM
Show HN: Running LLMs in one line of Python without Docker https://ift.tt/iDwaHbj
Related Articles
Show HN: Private, local data gathering tool, to make your Digital Twin https://ift.tt/gI8s2anShow HN: Private, local data gathering tool, to make your Digital Twin… Read More
Show HN: Use Spectre.Console Terminal Widgets in PowerShell https://ift.tt/MC0bIxPShow HN: Use Spectre.Console Terminal Widgets in PowerShell I like spe… Read More
Show HN: PeepDB – open-source CLI tool to quickly view SQL database tables https://ift.tt/ZOFCAPfShow HN: PeepDB – open-source CLI tool to quickly view SQL database ta… Read More
Show HN: Dump entire Git repos into a single file for LLM prompts https://ift.tt/aL470MzShow HN: Dump entire Git repos into a single file for LLM prompts Hey!… Read More
Show HN: DeutschlandAPI – A modern REST API to access information of Germany https://ift.tt/XgtI4zHShow HN: DeutschlandAPI – A modern REST API to access information of G… Read More
Show HN: Ki Editor - the multicursor syntactical editor https://ift.tt/SaRlJ1WShow HN: Ki Editor - the multicursor syntactical editor Hi everyone, I… Read More
Show HN: Museum Music – generate period-appropriate playlists from artwork https://ift.tt/k87V2wAShow HN: Museum Music – generate period-appropriate playlists from art… Read More
Show HN: SocialPredict, MIT Licensed, Easy to Deploy Prediction Market Platform https://ift.tt/blVrQGEShow HN: SocialPredict, MIT Licensed, Easy to Deploy Prediction Market… Read More
0 Comments: