Show HN: Running LLMs in one line of Python without Docker Hello Hacker News! We're Yangqing, Xiang and JJ from lepton.ai. We are building a platform to run any AI models as easy as writing local code, and to get your favorite models in minutes. It's like container for AI, but without the hassle of actually building a docker image. We built and contributed to some of the world's most popular AI software - PyTorch 1.0, ONNX, Caffe, etcd, Kubernetes, etc. We also managed hundreds of thousands of computers in our previous jobs. And we found that the AI software stack is usually unnecessarily complex - and we want to change that. Imagine if you are a developer who sees a good model on github, or HuggingFace. To make it a production ready service, the current solution usually requires you to build a docker image. But think about it - I have a few python code and a few python dependencies. That sounds like a huge overhead, right? lepton.ai is really a pythonic way to free you from such difficulties. You write a simple python scaffold around your PyTorch / TensorFlow code, and lepton launches it as a full-fledged service callable via python, javascript, or any language that understands OpenAPI. We use containers under the hood, but you don't need to worry about all the infrastructure nuts and bolts. One of the biggest challenge in AI is that it's really "all-stack": in addition to a plethora of models, AI applications usually involves GPUs, cloud infra, web services, DevOps, and SysOps. But we want you to focus on your job - and we take care of the rest "boring but essential" work. We're really excited we get to show this to you all! Please let us know your thoughts and questions in the comments. https://www.lepton.ai/ October 4, 2023 at 10:07PM
Show HN: Running LLMs in one line of Python without Docker https://ift.tt/iDwaHbj
Related Articles
Show HN: Whatcha – the social network for digital media https://ift.tt/nRlFckVShow HN: Whatcha – the social network for digital media https://ift.tt… Read More
Show HN: Piano Trainer – Learn piano scales, chords and more using MIDI https://ift.tt/q8SvT2LShow HN: Piano Trainer – Learn piano scales, chords and more using MID… Read More
Show HN: Movably – Protect your health, move more while you work https://ift.tt/UZ0vCmXShow HN: Movably – Protect your health, move more while you work https… Read More
Show HN: Headlamp – Follow the dots to find bugs https://ift.tt/EtAC2VmShow HN: Headlamp – Follow the dots to find bugs https://ift.tt/0GDAUB… Read More
Show HN: Old News – News that gets better the *less* you read it https://ift.tt/gKJBa5CShow HN: Old News – News that gets better the *less* you read it I’ve … Read More
Show HN: Heat Pump Cost Comparison https://ift.tt/Xt9ES5AShow HN: Heat Pump Cost Comparison https://ift.tt/rWjagBX June 30, 202… Read More
Show HN: Mapedia.org – A Crowdsourced Learning Map https://ift.tt/JvDpuqjShow HN: Mapedia.org – A Crowdsourced Learning Map Hi HN! We're happy … Read More
Show HN: Copy React code from any site https://ift.tt/8l1LE6YShow HN: Copy React code from any site https://ift.tt/AM4FfSu July 9, … Read More
0 Comments: