Show HN: StableBuild – make any Docker container deterministic Hi HN! I've posted this a few weeks back without much HN traction - today we've added a free community tier, so anyone can try it out. TL;DR: We’ve launched StableBuild, a new tool to easily freeze and pin Docker images, operating system packages, Python packages, and arbitrary build dependencies; in 5 lines of code: https://stablebuild.com . As the CTO at an ML startup w/ 75 people ( https://ift.tt/iYnvP61 ) I’ve grown incredibly frustrated with non-deterministic builds. Last year basically every week one of our containers (we have 40+ unique ones in prod) would stop working properly because some dependency was updated or removed. This ranges from Nvidia deleting cuda base images from Docker Hub, to Chromium being removed from the Ubuntu package registry in favor of the snap version, to pandas 2 being published with breaking APIs - while everyone just depends on e.g. pandas>=1.4. This has been super disruptive because builds break for no apparent reason: someone pushes some unrelated code change, a container needs to be rebuilt, now it gets the latest dependencies => boom, either a compile error or an integration test fails. Many times this even blocks deployment. If the build system has decided that a container on master needs to be rebuilt, we can’t deploy the complete system if a dependency has shifted. And, fixing this naturally falls on the most senior engineers. Anyway, to fix this I’ve funded (together w/ my Edge Impulse cofounder) StableBuild. It’s a set of mirrors and registries that let you easily freeze and pin Docker images, apt/apk packages, Python packages, and arbitrary files and URLs from the internet. It currently consists of: * A custom pull-through cache for Docker Hub, that makes any image pulled immutable. Protects against updated or removed images; and as a nice byproduct also bypasses pull-rate limits in Docker Hub. * Full daily copies of the Ubuntu, Debian and Alpine package registries + the most popular PPAs; so you can pin to a specific date (give me the package registry as it was on 2023-12-15). Essentially what snapshot.debian.org does, but fast and highly available (and for more repos). * Full daily copy of the PyPI registry, so you can also pin to a specific date. This has been super useful for resurrecting old Python code. Any Python example w/ dependencies is bitrotted the moment it gets published - StableBuild’s historic registry helps tremendously (see https://ift.tt/Yn5W1mZ ...) * A generic file / URL cache for arbitrary things you need to pull from the internet during builds. This has all been in production with SB’s first customers and has basically eliminated random build failures due to changed dependencies for them. Naturally you still want to upgrade dependencies (security patches are nice!) - but you can do it at their own pace, rather than whenever a container rebuilds. StableBuild is now available for everyone. There's a free Community tier (since today) that gives free access to all services and mirrors (although with a hard 15GB/month traffic limit), and commercial pricing starting at $199 (cheaper than running a high-available apt mirror on AWS - which we used to do at Edge Impulse). Would love to hear people's thoughts <3 Sign up: https://dashboard.stablebuild.com Docs: https://docs.stablebuild.com https://www.stablebuild.com/ March 13, 2024 at 01:49AM
Show HN: StableBuild – make any Docker container deterministic https://ift.tt/dc8lhsN
Related Articles
Show HN: Open Sourcing Our No-Code WebXR Editor After 5 Years of Development https://ift.tt/DS4QzFXShow HN: Open Sourcing Our No-Code WebXR Editor After 5 Years of Devel… Read More
Show HN: Invoice Detector – All your invoices in one place, every month https://ift.tt/JSphGufShow HN: Invoice Detector – All your invoices in one place, every mont… Read More
Show HN: I made a search engine for Hacker News https://ift.tt/iQJ8VDOShow HN: I made a search engine for Hacker News I love HN but always f… Read More
Show HN: I'm Building a Better Goodreads https://ift.tt/ptILqd7Show HN: I'm Building a Better Goodreads I'm Shreya and I just started… Read More
Show HN: Connect to more than 200 event sources (GreptimeDB as Telegraf Output) https://ift.tt/kRwEVnGShow HN: Connect to more than 200 event sources (GreptimeDB as Telegra… Read More
Show HN: Rise Tools. Server Defined Rendering for React Native https://ift.tt/9OHjlbdShow HN: Rise Tools. Server Defined Rendering for React Native https:/… Read More
Show HN: Piperiv.com – Data Tools for Natural Gas, Oil, and Power https://ift.tt/JxbnsaOShow HN: Piperiv.com – Data Tools for Natural Gas, Oil, and Power http… Read More
Show HN: I built a Home Server | NAS with LXD https://ift.tt/S6pLdOfShow HN: I built a Home Server | NAS with LXD https://ift.tt/UEpsYbi J… Read More
0 Comments: