Show HN: SlateDB – An embedded storage engine built on object storage SlateDB is an embedded storage engine built as a log-structured merge-tree. Unlike traditional LSM-tree storage engines, SlateDB writes data to object storage (S3, GCS, ABS, MinIO, Tigris, and so on). Leveraging object storage allows SlateDB to provide bottomless storage capacity, high durability, and easy replication. The trade-off is that object storage has a higher latency and higher API cost than local disk. To mitigate high write API costs (PUTs), SlateDB batches writes. Rather than writing every put() call to object storage, MemTables are flushed periodically to object storage as a string-sorted table (SST). The flush interval is configurable. To mitigate write latency, SlateDB provides an async put method. Clients that prefer strong durability can await on put until the MemTable is flushed to object storage (trading latency for durability). Clients that prefer lower latency can simply ignore the future returned by put. To mitigate read latency and read API costs (GETs), SlateDB will use standard LSM-tree caching techniques: in-memory block caches, compression, bloom filters, and local SST disk caches. https://ift.tt/jEYONsb August 15, 2024 at 12:47AM
Show HN: SlateDB – An embedded storage engine built on object storage https://ift.tt/Vrx5KDC
Related Articles
Show HN: A Story-Based Brainwallet https://ift.tt/1XtmPDJShow HN: A Story-Based Brainwallet https://ift.tt/TryAEMU April 7, 202… Read More
Show HN: I made a website for you to travel the world in first-person POV https://ift.tt/97sSKCQShow HN: I made a website for you to travel the world in first-person … Read More
Show HN: Svend3r, an interactive plug and play charting library for Svelte https://ift.tt/l7phZPoShow HN: Svend3r, an interactive plug and play charting library for Sv… Read More
Show HN: pync – Netcat for Python https://ift.tt/ulNzUtVShow HN: pync – Netcat for Python https://ift.tt/Msx4woH March 31, 202… Read More
Show HN: A tiling window manager like i3wm written entirely in C# https://ift.tt/kUFMQsyShow HN: A tiling window manager like i3wm written entirely in C# http… Read More
Show HN: I made Million – It's a Virtual DOM made for the future https://ift.tt/jwoKsxCShow HN: I made Million – It's a Virtual DOM made for the future Repo … Read More
Show HN: Multiplayer Demo Built with Elixir https://ift.tt/FuOAPVwShow HN: Multiplayer Demo Built with Elixir Hey HN, I’m an engineer at… Read More
Show HN: Paper Birch Notes https://ift.tt/UquHpPJShow HN: Paper Birch Notes https://ift.tt/UFmbCaf April 11, 2022 at 11… Read More
0 Comments: