Show HN: SlateDB – An embedded storage engine built on object storage SlateDB is an embedded storage engine built as a log-structured merge-tree. Unlike traditional LSM-tree storage engines, SlateDB writes data to object storage (S3, GCS, ABS, MinIO, Tigris, and so on). Leveraging object storage allows SlateDB to provide bottomless storage capacity, high durability, and easy replication. The trade-off is that object storage has a higher latency and higher API cost than local disk. To mitigate high write API costs (PUTs), SlateDB batches writes. Rather than writing every put() call to object storage, MemTables are flushed periodically to object storage as a string-sorted table (SST). The flush interval is configurable. To mitigate write latency, SlateDB provides an async put method. Clients that prefer strong durability can await on put until the MemTable is flushed to object storage (trading latency for durability). Clients that prefer lower latency can simply ignore the future returned by put. To mitigate read latency and read API costs (GETs), SlateDB will use standard LSM-tree caching techniques: in-memory block caches, compression, bloom filters, and local SST disk caches. https://ift.tt/jEYONsb August 15, 2024 at 12:47AM
Show HN: SlateDB – An embedded storage engine built on object storage https://ift.tt/Vrx5KDC
Related Articles
SHOW HN: I coded an espresso brewing app to keep all information at one place https://ift.tt/urUDKtWSHOW HN: I coded an espresso brewing app to keep all information at on… Read More
Show HN: Serverless VPN for Lifetime https://ift.tt/bHQWNe8Show HN: Serverless VPN for Lifetime https://ift.tt/NrJnYFH April 25, … Read More
Show HN: A blogging platform based on Markdown and HTML (written in Rust) https://ift.tt/ysJd6x2Show HN: A blogging platform based on Markdown and HTML (written in Ru… Read More
Show HN: Use LLMs to Interact with APIs https://ift.tt/xgrndOtShow HN: Use LLMs to Interact with APIs Recently featured in a LangCha… Read More
Show HN: Just launched the Bento creation tool to rule them all https://ift.tt/7wLcTurShow HN: Just launched the Bento creation tool to rule them all https:… Read More
Show HN: ReaperAI – Automatically delete dead code from your app https://ift.tt/5upJ3kXShow HN: ReaperAI – Automatically delete dead code from your app Hi al… Read More
Show HN: I've developed an open-source proxy server for front end applications https://ift.tt/UVpjQ4AShow HN: I've developed an open-source proxy server for front end appl… Read More
Show HN: Open-source alternative to HashiCorp/IBM Vault https://ift.tt/Ig5rYHqShow HN: Open-source alternative to HashiCorp/IBM Vault https://ift.tt… Read More
0 Comments: