Show HN: SlateDB – An embedded storage engine built on object storage SlateDB is an embedded storage engine built as a log-structured merge-tree. Unlike traditional LSM-tree storage engines, SlateDB writes data to object storage (S3, GCS, ABS, MinIO, Tigris, and so on). Leveraging object storage allows SlateDB to provide bottomless storage capacity, high durability, and easy replication. The trade-off is that object storage has a higher latency and higher API cost than local disk. To mitigate high write API costs (PUTs), SlateDB batches writes. Rather than writing every put() call to object storage, MemTables are flushed periodically to object storage as a string-sorted table (SST). The flush interval is configurable. To mitigate write latency, SlateDB provides an async put method. Clients that prefer strong durability can await on put until the MemTable is flushed to object storage (trading latency for durability). Clients that prefer lower latency can simply ignore the future returned by put. To mitigate read latency and read API costs (GETs), SlateDB will use standard LSM-tree caching techniques: in-memory block caches, compression, bloom filters, and local SST disk caches. https://ift.tt/jEYONsb August 15, 2024 at 12:47AM
Show HN: SlateDB – An embedded storage engine built on object storage https://ift.tt/Vrx5KDC
Related Articles
Show HN: Weekend art project Voronoi Virus https://ift.tt/5RZjh9iShow HN: Weekend art project Voronoi Virus https://ift.tt/7Ocb5uA Febr… Read More
Show HN: CLI for generating beautiful PDF for offline reading https://ift.tt/YMf761mShow HN: CLI for generating beautiful PDF for offline reading I've alw… Read More
Show HN: Open-Source Pong Game https://ift.tt/VizD165Show HN: Open-Source Pong Game Need feedback on this game folks! I'm o… Read More
Show HN: Molerat – Small Web Protocol https://ift.tt/obWFCTeShow HN: Molerat – Small Web Protocol https://ift.tt/FDofKbR February … Read More
Show HN: USD 0.99/TB/month cloud storage https://ift.tt/OWfmnqEShow HN: USD 0.99/TB/month cloud storage https://ift.tt/YhlVR7w Februa… Read More
Show HN: Unofficial Google Lens OCR API https://ift.tt/wpGkqMAShow HN: Unofficial Google Lens OCR API Default OCR in ShareX is prett… Read More
Show HN: CPU Prices on eBay https://ift.tt/ja0voBOShow HN: CPU Prices on eBay Tech stack: Go + templ + htmx There are so… Read More
Show HN: Aidely is AI powered thread; AI and Humans cooperation https://ift.tt/LFMWUNDShow HN: Aidely is AI powered thread; AI and Humans cooperation Discov… Read More
0 Comments: