Show HN: SlateDB – An embedded storage engine built on object storage SlateDB is an embedded storage engine built as a log-structured merge-tree. Unlike traditional LSM-tree storage engines, SlateDB writes data to object storage (S3, GCS, ABS, MinIO, Tigris, and so on). Leveraging object storage allows SlateDB to provide bottomless storage capacity, high durability, and easy replication. The trade-off is that object storage has a higher latency and higher API cost than local disk. To mitigate high write API costs (PUTs), SlateDB batches writes. Rather than writing every put() call to object storage, MemTables are flushed periodically to object storage as a string-sorted table (SST). The flush interval is configurable. To mitigate write latency, SlateDB provides an async put method. Clients that prefer strong durability can await on put until the MemTable is flushed to object storage (trading latency for durability). Clients that prefer lower latency can simply ignore the future returned by put. To mitigate read latency and read API costs (GETs), SlateDB will use standard LSM-tree caching techniques: in-memory block caches, compression, bloom filters, and local SST disk caches. https://ift.tt/jEYONsb August 15, 2024 at 12:47AM
Show HN: SlateDB – An embedded storage engine built on object storage https://ift.tt/Vrx5KDC
Related Articles
Show HN: Release 0.8 of sbctl, Secure Boot key manager https://ift.tt/3qaQ7l8Show HN: Release 0.8 of sbctl, Secure Boot key manager https://ift.tt/… Read More
Show HN: I built my own second brain software tool https://ift.tt/3sqSIKIShow HN: I built my own second brain software tool https://ift.tt/3Fi8… Read More
Show HN: Python decorator that enables arbitrarily-deep tail/non-tail recursion https://ift.tt/3pdjvIfShow HN: Python decorator that enables arbitrarily-deep tail/non-tail … Read More
Show HN: Describe SQL using natural language, and execute against real data https://ift.tt/3p3pDTaShow HN: Describe SQL using natural language, and execute against real… Read More
Show HN: Two uBlock Origin rules to improve your Twitch experience https://ift.tt/3pkGtgPShow HN: Two uBlock Origin rules to improve your Twitch experience It … Read More
Show HN: Turn a Raspberry Pi into a Bluetooth MIDI Device https://ift.tt/3H1yknrShow HN: Turn a Raspberry Pi into a Bluetooth MIDI Device https://ift.… Read More
Show HN: HTML/CSS Photo Sheet https://ift.tt/3mqfMoUShow HN: HTML/CSS Photo Sheet https://ift.tt/3Fp7oxB December 24, 2021… Read More
Show HN: A Chrome extension to generate markup link reference for current page https://ift.tt/3H2xfvHShow HN: A Chrome extension to generate markup link reference for curr… Read More
0 Comments: