summary refs log tree commit diff
path: root/ripple/fossil/src/lib.rs
AgeCommit message (Collapse)Author
2022-06-19ripple/fossil: box the bao decoderedef
Shuffling 3k around on the stack doesn't make anyone happy. Change-Id: I444fc22267380d9b99ca63ca148b9a9e85238b5a
2022-06-13ripple/fossil: use std::os::unix::preludeedef
We were accidentally grabbing things from the prelude module already, and this is the cleaner fix. Change-Id: I2c6b669164df11f467eae495d109e096eb267b59
2022-05-10ripple/fossil: permit importing trees from git reposedef
Change-Id: I8329f14bddaaa2d141e82c087821e4602bd1259a
2022-05-03ripple/fossil: handle missing blobs more gracefullyedef
Change-Id: I8a928b57ecc81bea31d757e73b9ece9474628db4
2022-05-03ripple/fossil: fix clippy nitedef
Change-Id: I2c9b2a15ac066ec2295d54665afd301f396efdc1
2022-05-03ripple/fossil: don't expose protobufs in the frontendedef
Previously, the CLI took Directory protobufs as input or wrote them as output. Now we just deal in store hashes. Change-Id: I5e0f0f33929ede43d971080c33bdb865f1832b2e
2022-05-03ripple/fossil: add digest_from_str and digest_stredef
These decode digests to and from zbase32 for user-facing uses. Change-Id: Ibd2db960044a97812d18d1a3c107521d78bd7f24
2022-05-03ripple/fossil: expose add_directoryedef
Change-Id: I8e976279bd7aaaaf325129dc5c68a6ca5c750dc6
2022-05-03ripple/fossil: don't recursively fsyncedef
`add` takes about 10 seconds to run on a full LLVM tree, unless it were to spend 4 minutes mostly waiting for a series of tiny fsyncs. It did. Change-Id: I492604bae68e3472f8626a112a33d023947e0e86
2022-05-01ripple/fossil: benchmark Chunker using Criterionedef
Performance hovers around 300MiB/s on my machine. Change-Id: I387ccbf065c0b667824ede0675e6a295722f6d4b
2022-04-28ripple/fossil: use to_be_bytes rather than write_u64edef
Change-Id: I7eb02482772f48ca9f486f514b89652a9c5730cd
2022-04-28ripple/fossil: ensure durability of add_pathedef
sled doesn't actually promise not to eat your data until you invoke flush. This is observable under normal circumstances as add_path occasionally just not committing anything to the store. While we're at it, ensure we're syncing the chunks file data to disk, so the database *might* actually be consistent. We're not going for full crash-correctness yet, mostly for performance and complexity reasons. Change-Id: I6cc69a013dc18cd50df26e73801b185de596565c
2022-04-28ripple/fossil: deduplicate using content-defined chunkingedef
This implements casync-style content-defined chunking and deduplication. Change-Id: I42a9b98e1140bed462a5ae1e0aba508bebc9fa0e
2022-04-28ripple/fossil: outline write_blob_inneredef
Change-Id: I1c4a5d16cf10c464f9835c961481da221aa0d12e
2022-04-26ripple/fossil: use the default sled tree for global metadata, not blobsedef
Change-Id: I358a5c354c19cc8cb0a75629758fda476629406d
2022-04-25ripple/fossil: implement incremental blob readingedef
Change-Id: I69ae53824e149133aa6bb61dda201f972c840b1f
2022-04-25ripple/fossil: store bao outboard tree in the blob metadataedef
One step closer to genuine incremental blob reads. Change-Id: I796710820c1b69baad91a6dc65f9d7f0dee311d3
2022-04-25ripple/fossil: store blob metadata as protobufedef
Change-Id: I12c590d842471bf543f16fd21056224d8a7c0857
2022-04-24ripple/fossil: implement io::Seek for RawBlobedef
Change-Id: I625530fe2f4db89be5889e46f0a5ed50727c8cd1
2022-04-23ripple/fossil: implement a more lightweight blob store backendedef
The sled chunk store works fine, but has pretty awful performance, since sled just doesn't make a particularly good large blob store. This change replaces it with a file-based store using sled purely as a metadata store. Adding LLVM's source tree takes about 120s before, and about 15s after. Change-Id: I5fb22ea79a006fa6bcf5351921038f57f2484112
2022-04-20ripple/fossil: prefer anyhow::Result over io::Resultedef
Change-Id: I4a94b84ef456b427422757a899fdce6198fd01a1
2022-04-19ripple/fossil: use bao to one-shot verify hashesedef
Change-Id: I77ace8ee9f69ccb92afaa0a41d69538d28f11583
2022-04-19ripple: upgrade blake3 (0.3.8 -> 1.3.1)edef
Change-Id: I75f2e0ff57e09b026fd1aaaeb86b041ddb8238f4
2022-04-19ripple/fossil: prepare for seekable, streaming blob readingedef
This implements blob reading in terms of RawBlob, a fairly naive streaming blob reader. For now, we still only use it for simple one-shot reads. Change-Id: Iecd4f926412b474ca6f3dde8c6055c0c3781301f
2022-04-19ripple/fossil: add read_write testedef
Change-Id: I88d13d9dd7055b8370706df7b3dd4479a0891399
2022-04-18ripple/fossil: use blake3::CHUNK_LEN as chunk sizeV
This will pave the way for BLAKE3 verified streaming, so we won't have to read objects into memory in their entirety. Change-Id: Ic68dee2ad81448db4969b8c423f0876f0e0272e0
2022-04-12ripple/fossil/mount: serve an in-memory filesystem treeedef
Change-Id: I92073aae32bcf603799b83717635a13edbbef190
2022-04-12ripple/fossil: correctly handle the empty blobedef
Change-Id: I6a91a998063e70ec92e8239361a37e3f8033b049
2022-04-11ripple/fossil: don't .ok().expect()edef
Result::expect already does the right thing, and discards less information. Change-Id: I2d2fbd1981d4c3b84bb287fb4d79d985f8e465c9
2022-04-11ripple/fossil: track file sizeedef
Change-Id: I424bd482d82471255f2ce2f17bb0b5c6eae3b77a
2022-04-10ripple/fossil: split out FileRef/DirectoryRefedef
Change-Id: I649c89ccc4e7fbc3ce42c86f6653d59c07cf58a9
2022-04-10ripple/fossil: track transitive inode countedef
Change-Id: Ia5893bc64f58e3a6b17aa96fdd27b6b49ebf5fa5
2022-04-10ripple/fossil: just copy the slice refedef
Thanks, Clippy! Change-Id: I88e6b94b8edd415d1ed79c288a4677e2040dde77
2022-04-10ripple/fossil: use [T]::is_emptyedef
Thanks, Clippy! Change-Id: I9d8d154db76c49807262bb07f37aec0c17725739
2021-08-22ripple/fossil: prefer anonymous type parametersedef
This is mostly a stylistic distinction, but it helps with readability. Change-Id: Ia6b3dca53638f12bcfd498b5f36e7abf16fd061a
2021-08-14ripple/fossil: a basic content-addressable storeedef
Fossil stores content-addressed blobs of file contents and Protobuf-encoded directory listings, backed by Sled. Change-Id: I8b49de6342218ca00755cec980b1d0cfb18878a7
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-22 20:08:15 +0000