Code

Add raw file backend#991

Code

Merged

mkeeter merged 62 commits intooxidecomputer:mainoxidecomputer/crucible:mainfrom mkeeter:raw-file-backendmkeeter/crucible:raw-file-backendCopy head branch name to clipboardNov 9, 2023

+3,829-332

Commits

Commits on Nov 7, 2023

• Add raw file backend

  Show description for 2362e5e
  mkeeter

  committed
  2362e5e

• Update comments

  mkeeter

  committed
  b4a6f68

• Bump version

  mkeeter

  committed
  a3bc2da

• Implement fast migration

  mkeeter

  committed
  a7461af

• Further optimization

  mkeeter

  committed
  3ae42f4

• Fix ordering

  mkeeter

  committed
  8a71b11

• Preallocate during migration

  mkeeter

  committed
  1e02809

• Better definition of EXTENT_META_SQLITE

  mkeeter

  committed
  e247dfb

• Updates per review

  mkeeter

  committed
  0f5ddca

• Rephrase

  mkeeter

  committed
  837ec44

• Remove outdated comment

  mkeeter

  committed
  01dc06b

• Clarify read-only snapshot situation

  mkeeter

  committed
  5400ae8

• extent-read-done probe is elsewhere

  mkeeter

  committed
  b52ca74

• Make active_context always valid (instead of optional)

  mkeeter

  committed
  cb9a772

• Shuffle array order

  mkeeter

  committed
  3ffbc0a

• Almost fixed reading file from disk

  mkeeter

  committed
  e14c5cd

• Fix migration

  mkeeter

  committed
  10ce166

• Preparing to coalesce context writes

  mkeeter

  committed
  a71b807

• Write metadata in (semi) bulk

  mkeeter

  committed
  4e5c823

• Add DTrace probe for write count

  mkeeter

  committed
  4412b84

• Add unit test for serialized sizes

  mkeeter

  committed
  600be6e

• Shrink metadata sizes

  mkeeter

  committed
  867fd56

• Coalesce block context reads

  mkeeter

  committed
  02ab716

• Remove seeking in favor of pread

  mkeeter

  committed
  ba02080

• Implement context defragmentation

  mkeeter

  committed
  badd2f9

• Only defragment if we're doing extra work

  mkeeter

  committed
  5563c2a

• Only hash the file when slot is ambiguous

  mkeeter

  committed
  0ce8cab

• Fix indentation

  mkeeter

  committed
  e7bae58

• Encode active context in metadata region

  mkeeter

  committed
  de5f1a0

• Improvements and bug fixes

  mkeeter

  committed
  7bad3dd

• Move metadata to the very end of the file

  mkeeter

  committed
  e1e28b1

• Only get block contexts once (hopefully)

  mkeeter

  committed
  e5f1d22

• Only rehash if this isn't read-only

  mkeeter

  committed
  caa5b50

• Move file size to a function

  mkeeter

  committed
  3667b3a

• Add defragmentation test

  mkeeter

  committed
  863a1dc

• Move migration file knowledge to RawInner

  mkeeter

  committed
  b84cb78

• Add RawLayout class to manage layout

  mkeeter

  committed
  ea777ca

• Add docstring

  mkeeter

  committed
  c3ed14d

• Add support for alternate backends to integration tests

  mkeeter

  committed
  9642e66

• Add raw-on-SQLite integration test

  mkeeter

  committed
  786d498

• Add migration test

  mkeeter

  committed
  bd91778

• serialize_into should be infallible

  mkeeter

  committed
  26b6b6d

• Make open() and create() consistent (taking directory)

  mkeeter

  committed
  88cb382

• Fix last failing test

  mkeeter

  committed
  7b883ba

• More specific errors

  mkeeter

  committed
  8583774

• Remove debug stuff

  mkeeter

  committed
  ade7392

• Fix comment

  mkeeter

  committed
  7c6c217

• Typo

  mkeeter

  committed
  ec5cb52

• More comments

  mkeeter

  committed
  856c18b

• Add some assertions

  mkeeter

  committed
  b6ef63a

• Add assertion

  mkeeter

  committed
  dc81096

• More defragment tests

  mkeeter

  committed
  e94ea4e

• Check that defragment is only called on a clean extent

  mkeeter

  committed
  fb59d22

• Add multiple-write-to-same-block test

  mkeeter

  committed
  b66519d

• Post-rebase fixes

  mkeeter

  committed
  57a52b6

• Restore buf after get_active_contexts

  mkeeter

  committed
  a8b3705

• Update comment

  mkeeter

  committed
  416094e

• Add some handling of partial pread/pwrite(v)

  mkeeter

  committed
  2a5ed27

• Explain dirty / context slot mismatch

  mkeeter

  committed
  971ed1e

• Remove buf (it doesn't make things faster and uses up RAM)

  mkeeter

  committed
  868bca4

• Simplify dirty context slot tracking to reduce RAM

  mkeeter

  committed
  0d4f573

• Remove more premature optimization

  mkeeter

  committed
  8fe775b