Skip to content

Latest commit

 

History

History
231 lines (206 loc) · 30.4 KB

CHANGELOG.md

File metadata and controls

231 lines (206 loc) · 30.4 KB

v0.1.0 - 2022-10-28

We’re on the board 🎉! This first release of iroh brings a new implementation of IPFS to the world.

Key things to highlight about the first release of iroh:

  • Exchange Data with Kubo. Iroh can interoperate with kubo nodes on the IPFS network, pushing & fetching data.
  • Service Oriented architecture. Don't want p2p? Turn it off! Iroh will still work without it.
  • Single CLI to control all services.
  • Built for efficiency. Iroh's memory footprint & CPU usage are either on par or better than the best interoperable IPFS implementations out there.
  • Runs well on a laptop. We have a custom installer script that will configure iroh for your laptop. See our install docs

v0.1.0 ships as 4 binaries for Linux & macOS with zero external dependencies:

  • iroh - command line client
  • iroh-store - data storage daemon
  • iroh-p2p - peer 2 peer networking
  • iroh-gateway - IPFS-to-HTTP bridge

You'll need to download all of them & put them on your $PATH to work with iroh. Our install docs have more info.

Please do give iroh a try, we'd love to hear your feedback. Thanks!

Benchmarks

We're runnning benchmarks on a relatively stock AWS box for ease-of-replication. We don't have benchmarks for IPFS network retrieval this round.

Adding Single Files

Add a single file with iroh add --no-wrap --offline:

file size real (s) user(s) sys(s) cpu % cpu/total %
100K 0.0102 0.0162 0.0317 469.6078431 7.337622549
1M 0.0167 0.004 0.0519 334.7305389 5.230164671
10M 0.01345 0.0101 0.0418 385.87 6.03
100M 0.01345 0.0101 0.0418 385.87 6.03
1G 0.01345 0.0101 0.0418 385.87 6.03

Fetching Cached Gateway Content

Repeatedly request the same CID of different file sizes to measure cached content throughput via HTTP request

File size cpu/total (%) rps throughput/s avg latency
100K 90.48072917 40,064.27 3.85GB 5.08ms
1M 87.66614583 6,926.95 6.77GB 29.08ms
10M 87.34010417 711.55 6.98GB 278.86ms
100M 88.00885417 66.52 6.82GB 2.85s
1G 87.84947917 6.65 6.85GB 0.00s

Production Readiness

Is this production ready? No. We need your help to kick tires & find bugs!.

The v0.1.0 moniker should testify to just how new this software is. Please don't deploy it anywhere mission critical until we've had at least a month or two to address bugs & write tests.

Things you should know about this release:

There are two problems we couldn't properly address

  • There is no delete command. Data added to iroh will stay there until a future release where we add content removal. See issue #432 for details. The workaround for now is to blow away the storage database.
  • We have one particularly nasty known bug that can cause iroh to not persist data for highly nested directories upon add we plan to address ASAP in a patch release. See issue #423.

Bug Fixes

  • check and watch should display the same service names (#101) (ef99fe5)
  • add --no-metrics support to p2p (#65) (a782fcb)
  • add macos builds to ci (#33) (a0e79fb)
  • add missing metrics and update to fixed libp2p (0f4f467)
  • allow Raw as UnixfsType (#327) (275d88b)
  • allow concurrent rpc calls & de-duplicate provider requests (#60) (a74f8dd)
  • auto-raise FD limits where possible (#215) (9e429bb)
  • avoid 0 divisions (#76) (8f491d5)
  • bitswap sessions shutdown (6de85c9)
  • bitswap: fetch provider loop (93517a3)
  • ci release aarch64 linux (#184) (4cc66aa)
  • CI, add macOS & windows builds, cargo audit (5dbd907)
  • correctly check headers parse result (#227) (82a1b82)
  • correctly map memory channels (0742fb2), closes #165
  • default off metrics & tracing (#174) (bbc2ec1)
  • don't error on empty providers in the wrong place (414a8f2)
  • drop openssl requirement (#72) (b72ce04)
  • error when someone tries to start an unknown service (#431) (9a57fb7)
  • extend metrics (#32) (34c53da)
  • fix builds (#173) (932161c)
  • Fix test that for some reason assumes sort order in a vec (d57b12a)
  • gateway readme (#31) (e94a7e0)
  • gateway upgrade (#36) (0bb4fe9)
  • gateway: do not panic on missing location info (c518ce7)
  • gateway: populate directory listings (c7217f1)
  • gha disk space hacks (#387) (a9d9c25)
  • hamt directory listings (5187773), closes #192
  • histogram metric assignment (#119) (b75bbbd)
  • Improve error handling (#379) (7a945ed)
  • Improve error management and reporting when using unix domain sockets (#228) (8d13e4f)
  • iroh-car: limit to 4MiB not 4GiB (34eedb8)
  • iroh-gateway: add FileType::Raw option (#332) (ff4ef02)
  • iroh-gateway: increase timeouts (c3d56c6)
  • iroh-one: use default store path when no --store-path flag present (#317) (643fd43), closes #255 #309
  • limit max memory allocations per node (9817b5a)
  • loger only on p2p (b24c5fe)
  • macos releases (#34) (2fc0109)
  • make trace_id visible only if in use (#359) (c29bf85)
  • metrics rework & update (#69) (6c1d003)
  • metrics toggle on store (2b12158)
  • move bitswap msg counters out of encode/decode (#374) (85d80b9)
  • p2p subcommand output cleanup (#427) (0ca029c)
  • p2p: cleanup sessions when they are canceled (#419) (efa9e82)
  • p2p: do not panic on shutdown of bs req (b05b94a)
  • p2p: enable rsa key keys again (afeafb4)
  • p2p: ensure default grpc connections work (eec218d)
  • p2p: Increase the size of the kad mem store (#320) (00f5660)
  • p2p: record Kad, Identify and Ping metrics (af8eaa9)
  • p2p: remove debug logs (#420) (23e1b3a)
  • parameters and connection handling (96514d3)
  • params (#128) (1f47d7c)
  • pass links to store when adding data (#299) (9476d9c)
  • prom metrics no longer ping on --no-metrics (#44) (7f9c81a)
  • properly saves filenames for hamt dirs (#422) (1fe8e69)
  • properly setup the default config for iroh-one (#256) (a1e4940)
  • protoc from release bins (#42) (28cc3ab)
  • require passing providers to fetch_bitswap (771ee00), closes #73
  • resolver: handle trailing slashes in the path with more sense (#414) (9ef95e8)
  • resolver: ipld, double-fetching, and trailing slash fixes (#408) (f861102)
  • resolver: stop sessions when requests are done (9c46cb0)
  • rpc creation no longer blocks the gateway (#111) (8f1d93c)
  • rpc-client: correct address format for clients (8cde5cf)
  • rpc-client: use multihash to query providers (2b11a14)
  • stop preventing any directory or file foo from existing in the project (#334) (3ef7ccc)
  • switch back to rust-libp2p master (1336e9e)
  • tracing on request level (#85) (562e6d8)
  • undo label change (#219) (3433340)
  • Update ipld in iroh-car to limit prost duplicates (#307) (ee43bd7)
  • use rustls (#64) (0c37cef)
  • use the uds-gateway feature only to control the http uds endpoint (#252) (9bcbb59)
  • windows symlink support (#386) (563aac0)

Features

  • iroh-ctl add & iroh-ctl get (#164) (979a36f)
  • iroh-ctl status -w watches the health of each iroh process (#84) (22b2bb5)
  • abstract over internal rpc mechanism & add support for UDS & Mem RPC (6e859c2)
  • add car importer (aabfb9a)
  • add iroh-car (#27) (26a1b86), closes #9
  • add placeholder iroh-bitswap and iroh-p2p crates (9c4ab10)
  • add [email protected] (b08ac2b)
  • add trace id to responses (#28) (9d9c89e)
  • Allow to pass custom resolvers (#220) (9d319c5)
  • bad bits implementation (#172) (bda0173)
  • bitswap: add dial timeouts (06e0104)
  • bitswap: add query errors and start cleanups (72ae431)
  • bitswap: allow tracking queries based on ids (21f75d5)
  • bitswap: custom protocol handler for bitswap (ff2ebc3)
  • bitswap: refactor & improve general strategy (0747310)
  • CLI get command improvements (#331) (b18b6c8), closes #269
  • CLI tests for iroh add, and API changes (#343) (fdf2170)
  • connect the dots (4b15df5)
  • connection pooling (#404) (b499751)
  • ctl ci (#175) (b7ebf02)
  • deps: update libp2p to 0.47 (9dadadc)
  • distributed tracing (#46) (3e555bc)
  • don't run 'dht_nice_tick()' when kadmelia is disabled (#242) (050ad48)
  • Enable the CompressionLayer middleware (#236) (9151fc1)
  • fetch providers via bitswap (b16be5a)
  • gateway upgrade (#87) (4732a76)
  • gateway: initial directory listing support (#328) (3316a53)
  • get config from files, environment vars, & command line flags (#112) (652b7ff)
  • implement basic iroh-store (a1b9586)
  • implement iroh-ctl find-provs (#182) (ec06238)
  • import flatfs-store and rocks-store crates (4ac82a8)
  • importer: add hash verification (96eda12)
  • improve add experience (#401) (4f86388)
  • improve CLI texts and flags (#342) (9e0a03e)
  • improve provider fetching (38ce246)
  • iroh-api rename, iroh get basics, rpc client construction (d24edc1)
  • iroh-api: implement lookup & connect (#372) (a93dfb4)
  • iroh-ctl: implement available commands (#123) (c6bdca3)
  • iroh-one (#212) (11fe705)
  • iroh-resolver: create a balanced tree dag (3285d1f)
  • iroh-resolver: implement unixfs dir listing (263b3c2)
  • iroh-store: add minimal main implementation (b1c49a1)
  • Map id to cid instead of multihash (#336) (092bdf7), closes #335
  • offline adding (#415) (7725508)
  • p2p api service calls with a p2p-specific error, better feedback (#402) (b565231)
  • p2p: add gossipsub to p2p node & gossipsub rpc methods (#132) (f18d726)
  • p2p: improve default configurations for dht and bs (791181a)
  • p2p: integrate autonat protocol (5c5c1b0)
  • p2p: integrate relay and dcutr (6555090)
  • p2p: make bitswap toglable (648bcbf)
  • p2p: make gossipsub configurable (071fd1b)
  • p2p: persist identity (8f57278)
  • p2p: refresh kad buckets & rebootstrap regularly (339ed13)
  • p2p: respond to bitswap requests (965e21c)
  • progress bar for iroh add command (#368) (bff3560)
  • provide a mechanism to check if an iroh program is already running. (#293) (bddbae9)
  • provide only root, and only at the end of add (#406) (96c6148)
  • race a http gateway with p2p in iroh-one's content loader (#338) (40cb232)
  • range request support (#330) (074f0c6)
  • recursive directories & directory link limits (#213) (be4e931)
  • relases for p2p, store & some refactoring (#45) (d101e5a)
  • rename the iroh-ctl crate to the iroh crate (#321) (8b02977)
  • resolver: add support for raw leaves in unixfs (1d6d57d)
  • resolver: expose metadata about the resolution result (a229875)
  • resolver: expose way to get directory content (9b59326)
  • resolver: handle symlinks properly (5cc84ea)
  • resolver: implement dsnlink resolution (5fa893d)
  • resolver: stream content (3d419b5)
  • resolver: support raw codec (25d5b28)
  • resolver: unixfs: resolve simple multi chunk files (fa94b6d)
  • rpc client that can handle out-going commands and incoming requests (#26) (f2fd5cb)
  • rpc: add rpc client and server for store process (#41) (854b605)
  • streaming providers (a810e07)
  • switch to [email protected] (f1dd5f7)
  • Unixfs Improvements (42dee06)
  • unixfs: add filesize and blocksizes for UnixfsNode::File nodes (#254) (6ba6b57)
  • unixfs: add symlink support (#337) (3157c15)
  • update libp2p to latest (39b25e8)
  • update rust-libp2p (c595c4d)
  • update to [email protected] (#391) (8a906ee)
  • upload gateway builds to s3 (linux/amd64) (#29) (1918a83)
  • use the store to cache received data (9dac53a)

Performance Improvements

  • add parallelism to hashing (#411) (12db7ed)
  • impl put_many and reuse column families (#412) (4cab5c7)
  • resolver: avoid some allocations (9f2d718)
  • use content_bytes instead of content_reader (9dd0c22)
  • Use the libipld built in way to scrape references (93546b1)

Reverts