Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement iroh blobs #9

Merged
merged 88 commits into from
Oct 8, 2024
Merged

Implement iroh blobs #9

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
88 commits
Select commit Hold shift + click to select a range
36835ea
Add veilid-iroh-blobs' initial branch as a dependency
tripledoublev Sep 18, 2024
a11dd30
Update iroh-blobs dependency to version 0.24.0
tripledoublev Sep 18, 2024
dc2d685
Add constants for communication protocol flags
tripledoublev Sep 27, 2024
8147789
Update dependencies in Cargo.toml
tripledoublev Sep 27, 2024
0d69d9e
Refactor backend.rs to initialize VeilidIrohBlobs and handle shutdown
tripledoublev Sep 27, 2024
31990ce
add make_route function for creating custom private routes
tripledoublev Sep 27, 2024
b14ff6a
include VeilidIrohBlobs in Group struct and implement upload_blob fun…
tripledoublev Sep 27, 2024
c3c144a
include VeilidIrohBlobs to Repo and implement download_blob function
tripledoublev Sep 27, 2024
5479ad7
Update test to include VeilidIrohBlobs
tripledoublev Sep 27, 2024
7324996
simplify result handling in make_route function
tripledoublev Sep 27, 2024
8f81efc
Use is_empty instead of len
tripledoublev Sep 27, 2024
fc17b3d
Refactor error handling and remove unnecessary returns
tripledoublev Sep 30, 2024
be8631d
Refactor make_route function for ordered sequencing
tripledoublev Sep 30, 2024
02b63f3
Add logs
tripledoublev Sep 30, 2024
24bc195
Remove unnecessary reference to veilid_api
tripledoublev Sep 30, 2024
12296ce
Remove download_blob and use download_file_from instead
tripledoublev Sep 30, 2024
e90f177
Add get_route_id_blob method to DHTEntity
tripledoublev Sep 30, 2024
f44bed9
Add get_route_id_blob method to Repo
tripledoublev Sep 30, 2024
3346309
Add get_route_id_blob method to Group
tripledoublev Sep 30, 2024
9a3a6e5
Serialize tests
tripledoublev Sep 30, 2024
953b934
Add new constants for data processing
tripledoublev Sep 30, 2024
827ff08
Create blob_transfer_test
tripledoublev Sep 30, 2024
58d1a0b
Import CryptoKey from veilid_core
tripledoublev Oct 1, 2024
c15bfea
Normalize test names
tripledoublev Oct 1, 2024
4542fa2
Create group_creation test
tripledoublev Oct 1, 2024
351462e
Create keypair_storage_and_retrieval test
tripledoublev Oct 1, 2024
fb06110
Create repo_creation test
tripledoublev Oct 1, 2024
d291cc4
Create message_sending_via_private_route
tripledoublev Oct 1, 2024
36e5640
Create group_name_persistence
tripledoublev Oct 1, 2024
0a8e326
Run tests with single thread and verbose output
tripledoublev Oct 1, 2024
5f52265
Reorder test flags for verbose output and single thread execution
tripledoublev Oct 1, 2024
e8005fa
Remove --verbose and --nocapture
tripledoublev Oct 2, 2024
06d9a37
Refactor Repo struct to remove id field and add id() method
tripledoublev Oct 2, 2024
8e85987
Refactor CommonKeypair struct to remove public_key field
tripledoublev Oct 2, 2024
ad8aaa1
Refactor CommonKeypair struct to remove public_key fieldin backend.rs
tripledoublev Oct 2, 2024
5549512
Update repo creation logic in backend.rs
tripledoublev Oct 2, 2024
9baf6bd
Refactor get_repo method in backend.rs to load keypair using repo ID …
tripledoublev Oct 2, 2024
670c499
Update tests, add repo_persistence test
tripledoublev Oct 2, 2024
7ce3e23
Reintroduce public_key field to CommonKeypair after premature removal
tripledoublev Oct 2, 2024
daac28b
Enable verbose output and prevent test output from being captured
tripledoublev Oct 2, 2024
dbfd247
Add iroh dependency to Cargo.toml
tripledoublev Oct 2, 2024
a334163
when creating a new group create a en empty iroh collection and set t…
tripledoublev Oct 2, 2024
93ba843
Add DHT writing logic to upload_blob in Group
tripledoublev Oct 2, 2024
5c4389b
Add test for upload_blob to verify file upload and DHT storage
tripledoublev Oct 2, 2024
018be9e
test will stop if it exceeds 10 minutes to prevent it from hanging in…
tripledoublev Oct 2, 2024
631f2fa
ensures that the async block returns Result
tripledoublev Oct 2, 2024
0953fc6
Create DHTRecordInfo struct for storage in protected store
tripledoublev Oct 2, 2024
e285bdd
Add DHTRecordInfo struct for storage in protected store in upload_blob
tripledoublev Oct 2, 2024
2899f4c
Create upload_blob_and_verify_protected_store test
tripledoublev Oct 2, 2024
f63a4aa
Increase timeout duration in sending_message_via_private_route test
tripledoublev Oct 3, 2024
8845bc9
Merge branch 'OpenArchive:main' into implement-iroh-blobs
tripledoublev Oct 3, 2024
8051152
Merge branch 'main' into implement-iroh-blobs
tripledoublev Oct 3, 2024
d50fd96
adds the `iroh_blobs` field to the `Group`, await the result of `self…
tripledoublev Oct 3, 2024
b578810
Remove duplicated imports in group.rs
tripledoublev Oct 3, 2024
842a67a
Remove duplicated import in lib.rs
tripledoublev Oct 3, 2024
8f1dff7
Restructure blob uploads, create_Repo takes group id, save route and …
Oct 4, 2024
ce2a50b
Download hash from peers
Oct 5, 2024
14a3018
get_stream_from_hash in group
Oct 5, 2024
587a941
Use latest blobs code
Oct 7, 2024
8fb296b
Fix clippy warnings; refactor iterator and pattern matching
tripledoublev Oct 7, 2024
9bc8168
Merge branch 'implement-iroh-blobs' of https://github.com/tripledoubl…
tripledoublev Oct 7, 2024
8e735ae
Add test for listing repositories in a group (list_repos_test)
tripledoublev Oct 7, 2024
987e4d6
Add test for getting own repository in a group (get_own_repo_test)
tripledoublev Oct 7, 2024
95be255
Add test for creating a collection and uploading a file via the backend
tripledoublev Oct 7, 2024
79ec187
Add test for deleting a file from a collection via the backend
tripledoublev Oct 7, 2024
d866d78
Add tests for downloading hash from peers and checking if peers have …
tripledoublev Oct 7, 2024
c478862
Merge pull request #3 from tripledoublev/add-more-tests
tripledoublev Oct 7, 2024
0c715a2
Merge pull request #2 from tripledoublev/add-new-tests
tripledoublev Oct 7, 2024
7bb79eb
Add load_known_groups to backend
Oct 7, 2024
9b4c85d
Add collection management methods for repo
tripledoublev Oct 7, 2024
bfa8cec
Add test for collection management methods for repo
tripledoublev Oct 7, 2024
cc9cef5
replace get_file_from_repo_collection with get_file_hash
tripledoublev Oct 7, 2024
7651e9d
Make list_files_in_repo_collection private
tripledoublev Oct 7, 2024
c4a38ee
Making it short
tripledoublev Oct 7, 2024
d97f376
Making it shorter
tripledoublev Oct 7, 2024
e52e9bc
Create public method list_files that checks for permissions
tripledoublev Oct 7, 2024
44b22e8
Create update_collection_on_dht method
tripledoublev Oct 7, 2024
e9f7ed3
Update DHT on file deletion
tripledoublev Oct 7, 2024
f2849aa
Update DHT on Upload
tripledoublev Oct 7, 2024
0c43a37
Making it private
tripledoublev Oct 7, 2024
e28ba0d
Update src/repo.rs
tripledoublev Oct 7, 2024
01c141d
Use upload instead of upload_blob
tripledoublev Oct 7, 2024
56f6168
Make get_or_create_collection private, skip set_file_in_repo_collecti…
tripledoublev Oct 8, 2024
84c684c
Use get_or_create_collection when you read or write
tripledoublev Oct 8, 2024
ccc013b
Update test to reflect latest implementation
tripledoublev Oct 8, 2024
002385b
Merge pull request #4 from tripledoublev/add-collection-to-repo
tripledoublev Oct 8, 2024
d01ab1f
Remove unnecessary use of `to_string`
tripledoublev Oct 8, 2024
e79ecfc
Change &file_name to file_name
tripledoublev Oct 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,10 @@ version = "0.1.0"
edition = "2021"

[dependencies]
iroh-blobs = "0.23.0"
veilid-core = { git = "https://gitlab.com/veilid/veilid.git", version = "0.3.4" }
eyre = "0.6"
iroh = "0.24.0"
iroh-blobs = "0.24.0"
veilid-core = { git = "https://gitlab.com/veilid/veilid.git", branch = "main" }
veilid-iroh-blobs = { git = "https://github.com/RangerMauve/veilid-iroh-blobs.git", branch = "default" }
tracing = "0.1"
xdg = "2.4"
tmpdir = "1"
Expand All @@ -15,7 +16,11 @@ serde_cbor = "0.11.2"
clap = "4.5.9"
anyhow = "1.0.86"
tokio = {version ="1.39.3", features=["full"] }
tokio-stream = "0.1.16"
async-stream = "0.3.5"
futures-core = "0.3.30"
bytes = "1.6.1"
serial_test = "3.1.1"
url = "2.5.2"
hex = "0.4.3"
rand = "0.8.5"
Loading
Loading