Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

routing: Add triert package with Xor Trie backed routing table #50

Merged
merged 32 commits into from
Jul 13, 2023
Merged
Changes from 1 commit
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
77203e1
routing: Add triert package with Xor Trie backed routing table
iand Jul 4, 2023
7480743
go mod tidy
iand Jul 4, 2023
c3ea4a4
Remove unused package
iand Jul 4, 2023
b1c73aa
Fix comment
iand Jul 4, 2023
f8be3dc
Remove unused bucket size arg from New
iand Jul 4, 2023
e3df916
Update comment
iand Jul 4, 2023
9edf486
Remove use of Hex in keyNodes map
iand Jul 4, 2023
c323ac3
Remove unused import
iand Jul 4, 2023
e01bef1
Remove context from closestAtDepth
iand Jul 5, 2023
f3338f8
Simplify NearestPeers
iand Jul 5, 2023
afc7f65
Fix CplSize
iand Jul 5, 2023
9b99b49
Add trie implementation
iand Jul 5, 2023
e882e36
Add trie implementation
iand Jul 5, 2023
535956b
Tidy up trie package
iand Jul 5, 2023
bfba04d
Improve trie test coverage
iand Jul 5, 2023
29e5c75
go mod tidy
iand Jul 5, 2023
ba30518
Add BenchmarkBuildTable
iand Jul 7, 2023
ff09661
Add mutable Add/Remove to trie
iand Jul 10, 2023
f7ef0ca
Add benchmarks for key/trie
iand Jul 10, 2023
cc37031
Add benchmarks for routing/triert
iand Jul 10, 2023
5b7a773
Add churn benchmark
iand Jul 10, 2023
e70e4ee
Convert trie README to doc.go
iand Jul 11, 2023
b5a1c51
Remove concurrent support
iand Jul 11, 2023
e92cc80
Fix build on older versions of Go
iand Jul 13, 2023
ce88d3b
Fix build on older versions of Go again
iand Jul 13, 2023
e262f29
Fix method signature of ReportTimePerItemMetric on pre 1.20 versions …
iand Jul 13, 2023
608229b
Add key filters
iand Jul 13, 2023
08b8669
Fix lint issues
iand Jul 13, 2023
29a8ef1
Panic if BitAt index out of range
iand Jul 13, 2023
5e3f1c9
Move merge keyutil package into testutil
iand Jul 13, 2023
0b284ff
Check input length of RandomWithPrefix
iand Jul 13, 2023
23f01c1
Random key generation uses bitlength instead of byte length
iand Jul 13, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Remove context from closestAtDepth
iand committed Jul 10, 2023
commit e01bef18f7cdf95849cb5d510d8324f312edce34
28 changes: 10 additions & 18 deletions routing/triert/table.go
Original file line number Diff line number Diff line change
@@ -121,10 +121,11 @@ func (rt *TrieRT) NearestPeers(ctx context.Context, kk key.KadKey, n int) ([]add
}

keys := rt.keys.Load().(*trie.Trie)
closestKeys, err := closestAtDepth(ctx, tkey.Key(kk), keys, 0, n, make([]tkey.Key, 0, n))
if err != nil {
return nil, err
closestKeys := closestAtDepth(tkey.Key(kk), keys, 0, n, make([]tkey.Key, 0, n))
if len(closestKeys) == 0 {
return []address.NodeID{}, nil
}

keyNodes := rt.keyNodes.Load().(map[string]address.NodeID)

nodes := make([]address.NodeID, 0, len(closestKeys))
@@ -137,14 +138,10 @@ func (rt *TrieRT) NearestPeers(ctx context.Context, kk key.KadKey, n int) ([]add
return nodes, nil
}

func closestAtDepth(ctx context.Context, k tkey.Key, t *trie.Trie, depth int, n int, found []tkey.Key) ([]tkey.Key, error) {
if ctx.Err() != nil {
return found, ctx.Err()
}

func closestAtDepth(k tkey.Key, t *trie.Trie, depth int, n int, found []tkey.Key) []tkey.Key {
// If we've already found enough peers, abort.
if n == len(found) {
return found, nil
return found
}

// Find the closest direction.
@@ -158,20 +155,15 @@ func closestAtDepth(ctx context.Context, k tkey.Key, t *trie.Trie, depth int, n
chosenDir = 1 - dir
} else if t.Key != nil {
// We've found a leaf
return append(found, t.Key), nil
return append(found, t.Key)
} else {
// We've found an empty node?
return found, nil
return found
}

var err error
// Add peers from the closest direction first, then from the other direction.
found, err = closestAtDepth(ctx, k, t.Branch[chosenDir], depth+1, n, found)
if err != nil {
return found, err
}

return closestAtDepth(ctx, k, t.Branch[1-chosenDir], depth+1, n, found)
found = closestAtDepth(k, t.Branch[chosenDir], depth+1, n, found)
return closestAtDepth(k, t.Branch[1-chosenDir], depth+1, n, found)
}

func (rt *TrieRT) Find(ctx context.Context, kk key.KadKey) (address.NodeID, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What will this function be used for? I can only find usages in tests (I checked this repo + the go-libp2p-kad-dht/kbucket repos)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be used to verify if a peer is already in the routing table or not, e.g in the context of libp2p/go-libp2p-kad-dht#811 where we send a kademlia request to the remote peer before adding it to the routing table.

In go-libp2p-kbucket this logic is implemented in https://github.com/libp2p/go-libp2p-kbucket/blob/6d85d4e63aa3b8e04b9b55e1de64be8a52532f21/table.go#L119. So there is no use case now (except testing), but it may be useful in the future.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's used in tests and potentially in other diagnostics / consistency checking. It's not part of the Table interface.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's possible we could un-export it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see the need to unexport it. It's a useful function for code that want to directly query the table