Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added client for tlog-tiles #49

Merged
merged 3 commits into from
Jul 17, 2024

Conversation

mhutchinson
Copy link
Contributor

@mhutchinson mhutchinson commented Jul 10, 2024

Added client for tlog-tiles logs.

This was largely copied from serverless-log and minimally changed to
bind this to the tlog-tiles API which has a few differences from the
serverless API. The obvious changes are the URL paths, but that is
largely hidden behind the layout API. The more significant delta is
that serverless returns fully populated tiles, where tlog-tiles returns
only the leaf nodes and thus the client needs to perform some
repopulation of the internal nodes in order to get at all node hashes
for proof generation.

The test data is largely the same as the serverless-log test data in
shape, though it needed to be completely regenerated due to the above
changes. This required the posix log implementation to be ported across
to the tlog-tiles API too. I have this in a local branch and will get it
up in its own PR after this one.

This client is intended to be a minimal foundation on which we can build.
It has known inefficiencies such as only returning a single leaf from an
entry bundle, and not storing the results of computed internal hashes
within a tile. These will be addressed as I write the hammer tool that uses
this client.

Towards #66.

@mhutchinson mhutchinson force-pushed the tiles-client branch 4 times, most recently from 9be7adc to 1e0076d Compare July 17, 2024 09:17
@mhutchinson mhutchinson marked this pull request as ready for review July 17, 2024 09:19
@mhutchinson mhutchinson requested a review from AlCutter July 17, 2024 09:19
@mhutchinson mhutchinson changed the title WIP: Added client for tlog-tiles Added client for tlog-tiles Jul 17, 2024
@AlCutter
Copy link
Collaborator

Would it be possible to insert a first "import" commit that just copies over the original code verbatim, and then rebase this commit on top? It'll make it a lot easier to spot the diffs looking at it commit-by-commit

This was largely copied from serverless-log and minimally changed to
bind this to the tlog-tiles API which has a few differences from the
serverless API. The obvious changes are the URL paths, but that is
largely hidden behind the `layout` API. The more significant delta is
that serverless returns fully populated tiles, where tlog-tiles returns
only the leaf nodes and thus the client needs to perform some
repopulation of the internal nodes in order to get at all node hashes
for proof generation.

The test data is largely the same as the serverless-log test data in
shape, though it needed to be completely regenerated due to the above
changes. This required the posix log implementation to be ported across
to the tlog-tiles API too. I have this in a local branch and will get it
up in its own PR after this one.
@mhutchinson
Copy link
Contributor Author

Would it be possible to insert a first "import" commit that just copies over the original code verbatim, and then rebase this commit on top? It'll make it a lot easier to spot the diffs looking at it commit-by-commit

Done. Right on the limits of my git-fu, but it's done.

@mhutchinson mhutchinson mentioned this pull request Jul 17, 2024
Copy link
Collaborator

@AlCutter AlCutter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed IRL: this is intended to be a quick client which unblocks the hammer/larger testing rather than an optimised client. The intention is for us to double back at a later date to improve it.

@mhutchinson
Copy link
Contributor Author

Discussed IRL: this is intended to be a quick client which unblocks the hammer/larger testing rather than an optimised client. The intention is for us to double back at a later date to improve it.

Yeah and this is now clear in this PR description and in #66, which will stay open until the known rough edges are resolved.

@mhutchinson mhutchinson merged commit 18d1999 into transparency-dev:main Jul 17, 2024
4 checks passed
@mhutchinson mhutchinson deleted the tiles-client branch July 17, 2024 10:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants