Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pczt: Define the structure and semantics of the PCZT format #1577

Merged
merged 18 commits into from
Dec 9, 2024
Merged

Conversation

str4d
Copy link
Contributor

@str4d str4d commented Oct 16, 2024

No description provided.

Copy link

codecov bot commented Oct 16, 2024

Codecov Report

Attention: Patch coverage is 3.97112% with 798 lines in your changes missing coverage. Please review.

Project coverage is 54.58%. Comparing base (f6040a1) to head (0cceac1).
Report is 25 commits behind head on main.

Files with missing lines Patch % Lines
pczt/src/sapling.rs 0.00% 98 Missing ⚠️
pczt/src/orchard.rs 0.00% 87 Missing ⚠️
pczt/src/roles/tx_extractor/mod.rs 0.00% 79 Missing ⚠️
pczt/src/transparent.rs 0.00% 72 Missing ⚠️
pczt/src/roles/creator/mod.rs 0.00% 54 Missing ⚠️
...transaction/components/transparent/pczt/updater.rs 0.00% 50 Missing ⚠️
zcash_primitives/src/transaction/builder.rs 0.00% 44 Missing ⚠️
...action/components/transparent/pczt/tx_extractor.rs 0.00% 40 Missing ⚠️
...c/transaction/components/transparent/pczt/parse.rs 0.00% 38 Missing ⚠️
pczt/src/roles/combiner/mod.rs 8.33% 33 Missing ⚠️
... and 15 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1577      +/-   ##
==========================================
- Coverage   56.38%   54.58%   -1.80%     
==========================================
  Files         149      168      +19     
  Lines       19476    20371     +895     
==========================================
+ Hits        10981    11119     +138     
- Misses       8495     9252     +757     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

pczt/src/common.rs Outdated Show resolved Hide resolved
pczt/src/common.rs Outdated Show resolved Hide resolved
pczt/src/sapling.rs Outdated Show resolved Hide resolved
pczt/src/lib.rs Show resolved Hide resolved
pczt/src/orchard.rs Outdated Show resolved Hide resolved
Copy link

@conradoplg conradoplg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, added some comments about FROST usage

pczt/src/orchard.rs Outdated Show resolved Hide resolved
@conradoplg
Copy link

For ZSA issuance I think we need the fields required for computing the issuance_digest?

@str4d
Copy link
Contributor Author

str4d commented Nov 8, 2024

Force-pushed to fill in some more parts:

  • Transparent bundle fields for signing (adapting from BIPs 174 and 370)
  • Optional ZIP 32 paths (that Updaters can set to help Signers determine which keys to use)
  • Decided on semantics for merging data in proprietary maps (equal keys are required to have equal values, same as every other field and map).

@str4d
Copy link
Contributor Author

str4d commented Nov 8, 2024

I'm currently inclined to leave out both FROST support and ZSA issuance support from the first version of PCZTs, as we don't have experience with either of them yet, and trying to get them right at the same time as the base PCZT format is likely to end in sorrow.

As @conradoplg notes, we can in some cases have FROST internally pass around a PCZT, but that won't necessarily help for all possible PCZT use cases, so we'll likely want a v2 that adds those fields. That would also be the right time to adjust the core fields if we find weaknesses in the v1 PCZT format.

@conradoplg
Copy link

I'm happy with leaving FROST out of v1, we can make it work for now by sending additional data along with the PCZT.

@str4d
Copy link
Contributor Author

str4d commented Nov 12, 2024

Force-pushed with more changes:

  • Anchors are now required and set by the Creator.
  • Ciphertexts are now Vec<u8> to accommodate upcoming plaintext changes.
  • Pczt struct no longer contains a Version; instead that will be explicitly part of the encoding.
  • Extracted some conversion logic from the Prover role to use it for consistency checks in the Signer role.

@str4d
Copy link
Contributor Author

str4d commented Nov 26, 2024

Force-pushed:

  • Added an IO Finalizer role, and started on a Constructor role.
  • Noted some bugs in the format that need to be fixed.
  • Started writing an end-to-end test.

Use of the Constructor role interacts heavily with the per-protocol crates, so I'm now working on the production impl of those, both to fix the format bugs and get us to a working client-side implementation.

@str4d
Copy link
Contributor Author

str4d commented Dec 3, 2024

Force-pushed:

  • Migrated to the protocol-specific PCZT implementations in the orchard and sapling-crypto crates.
  • Added a similar transparent PCZT impl in zcash_primitives.
  • Fixed a bunch of bugs in the format uncovered by writing end-to-end tests.
  • Wrote several end-to-end tests and got them passing.

I have a couple more changes to make to the Global struct (specifically it is missing the global flags), and then I'll add the postcard derives.

transparent = ["dep:secp256k1", "dep:zcash_primitives", "dep:zcash_protocol"]
zcp-builder = ["dep:zcash_primitives"]
io-finalizer = ["orchard", "sapling"]
prover = ["dep:rand_core", "sapling?/temporary-zcashd"]
Copy link
Contributor

@daira daira Dec 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the sapling/temporary-zcashd feature dependency strictly necessary? Consider filing an issue (if there isn't one already) against sapling to make the necessary APIs stable.

Copy link
Contributor

@daira daira Dec 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was previously necessary due to a use of SpendValidatingKey::temporary_zcash_from_bytes, but that appears to have been removed in a recent push, and this branch now compiles for me without the temporary-zcashd feature dependency (using --all-targets --all-features).

Suggested change
prover = ["dep:rand_core", "sapling?/temporary-zcashd"]
prover = ["dep:rand_core"]

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still relevant.

pczt/src/lib.rs Outdated Show resolved Hide resolved
pczt/src/lib.rs Outdated Show resolved Hide resolved
pczt/src/common.rs Outdated Show resolved Hide resolved
pczt/src/sapling.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@daira daira left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After having read BIP 370 thoroughly, I just don't understand the design of this API. BIP 370 says, for the Constructor:

Before any input or output may be added, the constructor must check the PSBT_GLOBAL_TX_MODIFIABLE field. Inputs may only be added if the Inputs Modifiable flag is True. Outputs may only be added if the Outputs Modifiable flag is True.

That is not enforced by the API. If it's supposed to be a client of this API that is responsible for enforcing it, then it doesn't have enough information to do so in general for the adaptation of the above rule to Zcash. The adapted rule would require that we not make any changes to effecting data outside the transparent inputs and outputs after signing any input. That isn't captured by the flags as they stand, and so a client would have to track that independently of the flags.

I can accept that we have little time to make any major design changes at this point. But please document the limitations (and maybe fix the most obvious loopholes that are easy to fix without a major design change).

daira
daira previously approved these changes Dec 8, 2024
Copy link
Contributor

@daira daira left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK with comments.

@str4d
Copy link
Contributor Author

str4d commented Dec 9, 2024

After having read BIP 370 thoroughly, I just don't understand the design of this API. BIP 370 says, for the Constructor:

Before any input or output may be added, the constructor must check the PSBT_GLOBAL_TX_MODIFIABLE field. Inputs may only be added if the Inputs Modifiable flag is True. Outputs may only be added if the Outputs Modifiable flag is True.

That is not enforced by the API.

We don't provide an independent Constructor implementation, and thus have nowhere that needs to enforce this requirement. (If we did, we'd indeed need to enforce this requirement in that API.)

What we provide in our implemented API is Builder::build_for_pczt, which is inherently both Creator and Constructor. For the lifetime of Builder, the modifiability flags are implicitly true, and then once Builder::build_for_pczt is called, we have no API that enables adding subsequent Spends, Outputs, or Actions. (I know this because I did try implementing an independent Constructor, and quickly ran into the API safety rails we enforced in the orchard crate; bypassing these would have taken much longer and been more risky than instead baking PCZT support into the existing Builders.)

str4d added 2 commits December 9, 2024 10:23
- The Inputs Modifiable Flag and Outputs Modifiable Flag are now only
  for transparent inputs and outputs.
- Bit 7 is now the Shielded Modifiable Flag (a single flag bit is fine
  because all shielded effects are always committed to at once).
- Fixed a bug in `Creator::build_from_parts` where the Has
  `SIGHASH_SINGLE` Flag was not being set correctly.
- The Signer role now updates `tx_modifiable` in case the IO Finalizer
  role has not yet run (which might be possible in some scenarios).
Comment on lines +164 to +166
if ((self.tx_modifiable & !FLAG_SHIELDED_MODIFIABLE) >> 3) != 0
|| ((tx_modifiable & !FLAG_SHIELDED_MODIFIABLE) >> 3) != 0
{
Copy link
Contributor

@daira daira Dec 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if ((self.tx_modifiable & !FLAG_SHIELDED_MODIFIABLE) >> 3) != 0
|| ((tx_modifiable & !FLAG_SHIELDED_MODIFIABLE) >> 3) != 0
{
if ((self.tx_modifiable & !(FLAG_TRANSPARENT_INPUTS_MODIFIABLE | FLAG_TRANSPARENT_OUTPUTS_MODIFIABLE | FLAG_SHIELDED_MODIFIABLE)) != 0) {

Non-blocking.

Copy link
Contributor

@nuttycom nuttycom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK 0cceac1; we should create an issue for adding some of the missing tests.

pczt/src/orchard.rs Show resolved Hide resolved
pczt/src/roles/combiner/mod.rs Show resolved Hide resolved
pczt/src/roles/creator/mod.rs Show resolved Hide resolved
pczt/src/roles/prover/sapling.rs Show resolved Hide resolved
zcash_primitives/CHANGELOG.md Show resolved Hide resolved
Copy link
Contributor

@daira daira left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK

@daira daira merged commit 9e51039 into main Dec 9, 2024
28 of 30 checks passed
@daira daira deleted the pczt-format branch December 9, 2024 16:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants