Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merklization audit #157

Merged
merged 3 commits into from
Mar 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 61 additions & 4 deletions commit_verify/src/bin/commit-stl.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,20 @@
// You should have received a copy of the Apache 2.0 License along with this
// software. If not, see <https://opensource.org/licenses/Apache-2.0>.

use commit_verify::stl;
use strict_types::parse_args;
use std::fs;
use std::io::Write;

use commit_verify::stl::commit_verify_stl;
use commit_verify::{mpc, CommitmentLayout, MerkleNode};
use strict_types::stl::{std_stl, strict_types_stl};
use strict_types::{parse_args, SystemBuilder};

fn main() {
let lib = stl::commit_verify_stl();
let lib = commit_verify_stl();
let (format, dir) = parse_args();
lib.serialize(
format,
dir,
dir.as_ref(),
"0.1.0",
Some(
"
Expand All @@ -31,4 +36,56 @@ fn main() {
),
)
.expect("unable to write to the file");

let dir = dir.unwrap_or_else(|| ".".to_owned());

let std = std_stl();
let st = strict_types_stl();
let cv = commit_verify_stl();

let sys = SystemBuilder::new()
.import(cv)
.unwrap()
.import(st)
.unwrap()
.import(std)
.unwrap()
.finalize()
.expect("not all libraries present");

let mut file = fs::File::create(format!("{dir}/Merkle.vesper")).unwrap();
writeln!(
file,
"{{-
Description: Merklization and MPC workflows
Author: Dr Maxim Orlovsky <[email protected]>
Copyright (C) 2024 LNP/BP Standards Association. All rights reserved.
License: Apache-2.0
-}}

Merklization vesper lexicon=types+commitments
"
)
.unwrap();
writeln!(file, "\n-- General merklization workflows\n").unwrap();
let layout = MerkleNode::commitment_layout();
writeln!(file, "{layout}").unwrap();
let tt = sys.type_tree("CommitVerify.MerkleNode").unwrap();
writeln!(file, "{tt}").unwrap();

writeln!(file, "\n-- Multi-protocol commitment workflows\n").unwrap();
let layout = mpc::Leaf::commitment_layout();
writeln!(file, "{layout}").unwrap();
let tt = sys.type_tree("CommitVerify.Leaf").unwrap();
writeln!(file, "{tt}").unwrap();

let layout = mpc::MerkleBlock::commitment_layout();
writeln!(file, "{layout}").unwrap();
let tt = sys.type_tree("CommitVerify.MerkleBlock").unwrap();
writeln!(file, "{tt}").unwrap();

let layout = mpc::MerkleTree::commitment_layout();
writeln!(file, "{layout}").unwrap();
let tt = sys.type_tree("CommitVerify.MerkleTree").unwrap();
writeln!(file, "{tt}").unwrap();
}
27 changes: 19 additions & 8 deletions commit_verify/src/mpc/block.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,9 @@
use std::cmp::Ordering;
use std::collections::{BTreeMap, BTreeSet};

use amplify::confinement::{Confined, LargeVec};
use amplify::confinement::{Confined, NonEmptyVec, U32 as U32MAX};
use amplify::num::u5;
use strict_encoding::{StrictDeserialize, StrictEncode, StrictSerialize};
use strict_encoding::{StrictDeserialize, StrictDumb, StrictEncode, StrictSerialize};

use crate::id::CommitId;
use crate::merkle::{MerkleBuoy, MerkleHash};
Expand Down Expand Up @@ -130,7 +130,7 @@ impl TreeNode {
}

/// Partially-concealed merkle tree data.
#[derive(Getters, Clone, PartialEq, Eq, Hash, Debug, Default)]
#[derive(Getters, Clone, PartialEq, Eq, Hash, Debug)]
#[derive(StrictType, StrictEncode, StrictDecode)]
#[strict_type(lib = LIB_NAME_COMMIT_VERIFY)]
#[derive(CommitEncode)]
Expand All @@ -148,14 +148,25 @@ pub struct MerkleBlock {

/// Tree cross-section.
#[getter(skip)]
cross_section: LargeVec<TreeNode>,
cross_section: NonEmptyVec<TreeNode, U32MAX>,

/// Entropy used for placeholders. May be unknown if the message is provided
/// by a third-party, wishing to conceal that information.
/// by a third party, wishing to conceal that information.
#[getter(as_copy)]
entropy: Option<u64>,
}

impl StrictDumb for MerkleBlock {
fn strict_dumb() -> Self {
MerkleBlock {
depth: u5::ONE,
cofactor: 0,
cross_section: NonEmptyVec::with(TreeNode::strict_dumb()),
entropy: Some(8845),
}
}
}

impl StrictSerialize for MerkleBlock {}
impl StrictDeserialize for MerkleBlock {}

Expand All @@ -177,7 +188,7 @@ impl From<&MerkleTree> for MerkleBlock {
})
});
let cross_section =
LargeVec::try_from_iter(iter).expect("tree width guarantees are broken");
NonEmptyVec::try_from_iter(iter).expect("tree width guarantees are broken");

MerkleBlock {
depth: tree.depth,
Expand Down Expand Up @@ -237,7 +248,7 @@ impl MerkleBlock {
});
cross_section.extend(rev.into_iter().rev());
let cross_section =
LargeVec::try_from(cross_section).expect("tree width guarantees are broken");
NonEmptyVec::try_from(cross_section).expect("tree width guarantees are broken");

Ok(MerkleBlock {
depth: u5::with(path.len() as u8),
Expand Down Expand Up @@ -471,7 +482,7 @@ impl MerkleBlock {
cross_section.extend(b);

self.cross_section =
LargeVec::try_from(cross_section).expect("tree width guarantees are broken");
NonEmptyVec::try_from(cross_section).expect("tree width guarantees are broken");

assert_eq!(
self.cross_section
Expand Down
3 changes: 2 additions & 1 deletion commit_verify/src/stl.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ use strict_types::{CompileError, LibBuilder, TypeLib};
use crate::{mpc, MerkleHash, MerkleNode, StrictHash, LIB_NAME_COMMIT_VERIFY};

pub const LIB_ID_COMMIT_VERIFY: &str =
"urn:ubideco:stl:57sPvZcwQaziec3ux249XoCMhziQpKB8Yw99U5oRwfqW#deluxe-safari-random";
"urn:ubideco:stl:7qvjR4HCwJKF3mxE5GqsAaADces5JDRwb8ajAse9mkz3#exhibit-karate-ritual";

fn _commit_verify_stl() -> Result<TypeLib, CompileError> {
LibBuilder::new(libname!(LIB_NAME_COMMIT_VERIFY), tiny_bset! {
Expand All @@ -33,6 +33,7 @@ fn _commit_verify_stl() -> Result<TypeLib, CompileError> {
.transpile::<mpc::MerkleTree>()
.transpile::<mpc::MerkleBlock>()
.transpile::<mpc::MerkleProof>()
.transpile::<mpc::Leaf>()
.transpile::<mpc::Commitment>()
.transpile::<MerkleNode>()
.transpile::<MerkleHash>()
Expand Down
51 changes: 27 additions & 24 deletions stl/[email protected]
Original file line number Diff line number Diff line change
@@ -1,33 +1,36 @@
-----BEGIN STRICT TYPE LIB-----
Id: 57sPvZcwQaziec3ux249XoCMhziQpKB8Yw99U5oRwfqW#deluxe-safari-random
Id: 7qvjR4HCwJKF3mxE5GqsAaADces5JDRwb8ajAse9mkz3#exhibit-karate-ritual
Name: CommitVerify
Dependencies:
DzTvt9gGhPUKD8Dkkjk9PDBhkJ4gtWxXWQjxnmUYLNrs#voyage-kimono-disco

3`1{iZE18?WpZg|c>%!$%tl@;WNu_lcw-g2$nLVb<*lb*ENIx0;35YAw}=B%bYuY
oQ*>kj0A^Tl*p6J$36SYb7g#;qpQBTpwL(~+!(f@;t~vt?k^)sV3jhj3Z*6U9bZu
pBbOiwb2LJ#-AOH(ZWpZn5WkPIkV`~Nk1!QG#bZ7#>1k6TWEM#tEO?YD!yU6acy5
+5>Uo2?YlHei-0Jn%{Sa8^mT+s=T=}Z?`J=~w8Q=GLzSfImTncuED0}GM}V{c|*V
{~tF0006EV{&hEb5mtwbZKvH00;rtwYiq_Rlwao{(T?aUNqayF^88E^#IUpx^^~;
)zDV}0000000030|Ns9000007Wo~qGZ*X}41OfmAZf|a7000011aog~WdH>M000O
IO=WUxY-LDcb7%zt00#g7Kp+4LO=WUxY-LVwWMu{g31V_#ZewU^Zf60zOl}Js;Lv
V7e1xlC`fl^8AoO?D+|y;s#+-J4VoSmWWMy!4XaE2K1$Sv=bZ7toAO&u3WMwe{IG
#g>Clv)aMjKgwAH@`bu1x<7g|G$};xvA~n-$_S1#WL-WikOco<oKw6$Lm(8(DuJ#
T0?AO#cjpum%_6G=9FD72-1sO=WUxY-Lb#Z*OJ>0|Rhxa{vGY2xD(%VPkY}asU7V
1aM(=XaEQSIG#g>Clv)aMjKgwAH@`bu1x<7g|G$};xvA~n-$_S0000000000AOHX
W000003Qc8lYiwmya%E)(1qEbfaCB$_!34}kUMys8WKDQu6}!mpvbyE1r(Y~+*pl
EP2LQK-W>|38j$F|Rkm*bpSUudIqf?x<LRg@~V42^pIs*%m2W4(_a&K^X000OGV{
c|*V{~tF0006AZDn(FVP|D?015#xm44<OVKiC01qkHfuRUrZzt;Qv9WjEZdF4fP;
8w8#H8-hI70Bv^+*0?ef%0)>Q3WPbltNdpi4*91)SI!>0000000000|Ns9000000
18re&015yA1ONpB0Wg(*<{e=)S-S-Y<l(P9Y9YVY`}-X+f~R@qMRed+u>mzVsZkZ
k>V@1=_p5>Oab-~jCR3C`SFec^=zG+gvC{wm0000000960{{R30000L~Wpi_3XJr
Ke00#g7Kp+4OPH$voLULhlV`yn^X9EKSc5i8900whuZf9&|0S01nVQyn+0t!%aZ*
*^CZ){0q1pxpD002NB018uda%p39NMUnm1pxpD002NB00>laWo1rpWMu>b01abrZ
ewL(Y-MClZ)9Zv1_A|SWpH$80>K2#MqVsrZe&e(V->r|?y|b&t*2isXxNhAA_oAs
h-O%D*p6J$36SYb7g#;qpQBTpwL(~+!(f@;t~vt?k_2dBb7%oLo<oKw6$Lm(8(Du
J#T0?AO#cjpum%_6G=9FD72-1i4r6a^ZE19EWo~p#WnpFj1_BCja&L5RV{dFpWC1
Xhe&!uvG+Da^2;||fJ!&Dp*8BS%F@mRg<wbPhR<Q?dWpi_3XJr93H>pt-$m)gMQu
nKY@^NKR1twFJLRYVe6X<)?o3Ya
oQ*>kj0A^Tl*p6J$36SYb7g#;qpQBTpwL(~+!(f@;t~vt?k^)sV3;+s4Z*6U9bZu
pBbOiwb2LJ#-AOHkRWnpFn0uTvlZfId*X>?^|00sgGaB^>SZ)0z40Wg(*<{e=)S-
S-Y<l(P9Y9YVY`}-X+f~R@qMRed+u?KBsb8}&5WdSuesZkZk>V@1=_p5>Oab-~jC
R3C`SFec^=zG+gvC|O;Wo~qGZ*X}41_B3VZgg^QaCra#2m^3$a{vGY3r%HmYiwmg
Y;R+01_T9UWpH$80>K2#MqVsrZe&e(V->r|?y|b&t*2isXxNhAA_oAsh-O%D*p6J
$36SYb7g#;qpQBTpwL(~+!(f@;t~vt?k_cmOW?^G=Z*l+t0t{nvZ*y}~Wn*+{Z*B
kx0ob*<mh)A>-9G+(AKhLw+s!eDmlO2>&}_PPHjCBJR{;P3000000RR90{{R3000
(7mbaHQSc>n|g00eGtZe;)f009JZZ*64&1pxp62ntPQa%*g5NMUnm1pxpD002NB0
18cIa%*g5PH$vo1_cRXa$#;`XlZU|0lQ3Y3mf3jZajR1t6%zV^Qs{9ch%g}Wy;2!
c7I|^!UbeyaCB$@009MeX=HS0001BbZf|5|F#$N9Lxv|61vo|<S$`kJ6oIZx{|tq
&1{dNqe!iO(;xh$qZ)9aM0XUvRh9?yTI7S;;e;>sZfv!yd427@;7veO2zMB=|GYd
^+a%*g5P;zf?W(ETTaBp(}00anQZ)Ra*bZ>G100IPXVRUE!2mv^rLxv|61vo|<S$
`kJ6oIZx{|tq&1{dNqe!iO(;xhmM00000001BW00000000V2WpZn5WmIxyWd;QWW
My!4Xad0m%tl@;WNu_lcw-g2$nLVb<*lb*ENIx0;35YAw}@s~aM+Gq(Fu_0Ocz)^
+@GUUoV7w&pu=F9->y0X3z7$AZgg^QaCra#2nb_uW?^G=Z*l+t0tjtob8}&5Wpe-
u0Wg(*<{e=)S-S-Y<l(P9Y9YVY`}-X+f~R@qMRed+u>mzVsZkZk>V@1=_p5>Oab-
~jCR3C`SFec^=zG+gvC{wm0000000960{{R30000ARVQ>Hn000C41p)yum44<OVK
iC01qkHfuRUrZzt;Qv9WjEZdF4fP;8w8#H8-hI70Bv^+*0?ef%0)>Q3WPbltNdpi
4*91)SI!>0000000000|Ns90000002Tf&jb75y?1pxpD002NB01ZxWWMx8fVQyn+
X>Ml&0|a(&X=DHfb7^j8Y-IrkVsc?_V`u^jP;zf{Z)0z4Nn`~900#g7Kp+4LQ*?4
^V{}Mib7%zt00#g7Kp+4JRB~lyPH$vo1OfmJV{dL_WnpY(WKM5nWdH^O1!QG#bZ7
#>1k6TWEM#tEO?YD!yU6acy5+5>Uo2?YlHei-0Jn%{Sa8^mT+s=T=}Z?`J=~w8Q=
GLzSfImTncuED0}GM_Xkl|`0XUvRh9?yTI7S;;e;>sZfv!yd427@;7veO2zMB=|G
XV}`Z*6U9bZupBbWCMoW&j2P3UG37bZ=vCY)NDRFqMAh9bq(Cy9Eg3;jcYvA-~r9
`yDZYr+MW?bl_I82W@3@b75y?0W~+NQ5DGQh1^p2tAX-yWl;qtQ<OqiuZa`rd(@k
;(*

-----END STRICT TYPE LIB-----

Binary file modified stl/[email protected]
Binary file not shown.
10 changes: 7 additions & 3 deletions stl/[email protected]
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{-
Id: urn:ubideco:stl:57sPvZcwQaziec3ux249XoCMhziQpKB8Yw99U5oRwfqW#deluxe-safari-random
Id: urn:ubideco:stl:7qvjR4HCwJKF3mxE5GqsAaADces5JDRwb8ajAse9mkz3#exhibit-karate-ritual
Name: CommitVerify
Version: 0.1.0
Description: Client-side-validation deterministic commitments
Expand All @@ -18,10 +18,14 @@ import Std#DzTvt9gGhPUKD8Dkkjk9PDBhkJ4gtWxXWQjxnmUYLNrs#voyage-kimono-disco
@mnemonic(ventura-equal-think)
data Commitment : [Byte ^ 32]

@mnemonic(iris-explore-script)
@mnemonic(pony-audio-mozart)
data Leaf : inhabited#16 (protocol ProtocolId, message Message)
| entropy (entropy U64, pos U32)

@mnemonic(owner-albert-lexicon)
data MerkleBlock : depth Std.U5#nitro-george-nebula
, cofactor U16
, crossSection [TreeNode ^ ..0xffffffff]
, crossSection [TreeNode ^ 1..0xffffffff]
, entropy U64?

@mnemonic(culture-metro-modular)
Expand Down
83 changes: 83 additions & 0 deletions stl/Merkle.vesper
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
{-
Description: Merklization and MPC workflows
Author: Dr Maxim Orlovsky <[email protected]>
Copyright (C) 2024 LNP/BP Standards Association. All rights reserved.
License: Apache-2.0
-}

Merklization vesper lexicon=types+commitments


-- General merklization workflows

MerkleHash commitment hasher=SHA256 tagged=urn:ubideco:merkle:node#2024-01-31
MerkleNode serialized

MerkleNode rec
branching enum void=0 single=1 branch=2 -- NodeBranching
depth is U8
width is U256
node1 bytes len=32 aka=MerkleHash
node2 bytes len=32 aka=MerkleHash


-- Multi-protocol commitment workflows

MerkleHash commitment hasher=SHA256 tagged=urn:ubideco:merkle:node#2024-01-31
Leaf serialized

Leaf union
inhabited rec tag=0
protocol bytes len=32 aka=ProtocolId
message bytes len=32 aka=Message
entropy rec tag=1
entropy is U64
pos is U32

Commitment commitment hasher=SHA256 tagged=urn:ubideco:mpc:commitment#2024-01-31
MerkleBlock concealed concealed=MerkleBlock

MerkleBlock rec
depth enum {
_0=0 _1=1 _2=2 _3=3 _4=4 _5=5 _6=6 _7=7 _8=8
_9=9 _10=10 _11=11 _12=12 _13=13 _14=14 _15=15 _16=16
_17=17 _18=18 _19=19 _20=20 _21=21 _22=22 _23=23 _24=24
_25=25 _26=26 _27=27 _28=28 _29=29 _30=30 _31=31
} -- U5
cofactor is U16
crossSection list len=1..MAX32
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just confirming:

I reread LNPBP-4 because of this change and in fact, we will always have at least one commiment in the MPC. This change reflects the statement above, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correct

TreeNode union
concealedNode rec tag=0
depth enum {
_0=0 _1=1 _2=2 _3=3 _4=4 _5=5 _6=6 _7=7 _8=8
_9=9 _10=10 _11=11 _12=12 _13=13 _14=14 _15=15 _16=16
_17=17 _18=18 _19=19 _20=20 _21=21 _22=22 _23=23 _24=24
_25=25 _26=26 _27=27 _28=28 _29=29 _30=30 _31=31
} -- U5
hash bytes len=32 aka=MerkleHash
commitmentLeaf rec tag=1
protocolId bytes len=32 aka=ProtocolId
message bytes len=32 aka=Message
some is U64 option wrapped tag=1

Commitment commitment hasher=SHA256 tagged=urn:ubideco:mpc:commitment#2024-01-31
MerkleBlock concealed concealed=MerkleBlock

MerkleTree rec
depth enum {
_0=0 _1=1 _2=2 _3=3 _4=4 _5=5 _6=6 _7=7 _8=8
_9=9 _10=10 _11=11 _12=12 _13=13 _14=14 _15=15 _16=16
_17=17 _18=18 _19=19 _20=20 _21=21 _22=22 _23=23 _24=24
_25=25 _26=26 _27=27 _28=28 _29=29 _30=30 _31=31
} -- U5
entropy is U64
cofactor is U16
messages map len=0..MAX24
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to question above, we dont need at least one message in the merkle tree?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm.. On one side no: we may have a merkle tree made of entropy (and its cross-section still will not be an empty set). However this has low utility, so it could be we can prohibit that...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Giving a second thought: there might be a need to tress with no messages and just an entropy. While it's marginal, even if we require a non-empty message set, people can fake it and still produce trees containing no real messages. Thus, while it is mathematically impossible to have a tree cross-section which is empty, it is mathematically possible to have a tree with no messages, made only of entropy - thus, the difference in these types.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks for explanation.

key bytes len=32 aka=ProtocolId
value bytes len=32 aka=Message
map map len=0..MAX24
key is U32
value tuple
_ bytes len=32 aka=ProtocolId
_ bytes len=32 aka=Message

Loading