Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unify & optimise serialization in v0.7 #141

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ version = "0.5"
features = ["ecdsa"]

[dev-dependencies]
serde_test = "1.0"
bincode = "1.1"
serde_json = "1.0"
paste = "1.0.2"
Expand Down
4 changes: 1 addition & 3 deletions src/arithmetic/big_gmp.rs
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ use std::{fmt, ops, ptr};
use gmp::mpz::Mpz;
use gmp::sign::Sign;
use num_traits::{One, Zero};
use serde::{Deserialize, Serialize};
use zeroize::Zeroize;

use super::errors::*;
Expand All @@ -35,8 +34,7 @@ type BN = Mpz;
/// very limited API that allows easily switching between implementations.
///
/// Set of traits implemented on BigInt remains the same regardless of underlying implementation.
#[derive(PartialOrd, PartialEq, Ord, Eq, Clone, Serialize, Deserialize)]
#[serde(transparent)]
#[derive(PartialOrd, PartialEq, Ord, Eq, Clone)]
pub struct BigInt {
gmp: Mpz,
}
Expand Down
4 changes: 1 addition & 3 deletions src/arithmetic/big_native.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ use std::convert::{TryFrom, TryInto};
use std::{fmt, ops};

use num_traits::Signed;
use serde::{Deserialize, Serialize};

use super::errors::*;
use super::traits::*;
Expand All @@ -18,8 +17,7 @@ mod primes;
/// very limited API that allows easily switching between implementations.
///
/// Set of traits implemented on BigInt remains the same regardless of underlying implementation.
#[derive(PartialOrd, PartialEq, Ord, Eq, Clone, Serialize, Deserialize)]
#[serde(transparent)]
#[derive(PartialOrd, PartialEq, Ord, Eq, Clone)]
pub struct BigInt {
num: BN,
}
Expand Down
12 changes: 12 additions & 0 deletions src/arithmetic/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
mod errors;
mod macros;
mod samplable;
mod serde_support;
pub mod traits;

#[cfg(not(any(feature = "rust-gmp-kzen", feature = "num-bigint")))]
Expand All @@ -31,6 +32,7 @@ pub use big_gmp::BigInt;

#[cfg(feature = "num-bigint")]
mod big_native;

#[cfg(feature = "num-bigint")]
pub use big_native::BigInt;

Expand All @@ -45,6 +47,16 @@ mod test {

use super::*;

#[test]
fn serde() {
use serde_test::{assert_tokens, Token::*};
for bigint in [BigInt::zero(), BigInt::sample(1024)] {
let bytes = bigint.to_bytes();
let tokens = vec![Bytes(bytes.leak())];
assert_tokens(&bigint, &tokens)
}
}

#[test]
fn serializing_to_hex() {
let n = BigInt::from(1_000_000_u32);
Expand Down
43 changes: 43 additions & 0 deletions src/arithmetic/serde_support.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
use std::fmt;

use serde::de::Visitor;
use serde::{Deserialize, Deserializer, Serialize, Serializer};

use super::traits::Converter;
use super::BigInt;

impl Serialize for BigInt {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let bytes = self.to_bytes();
serializer.serialize_bytes(&bytes)
}
}

impl<'de> Deserialize<'de> for BigInt {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: Deserializer<'de>,
{
struct BigintVisitor;

impl<'de> Visitor<'de> for BigintVisitor {
type Value = BigInt;

fn expecting(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "bigint")
}

fn visit_bytes<E>(self, v: &[u8]) -> Result<Self::Value, E>
where
E: serde::de::Error,
{
Ok(BigInt::from_bytes(v))
}
}

deserializer.deserialize_bytes(BigintVisitor)
}
}
43 changes: 14 additions & 29 deletions src/elliptic/curves/secp256_k1.rs
Original file line number Diff line number Diff line change
Expand Up @@ -543,9 +543,8 @@ impl Serialize for Secp256k1Point {
where
S: Serializer,
{
let mut state = serializer.serialize_struct("Secp256k1Point", 2)?;
state.serialize_field("x", &self.x_coor().unwrap().to_hex())?;
state.serialize_field("y", &self.y_coor().unwrap().to_hex())?;
let mut state = serializer.serialize_struct("Secp256k1Point", 1)?;
state.serialize_field("p", &self.bytes_compressed_to_big_int().to_hex())?;
Copy link
Contributor

@survived survived Sep 22, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit:

  1. Since serialized structure Secp256k1Point consist of only one field, maybe just serialize it as string, not as structure? Ie. simple serializer.serialize_str(...) can be used here
  2. In curv v0.8 we serialize point as raw bytes, not as hex-encoded bytes. This removes verbosity, but makes it highly efficient for binary data formats like bincode. Simply saying, it reduces size of serialized point by half. Do you prefer hex format?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. True, it is shorter and simpler than my idea about newtype struct.
  2. As a matter of fact, I prefer raw bytes as well. I'll change the code soon.

state.end()
}
}
Expand All @@ -555,7 +554,7 @@ impl<'de> Deserialize<'de> for Secp256k1Point {
where
D: Deserializer<'de>,
{
let fields = &["x", "y"];
let fields = &["p"];
deserializer.deserialize_struct("Secp256k1Point", fields, Secp256k1PointVisitor)
}
}
Expand All @@ -573,38 +572,25 @@ impl<'de> Visitor<'de> for Secp256k1PointVisitor {
where
V: SeqAccess<'de>,
{
let x = seq
let p = seq
.next_element()?
.ok_or_else(|| V::Error::invalid_length(0, &"a single element"))?;
let y = seq
.next_element()?
.ok_or_else(|| V::Error::invalid_length(0, &"a single element"))?;

let bx = BigInt::from_hex(x).map_err(V::Error::custom)?;
let by = BigInt::from_hex(y).map_err(V::Error::custom)?;

Ok(Secp256k1Point::from_coor(&bx, &by))
let bp = BigInt::from_hex(p).map_err(V::Error::custom)?;
Ok(Secp256k1Point::from_bytes(&bp.to_bytes()[1..33]).unwrap())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same as visit_map:

  • Ensure that bp.to_bytes() is large enough
  • Propagate error returned by Secp256k1Point::from_bytes
  • Consider using hex::decode/hex::decode_slice over BigInt::from_hex

}

fn visit_map<E: MapAccess<'de>>(self, mut map: E) -> Result<Secp256k1Point, E::Error> {
let mut x = String::new();
let mut y = String::new();

let mut p = String::new();
while let Some(ref key) = map.next_key::<String>()? {
let v = map.next_value::<String>()?;
if key == "x" {
x = v
} else if key == "y" {
y = v
if key == "p" {
p = v
} else {
return Err(E::Error::unknown_field(key, &["x", "y"]));
return Err(E::Error::unknown_field(key, &["p"]));
}
}

let bx = BigInt::from_hex(&x).map_err(E::Error::custom)?;
let by = BigInt::from_hex(&y).map_err(E::Error::custom)?;

Ok(Secp256k1Point::from_coor(&bx, &by))
let bp = BigInt::from_hex(&p).map_err(E::Error::custom)?;
Ok(Secp256k1Point::from_bytes(&bp.to_bytes()[1..33]).unwrap())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Malformed input that comes from untrusted source may cause this method to panic. Deserialize implementation must validate input and return error if it's invalid. You need to:

  • Ensure that bp.to_bytes() >= 33 (otherwise &bp.to_bytes()[1..33] may panic)
  • Properly handle error returned by Secp256k1Point::from_bytes

Something like that should work:

Suggested change
Ok(Secp256k1Point::from_bytes(&bp.to_bytes()[1..33]).unwrap())
let bp = bp.to_bytes();
if bp.len() < 33 {
return E::Error::invalid_length(bp.len(), &"33 bytes")
}
Secp256k1Point::from_bytes(&bp[1..33])
.map_err(|_e| E::Error::custom("invalid point"))

Copy link
Contributor

@survived survived Sep 22, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit (not important): I think BigInt can be removed from this function by using hex::decode - this would save a couple allocations. You can even remove all the allocations in this function by using hex::decode_to_slice. That would optimise deserialization.

}
}

Expand Down Expand Up @@ -673,13 +659,12 @@ mod tests {
fn serialize_pk() {
let pk = Secp256k1Point::generator();
let x = pk.x_coor().unwrap();
let y = pk.y_coor().unwrap();
let s = serde_json::to_string(&pk).expect("Failed in serialization");

let expected = format!("{{\"x\":\"{}\",\"y\":\"{}\"}}", x.to_hex(), y.to_hex());
let expected = format!("{{\"p\":\"{}{}\"}}", 2, x.to_hex());
assert_eq!(s, expected);

let des_pk: Secp256k1Point = serde_json::from_str(&s).expect("Failed in serialization");
let des_pk: Secp256k1Point = serde_json::from_str(&s).expect("Failed in deserialization");
assert_eq!(des_pk.ge, pk.ge);
}

Expand Down