Accessing raw bitpacked sequences for writing?

[vsbuffalo]

Hi,

Thank for writing this beautiful and powerful Rust library! I was curious if it would work well for what I want to do — bitpacking IUPAC nucleotide sequences, and then writing/reading them to a memory mapped file. But as far as I can tell, there isn't a way to take write/read the raw underlying u8 sequence to disk. Is there support for this? My apologies if I'm overlooking something.

Thanks,
Vince

[jeff-k]

Hey Vince, thanks for the kind words!

There are two options:

• using Seq::into_raw()/Seq::from_raw(): https://docs.rs/bio-seq/latest/bio_seq/seq/struct.Seq.html#method.from_raw. This will give you access to the raw bits as usize arrays. You'll need to store the length of the sequence and codec separately. There may be endianess pitfalls if you break the usizes in [u8]s (eg with https://doc.rust-lang.org/stable/std/primitive.usize.html#method.to_ne_bytes) so I put a scary "Experimental" notice in the docs.

• using serde. This needs to be enabled in Cargo.toml, bio-seq = { version="0.13", features=["serde"] }. You'll also need something like bincode to encode the serialised object into a Vec<u8>:

  bio-seq/bio-seq/examples/seq2bin.rs

  Line 61 in ac526e3

  let bs: Vec<u8> = bincode::serialize(seq).unwrap();

I put together a little example here: https://github.com/jeff-k/bio-seq/blob/main/bio-seq/examples/seq2bin.rs

If you're using memory mapping to handle very large sequences please let me know if there are any issues, I haven't tried this. I'm wondering if you'll need to view the &[usize] as a &[u8] without any copying. It could be a very nice example to include!