BitNet.cpp for Rust

Almost safe wrapper for BitNet-cpp based on llama-cpp

Development

See bitnet-cpp-sys

Roadmap

minimize crate size
rewrite sampler PR 9294
look into Metal implementation of MatMul kernerls for BitNet
add more examples
replace all remaining python from BitNet repo
add better division between arm64 and x86_64
move python code generation into patch files

Platform support

arm64	x64	OS	comments
✅	❌	MacOS	tested fully on Apple Silicon macs
⚠️	❌	Linux	should test on ARM based linux
❓	❌	Windows	should test on ARM based windows

Thanks

Heavily inspired by llama-cpp-rs

License

Copyright

© 2024, Eugene Hauptmann