Miscellaneous fixes (#17)

* moving docs around all over the place * crate features to compile out debug/trace logging
jmwample · Apr 1, 2024 · cdfd9e4 · cdfd9e4
1 parent b52e634
commit cdfd9e4
Show file tree

Hide file tree

Showing 41 changed files with 389 additions and 194 deletions.
diff --git a/Cargo.toml b/Cargo.toml
@@ -11,3 +11,5 @@ members = [
 resolver = "2"
 
 
+[profile.test]
+default = ["ptrs/debug", "obfs4/debug", "lyrebird/debug"]
diff --git a/README.md b/README.md
@@ -1,74 +1,44 @@
-# o7 Proxy
+# Pluggable Transports in Rust (PTRS)
 
 <p>
   <a href="https://github.com/jmwample/o7/actions/workflows/rust.yml">
     <img src="https://github.com/jmwample/o7/actions/workflows/rust.yml/badge.svg?branch=main" alt="Build Status">
-  <a href="https://codecov.io/gh/jmwample/o7" > 
-    <img src="https://codecov.io/gh/jmwample/o7/graph/badge.svg?token=0lMlrA32xd"/> 
+  <a href="https://codecov.io/gh/jmwample/o7" >
+    <img src="https://codecov.io/gh/jmwample/o7/graph/badge.svg?token=0lMlrA32xd"/>
   </a>
   <a href="https://deps.rs/repo/github/jmwample/o7">
     <img src="https://deps.rs/repo/github/jmwample/o7/status.svg">
   </a>
-  <a href="https://crates.io/crates/o7">
-    <img src="https://img.shields.io/crates/v/o7.svg">
-  </a>
-  <a href="https://docs.rs/o7">
-    <img src="https://docs.rs/o7/badge.svg">
-  </a>
   <a href="https://doc.rust-lang.org/1.6.0/complement-project-faq.html#why-dual-mitasl2-license">
     <img src="https://img.shields.io/badge/license-MIT%2FApache--2.0-blue" alt="License: MIT/Apache 2.0">
   </a>
 </p>
 
 
 This repository contains multiple related crates implementing the lyrebird (obfs4) library,
-lyrebird binary, and Pluggable Transports in Rust (PTRS) library. 
-
-Things to keep an eye on:
-
-- [ ] PR implementating elligator2 for the `dalek` ed25519 library. [PR Here](https://github.com/dalek-cryptography/curve25519-dalek/pull/612)
-
-
-## Examples
-
-<details>
-<summary>Obfs4 Client Example</summary>
-
-```rs
-let client = Client::from_param_str("");
+lyrebird binary, and Pluggable Transports in Rust (PTRS) library.
 
-let mut conn = tokio::net::TcpStream::Connect();
 
-c = client.wrap(&mut conn);
 
-```
+|                 Crate                    |   Description  | Crates.io | Docs |
+-------------------------------------------|----------------|-----------|------|
+| [`ptrs`](./crate/ptrs) | A library supporting implementation and integration of Pluggable Transport protocols. | [![](https://img.shields.io/crates/v/ptrs.svg)](https://crates.io/crates/ptrs) | [![](https://img.shields.io/docsrs/ptrs)](https://docs.rs/ptrs) |
+| [`lyrebird`](./crates/lyrebird) | Implementation of the `Lyrebird` Tor bridge and a forward proxy compatible with `ptrs`. | [![](https://img.shields.io/crates/v/lyrebird.svg)](https://crates.io/crates/lyrebird) | [![](https://docs.rs/lyrebird/badge.svg)](https://docs.rs/lyrebird) |
+| [`obfs4`](./crates/obfs4) | An implementation of obfs4 pluggable transport library in pure rust. | [![](https://img.shields.io/crates/v/obfs4.svg)](https://crates.io/crates/obfs4) | [![](https://docs.rs/obfs4/badge.svg)](https://docs.rs/obfs4) |
 
-</details>
 
-## Command Line Interface
+## MSRV
 
+Current Minimum Supported Rust Version (MSRV) is 1.60.
 
-<details>
-<summary>CLI Options</summary>
+MSRV can be changed in the future, but it will be done with a minor version bump.
 
-can be compiled and run, or run using the rust binary
+## Related
 
-```sh
-cargo install .....
-```
-
-</details>
-
-## FAQ
-
-* Why shift from the obfs4 style naming and use o7? 
-
-    I wrote the library and I like it that way. Don't like the name? Fork it
-    and maintain it yourself.
+Things to keep an eye on:
 
-* What happened to o6? 
+- [ ] PR implementating elligator2 for the `dalek` ed25519 library. [PR Here](https://github.com/dalek-cryptography/ptrs/pull/612)
 
-    See the answer above.
 
 ## Open Source License
 
@@ -77,10 +47,7 @@ community and has been used for both the compiler and many public libraries sinc
 [Why dual MIT/ASL2license?](https://doc.rust-lang.org/1.6.0/complement-project-faq.html#why-dual-mitasl2-license)).
 In order to match the community standards, o7 is using the dual MIT+Apache-2.0 license.
 
-## MSRV Policy
-
-Minimum Supported Rust Version (MSRV) can be changed in the future, but it will be done with a minor version bump.
-
 ## Contributing
 
-Contributors, Issues, and Pull Requests are Welcome!
+Contributions, Issues, and Pull Requests are welcome!
+
diff --git a/crates/lyrebird/Cargo.toml b/crates/lyrebird/Cargo.toml
@@ -10,6 +10,9 @@ keywords = ["tor", "censorship", "pluggable", "transports"]
 categories = ["network-programming", "cryptography"]
 repository = "https://github.com/jmwample/o7"
 
+[features]
+default = []
+debug = ["ptrs/debug", "obfs4/debug"]
 
 [[bin]]
 name = "fwd"

diff --git a/crates/lyrebird/README.md b/crates/lyrebird/README.md
@@ -1,9 +1,82 @@
-# Pluggable Transport Proxy Applications
+<p align="center">
+<img
+ alt="lyrebird logo: a lyrebird making noises you would never expect from a bird."
+ width="200px"
+ src="../../doc/lyrebird_logo.png"/>
+</p>
+
+# Lyrebird - Pluggable Transport Proxy Applications
 
 This crates contains multiple binary exceutables designed specifically to work
 with and pluggable transport library implementing the ['ptrs'] interface.
 
 
+## Lyrebird Pluggable Transport Bridge
+
+['lyrebird'] provides an executable program designed to manage the calling
+interface used by the Tor libraries when launching pluggable transports (see `pt-spec.txt`).
+
+`... [tor_client] <---> [pt_client] <====> [pt_bridge] <---> [tor_orport] ...`
+
+Usage info:
+
+```txt
+Tunnel Tor SOCKS5 traffic through pluggable transport connections
+
+Usage: lyrebird [OPTIONS]
+
+Options:
+      --enable-logging         Log to {TOR_PT_STATE_LOCATION}/obfs4proxy.log
+      --log-level <LOG_LEVEL>  Log Level (ERROR/WARN/INFO/DEBUG/TRACE) [default: ERROR]
+      --unsafe-logging         Disable the address scrubber on logging
+  -h, --help                   Print help
+  -V, --version                Print version
+```
+
+### Installation
+
+To install:
+
+    `cargo install lyrebird`
+
+This installs in the configured Rust location (i.e. `$HOME/.cargo/bin`). You may
+wish to copy `./lyrebird` to a permanent location (e.g. `/usr/local/bin`).
+
+Client side torrc configuration:
+```
+ClientTransportPlugin obfs4 exec /usr/local/bin/lyrebird
+```
+
+Bridge side torrc configuration:
+```
+# Act as a bridge relay.
+BridgeRelay 1
+
+# Enable the Extended ORPort
+ExtORPort auto
+
+# Use lyrebird to provide the obfs4 protocol.
+ServerTransportPlugin obfs4 exec /usr/local/bin/lyrebird
+
+# (Optional) Listen on the specified address/port for obfs4 connections as
+# opposed to picking a port automatically.
+#ServerTransportListenAddr obfs4 0.0.0.0:443
+```
+
+### Tips and tricks
+
+ * On modern Linux systems it is possible to have lyrebird bind to reserved
+   ports (<=1024) even when not running as root by granting the
+   `CAP_NET_BIND_SERVICE` capability with setcap:
+
+   `# setcap 'cap_net_bind_service=+ep' /usr/local/bin/lyrebird`
+
+ * The autogenerated obfs4 bridge parameters are placed in
+   `DataDir/pt_state/obfs4_state.json`.  To ease deployment, the client side
+   bridge line is written to `DataDir/pt_state/obfs4_bridgeline.txt`.
+
+
+
 ## Forward proxy
 
 ` [client] <---> [fwd\_client] <====> [fwd\_server] <---> [target] `
@@ -25,30 +98,27 @@ Arguments:
 
 Options:
   -a, --args <ARGS>            Transport argument string
+  -s, --state-dir <DIR>        Path to a directory where launch state is located.
   -l, --log-level <LOG_LEVEL>  Log Level (ERROR/WARN/INFO/DEBUG/TRACE) [default: INFO]
   -x, --unsafe-logging         Disable the address scrubber on logging
   -h, --help                   Print help
   -V, --version                Print version
 ```
 
-## Pluggable Transport Bridge
+Examples
 
-['lyrebird'] provides an executable program designed to manage the calling
-interface used by the Tor libraries when launching pluggable transports (see `pt-spec.txt`).
 
-`... [tor_client] <---> [pt_client] <====> [pt_bridge] <---> [tor_orport] ...`
+```sh
+fwd -s ./state/ server fwd "127.0.0.1:5201"
+```
 
-Usage info:
+```sh
+fwd -a "cert=AAAAAAAAAAAAAAAAAAAAAAAAAADTSFvsGKxNFPBcGdOCBSgpEtJInG9zCYZezBPVBuBWag;iat-mode=0" -l DEBUG 127.0.0.1:9000 client 127.0.0.1:9001
+```
 
-```txt
-Tunnel Tor SOCKS5 traffic through pluggable transport connections
 
-Usage: lyrebird [OPTIONS]
+## Potential Features
+
+- [ ] geoip for obvious signs of censorship
+- [ ] double check the bit randomization and clearing for high two bits in the `dalek` representative
 
-Options:
-      --enable-logging         Log to {TOR_PT_STATE_LOCATION}/obfs4proxy.log
-      --log-level <LOG_LEVEL>  Log Level (ERROR/WARN/INFO/DEBUG/TRACE) [default: ERROR]
-      --unsafe-logging         Disable the address scrubber on logging
-  -h, --help                   Print help
-  -V, --version                Print version
-```
diff --git a/crates/lyrebird/src/fwd/backend.rs b/crates/lyrebird/src/fwd/backend.rs
@@ -1,10 +1,10 @@
 use anyhow::{Context, Result};
 use clap::Parser;
 use futures::Future;
+use ptrs::{info, warn};
 use safelog::sensitive;
 use tokio::io::{copy_bidirectional, AsyncRead, AsyncWrite};
 use tokio::net::TcpStream;
-use tracing::{info, warn};
 
 use std::net::SocketAddr;
 use std::ops::Deref;

diff --git a/crates/lyrebird/src/fwd/main.rs b/crates/lyrebird/src/fwd/main.rs
@@ -10,6 +10,7 @@
 use futures::Future;
 use obfs4::{obfs4::ClientBuilder, Obfs4PT};
 use ptrs::{args::Args, ClientTransport, PluggableTransport, ServerBuilder, ServerTransport};
+use ptrs::{debug, error, info, warn};
 
 use anyhow::{anyhow, Context, Result};
 use clap::{Parser, Subcommand, ValueEnum};
@@ -27,7 +28,7 @@ use tokio::{
 };
 use tokio::{net::ToSocketAddrs, task::JoinSet};
 use tokio_util::sync::CancellationToken;
-use tracing::{debug, error, info, warn, Level};
+use tracing::Level;
 use tracing_subscriber::{filter::LevelFilter, prelude::*};
 
 use std::{
@@ -128,7 +129,6 @@ fn ingest_args(cli_args: &CliArgs) -> Option<Args> {
 #[tokio::main]
 async fn main() -> Result<()> {
     let args = CliArgs::parse();
-    // obfs4::dev::print_dev_args();
 
     // launch tracing subscriber with filter level
     let _guard = init_logging_recvr(args.unsafe_logging, &args.log_level)?;

diff --git a/crates/lyrebird/src/main.rs b/crates/lyrebird/src/main.rs
@@ -8,6 +8,7 @@
 #![allow(unused, dead_code)]
 
 use obfs4::{obfs4::ClientBuilder, Obfs4PT};
+use ptrs::{error, info, warn};
 use ptrs::{ClientTransport, PluggableTransport, ServerBuilder, ServerTransport};
 
 use anyhow::{anyhow, Context, Result};
@@ -26,18 +27,7 @@ use tokio::{
     sync::oneshot,
 };
 use tokio_util::sync::CancellationToken;
-// use tokio_stream::StreamExt;
-// use tor_chanmgr::transport::proxied::{settings_to_protocol, Protocol};
-// use tor_linkspec::PtTransportName;
-// use tor_ptmgr::ipc::{
-//     PtClientParameters,
-//     PtCommonParameters,
-//     PtServerParameters,
-//     // PluggableClientTransport, PluggableServerTransport, // PluggableTransport
-// };
-// use tor_rtcompat::PreferredRuntime;
-// use tor_socksproto::{SocksAuth, SocksVersion};
-use tracing::{error, info, warn, Level};
+use tracing::Level;
 use tracing_subscriber::{filter::LevelFilter, prelude::*};
 
 use std::{env, net::SocketAddr, str::FromStr, sync::Arc};

diff --git a/crates/o5/src/README.md → crates/o5/README.md b/crates/o5/src/README.md → crates/o5/README.md
@@ -1,3 +1,9 @@
+# o5 Pluggable Transport Library
+
+A randomizing look like nothing pluggable transport library, spiritually a successor
+to `obfs4`.
+
+
 
 changes from obfs4:
 
@@ -19,8 +25,9 @@ changes from obfs4:
 
 Goals
 * Stick closer to Codec / Framed implementation for all packets (hadshake included)
-* use the tor/arti ntor implementation
+* use the tor/arti ntor v3 implementation
 
 
 Features to keep
 - once a session is established, unrecognized frame types are ignored
+
diff --git a/crates/o7/Cargo.toml b/crates/o7/Cargo.toml
@@ -0,0 +1,6 @@
+[package]
+name = "o7"
+version = "0.1.0"
+edition = "2021"
+
+[dependencies]
diff --git a/crates/o7/README.md b/crates/o7/README.md
@@ -0,0 +1,52 @@
+# o7 Pluggable Transport
+
+Experimental randomizing look-like-nothing pluggable transport library in the spirit of obfs4.
+
+### Features for o7
+
+* Fully Encrypted
+* quantum forward secrecy (i.e. ed25519 + Kyber)
+* reliability layer inside encryption layer
+* if outer transport is reliable (i.e. TCP) inner reliability can be disabled
+* if reliability is not required in general, it can be disabled
+* operates as:
+  - Wrap (client, or server)
+  - Dialer
+  - Peer
+
+
+**Supporting**
+
+- kcp / quic / sctp for ordering / reliability
+    - trim quiche down to just quic (no tls / H3)
+
+- kyber (512 / 768 / 1024) keys mapping to uniform random? i.e. elligator2 for kyber
+
+- browser extension for capturing traffic shapes.
+
+- multi-path transport (multipath quic / kcp) with modern congestion control options
+
+- water wasm transport integration
+
+
+### Changes from o5
+
+* Designed for udp
+    - or any transport really, just don't require our reliability layer to be the outermost
+* allow reliable or unreliable traffic modes
+* multiple streams per tunnel
+    - reliability set per stream not per tunnel
+* connection resumption + tickets
+* traffic shaping driven send rates & padding
+    - browser extension for capturing traffic shapes of real sessions securely.
+
+---
+
+* Why shift from the obfs4 style naming and use o7? 
+
+    I am writing the library and I like it that way.
+
+* What happened to o6? 
+
+    See the answer above.
+
diff --git a/crates/o7/src/main.rs b/crates/o7/src/main.rs
@@ -0,0 +1,3 @@
+fn main() {
+    println!("Hello, world!");
+}