rustgraz_talk_07.md

author	title	date
Lukas Prokop	Rust Graz – 07 unsafe	29th of January, 2020

Prologue

---

Prologue: Is UTF-8 robust w.r.t. to bit flips?

Consider the following [contrived] notion of robustness.

Let {X} be a valid UTF-8 string. Is [A, {X}, A] a valid UTF-8 string if one arbitrary bit flip occurs?

---

Prologue: Is UTF-8 robust w.r.t. to bit flips?

// Consider a byte array like [0b1000001, 0b11110xxx,
// 0b10xxxxxx, 0b10xxxxxx, 0b10xxxxxx, 0b1000001]
// where x denote arbitrary bits.
// Continue only if the array is a valid UTF-8 string.
// Now introduce one arbitrary bit flip in any bit of
// the 4 mid bytes. Does it give a valid UTF-8 string?
// Returns the total number of tests and invalid strings.

---

Prologue: Is UTF-8 robust w.r.t. to bit flips?

fn flips_in_4_bytes() -> (u64, u64) {
  let mut total = 0u64;
  let mut invalid = 0u64;

  let mut text = [
    'A' as u8,
    0b1111_0000u8, 0b10_000000,
    0b10_000000, 0b10_000000,
    'A' as u8,
  ];
  // TODO next slide

  (total, invalid)
}

---

Prologue: Is UTF-8 robust w.r.t. to bit flips?

// base gives x-values in 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx
for base in 0..=2097152u32 {
  // bit_flip defines the index of the bit to flip
  for bit_flip in 0..32 {
    text[1] = 0b1111_0000 | (base & 0b111) as u8;
    text[2] = 0b1000_0000 | ((base >> 3) & 0b0011_1111) as u8;
    text[3] = 0b1000_0000 | ((base >> 9) & 0b0011_1111) as u8;
    text[4] = 0b1000_0000 | ((base >> 15) & 0b0011_1111) as u8;
    if str::from_utf8(&text).is_err() {
      continue
    }
    // TODO next slide
    total += 1;
  }
}

---

Prologue: Is UTF-8 robust w.r.t. to bit flips?

      // apply bit flip
      if bit_flip < 8 {
        text[1] = text[1] ^ (1 << bit_flip);
      } else if bit_flip < 16 {
        text[2] = text[2] ^ (1 << (bit_flip - 8));
      } else if bit_flip < 24 {
        text[3] = text[3] ^ (1 << (bit_flip - 16));
      } else {
        text[4] = text[4] ^ (1 << (bit_flip - 24));
      }
      match str::from_utf8(&text) {
        Ok(_) => {},
        Err(_) => invalid += 1,
      };

---

Prologue: Is UTF-8 robust w.r.t. to bit flips?

bytes	strings	turned invalid	%
1	1024	128	12.5 %
2	30,720	10,112	33.0 %
3	1,474,560	512,000	34.7 %
4	33,554,432	13,107,200	39.1 %
	35,060,736	13,629,440	38.9 %

---

Prologue: Is UTF-8 robust w.r.t. to bit flips?

Is [A, {X}, A] a valid UTF-8 string if one arbitrary bit flip occurs? Yes, with 61.6 % chance.

Dialogue: actix

---

actix

https://actix.rs/

• “rust's powerful actor system and most fun web framework”
• Uses tokio (“asynchronous run-time for Rust”)
• Uses futures (“zero-cost asynchronous programming in Rust”)
• async/sync actors
• Actor supervision
• Typed messages
• HTTP 1 / HTTP 2 support ⇒ actix_web

---

actix_web

“Actix web is a small, pragmatic, and extremely fast rust web framework.”

• routing
• GET/POST parameters
• form handling
• serde for serialization
• many dependencies, but fastest web framework

---

actix_web

---

actix_web example

use actix_web::{web, App, Responder, HttpServer};

async fn index(info: web::Path<String>) -> impl Responder {
    format!("Hello {}!", info)
}

#[actix_rt::main]
async fn main() -> std::io::Result<()> {
    HttpServer::new(|| App::new().service(
        web::resource("/{name}").to(index))
    ).bind("localhost:8080")?.run().await
}

---

actix_web example

Dialogue: The actix-web incident

---

github page (2019-12-17)

---

github page (2019-12-17)

---

github page (2020-01-17)

---

github page (2020-01-17)

---

Tweet by @fafhrd91

---

README text excerpts

Another day, another "unsafe shitstorm”, I guess I get used to it.

---

README text excerpts

I’ve never used unsafe unintentionally, I use it because I believe usage is safe. There was no any malicious intentions. I believed it held mutable aliasing invariant and I was very happy that someone found real problem.

---

README text excerpts

Be a maintainer of large open source project is not a fun task. You alway face with rude and hate, everyone knows better how to build software, nobody wants to do home work and read docs and think a bit and very few provide any help.

---

README text excerpts

For example, async/await took three weeks 12 hours/day work stint, quite exhausting, and what happened after release, I started to receive complaints that docs are not updated and i have to go fix my shit.

---

README text excerpts

[…] But damage to the project's reputation is done and I don’t think it is possible to recover. Actix always will be “shit full of UB” and “benchmark cheater”.

---

README text excerpts

(Btw, with tfb benchmark I just wanted to push rust to the limits, I wanted it to be on the top, I didn’t want to push other rust frameworks down.)

---

README text excerpts

Everything started with actix, then actix-web and then actix-net. It took a lot of time to design api and architecture. Each of this projects was rewritten from scratch at least 4-5 time

---

README text excerpts

Nowadays supporting actix project is not fun, and be part of rust community is not fun as well.

I am done with open source. […] Everything has to come to the end. It was fun trip but now is time to move on. Life should be fun.

---

Summary

• Somehow related to some shitstorm
• Shitstorm was somehow related to some unsafe feature of rust
• github repo was wiped (incl. issues) and README with post mortem text appeared
• @fafhrd91 used to be incredibly active and supportive towards users

→ An attempt of a reconstruction

---

2018/06/08

• github issue #289
• title “Unsound uses of Unsafe in API”
• “Right now the actix-web code contains 100+ uses of unsafe. Presumably this is in order to achieve the best possible performance in hot parts of the code.” …

---

2018/06/12

• github issue #301
• issue is related to a memory-safety bug. Technical discussion is started.
• comment by @fafhrd91 “@seanmonstar do you think I don’t understand impl Send? :)” 11× 👎

---

2018/07/06

• reddit thread references issue #289
• title “actix-web has removed all unsound use of unsafe in its codebase. It's down to less than 15 occurences of unsafe from 100+”

---

2020/01/17

• reddit thread “Actix-net unsoundness patch "is boring"”

• by user Code-Sandwich

  • brings github issue #83 to reddit user's attention (is deleted these days)
  • Summarizes issue #83
  • Asks “I hope it's an objective summary. Any thoughts?”

---

Issue #83 2020-01-08 and later

@Shnatsel opens the issue:

The following code is unsound: [actix-net/actix-service/src/cell.rs Lines 34 to 36 in 7c5fa25] This uses Rc::as_ref() to obtain a reference to the underlying data, which does not guarantee uniqueness. It is possible to obtain several mutable references to the same memory location by calling this function repeatedly: […]

---

Issue #83 excerpts

@fafhrd91

This is internal code. There is no repeated call to get_mut() anywhere in code

Closes issue.

@fafhrd91

Please, don’t start

---

Issue #83 excerpts

@Shnatsel

These two references do not need to exist in the same function to trigger undefined behavior, they only need to exist at the same point in time. An easy way to see if this is a problem in practice is replace .as_ref() with .get_mut().unwrap() and see if anything panics.

---

Issue #83 excerpts

@cdbattags

I don't think "unsound" was meant to be personal or offensive. Why close this so quickly?

@fafhrd91

I need unit test that shows UB. Unit test must use public api.

---

Issue #83 excerpts

@Nemo157

[provides such a unit test]

@JohnTitor

I think it's worth to fix, re-opening.

---

Issue #83 excerpts

@fafhrd91

@Nemo157 @repnop nice! finally some real code!

[provides patch]

should be fixed in master

---

Issue #83 excerpts

Discussion about incompleteness of patch. Nemo157 provides patch using Rc<RefCell<A>>, not Cell<A> to ensure borrow safety at runtime.

@fafhrd91

this patch is boring

---

Issue #83 excerpts

@CJKay

quote[this patch is boring] So is resolving silent data corruption.

---

Issue #83 excerpts

@bbqsrc

@fafhrd91 seriously? Please just stop writing Rust. You do not respect semver, you do not respect soundness, so why are you using a language predominantly based around doing these things right?

---

Community responses

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

→ “The Soundness Pledge”

---

The Soundness Pledge

“The intent of this crate is to be free of soundness bugs. The developers will do their best to avoid them, and welcome help in analyzing and fixing them.”

---

---

Github issues on 2020-01-20

• lmao
• a sad day for rust ... and me
• actix-web is great
• jealous assholes
• Cheer up
• Please continue....
• Proud of your Awesome work!
• Don't take shit from anyone
• Condorcet

---

Github issues on 2020-01-20

• It's fair
• fork?
• We love actix, thank you for creating it!
• Don't care about the prejudices of those few people, support you！
• Actix is still awesome
• Consider applying for GitHub sponsorship?
• Please archive the project instead of deleting it.
• Thank you for your service

---

… with purely positive replies.

---

Resources

• Wiped repo
• HackerNews thread
• Support letter repo for Nikolay
• Steve Klabnik's blog post
• “The Soundness pledge”
• reddit discussion

---

Questions to contemplate

• What are the expectations/responsibilities of a community/maintainer?
• Is there a difference between a commercial and a hobby project?
• When shall I contribute or fork a project?
• How many maintainers/contributors does a project need?
• How much shall we consider differences in cultural norms?
• What non-technical aspects do we need to discuss/ensure in a project?

---

Unquestionably good advices

• Be lenient in what you accept. Be strict in what you send out.
• Try to understand each other. Expect the other has good intentions. In short: be respectful.
• Ask for help if you need advice.
• “Code of Conducts” make is explicit that “do whatever is legally allowed” sets the bar too low

Dialogue: unsafe rust

---

unsafe

See Unsafe Rust section in the Rust book and nomicon.

1. Is a keyword in rust: unsafe { let a = 42; }
2. Applies to blocks of code, incl. functions, traits.
3. Does not turn off borrow checker or alike
4. Semantics are basic: Some responsibilities move from rust to the programmer!

---

unsafe rationale

1. Static analysis is conservative. When the compiler tries to determine whether or not code upholds the guarantees, it’s better for it to reject some valid programs rather than accept some invalid programs.
2. The underlying computer hardware is inherently unsafe. Optimizations sometimes requires breaking strong guarantees.

---

unsafe superpowers

1. Dereference a raw pointer
2. Call an unsafe function or method
3. Access or modify a mutable static variable
4. Implement an unsafe trait
5. Access fields of unions

---

Raw pointers

→ does not require unsafe rust. But we can’t dereference raw pointers outside an unsafe block.

1. Are allowed to ignore the borrowing rules by having both immutable and mutable pointers or multiple mutable pointers to the same location
2. Aren’t guaranteed to point to valid memory
3. Are allowed to be null
4. Don’t implement any automatic cleanup

---

In practice: raw pointers

fn main() {
    let addr = 0x00;
    let ptr = addr as *const i32;
    println!("{}", *ptr);
}

ptr is a raw pointer. Does it compile?

---

error[E0133]: dereference of raw pointer is unsafe and
              requires unsafe function or block
 --> unsafe_tests.rs:4:20
  |
4 |     println!("{}", *ptr);
  |                    ^^^^ dereference of raw pointer
  |
  = note: raw pointers may be NULL, dangling or unaligned;
    they can violate aliasing rules and cause data races:
    all of these are undefined behavior

error: aborting due to previous error

---

In practice

fn main() {
    let addr = 0x00;
    let ptr = addr as *const i32;
    unsafe {
        println!("{}", *ptr);
    }
}

---

In practice

meisterluk@gardner ~ % ./unsafe_block
[1]    29915 segmentation fault  ./unsafe_block

---

In practice

unsafe fn deref() {
    let addr = 0x00;
    let ptr = addr as *const i32;
    println!("{}", *ptr);
}

fn main() {
    unsafe {
        deref();
    }
}

If unsafe block is omitted, “call to unsafe function is unsafe and requires unsafe function or block”

---

In practice: mut static variable

static MY_INT: u64 = 42;
  
fn main() {
    println!("{}", MY_INT);
}

→ type specifier is required

---

In practice

static MY_INT: u64 = 42;

fn main() {
    MY_INT += 1;
    println!("{}", MY_INT);
}

---

In practice

static MY_INT: u64 = 42;

fn main() {
    MY_INT += 1;
    println!("{}", MY_INT);
}

Does it compile?

---

In practice

error[E0594]: cannot assign to immutable static item `MY_INT`
 --> mut_static_var.rs:4:5
  |
4 |     MY_INT += 1;
  |     ^^^^^^^^^^^ cannot assign

error: aborting due to previous error

---

In practice

static mut MY_INT: u64 = 42;
  
fn main() {
    MY_INT += 1;
    println!("{}", MY_INT);
}

Does it compile?

---

In practice

error[E0133]: use of mutable static is unsafe
              and requires unsafe function or block
 --> mut_static_var.rs:4:5
  |
4 |     MY_INT += 1;
  |     ^^^^^^^^^^^ use of mutable static
  |
  = note: mutable statics can be mutated by multiple
    threads: aliasing violations or data races will
    cause undefined behavior

error[E0133]: use of mutable static is unsafe and
              requires unsafe function or block
 --> mut_static_var.rs:5:20
  |
5 |     println!("{}", MY_INT);
  |                    ^^^^^^ use of mutable static
  |
  = note: mutable statics can be mutated by multiple
    threads: aliasing violations or data races will
    cause undefined behavior

---

In practice

static mut MY_INT: u64 = 42;
  
fn main() {
    unsafe {
        MY_INT += 1;
    }
    println!("{}", MY_INT);
}

Does it compile?

---

In practice

error[E0133]: use of mutable static is unsafe and
              requires unsafe function or block
 --> mut_static_var.rs:7:20
  |
7 |     println!("{}", MY_INT);
  |                    ^^^^^^ use of mutable static
  |
  = note: mutable statics can be mutated by multiple
    threads: aliasing violations or data races will
    cause undefined behavior

---

In practice

static mut MY_INT: u64 = 42;
  
fn main() {
    unsafe {
        MY_INT += 1;
        println!("{}", MY_INT);
    }
}

---

In practice: union

#[repr(C)]
union Ambiguous {
    my_int: u32,
    addr: *const u32,
}

fn main() {
    let a = Ambiguous { my_int: 0x00 };
    println!("{}", *a.addr);
}

---

In practice

error[E0133]: access to union field is unsafe and
              requires unsafe function or block
 --> enum.rs:9:20
  |
9 |     println!("{}", *a.addr);
  |                    ^^^^^^^ access to union field
  |
  = note: the field may not be properly initialized: using uninitialized data will cause undefined behavior

error[E0133]: dereference of raw pointer is unsafe
              and requires unsafe function or block
 --> enum.rs:9:20
  |
9 |     println!("{}", *a.addr);
  |                    ^^^^^^^ dereference of raw pointer
  |
  = note: raw pointers may be NULL, dangling or unaligned;
    they can violate aliasing rules and cause data races:
    all of these are undefined behavior

---

Do I see unsafe rust in the documentation?

Yes, because you need to call it unsafely.

Example: Rc::assume_init

---

So, how to consider unsafe rust?

---

Compiler guarantees

C ⊂ unsafe rust ⊂ safe rust

---

via github: safety dance

---

RustBelt: Logical Foundations for the Future of Safe Systems Programming by Derek Dreyer

---

RustBelt: Logical Foundations for the Future of Safe Systems Programming by Derek Dreyer

Quiz

---

A bit flip occurs in a valid UTF-8 string. How “likely” is it still valid?

61.6 %

What is actix / actix-web?

Actor system in rust / web framework

Lexically, unsafe is …

a keyword

What are the differences of (un)safe rust?

superpowers = {deref raw pointer, call unsafe fn, read/write mut static var, impl unsafe trait, read union fields}

---

Thanks!