Type-inferred scope guard Allow and Subscribe API design

[jrvanwhy]

Rust Playground link with this design.

Scope guards

One way in Rust to guarantee that cleanup is performed is to use a "scope guard". An example of a scope guard is crossbeam::scope. Using a scope guard looks like the following:

run(|scope| {
    scope.do_thing_requiring_cleanup();
});

In the example, run creates the scope and guarantees it is dropped correctly. The calling code does not own the scope, and therefore cannot move or forget it.

In the case of libtock-rs, the scope variable needs to hold the data required to clean up system calls. For Subscribe, no data is required, as the core kernel guarantees the swapping behavior. However, for Allow, the scope variable needs a copy of the shared slice reference, so it can verify the driver returns the correct buffer during cleanup.

Unfortunately, that means the size and type of the scope varies depending on what system calls are performed. Without using heap-based memory allocation, that requires run to know what system calls might be called.

Per-syscall data types

Lets define some structs that know how to clean up after system calls. Note we don't define a constructor yet, because constructing these types is unsafe (the owning code must guarantee that Drop::drop is called before the lifetime 'k ends):

// Safety invariant: RoAllow must be dropped before the 'k lifetime may end.
struct RoAllow<'k, const ID: u32> {
    data: Cell<Option<&'k [u8]>>,
}

impl<'k, const ID: u32> Drop for RoAllow<'k, ID> {
    fn drop(&mut self) {
        if let Some(data) = self.data.take() {
            // Perform the unallow call here.
        }
    }
}

impl<'k, const ID: u32> RoAllow<'k, ID> {
    fn allow_ro(&self, buffer: &'k [u8]) {
        // Perform the allow call here.
        self.data.set(Some(buffer));
    }
}

trait Callback {
    fn invoke(&self, arg: u32);
}

// Safety invariant: Subscribe must be dropped before the 'k lifetime may end.
struct Subscribe<'k, const ID: u32> {
    _phantom: PhantomData<Cell<&'k ()>>,
}

impl<'k, const ID: u32> Drop for Subscribe<'k, ID> {
    fn drop(&mut self) {
        // Do the unsubscribe kernel call here.
    }
}

impl<'k, const ID: u32> Subscribe<'k, ID> {
    fn subscribe<C: Callback>(&self, callback: &'k C) {
        // Do the subscribe kernel call here.
    }
}

Example driver

Although it isn't yet clear how we can instantiate these types, we can go ahead and implement a driver using them:

#[derive(Default)]
struct ConsoleWriter { done: Cell<bool> }

impl Callback for ConsoleWriter {
    fn invoke(&self, _arg: u32) { self.done.set(true); }
}

impl ConsoleWriter {
    fn start_write<'k>(&'k self, &(ref allow, ref subscribe): &(RoAllow<'k, 1>, Subscribe<'k, 1>), buffer: &'k [u8]) {
        allow.allow_ro(buffer);
        subscribe.subscribe(self);
    }

    fn is_done(&self) -> bool { self.done.get() }
}

Application code

Application code would want to perform a write using something like:

fn main() {
    let buffer = b"Hello, World!";
    let console: ConsoleWriter = Default::default();
    run(|syscall_list| {
        console.start_write(syscall_list, buffer);
    });
}

run implementation

With the above app, run would need to look something like:

fn run<Scope, F: FnOnce(&Scope)>(fcn: F) {
    let scope = /* todo: figure out how to construct scope */;
    fcn(&scope)
}

In the example application, scope would need to have the type (RoAllow<1>, Subscribe<1>). To allow run to construct scopes, we create a trait that is implemented for every type that scope could have. We'll call the trait SyscallList because each scope is a list of syscalls:

trait SyscallList<'k> {
    // Safety: The caller must invoke Drop::drop on the returned object before
    // the 'k lifetime ends.
    unsafe fn new() -> Self;
}

Interpreting a single system call as a list of one system call, we can implement SyscallList for RoAllow, Subscribe, and some tuple types:

impl<'k, const ID: u32> SyscallList<'k> for RoAllow<'k, ID> {
    unsafe fn new() -> RoAllow<'k, ID> {
        RoAllow { data: Default::default() }
    }
}

impl<'k, const ID: u32> SyscallList<'k> for Subscribe<'k, ID> {
    unsafe fn new() -> Subscribe<'k, ID> {
        Subscribe { _phantom: PhantomData }
    }
}

impl<'k, A: SyscallList<'k>> SyscallList<'k> for (A,) {
    unsafe fn new() -> (A,) {
        (A::new(),)
    }
}

impl<'k, A: SyscallList<'k>, B: SyscallList<'k>> SyscallList<'k> for (A, B) {
    unsafe fn new() -> (A, B) {
        (A::new(), B::new())
    }
}

We can now modify run's definiton to infer the system call list type from the passed callback function, and instantiate it using SyscallList::new():

fn run<'k, L: SyscallList<'k>, F: FnOnce(&L)>(fcn: F) {
    let list = unsafe { L::new() };
    fcn(&list)
}

[jrvanwhy]

I realize the above writeup is rather rough (probably hard to read/understand) -- if I think it's worth pursuing I'll refine it further.

Future improvements:

1. See if it can work with 'static callbacks and/or buffers.
2. See if some form of "operation" trait can be introduced to make basic synchronous use easier.

[jettr]

I like this idea and the use of ZST to guarding things. I didn't know the best way to give feedback on the code, so I made edits from your playground here.

Basically my comments are:

• Add convert for sycall_list all the way up to 6 (I know it was just a prototype and we probably would have done this)
• Added a return value to the run command
• Made RoAllow hold only PhantomData and always unallow. This makes RoAllow ZST just like Subscribe, and Subscribe always unsubscribes when dropping because it is the safest thing to do, and RoAllow can follow that as well. The caller must know that whenever they call run they are going to get unallow and unsubscribes for all of the syscall items they request, regardless of if they are used (but why request them if you aren't going to use them)
• Added a Timer driver to the example to see how multiple drivers could interact
• Change the order of how syscall objects are packaged for console.start_write to tie buffer and allow object more closely together (very subjective, I know)
• Added a comment that we need to use a read_volatile when reading the timer.done or console.done because of yieldk stuff.

Thanks for putting together the playground to work with!

[jrvanwhy]

I like this idea and the use of ZST to guarding things. I didn't know the best way to give feedback on the code, so I made edits from your playground here.

Basically my comments are:

• Add convert for sycall_list all the way up to 6 (I know it was just a prototype and we probably would have done this)
• Added a return value to the run command

Yes, I intended to do both of those.

• Made RoAllow hold only PhantomData and always unallow. This makes RoAllow ZST just like Subscribe, and Subscribe always unsubscribes when dropping because it is the safest thing to do, and RoAllow can follow that as well. The caller must know that whenever they call run they are going to get unallow and unsubscribes for all of the syscall items they request, regardless of if they are used (but why request them if you aren't going to use them)

That change isn't compatible with the Tock threat model, because it allows malicious capsules to read memory that was not explicitly shared with them. In particular: a capsule can hold on to the buffer past the unallow call (returning a different buffer). The only thing libtock-rs can do about that is to check the return value from Allow against the expected value and terminate if an unexpected buffer was returned.

• Added a Timer driver to the example to see how multiple drivers could interact
• Change the order of how syscall objects are packaged for console.start_write to tie buffer and allow object more closely together (very subjective, I know)

By doing that, you're making the caller specify both separately rather than inferring them as a pair. That leaks implementation details into the calling code, which I'd rather not do.

• Added a comment that we need to use a read_volatile when reading the timer.done or console.done because of yieldk stuff.

That issue is outside the scope of this PR, but I'm fairly confident that read_volatile is not the correct solution.

Thanks for putting together the playground to work with!

[alexandruradovici]

Moved comment to #340

[jettr]

That change isn't compatible with the Tock threat model, because it allows malicious capsules to read memory that was not explicitly shared with them

Are malicious capsule not allowed to have unsafe? If not, I don't see how the malicious capsule gets a reference to the memory is can't read. I don't see how unallowing a buffer always let's a capsule get access. If you nested allows, then the outer unallow wouldn't be the buffer you were expecting anyway.

If we aren't going to implement these checks (which I don't think we should unless it is under and optional feature), then we don't need to waste the ram space to store if unallow needs to be called.

By doing that, you're making the caller specify both separately rather than inferring them as a pair. That leaks implementation details into the calling code, which I'd rather not do.

My example is inferring just as much as the previous example (e.g. there are no manual types listed on run). I don't understand your point. Nothing more about impl details have leaked. Can you write an example were you get different syscall dependencies for multiple drivers to show me how to leak less, please?

[jrvanwhy]

That change isn't compatible with the Tock threat model, because it allows malicious capsules to read memory that was not explicitly shared with them

Are malicious capsule not allowed to have unsafe? If not, I don't see how the malicious capsule gets a reference to the memory is can't read. I don't see how unallowing a buffer always let's a capsule get access. If you nested allows, then the outer unallow wouldn't be the buffer you were expecting anyway.

Consider the following capsule:

struct BufferEater {
    has_buffer: Cell<bool>,
    buffer: Cell<ReadOnlyProcessBuffer>
}

impl SyscallDriver for BufferEater {
    fn allow_readonly(&self, _: ProcessId, _: usize, buffer: ReadOnlyProcessBuffer)
        -> Result<ReadOnlyProcessBuffer, (ReadOnlyProcessBuffer, ErrorCode)>
    {
        if self.has_buffer.get() {
            // Refuse to swap process buffers, return a null buffer.
            return Ok(Default::default());
        }

        // Store the buffer forever.
        self.buffer.set(buffer);
        self.has_buffer.set(true);
    }
}

Then BufferEater gets access to deallocated memory with the following application code:

fn foo() {
    let buffer = b"to be deallocated";
    run(|ro_allow: RoAllow<1>| {
        // Share the buffer with the buffer eater.
        ro_allow.allow_ro(&buffer);
    });
    // When `run` finishes, it calls Read-Only Allow to retrieve the buffer. The syscall succeeds, but
    // does not revoke `BufferEater`'s access to `buffer`.
    // After foo returns, `BufferEater` will have access to something it shouldn't.
}

The above BufferEater could be identified by checking the returned buffer against null, but a more complex capsule could implement a FIFO queue of buffers which RoAllow would be unable to identify as incorrect behavior.

Does that make sense?

If we aren't going to implement these checks (which I don't think we should unless it is under and optional feature), then we don't need to waste the ram space to store if unallow needs to be called.

These checks should be there by default. I'm willing to add an interface that skips the checks, but it will be unsafe to use. Note that for Read-Write Allow a misbehaving/malicious capsule could trigger undefined behavior in a process if the checks are missing.

By doing that, you're making the caller specify both separately rather than inferring them as a pair. That leaks implementation details into the calling code, which I'd rather not do.

My example is inferring just as much as the previous example (e.g. there are no manual types listed on run). I don't understand your point. Nothing more about impl details have leaked. Can you write an example were you get different syscall dependencies for multiple drivers to show me how to leak less, please?

Playground link

By grouping the syscall handles as I had originally done, I was able to simplify the run call to:

    run(|(console_syscalls, timer_syscalls)| {
        timer.start_timer(timer_syscalls);
        console.start_write(console_syscalls, buffer);

The full diff relative to your playground is:

100c100
<     fn start_write<'k>(&'k self, subscribe: &Subscribe<'k, 1>, (buffer, allow): (&'k [u8], &RoAllow<'k, 1>)) {
---
>     fn start_write<'k>(&'k self, &(ref subscribe, ref allow): &(Subscribe<'k, 1>, RoAllow<'k, 1>), buffer: &'k [u8]) {
138,140c138,140
<     run(|(console_data, console_done, timer_done)| {
<         timer.start_timer(timer_done);
<         console.start_write(console_done, (buffer, console_data));
---
>     run(|(console_syscalls, timer_syscalls)| {
>         timer.start_timer(timer_syscalls);
>         console.start_write(console_syscalls, buffer);

[hudson-ayers]

The problem for allow is just that the kernel does not have allow-buffer-swapping-prevention in the way that it has callback-swapping prevention, right? But if that was implemented one day (and we have talked about wanting this for awhile) we would probably want to be able to remove this extra overhead from libtock-rs.

[jrvanwhy]

The problem for allow is just that the kernel does not have allow-buffer-swapping-prevention in the way that it has callback-swapping prevention, right? But if that was implemented one day (and we have talked about wanting this for awhile) we would probably want to be able to remove this extra overhead from libtock-rs.

Yes.

[jettr]

Thank you again for such thorough examples! I do now see that we should do the check to make sure the kernel does let go. Granted the kernel could still do something to that memory even if it gives you back the expected response, so you do have to draw the trust line somewhere.

I think having that check on all unallows might cost valuable flash space, especially for builds where we trust the kernel. At the risk of adding feature creep, this would probably be a really nice thing to be able to disable runtime checks for. This really would be handled better with build time checks (when the kernel does the memory allowing and unallowing)

I see your new example, and that is what I came up with locally after posting my shared playground. That looks good to me. To me, it seems like it leaks/ doesn't leak the same amount of implementation details, but I am not caught up on it. Either way is fine with me.

Overall this approach looks good to me.

[jrvanwhy]

I'm beginning to implement this design in #342

[jrvanwhy]

This design has been merged into libtock-rs.