So far messages have just been passing a single 64-bit register, but we’re getting to the point where we might want to pass more information, and different kinds of information, between threads. Like messages in Xous we’re going to have a short (and hopefully fast) message type, and a long type for larger or more complex messages.
Short messages just copy registers between threads. We use rax for
the syscall number and error return code, rdi for the handle, and we
can’t use rcx or r11 because these are used by syscall/sysret.
The Linux syscall interface uses rdi, rsi, rdi, r10, r8 and
r9 registers, so these are reasonable choices.
There is a penalty to using too many registers: The more registers we clobber with message passing, the more the user program will have to avoid using those registers, and perhaps push and pop from the stack instead. The optimum number of registers will depend on the typical message sizes, so tuning will have to wait until later.
We’ll need a register to store information about the kind of data the
message contains. The rax register seems like a good place to do
this: We’re using rax for the syscall number, but we don’t need 64
bits for that. We’re not likely to need more than 256 syscalls (8
bits), leaving 56 bits for other things. We can start by putting the
rendezvous handle, which is currently in rdi, into the high 32 bits
of rax. This will limit each program to “only” 4 billion handles,
which is probably enough.
Somewhat arbitrarily we’re going to use three registers for the data
(for now): rdi, rsi and rdx. We’ll make the Short message
type larger, and move it into a new file kernel/src/message.rs:
pub enum Message {
Short(u64, u64, u64),
}We can also define Message in the user library in a new file
euralios_std/src/message.rs with the same code, and add
pub mod message; to euralios_std/src/lib.rs. We’ll keep the
definitions separate because the storage of messages in the kernel and
user code will be different for the long messages.
The library code (euralios_std/src/syscalls.rs) is now going to put
the syscall number and handle into rax, and the three values into
rdi, rsi and rdx.
use crate::message::Message;
pub fn send(
handle: u32,
message: Message
) -> Result<(), u64> {
match message {
Message::Short(data1, data2, data3) => {
let err: u64;
unsafe {
asm!("syscall",
in("rax") 4 + ((handle as u64) << 32),
in("rdi") data1,
in("rsi") data2,
in("rdx") data3,
lateout("rax") err,
out("rcx") _,
out("r11") _);
}
if err == 0 {
return Ok(());
}
Err(err)
},
_ => return Err(0)
}
}Note that we signal to the user code that the handle is only 32 bits,
but convert to 64 bits to shift and put into rax.
The syscall instruction then switches to kernel code. To get the
data values into sys_send function we first need to add to
handle_syscall(). In the C calling convention the first six function
arguments are stored in registers rdi, rsi, rdx, rcx, r8 and
r9. The fifth argument is therefore r8, which will contain
data3:
"mov r8, rdx", // New
"mov rcx, rsi",
"mov rdx, rdi",
"mov rsi, rax",and dispatch_syscall() we’re now just using 8 bits for the syscall
number so we now match on syscall_id & 0xFF:
extern "C" fn dispatch_syscall(context_ptr: *mut Context, syscall_id: u64,
arg1: u64, arg2: u64, arg3: u64) {
...
match syscall_id & 0xFF { // New
...
4 => sys_send(context_ptr, syscall_id,
arg1, arg2, arg3), // New
...
}
}and sys_send()
fn sys_send(
context_ptr: *mut Context,
syscall_id: u64,
data1: u64,
data2: u64,
data3: u64) {
let handle = syscall_id >> 32; // New
...
let (thread1, thread2) = rdv.write().send(
Some(thread),
Message::Short(data1,
data2, // New
data3)); // New
}The other place we send messages is in the keyboard_handler_inner
function (interrupts.rs);
let (thread1, thread2) =
KEYBOARD_RENDEZVOUS.write().send(
None,
Message::Short(character as u64,
0, 0)); // NewTo get the data out of the kernel to the receiving thread, we need to
modify return_message() in process.rs which consumes a Message object
and puts the values into a receiving thread’s registers:
pub fn return_message(&self, message: Message) {
let context = self.context_mut();
context.rax = 0;
match message {
Message::Short(data1, data2, data3) => {
context.rdi = data1 as usize;
context.rsi = data2 as usize; // New
context.rdx = data3 as usize; // New
},
_ => {}
}
}A long message has to handle everything that we might want to send between processes. That includes values, as in short messages, but also rendezvous handles, and probably other things later like memory chunk handles, which will refer to a set of pages for transferring large amounts of data. For any kind of handle we might want to either copy or move/assign to the other process.
To keep things simple we’ll use the same three registers as short
messages (rdi, rsi and rdx), and just send three things. Each
register can contain either a value or a rendezvous handle.
rax will contain
- 8 bits for the syscall number
- 32 bits for the handle
- 1 bit to specify if it’s a long or short message. If a short message then the kernel skips any other checks and just copies the values.
- 1 bit per register (3 total) specifying the type (value or handle)
- 1 bit per register (3 total) to specify copy or move
- 17 remaining bits for future expansion
In rendezvous.rs we can define the long message as it will be stored
in the kernel:
use alloc::{boxed::Box, sync::Arc};
use spin::RwLock;
pub enum MessageData {
Value(u64),
Rendezvous(Arc<RwLock<Rendezvous>>),
}
pub enum Message {
Short(u64, u64, u64),
Long(u64, MessageData, MessageData), // New
}In syscalls.rs the flags which will be used in rax:
pub const MESSAGE_LONG: u64 = 2 << 8;
pub const MESSAGE_DATA2_RDV: u64 = 2 << 9;
const MESSAGE_DATA2_TYPE: u64 = MESSAGE_DATA2_RDV; // Bit mask
const MESSAGE_DATA2_MOVE: u64 = 2 << 10;
pub const MESSAGE_DATA3_RDV: u64 = 2 << 11;
const MESSAGE_DATA3_TYPE: u64 = MESSAGE_DATA3_RDV; // Bit mask
const MESSAGE_DATA3_MOVE: u64 = 2 << 12;fn sys_send(
context_ptr: *mut Context,
syscall_id: u64,
data1: u64,
data2: u64,
data3: u64) {
...
if let Some(rdv) = thread.rendezvous(handle) {
let message = if syscall_id & MESSAGE_LONG == 0 {
Message::Short(data1,
data2,
data3)
} else {
// Long message
let message = Message::Long(
data1,
if syscall_id & MESSAGE_DATA2_TYPE == MESSAGE_DATA2_RDV {
// Moving or copying a handle
// First copy, then drop if message is valid
if let Some(rdv) = thread.rendezvous(data2) {
MessageData::Rendezvous(rdv)
} else {
// Invalid handle
thread.return_error(SYSCALL_ERROR_INVALID_HANDLE);
process::set_current_thread(thread);
return;
}
} else {
MessageData::Value(data2)
},
if syscall_id & MESSAGE_DATA3_TYPE == MESSAGE_DATA3_RDV {
if let Some(rdv) = thread.rendezvous(data3) {
MessageData::Rendezvous(rdv)
} else {
// Invalid handle.
// If we moved data2 we would have to put it back here
thread.return_error(SYSCALL_ERROR_INVALID_HANDLE);
process::set_current_thread(thread);
return;
}
} else {
MessageData::Value(data3)
});
// Message is valid => Remove handles being moved
if (syscall_id & MESSAGE_DATA2_TYPE == MESSAGE_DATA2_RDV) &&
(syscall_id & MESSAGE_DATA2_MOVE != 0) {
let _ = thread.take_rendezvous(data2);
}
if (syscall_id & MESSAGE_DATA3_TYPE == MESSAGE_DATA3_RDV) &&
(syscall_id & MESSAGE_DATA3_MOVE != 0) {
let _ = thread.take_rendezvous(data3);
}
message
};
let (thread1, thread2) = rdv.write().send(
Some(thread),
message);
...
}Then in process.rs we need to be able to modify the vector of handles, but Thread.process is an
Arc<Process> which doesn’t allow modification. We need to use a mutex such as a spin lock:
struct Thread {
...
process: Arc<RwLock<Process>>,
...
}use crate::rendezvous::{Rendezvous, MessageData};
impl Thread {
pub fn return_message(&self, message: Message) {
let context = self.context_mut();
context.rax = 0; // No error
match message {
Message::Short(data1, data2, data3) => {
context.rdi = data1 as usize;
context.rsi = data2 as usize;
context.rdx = data3 as usize;
},
Message::Long(data1, data2, data3) => {
context.rdi = data1 as usize;
context.rsi = match data2 {
MessageData::Value(value) => value,
MessageData::Rendezvous(rdv) => {
context.rax |= (syscalls::MESSAGE_DATA2_RDV |
syscalls:: MESSAGE_LONG) as usize;
self.give_rendezvous(rdv)
}
} as usize;
context.rdx = match data3 {
MessageData::Value(value) => value,
MessageData::Rendezvous(rdv) => {
context.rax |= (syscalls::MESSAGE_DATA3_RDV |
syscalls::MESSAGE_LONG) as usize;
self.give_rendezvous(rdv)
}
} as usize;
}
}
}
}so in new_kernel_thread and new_user_thread we now need to
construct this with process: Arc::new(RwLock::new(Process {...}))
pub fn rendezvous(&self, id: u64)
-> Option<Arc<RwLock<Rendezvous>>> {
self.process.read().handles.get(id as usize) // Option<&Option<Arc<>>>
.unwrap_or(&None) // &Option<Arc<>>
.as_ref() // Option<&Arc<>>
.map(|rv| rv.clone()) // Option<Arc<>>
}
/// Take the rendezvous, leaving handle empty (None)
pub fn take_rendezvous(&self, id: u64)
-> Option<Arc<RwLock<Rendezvous>>> {
self.process.write().handles.get_mut(id as usize).map_or(None, |elem| elem.take())
}
/// Add a rendezvous to the process, returning the handle
pub fn give_rendezvous(&self, rendezvous: Arc<RwLock<Rendezvous>>) -> u64 {
// Lock the handles
let handles = &mut self.process.write().handles;
// Find empty handle slot
for (pos, handle) in handles.iter().enumerate() {
if handle.is_none() {
// Found empty slot => Store rendezvous
handles[pos] = Some(rendezvous);
return pos as u64;
}
}
// All full => Add new handle
handles.push(Some(rendezvous));
(handles.len() - 1) as u64
}Ok, enough messaging for now (hurray!). Next it’s time for the operating system to start doing something useful, so we’ll start work on accessing devices and storage next time.