Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

System V shared memory APIs (2nd draft) #2314

Open
wants to merge 58 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
096cb21
First try at implementing SystemV
IzawGithub Feb 13, 2024
072352c
Add permission struct
IzawGithub Feb 14, 2024
adeaaa2
Add crude semaphore support
IzawGithub Feb 14, 2024
135488e
Add documentation that was missing
IzawGithub Feb 14, 2024
ac357e2
Add PR changelog file
IzawGithub Feb 14, 2024
3f6a4c3
Move SystemV to it's own file
IzawGithub Feb 19, 2024
5047036
Fix not compiling on Musl because of missing struct
IzawGithub Feb 20, 2024
b428de0
Fix clippy unsafe ptr arg dereference lint
IzawGithub Feb 20, 2024
07a6f33
Formatting parse
IzawGithub Feb 20, 2024
ac33ca0
Add missing import that I forgot to commit
IzawGithub Feb 20, 2024
11aad71
Update doctest path that changed
IzawGithub Feb 20, 2024
d8db9fa
Add basic IPC test.
IzawGithub Feb 20, 2024
bd938e6
Fix using latest Rust feature that was unstable before
IzawGithub Feb 20, 2024
b9a5905
Forgot a formatting parse
IzawGithub Feb 20, 2024
50916a4
Enable SystemV only for linux, but not Android
IzawGithub Feb 20, 2024
f3e4771
Update conditional compilation for GNU extension
IzawGithub Feb 20, 2024
9c71d97
Fix macro not compiling for NetBSD
IzawGithub Feb 20, 2024
5d0d37c
Fix moving shmid_ds to GNU extension when it's not
IzawGithub Feb 20, 2024
315eb31
Fix mistake in conditional compilation that didn't disable SystemV fo…
IzawGithub Feb 20, 2024
9374dcf
Add back cfg that disable some flags for BSD
IzawGithub Feb 20, 2024
bb07d5b
Final (I hope) formatting pass
IzawGithub Feb 20, 2024
19c713d
Disable test for non systemV platform
IzawGithub Feb 20, 2024
17561e6
Fix wrong changelog number and content
IzawGithub Mar 21, 2024
2b76711
Remove first try at SystemV, replacing with refractor
IzawGithub Mar 22, 2024
94e4e11
First try at refractoring SystemV to be more Rusty
IzawGithub Mar 22, 2024
aea8588
Add a feature gate
IzawGithub Mar 22, 2024
02b0de2
Add missing doc
IzawGithub Mar 22, 2024
ed99763
Fix doctest copy and paste error
IzawGithub Mar 22, 2024
b614d81
Remove Clone derive, wasn't working correctly
IzawGithub Mar 22, 2024
2629192
Update test to create an entirely new SharedMemory object
IzawGithub Mar 22, 2024
b7b8818
Misc formatting
IzawGithub Mar 22, 2024
930f352
Remove unused static mutex for system_v
IzawGithub Mar 24, 2024
4bd41cf
Add `Shm` wrapper for creating a shared memory segment
IzawGithub Mar 24, 2024
2c714c2
Remove function that create a memory segment from `SharedMemory`
IzawGithub Mar 24, 2024
6877712
Make `buf` parameter pass by ref
IzawGithub Mar 24, 2024
b7f6a09
Remove the `shmat` enum value that work with incompatible void ptr
IzawGithub Mar 24, 2024
d07c9b8
Update unit test
IzawGithub Mar 24, 2024
5697ffd
Update imported function
IzawGithub Mar 24, 2024
3d70ae2
Remove stable ptr_from_ref feature that isn't stable on Rust 1.69
IzawGithub Mar 24, 2024
238fe62
Remove unused comment mark
IzawGithub Mar 25, 2024
0ff180a
Add back `shmat` address. Use a raw pointer.
IzawGithub Mar 25, 2024
f5f2f59
Fix mismatched target os
IzawGithub Mar 25, 2024
2803e31
Fix wrong comment formatting
IzawGithub Mar 25, 2024
35c2f4e
Move `shmctl` function to `Shm`.
IzawGithub Mar 25, 2024
d05db61
Update doctest
IzawGithub Mar 25, 2024
ef1a839
Remove `id` from `SharedMemory` don't need it now that `shmctl` isn't…
IzawGithub Mar 25, 2024
e91efa3
Move test to independent file
IzawGithub Mar 25, 2024
d96d5ee
Follow Nix module convention
IzawGithub Mar 25, 2024
6aa8740
Update independent test import
IzawGithub Mar 25, 2024
fd108e4
Rename `system_v` feature to `sysvipc`, like Linux
IzawGithub Mar 25, 2024
2fde2c5
Update doctest import path
IzawGithub Mar 25, 2024
e531e8b
Remove `IPC_PRIVATE` value from `ShmgetFlag`, because it wasn't actua…
IzawGithub Mar 25, 2024
fee5acb
Formatting parse
IzawGithub Mar 25, 2024
52473ce
Update comment of function that moved
IzawGithub Mar 25, 2024
d526780
Fix typo `test_smh` to `test_shm`
IzawGithub Mar 26, 2024
400f71e
Add a test for unsafe `shmget`. Branch coverage is now 89.29%
IzawGithub Mar 26, 2024
d619c48
Improve code comment
IzawGithub Mar 26, 2024
ca0c5f9
Formatting parse
IzawGithub Mar 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog/2313.added.md
IzawGithub marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add SystemV IPC support
SteveLauC marked this conversation as resolved.
Show resolved Hide resolved
4 changes: 4 additions & 0 deletions src/sys/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,10 @@ feature! {
#[allow(missing_docs)]
pub mod sysinfo;

#[allow(missing_docs)]
IzawGithub marked this conversation as resolved.
Show resolved Hide resolved
#[cfg(any(bsd, target_os = "linux", not(target_os = "android")))]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this not(target_os = "android") necessary as any(bsd, target_os = "linux") does not include android

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And it is always ok to not support all the platforms, we can add them incrementally

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed the redundant android target. BSD is still not building and I don't really understand why. Something to do with macro? I'm not proficient with them yet.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BSD is still not building and I don't really understand why.

The OpenBSD and NetBSD CI failed because some missing constants in the libc crate, which means we need to add them before merging this PR if we want to have #[cfg(bsd)s supported.

Let me take a closer look at this and probably give it a fix!

pub mod system_v;

feature! {
#![feature = "term"]
#[allow(missing_docs)]
Expand Down
385 changes: 385 additions & 0 deletions src/sys/system_v.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,385 @@
use std::ptr::{null, null_mut};

use crate::errno::Errno;
use crate::Result;

use libc::{self, c_int, c_short, c_void, key_t, shmid_ds, size_t};
#[cfg(target_env = "gnu")]
use libc::{semid_ds, seminfo};

#[derive(Debug, Default, Clone, Copy)]
/// Type used to transform a raw number to an octal permission, while performing a clamp to u9
///
/// # Example
///
/// ```
/// # use nix::errno::Errno;
/// # use nix::sys::system_v::Permissions;
///
/// # fn main() -> Result<(), Errno> {
/// assert_eq!(Permissions::new(511)?.get_permission(), &(0o0777 as u16));
/// assert_eq!(Permissions::new(512).expect_err("512 is bigger than what u9 can store"), Errno::E2BIG);
/// # Ok(())
/// # }
/// ```
pub struct Permissions {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the last 9 bits of those flg arguments, Nix already has a type for it,
nix::sys::stat::Mode, so a new type is probably not necessary.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, wasn't aware of this type. This makes it depend on the fs module but I'm assuming it's fine?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes it depend on the fs module but I'm assuming it's fine?

Yeah, this is unfortunate, ideally, this Mode type should be available if #[cfg(any(feature="fs", feature="system_v")], doing this will require some extra work, so I think it is ok to make system_v depend on fs.

permission: u16,
}

impl Permissions {
/// Create a new Permissions object
///
/// Clamp to a u9 size, return Errno::E2BIG if it fails
///
pub fn new(octal: u16) -> Result<Self> {
if octal >= 2_u16.pow(9) {
return Err(Errno::E2BIG);
}
Ok(Permissions { permission: octal })
}

/// Getter for permission
///
pub fn get_permission(&self) -> &u16 {
&self.permission
}

/// Using the current stored permission, do a bitor operation on the
/// bitflags enums given
///
pub fn to_octal<T: bitflags::Flags<Bits = i32>>(
&self,
vec_flags: Vec<T>,
) -> c_int {
let mut flags: c_int = T::empty().bits();
for flag in vec_flags {
flags |= flag.bits();
}
flags |= self.permission as i32;
flags
}
}

libc_bitflags!(
/// Valid flags for the third parameter of the function [`shmget`]
pub struct ShmgetFlag: c_int
{
/// A new shared memory segment is created if key has this value.
IPC_PRIVATE;
/// Create a new segment.
/// If this flag is not used, then shmget() will find the segment
/// associated with key and check to see if the user has permission
/// to access the segment.
IPC_CREAT;
/// This flag is used with IPC_CREAT to ensure that this call creates
/// the segment. If the segment already exists, the call fails.
IPC_EXCL;
/// Allocate the segment using "huge" pages. See the Linux kernel
/// source file Documentation/admin-guide/mm/hugetlbpage.rst for
/// further information.
#[cfg(linux)]
SHM_HUGETLB;
// TODO: Does not exist in libc/linux, but should? Maybe open an issue in their repo
// SHM_HUGE_2MB;
// TODO: Same for this one
// SHM_HUGE_1GB;
/// This flag serves the same purpose as the mmap(2) MAP_NORESERVE flag.
/// Do not reserve swap space for this segment. When swap space is
/// reserved, one has the guarantee that it is possible to modify the
/// segment. When swap space is not reserved one might get SIGSEGV upon
/// a write if no physical memory is available. See also the discussion
/// of the file /proc/sys/vm/overcommit_memory in proc(5).
#[cfg(linux)]
SHM_NORESERVE;
}
);
/// Creates and returns a new, or returns an existing, System V shared memory
/// segment identifier.
///
/// For more information, see [`shmget(2)`].
///
/// [`shmget(2)`]: https://man7.org/linux/man-pages/man2/shmget.2.html
pub fn shmget(
key: key_t,
size: size_t,
shmflg: Vec<ShmgetFlag>,
IzawGithub marked this conversation as resolved.
Show resolved Hide resolved
permission: Permissions,
) -> Result<i32> {
let flags = permission.to_octal(shmflg);
Errno::result(unsafe { libc::shmget(key, size, flags) })
}

libc_bitflags!(
/// Valid flags for the third parameter of the function [`semget`]
pub struct SemgetFlag: c_int
{
/// A new shared memory segment is created if key has this value
IPC_PRIVATE;
/// Create a new segment.
/// If this flag is not used, then shmget() will find the segment
/// associated with key and check to see if the user has permission
/// to access the segment.
IPC_CREAT;
/// This flag is used with IPC_CREAT to ensure that this call creates
/// the segment. If the segment already exists, the call fails.
IPC_EXCL;
}
);
/// Creates and return a new, or returns an existing, System V shared memory
/// semaphore identifier.
///
/// For more information, see [`semget(2)`].
///
/// [`semget(2)`]: https://man7.org/linux/man-pages/man2/semget.2.html
pub fn semget(
key: key_t,
size: i32,
semflg: Vec<SemgetFlag>,
permission: Permissions,
) -> Result<i32> {
let flags = permission.to_octal(semflg);
Errno::result(unsafe { libc::semget(key, size, flags) })
}

libc_bitflags! {
/// Valid flags for the third parameter of the function [`shmat`]
pub struct ShmatFlag: c_int
{
/// Allow the contents of the segment to be executed. The caller must
/// have execute permission on the segment.
#[cfg(linux)]
SHM_EXEC;
/// This flag specifies that the mapping of the segment should replace
/// any existing mapping in the range starting at shmaddr and
/// continuing for the size of the segment.
/// (Normally, an EINVAL error would result if a mapping already exists
/// in this address range.)
/// In this case, shmaddr must not be NULL.
#[cfg(linux)]
SHM_REMAP;
/// Attach the segment for read-only access. The process must have read
/// permission for the segment. If this flag is not specified, the
/// segment is attached for read and write access, and the process must
/// have read and write permission for the segment.
/// There is no notion of a write-only shared memory segment.
SHM_RDONLY;
/// TODO: I have no clue at what this does
SHM_RND;
}
}
/// Attaches the System V shared memory segment identified by `shmid` to the
/// address space of the calling process.
///
/// For more information, see [`shmat(2)`].
///
/// # Safety
///
/// `shmid` should be a valid shared memory identifier and
/// `shmaddr` must meet the requirements described in the [`shmat(2)`] man page.
///
/// [`shmat(2)`]: https://man7.org/linux/man-pages/man2/shmat.2.html
pub fn shmat(
shmid: c_int,
shmaddr: Option<c_void>,
shmflg: Vec<ShmatFlag>,
permission: Permissions,
) -> Result<*mut c_void> {
let shmaddr_ptr: *const c_void = match shmaddr {
Some(_) => &mut shmaddr.unwrap(),
None => null(),
};
let flags = permission.to_octal(shmflg);
Errno::result(unsafe { libc::shmat(shmid, shmaddr_ptr, flags) })
}

/// Performs the reverse of [`shmat`], detaching the shared memory segment at
/// the given address from the address space of the calling process.
///
/// For more information, see [`shmdt(2)`].
///
/// # Safety
///
/// `shmaddr` must meet the requirements described in the [`shmdt(2)`] man page.
///
/// [`shmdt(2)`]: https://man7.org/linux/man-pages/man2/shmdt.2.html
pub fn shmdt(shmaddr: c_void) -> Result<()> {
let shmaddr_ptr: *const c_void = &shmaddr;
Errno::result(unsafe { libc::shmdt(shmaddr_ptr) }).map(drop)
}

libc_bitflags!(
/// Valid flags for the second parameter of the function [`shmctl`]
pub struct ShmctlFlag: c_int {
/// Returns the index of the highest used entry in the kernel's internal
/// array recording information about all shared memory segment
#[cfg(linux)]
IPC_INFO;
/// Write the values of some members of the shmid_ds structure pointed
/// to by buf to the kernel data structure associated with this shared
/// memory segment, updating also its shm_ctime member.
///
/// The following fields are updated: shm_perm.uid,
/// shm_perm.gid, and (the least significant 9 bits of)
/// shm_perm.mode.
///
/// The effective UID of the calling process must match the owner
/// (shm_perm.uid) or creator (shm_perm.cuid) of the shared memory
/// segment, or the caller must be privileged.
IPC_SET;
/// Copy information from the kernel data structure associated with
/// shmid into the shmid_ds structure pointed to by buf.
/// The caller must have read permission on the shared memory segment.
IPC_STAT;
/// Mark the segment to be destroyed. The segment will actually be
/// destroyed only after the last process detaches it
/// (i.e., when the shm_nattch member of the associated structure
/// shmid_ds is zero).
/// The caller must be the owner or creator of the segment,
/// or be privileged. The buf argument is ignored.
///
/// If a segment has been marked for destruction, then the
/// (nonstandard) SHM_DEST flag of the shm_perm.mode field in the
/// associated data structure retrieved by IPC_STAT will be set.
///
/// The caller must ensure that a segment is eventually destroyed;
/// otherwise its pages that were faulted in will remain in memory
/// or swap.
///
/// See also the description of /proc/sys/kernel/shm_rmid_forced
/// in proc(5).
IPC_RMID;
// not available in libc/linux, but should be?
// SHM_INFO;
// SHM_STAT;
// SHM_STAT_ANY;
/// Prevent swapping of the shared memory segment. The caller must
/// fault in any pages that are required to be present after locking is
/// enabled.
/// If a segment has been locked, then the (nonstandard) SHM_LOCKED
/// flag of the shm_perm.mode field in the associated data structure
/// retrieved by IPC_STAT will be set.
#[cfg(linux)]
SHM_LOCK;
/// Unlock the segment, allowing it to be swapped out.
#[cfg(linux)]
SHM_UNLOCK;
}
);
/// Performs control operation specified by `cmd` on the System V shared
/// memory segment given by `shmid`.
///
/// For more information, see [`shmctl(2)`].
///
/// # Safety
///
/// All arguments should be valid and meet the requirements described in the [`shmctl(2)`] man page.
///
/// [`shmctl(2)`]: https://man7.org/linux/man-pages/man2/shmctl.2.html
pub fn shmctl(
shmid: c_int,
cmd: ShmctlFlag,
buf: Option<shmid_ds>,
permission: Permissions,
) -> Result<c_int> {
let buf_ptr: *mut shmid_ds = match buf {
Some(_) => &mut buf.unwrap(),
None => null_mut(),
};
let command = permission.to_octal(vec![cmd]);
Errno::result(unsafe { libc::shmctl(shmid, command, buf_ptr) })
}

libc_bitflags! {
/// Valid flags for the fourth parameter of the function [`semop`]
pub struct SemopFlag: c_int {
/// Fail the operation instead of blocking if it can't be done.
/// If it fails, return [`Errno::EAGAIN`]
IPC_NOWAIT;
// TODO: Not available in libc
// SEM_UNDO;
}
}
// pub fn semop(semid: c_int, semopflg: Vec<SemopFlag>, sops: *mut sembuf, nsops: size_t) -> Result<()> {
// Errno::result(unsafe { libc::semop(semid, sops, nsops) })
// }

#[derive(Debug)]
/// Called as the fourth parameter of the function [`semctl`]
///
pub enum Semun {
/// Value for SETVAL
val(c_int),
/// Buffer for IPC_STAT, IPC_SET
#[cfg(target_env = "gnu")]
buf(*mut semid_ds),
/// Array for GETALL, SETALL
array(*mut c_short),
/// Buffer for IPC_INFO
#[cfg(target_env = "gnu")]
__buf(*mut seminfo),
}
libc_bitflags! (
/// Valid flags for the third parameter of the function [`shmctl`]
pub struct SemctlCmd: c_int {
/// Copy information from the kernel data structure associated with
/// shmid into the shmid_ds structure pointed to by buf.
/// The caller must have read permission on the shared memory segment.
IPC_STAT;
/// Write the values of some members of the semid_ds structure pointed
/// to by arg.buf to the kernel data structure associated with this
/// semaphore set, updating also its sem_ctime member.
///
/// The following members of the structure are updated:
/// sem_perm.uid, sem_perm.gid, and (the least significant 9 bits of)
/// sem_perm.mode.
///
/// The effective UID of the calling process must match the owner
/// (sem_perm.uid) or creator (sem_perm.cuid) of the semaphore set,
/// or the caller must be privileged. The argument semnum is ignored.
IPC_SET;
/// Immediately remove the semaphore set, awakening all processes
/// blocked in semop(2) calls on the set
/// (with an error return and errno set to EIDRM).
/// The effective user ID of the calling process must match the creator
/// or owner of the semaphore set, or the caller must be privileged.
/// The argument semnum is ignored.
IPC_RMID;
/// Return information about system-wide semaphore limits and
/// parameters in the structure pointed to by arg.__buf. This structure
/// is of type [`seminfo`].
#[cfg(linux)]
IPC_INFO;
// TODO: None of the one following are defined in libc/linux
// SEM_INFO;
// SEM_STAT;
// SEM_STAT_ANY;
// GETALL;
// GETNCNT;
// GETPID;
// GETVAL;
// GETZCNT;
// SETALL;
// SETVAL;
}
);
/// Performs control operation specified by `cmd` on the System V shared
/// semaphore segment given by `semid`.
///
/// For more information, see [`semctl(2)`].
///
/// #
///
/// [`semctl(2)`]: https://man7.org/linux/man-pages/man2/semctl.2.html
pub fn semctl(
semid: c_int,
semnum: c_int,
cmd: SemctlCmd,
permission: Permissions,
semun: Option<Semun>,
) -> Result<c_int> {
let command = permission.to_octal(vec![cmd]);
if semun.is_none() {
return Errno::result(unsafe { libc::semctl(semid, semnum, command) });
}
Errno::result(unsafe { libc::semctl(semid, semnum, command, semun) })
}
Loading
Loading