Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 0.2.0 #44

Merged
merged 5 commits into from
Nov 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/workflows/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,9 @@ jobs:
- name: Install dependencies
run: dnf install -y cargo clang clippy kernel-devel libbpf-devel llvm-devel rustfmt
- name: Copy vmlinux.h
run: cp $(rpm -ql kernel-devel | grep '/vmlinux.h$' | tail -1) agent/src/bpf
run: |
cp $(rpm -ql kernel-devel | grep '/vmlinux.h$' | tail -1) agent/src/bpf
cp $(rpm -ql kernel-devel | grep '/vmlinux.h$' | tail -1) agent/tests/agenttest/src/bpf
- name: Build
run: cargo build --verbose
- name: Run tests
Expand Down
54 changes: 45 additions & 9 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

48 changes: 46 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,4 +1,48 @@
[workspace]

members = ["agent", "client", "crypto-auditing", "event-broker", "log-parser"]
members = [
"agent",
"agent/tests/agenttest",
"client",
"crypto-auditing",
"event-broker",
"log-parser"
]
resolver = "2"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I know it's not changed in here, but I'd still like to ask)

Features for target-specific dependencies are not enabled if the target is not currently being built.

is either a rather confusing feature description or a rather confusing feature. My testing seems to support the idea that the enabled feature set becomes the superset of the packages' features of the invocation. That suggests that 1. cargo build result would differ from make, 2. library featureset depends on whichever application target was built first, etc.

a. Should we strive for more isolation, e.g., by building the library first in a separate invocation?
b. Given that it still doesn't guarantee we've specified the features correctly, should we, maybe, give up and define just the all-encompassing superset at the top level and at least have the consistency?

Copy link
Contributor Author

@ueno ueno Nov 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if I get it right, but I can for example turn off io_uring in the agent while keeping libsystemd in the event-broker with make RELEASE=1 CARGO_ARGS="--no-default-features --features=libsystemd". Isn't that sufficient?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(clarified off-PR. We don't have builds that go "one app + library" and library inheriting a featureset of whatever app is built first, as I've mistakenly thought after seeing four app targets in Makefile. We'll either build the entire workspace at once with the union-of-all featureset or build subprojects one-by-one with their individual featuresets. As long as we test all of the approaches we use, this is fine.)


[workspace.package]
version = "0.2.0"
edition = "2021"
license = "GPL-3.0-or-later"
authors = ["The crypto-auditing developers"]

[workspace.dependencies]
anyhow = "1.0"
bindgen = "0.63"
bytes = "1.2"
clap = "4"
crypto-auditing = { version = "=0.2.0", path = "crypto-auditing" }
futures = "0.3"
hex = "0.4"
inotify = "0.10.2"
libbpf-rs = { version = "0.21", features = ["novendor"] }
libbpf-cargo = { version = "0.21", features = ["novendor"] }
libc = "0.2"
nix = "0.26"
openssl = "0.10"
page_size = "0.6"
probe = "0.5"
plain = "0.2"
serde = { version = "1.0", features = ["derive"] }
serde_cbor = "0.11"
serde_json = "1.0"
serde_with = "3"
tempfile = "3"
thiserror = "1.0"
time = "0.3"
tokio = "1.23"
tokio-serde = { version = "0.8", features = ["cbor"] }
tokio-stream = "0.1"
tokio-util = { version = "0.7", features = ["codec"] }
toml = "0.7"
tracing = "0.1"
tracing-subscriber = "0.3"
4 changes: 2 additions & 2 deletions GNUmakefile
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@ conffiles = \
.PHONY: all
all: $(programs)

agent/src/bpf/vmlinux.h:
agent/src/bpf/vmlinux.h agent/tests/agenttest/src/bpf/vmlinux.h:
bpftool btf dump file /sys/kernel/btf/vmlinux format c > $@-t && mv $@-t $@

$(programs): agent/src/bpf/vmlinux.h
$(programs): agent/src/bpf/vmlinux.h agent/tests/agenttest/src/bpf/vmlinux.h
cargo build --target-dir="${TARGETDIR}" ${CARGO_ARGS}

.PHONY: install-programs
Expand Down
50 changes: 25 additions & 25 deletions agent/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,39 +1,39 @@
[package]
name = "crypto-auditing-agent"
description = "Event collector agent for crypto-auditing project"
version = "0.1.0"
edition = "2021"
license = "GPL-3.0-or-later"
authors = ["The crypto-auditing developers"]
version.workspace = true
edition.workspace = true
license.workspace = true
authors.workspace = true

[features]
default = ["tokio-uring"]

[dependencies]
anyhow = "1.0"
bytes = "1.2"
clap = { version = "4", features = ["cargo", "derive"] }
crypto-auditing = { version = "0.1", path = "../crypto-auditing" }
futures = "0.3"
libbpf-rs = { version = "0.20", features = ["novendor"] }
libc = "0.2"
nix = "0.26"
openssl = "0.10"
page_size = "0.5"
serde = "1.0"
serde_cbor = "0.11"
time = { version = "0.3", features = ["formatting", "local-offset", "macros"] }
tokio = { version = "1.23", features = ["fs", "io-util", "signal"] }
anyhow.workspace = true
bytes.workspace = true
clap = { workspace = true, features = ["cargo", "derive"] }
crypto-auditing.workspace = true
futures.workspace = true
libbpf-rs.workspace = true
libc.workspace = true
nix.workspace = true
openssl.workspace = true
page_size.workspace = true
serde.workspace = true
serde_cbor.workspace = true
time = { workspace = true, features = ["formatting", "local-offset", "macros"] }
tokio = { workspace = true, features = ["fs", "io-util", "signal"] }
tokio-uring = { version = "0.4", optional = true }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any reason to single it out?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to the documentation, optional dependencies cannot be declared in workspace.dependencies.

toml = "0.6"
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
probe = "0.3"
toml.workspace = true
tracing.workspace = true
tracing-subscriber = { workspace = true, features = ["env-filter"] }
probe.workspace = true

[build-dependencies]
libbpf-cargo = { version = "0.20", features = ["novendor"] }
libbpf-cargo.workspace = true

[dev-dependencies]
tempfile = "3"
plain = "0.2"
tempfile.workspace = true
plain.workspace = true
agenttest = { path = "tests/agenttest" }
40 changes: 32 additions & 8 deletions agent/build.rs
Original file line number Diff line number Diff line change
@@ -1,17 +1,41 @@
// SPDX-License-Identifier: GPL-2.0

use libbpf_cargo::SkeletonBuilder;
use std::{env, path::PathBuf};

const SRC: &str = "src/bpf/audit.bpf.c";
use std::{
env,
fs::{self, File},
path::PathBuf,
process::Command,
};

fn main() {
let mut out =
let builddir =
PathBuf::from(env::var_os("OUT_DIR").expect("OUT_DIR must be set in build script"));
out.push("audit.skel.rs");
let srcdir = PathBuf::from(
env::var_os("CARGO_MANIFEST_DIR").expect("CARGO_MANIFEST_DIR must be set in build script"),
);

let vmlinux_h = srcdir.join("src").join("bpf").join("vmlinux.h");
if vmlinux_h.exists() {
fs::copy(&vmlinux_h, &builddir.join("vmlinux.h")).expect("unable to copy vmlinux.h");
} else {
let file = File::create(&builddir.join("vmlinux.h")).expect("unable to create vmlinux.h");
Command::new("bpftool")
.arg("btf")
.arg("dump")
.arg("file")
.arg("/sys/kernel/btf/vmlinux")
.arg("format")
.arg("c")
.stdout(file)
.status()
.expect("unable to run bpftool");
}
let src = srcdir.join("src").join("bpf").join("audit.bpf.c");
SkeletonBuilder::new()
.source(SRC)
.build_and_generate(&out)
.source(&src)
.clang_args(&format!("-I{}", builddir.display()))
.build_and_generate(&builddir.join("audit.skel.rs"))
.unwrap();
println!("cargo:rerun-if-changed={}", SRC);
println!("cargo:rerun-if-changed={}", src.display());
}
1 change: 1 addition & 0 deletions agent/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ use anyhow::{bail, Context as _, Result};
use bytes::BytesMut;
use core::future::Future;
use crypto_auditing::types::{ContextID, EventGroup};
use libbpf_rs::skel::{OpenSkel, SkelBuilder};
use openssl::{
rand::rand_bytes,
symm::{Cipher, Crypter, Mode},
Expand Down
13 changes: 7 additions & 6 deletions agent/src/ringbuf.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@
// Licensed under LGPL-2.1 or BSD-2-Clause.

use core::task::{Context, Poll};
use libbpf_rs::query::MapInfoIter;
use libbpf_rs::{query::MapInfoIter, Map};
use std::io::Result;
use std::num::NonZeroUsize;
use std::os::unix::io::RawFd;
use std::os::fd::{AsFd, AsRawFd, RawFd};
use tokio::io::unix::AsyncFd;
use tokio::io::{AsyncRead, ReadBuf};

Expand All @@ -23,21 +23,22 @@ pub struct RingBuffer {
}

impl RingBuffer {
pub fn new(map: &libbpf_rs::Map) -> Self {
pub fn new(map: &Map) -> Self {
let mut max_entries = 0;
for m in MapInfoIter::default() {
if m.name == map.name() {
max_entries = m.max_entries;
}
}
let psize = page_size::get();
let fd = map.as_fd().as_raw_fd();
let consumer = unsafe {
nix::sys::mman::mmap(
None,
NonZeroUsize::new(psize).expect("page size must not be zero"),
nix::sys::mman::ProtFlags::PROT_WRITE | nix::sys::mman::ProtFlags::PROT_READ,
nix::sys::mman::MapFlags::MAP_SHARED,
map.fd(),
fd,
0,
)
.unwrap()
Expand All @@ -49,15 +50,15 @@ impl RingBuffer {
.expect("page size + 2 * max_entries must not be zero"),
nix::sys::mman::ProtFlags::PROT_READ,
nix::sys::mman::MapFlags::MAP_SHARED,
map.fd(),
fd,
psize as i64,
)
.unwrap()
};

RingBuffer {
mask: (max_entries - 1) as u64,
async_fd: AsyncFd::with_interest(map.fd(), tokio::io::Interest::READABLE).unwrap(),
async_fd: AsyncFd::with_interest(fd, tokio::io::Interest::READABLE).unwrap(),
consumer,
producer,
data: unsafe { producer.add(psize) },
Expand Down
Loading