Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[inventory] Add Sled Agent datasets to inventory #6167

Merged
merged 70 commits into from
Sep 6, 2024
Merged
Show file tree
Hide file tree
Changes from 66 commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
5c2e5b0
[wip] Starting sled agent API to manage datasets explicitly
smklein Jul 18, 2024
1709d80
Merge branch 'main' into explicit-datasets
smklein Jul 22, 2024
a80313d
list implementation
smklein Jul 22, 2024
7193d1b
Tests
smklein Jul 22, 2024
9464cc1
schemas n stuff
smklein Jul 22, 2024
fb23666
Merge branch 'main' into explicit-datasets
smklein Jul 23, 2024
817a397
The sled agent side of datasets in inventory
smklein Jul 24, 2024
54b4658
Plumb inventory into nexus, omdb
smklein Jul 25, 2024
29c4eb7
Merge branch 'main' into explicit-datasets
smklein Jul 25, 2024
ff27801
Merge branch 'explicit-datasets' into inventory-datasets
smklein Jul 25, 2024
75f006b
Merge branch 'main' into explicit-datasets
smklein Jul 26, 2024
559db4f
Merge branch 'explicit-datasets' into inventory-datasets
smklein Jul 26, 2024
24d93a8
Fix mismerge
smklein Jul 26, 2024
e2dd6d7
Merge branch 'explicit-datasets' into inventory-datasets
smklein Jul 26, 2024
178e20e
Clippy and helios
smklein Jul 26, 2024
530bf08
Merge branch 'explicit-datasets' into inventory-datasets
smklein Jul 26, 2024
cf3f35c
openapi
smklein Jul 26, 2024
021aa2f
Merge branch 'explicit-datasets' into inventory-datasets
smklein Jul 26, 2024
67bf6fa
schema upgrade
smklein Jul 26, 2024
11f49fb
More broad support for datasets
smklein Jul 30, 2024
b999bf7
make it a schema change
smklein Jul 30, 2024
fac6456
Merge branch 'main' into explicit-datasets
smklein Jul 30, 2024
9298d83
Merge branch 'explicit-datasets' into inventory-datasets
smklein Jul 30, 2024
1baf7dd
welp I guess I changed some db queries
smklein Jul 30, 2024
fdf0644
queries
smklein Jul 30, 2024
a042439
Merge branch 'main' into explicit-datasets
smklein Jul 31, 2024
d70f9bc
Merge branch 'explicit-datasets' into inventory-datasets
smklein Jul 31, 2024
9c27213
Merge branch 'main' into explicit-datasets
smklein Aug 7, 2024
c562125
Optional properties still get set
smklein Aug 7, 2024
c7a5adf
clippy
smklein Aug 7, 2024
181507c
fmt
smklein Aug 7, 2024
8e1df07
str not string
smklein Aug 7, 2024
93134c2
cockroachdb, not cockroach_db
smklein Aug 7, 2024
d88d541
generation numbers
smklein Aug 7, 2024
95ebbb8
review feedback
smklein Aug 7, 2024
994757e
Serialize via string, not tag
smklein Aug 7, 2024
7156ba0
safer API for Dataset::new
smklein Aug 8, 2024
5a91c8f
Merge branch 'explicit-datasets' into inventory-datasets
smklein Aug 8, 2024
14a2dc4
merge main into branch, manually impl JsonSchema
sunshowers Aug 8, 2024
aa60254
update schema JSONs, add replace
sunshowers Aug 8, 2024
67c5d7e
Merge branch 'main' into explicit-datasets
smklein Aug 10, 2024
927efd1
Merge remote-tracking branch 'origin/explicit-datasets' into explicit…
smklein Aug 10, 2024
3e14cc4
Fix schema
smklein Aug 10, 2024
4d41c5c
Merge branch 'explicit-datasets' into inventory-datasets
smklein Aug 10, 2024
40f93ae
Merge branch 'main' into explicit-datasets
smklein Aug 22, 2024
0f8150f
Merge branch 'explicit-datasets' into inventory-datasets
smklein Aug 22, 2024
077908d
Fix clickhouseserver datasetkind serialization
smklein Aug 26, 2024
5d8f513
Merge branch 'explicit-datasets' into inventory-datasets
smklein Aug 26, 2024
a600494
Merge branch 'main' into explicit-datasets
smklein Aug 26, 2024
6f9cd0d
Merge branch 'explicit-datasets' into inventory-datasets
smklein Aug 26, 2024
a83450e
Add docs, use ByteCount more, better simulated inventory
smklein Aug 26, 2024
e8504a5
schema and openapi
smklein Aug 27, 2024
28e6497
Fix helios tests
smklein Aug 27, 2024
b9d3dcc
fmt
smklein Aug 27, 2024
0a63161
Merge branch 'main' into explicit-datasets
smklein Aug 28, 2024
c9f170e
Make CompressionAlgorithm strongly typed
smklein Aug 28, 2024
f5bc35a
Fixing helios-only tests, clippy
smklein Aug 28, 2024
474ec13
schemas
smklein Aug 28, 2024
8c52f4b
Merge branch 'explicit-datasets' into inventory-datasets
smklein Aug 28, 2024
c4d5b3a
Merge branch 'main' into explicit-datasets
smklein Aug 28, 2024
042f786
Merge branch 'explicit-datasets' into inventory-datasets
smklein Aug 28, 2024
5c5a856
Merge branch 'main' into explicit-datasets
smklein Aug 28, 2024
7b4083f
Merge branch 'explicit-datasets' into inventory-datasets
smklein Aug 28, 2024
77b3721
Pass byte values as single integers (no 'GiB' suffixes)
smklein Aug 28, 2024
70c8ede
Merge branch 'main' into explicit-datasets
smklein Sep 4, 2024
7693ef0
Merge branch 'explicit-datasets' into inventory-datasets
smklein Sep 4, 2024
830b67b
Merge branch 'main' into explicit-datasets
smklein Sep 6, 2024
bc29752
Merge branch 'explicit-datasets' into inventory-datasets
smklein Sep 6, 2024
67c2181
review feedback
smklein Sep 6, 2024
557dc22
Fix error wrapping in tests
smklein Sep 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions clients/sled-agent-client/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ progenitor::generate_api!(
SledRole = nexus_sled_agent_shared::inventory::SledRole,
SourceNatConfig = omicron_common::api::internal::shared::SourceNatConfig,
SwitchLocation = omicron_common::api::external::SwitchLocation,
TypedUuidForDatasetKind = omicron_uuid_kinds::DatasetUuid,
TypedUuidForInstanceKind = omicron_uuid_kinds::InstanceUuid,
TypedUuidForPropolisKind = omicron_uuid_kinds::PropolisUuid,
TypedUuidForZpoolKind = omicron_uuid_kinds::ZpoolUuid,
Expand Down
13 changes: 12 additions & 1 deletion common/src/api/external/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -593,7 +593,18 @@ impl JsonSchema for RoleName {
//
// TODO: custom JsonSchema impl to describe i64::MAX limit; this is blocked by
// https://github.com/oxidecomputer/typify/issues/589
#[derive(Copy, Clone, Debug, Serialize, JsonSchema, PartialEq, Eq)]
#[derive(
Copy,
Clone,
Debug,
Serialize,
JsonSchema,
Hash,
PartialEq,
Eq,
PartialOrd,
Ord,
)]
pub struct ByteCount(u64);

impl<'de> Deserialize<'de> for ByteCount {
Expand Down
6 changes: 3 additions & 3 deletions common/src/disk.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ use std::fmt;
use uuid::Uuid;

use crate::{
api::external::Generation,
api::external::{ByteCount, Generation},
ledger::Ledgerable,
zpool_name::{ZpoolKind, ZpoolName},
};
Expand Down Expand Up @@ -244,10 +244,10 @@ pub struct DatasetConfig {
pub compression: CompressionAlgorithm,

/// The upper bound on the amount of storage used by this dataset
pub quota: Option<usize>,
pub quota: Option<ByteCount>,

/// The lower bound on the amount of storage usable by this dataset
pub reservation: Option<usize>,
pub reservation: Option<ByteCount>,
}

#[derive(
Expand Down
38 changes: 38 additions & 0 deletions dev-tools/omdb/src/bin/omdb/db.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4744,6 +4744,44 @@ fn inv_collection_print_sleds(collection: &Collection) {
sled.reservoir_size.to_whole_gibibytes()
);

if !sled.zpools.is_empty() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any chance you could paste some example output?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, here's the (mildly redacted) output of omdb db inventory collections ...

sled e68c45ba-1e10-4ae3-9bfb-a770ed7dc643 (role = Gimlet, serial unknown)                                                                                                                                                                                                                                                                                                                                                                                                                     
    found at:    2024-09-06 02:39:19.150884 UTC from http://[fd00:1122:3344:102::1]:12345                                                                                                                                                                                                                                                                                                                                                                                                     
    address:     [fd00:1122:3344:102::1]:12345                                                                                                                                                                                                                                                                                                                                                                                                                                                
    usable hw threads:   8                                                                                                                                                                                                                                                                                                                                                                                                                                                                    
    usable memory (GiB): 7                                                                                                                                                                                                                                                                                                                                                                                                                                                                    
    reservoir (GiB):     0                                                                                                                                                                                                                                                                                                                                                                                                                                                                    
    physical disks:                                                                                                                                                                                                                                                                                                                                                                                                                                                                           
      M2: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-g1_0" } in 1024                                                                                                                                                                                                                                                                                                                                                                   
      M2: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-g1_1" } in 1025                                                                                                                                                                                                                                                                                                                                                                   
      U2: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-g1_0" } in 1026                                                                                                                                                                                                                                                                                                                                                                   
      U2: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-g1_1" } in 1027                                                                                                                                                                                                                                                                                                                                                                   
      U2: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-g1_2" } in 1028                                                                                                                                                                                                                                                                                                                                                                   
      U2: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-g1_3" } in 1029                                                                                                                                                                                                                                                                                                                                                                   
      U2: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-g1_4" } in 1030                                                                                                                                                                                                                                                                                                                                                                   
    zpools                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    
      96bb0a39-cca1-46ff-b33c-31425fce84f0: total size: 19968 MiB                                                                                                                                                                                                                                                                                                                                                                                                                             
      aee67881-c5ef-469d-baf1-9b57af906c11: total size: 19968 MiB                                                                                                                                                                                                                                                                                                                                                                                                                             
      b74c535a-5044-4d53-9b0a-37b966baf243: total size: 19968 MiB                                                                                                                                                                                                                                                                                                                                                                                                                             
      e1351784-80a3-40e9-b3c3-a200fa3010e9: total size: 19968 MiB                                                                                                                                                                                                                                                                                                                                                                                                                             
      fdc99717-dfb7-4b60-99bb-a1f4db11ba93: total size: 19968 MiB                                                                                                                                                                                                                                                                                                                                                                                                                             
    datasets:                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 
      oxp_96bb0a39-cca1-46ff-b33c-31425fce84f0 - id: None, compression: off                                                                                                                                                                                                                                                                                                                                                                                                                   
        available: 19162120 KiB, used: 646136 KiB                                                                                                                                                                                                                                                                                                                                                                                                                                             
        reservation: None, quota: None                                                                                                                                                                                                                                                                                                                                                                                                                                                        
      oxp_96bb0a39-cca1-46ff-b33c-31425fce84f0/crucible - id: Some(7c23501e-a7bf-4614-93c7-2fdbe55bd706 (dataset)), compression: off                                                                                                                                                                                                                                                                                                                                                          
        available: 19162120 KiB, used: 192 KiB                                                                                                                                                                                                                                                                                                                                                                                                                                                
        reservation: None, quota: None                                                                                 
      oxp_96bb0a39-cca1-46ff-b33c-31425fce84f0/crypt - id: None, compression: off
        available: 19162120 KiB, used: 644996 KiB                                                                      
        reservation: None, quota: None                                                                                 
      oxp_96bb0a39-cca1-46ff-b33c-31425fce84f0/crypt/debug - id: None, compression: gzip-9
        available: 19162120 KiB, used: 200 KiB                                                                         
        reservation: None, quota: Some(ByteCount(107374182400))
      oxp_96bb0a39-cca1-46ff-b33c-31425fce84f0/crypt/zone - id: None, compression: off
        available: 19162120 KiB, used: 644580 KiB                                                                      
        reservation: None, quota: None                                                                                 
      oxp_96bb0a39-cca1-46ff-b33c-31425fce84f0/crypt/zone/oxz_crucible_7c23501e-a7bf-4614-93c7-2fdbe55bd706 - id: None, compression: off
        available: 19162120 KiB, used: 644372 KiB                                                                      
        reservation: None, quota: None                                                                         
...
      oxp_fdc99717-dfb7-4b60-99bb-a1f4db11ba93 - id: None, compression: off
        available: 19162072 KiB, used: 646184 KiB
        reservation: None, quota: None
      oxp_fdc99717-dfb7-4b60-99bb-a1f4db11ba93/crucible - id: Some(cc7926c3-35b7-4bc3-b68a-2ebe2073f33e (dataset)), compression: off
        available: 19162072 KiB, used: 192 KiB
        reservation: None, quota: None
      oxp_fdc99717-dfb7-4b60-99bb-a1f4db11ba93/crypt - id: None, compression: off
        available: 19162072 KiB, used: 645044 KiB
        reservation: None, quota: None
      oxp_fdc99717-dfb7-4b60-99bb-a1f4db11ba93/crypt/debug - id: None, compression: gzip-9
        available: 19162072 KiB, used: 200 KiB
        reservation: None, quota: Some(ByteCount(107374182400))
      oxp_fdc99717-dfb7-4b60-99bb-a1f4db11ba93/crypt/zone - id: None, compression: off
        available: 19162072 KiB, used: 644628 KiB
        reservation: None, quota: None
      oxp_fdc99717-dfb7-4b60-99bb-a1f4db11ba93/crypt/zone/oxz_crucible_cc7926c3-35b7-4bc3-b68a-2ebe2073f33e - id: None, compression: off
        available: 19162072 KiB, used: 644420 KiB
        reservation: None, quota: None
    zones collected from http://[fd00:1122:3344:102::1]:12345 at 2024-09-06 02:39:19.154778 UTC
    zones generation: 5 (count: 11)
    ZONES FOUND
      zone 078b2df5-4dd7-447f-8bba-61a7359bcad5 (type crucible_pantry)
      zone 554086cc-b795-4b57-81e2-8c3e740e82aa (type cockroach_db)
      zone 5c08a890-733b-4433-90e7-54364e7b6a39 (type boundary_ntp)
      zone 66a572bc-8122-4d88-a8f4-4e1e6bb99e57 (type oximeter)
      zone 7c23501e-a7bf-4614-93c7-2fdbe55bd706 (type crucible)
      zone 8131b4fb-a3f3-4da2-a7b4-0d3c37bf5004 (type internal_dns)
      zone b89bfa4d-2419-48f3-ab96-fc1863e9e43a (type crucible)
      zone befda934-f3bc-4e9d-900d-04c6a5c3e537 (type crucible)
      zone c698e9ab-96e4-4eb5-be4a-9f900faa1da5 (type external_dns)
      zone cc7926c3-35b7-4bc3-b68a-2ebe2073f33e (type crucible)
      zone e87e7caf-7b8e-4e46-afc3-a55a7912282d (type crucible)

println!(" physical disks:");
}
for disk in &sled.disks {
let nexus_types::inventory::PhysicalDisk {
identity,
variant,
slot,
} = disk;
println!(" {variant:?}: {identity:?} in {slot}");
}

if !sled.zpools.is_empty() {
println!(" zpools");
}
for zpool in &sled.zpools {
let nexus_types::inventory::Zpool { id, total_size, .. } = zpool;
println!(" {id}: total size: {total_size}");
}

if !sled.datasets.is_empty() {
println!(" datasets:");
}
for dataset in &sled.datasets {
let nexus_types::inventory::Dataset {
id,
name,
available,
used,
quota,
reservation,
compression,
} = dataset;
println!(" {name} - id: {id:?}, compression: {compression}");
smklein marked this conversation as resolved.
Show resolved Hide resolved
println!(" available: {available}, used: {used}");
println!(" reservation: {reservation:?}, quota: {quota:?}");
}

if let Some(zones) = collection.omicron_zones.get(&sled.sled_id) {
println!(
" zones collected from {} at {}",
Expand Down
37 changes: 37 additions & 0 deletions dev-tools/omdb/src/bin/omdb/sled_agent.rs
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@ enum SledAgentCommands {
#[clap(subcommand)]
Zpools(ZpoolCommands),

/// print information about datasets
#[clap(subcommand)]
Datasets(DatasetCommands),

/// print information about the local bootstore node
#[clap(subcommand)]
Bootstore(BootstoreCommands),
Expand All @@ -55,6 +59,16 @@ enum ZpoolCommands {
List,
}

#[derive(Debug, Subcommand)]
enum DatasetCommands {
/// Print list of all datasets the sled agent is configured to manage
///
/// Note that the set of actual datasets on the sled may be distinct,
/// use the `omdb db inventory collections show` command to see the latest
/// set of datasets collected from sleds.
List,
}

#[derive(Debug, Subcommand)]
enum BootstoreCommands {
/// Show the internal state of the local bootstore node
Expand Down Expand Up @@ -86,6 +100,9 @@ impl SledAgentArgs {
SledAgentCommands::Zpools(ZpoolCommands::List) => {
cmd_zpools_list(&client).await
}
SledAgentCommands::Datasets(DatasetCommands::List) => {
cmd_datasets_list(&client).await
}
SledAgentCommands::Bootstore(BootstoreCommands::Status) => {
cmd_bootstore_status(&client).await
}
Expand Down Expand Up @@ -130,6 +147,26 @@ async fn cmd_zpools_list(
Ok(())
}

/// Runs `omdb sled-agent datasets list`
async fn cmd_datasets_list(
client: &sled_agent_client::Client,
) -> Result<(), anyhow::Error> {
let response = client.datasets_get().await.context("listing datasets")?;
let response = response.into_inner();

println!("dataset configuration @ generation {}:", response.generation);
let datasets = response.datasets;

if datasets.is_empty() {
println!(" <none>");
}
for dataset in &datasets {
println!(" {:?}", dataset);
}

Ok(())
}

/// Runs `omdb sled-agent bootstore status`
async fn cmd_bootstore_status(
client: &sled_agent_client::Client,
Expand Down
1 change: 1 addition & 0 deletions dev-tools/omdb/tests/usage_errors.out
Original file line number Diff line number Diff line change
Expand Up @@ -642,6 +642,7 @@ Usage: omdb sled-agent [OPTIONS] <COMMAND>
Commands:
zones print information about zones
zpools print information about zpools
datasets print information about datasets
bootstore print information about the local bootstore node
help Print this message or the help of the given subcommand(s)

Expand Down
Loading
Loading