omdb could interpret status of blueprint tasks #6440

davepacheco · 2024-08-26T20:30:04Z

I opted not to do #6421 here.

davepacheco · 2024-08-26T20:47:32Z

Some example output for blueprint_loader:

$ cargo run --bin=omdb -- nexus background-tasks show blueprint_loader
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.00s
     Running `target/debug/omdb nexus background-tasks show blueprint_loader`
note: using Nexus URL http://[::1]:12221
task: "blueprint_loader"
  configured period: every 10s
  currently executing: no
  last completed activation: iter 5, triggered by a periodic timer firing
    started at 2024-08-26T20:40:11.771Z (1s ago) and ran for 26ms
    target blueprint: 7c2f73da-8d50-48b8-99a8-80335cfe5e19
    execution:        disabled
    created at:       2024-08-26T20:39:32.789Z
    status:           target blueprint unchanged

$ cargo run --bin=omdb -- -w nexus blueprints target set 95e34045-2469-4341-8a9d-162f49544500 inherit; cargo run --bin=omdb -- nexus background-tasks show blueprint_loader; sleep 1; cargo run --bin=omdb -- nexus background-tasks show blueprint_loader; sleep 1; cargo run --bin=omdb -- nexus background-tasks show blueprint_loader
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.00s
     Running `target/debug/omdb -w nexus blueprints target set 95e34045-2469-4341-8a9d-162f49544500 inherit`
note: using Nexus URL http://[::1]:12221
set target blueprint to 95e34045-2469-4341-8a9d-162f49544500
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.00s
     Running `target/debug/omdb nexus background-tasks show blueprint_loader`
note: using Nexus URL http://[::1]:12221
task: "blueprint_loader"
  configured period: every 10s
  currently executing: no
  last completed activation: iter 6, triggered by an explicit signal
    started at 2024-08-26T20:42:25.337Z (1s ago) and ran for 20ms
    target blueprint: 95e34045-2469-4341-8a9d-162f49544500
    execution:        disabled
    created at:       2024-08-26T20:41:45.774Z
    status:           target blueprint updated

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.00s
     Running `target/debug/omdb nexus background-tasks show blueprint_loader`
note: using Nexus URL http://[::1]:12221
task: "blueprint_loader"
  configured period: every 10s
  currently executing: no
  last completed activation: iter 6, triggered by an explicit signal
    started at 2024-08-26T20:42:25.337Z (3s ago) and ran for 20ms
    target blueprint: 95e34045-2469-4341-8a9d-162f49544500
    execution:        disabled
    created at:       2024-08-26T20:41:45.774Z
    status:           target blueprint updated

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.03s
     Running `target/debug/omdb nexus background-tasks show blueprint_loader`
note: using Nexus URL http://[::1]:12221
task: "blueprint_loader"
  configured period: every 10s
  currently executing: no
  last completed activation: iter 7, triggered by a periodic timer firing
    started at 2024-08-26T20:42:29.152Z (1s ago) and ran for 20ms
    target blueprint: 95e34045-2469-4341-8a9d-162f49544500
    execution:        disabled
    created at:       2024-08-26T20:41:45.774Z
    status:           target blueprint unchanged

$ cargo run --bin=omdb -- -w nexus blueprints target enable 95e34045-2469-4341-8a9d-162f49544500; cargo run --bin=omdb -- nexus background-tasks show blueprint_loader; sleep 1; cargo run --bin=omdb -- nexus background-tasks show blueprint_loader; sleep 1; cargo run --bin=omdb -- nexus background-tasks show 
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.98s
     Running `target/debug/omdb -w nexus blueprints target enable 95e34045-2469-4341-8a9d-162f49544500`
note: using Nexus URL http://[::1]:12221
set target blueprint 95e34045-2469-4341-8a9d-162f49544500 to enabled

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.05s
     Running `target/debug/omdb nexus background-tasks show blueprint_loader`
note: using Nexus URL http://[::1]:12221
task: "blueprint_loader"
  configured period: every 10s
  currently executing: no
  last completed activation: iter 11, triggered by an explicit signal
    started at 2024-08-26T20:43:03.664Z (1s ago) and ran for 22ms
    target blueprint: 95e34045-2469-4341-8a9d-162f49544500
    execution:        enabled
    created at:       2024-08-26T20:41:45.774Z
    status:           target blueprint enabled

Some example output for blueprint_executor:

$ cargo run --bin=omdb -- nexus background-tasks show blueprint_executor
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.96s
     Running `target/debug/omdb nexus background-tasks show blueprint_executor`
note: using Nexus URL http://[::1]:12221
task: "blueprint_executor"
  configured period: every 1m
  currently executing: no
  last completed activation: iter 7, triggered by a dependent task completing
    started at 2024-08-26T20:44:27.954Z (5s ago) and ran for 0ms
    target blueprint: 95e34045-2469-4341-8a9d-162f49544500
    execution:        disabled
    errors:           0

$ cargo run --bin=omdb -- nexus -w blueprints target enable 95e34045-2469-4341-8a9d-162f49544500
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.02s
     Running `target/debug/omdb nexus -w blueprints target enable 95e34045-2469-4341-8a9d-162f49544500`
note: using Nexus URL http://[::1]:12221
set target blueprint 95e34045-2469-4341-8a9d-162f49544500 to enabled

$ cargo run --bin=omdb -- nexus background-tasks show blueprint_executor
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.03s
     Running `target/debug/omdb nexus background-tasks show blueprint_executor`
note: using Nexus URL http://[::1]:12221
task: "blueprint_executor"
  configured period: every 1m
  currently executing: no
  last completed activation: iter 10, triggered by a dependent task completing
    started at 2024-08-26T20:45:39.481Z (4s ago) and ran for 486ms
    target blueprint: 95e34045-2469-4341-8a9d-162f49544500
    execution:        enabled
    errors:           0

$ cargo run --bin=omdb -- nexus background-tasks show blueprint_executor
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.99s
     Running `target/debug/omdb nexus background-tasks show blueprint_executor`
note: using Nexus URL http://[::1]:12221
task: "blueprint_executor"
  configured period: every 1m
  currently executing: no
  last completed activation: iter 3, triggered by a dependent task completing
    started at 2024-08-26T20:47:11.989Z (4s ago) and ran for 346ms
    target blueprint: 8810d8a5-75e0-4f41-ad1e-684c3f32cb63
    execution:        enabled
    errors:           1
        error 0: failed to insert dataset record for dataset e3881066-ebef-4764-9c13-e940e8f567f3: Object (of type ById(ab0f71a1-54ea-4778-9540-f0abe714e5c2)) not found: zpool

Before the initial blueprint has been loaded:

$ cargo run --bin=omdb -- nexus background-tasks show blueprint_executor
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.01s
     Running `target/debug/omdb nexus background-tasks show blueprint_executor`
note: using Nexus URL http://[::1]:12221
task: "blueprint_executor"
  configured period: every 1m
  currently executing: no
  last completed activation: iter 1, triggered by a periodic timer firing
    started at 2024-08-26T20:46:26.783Z (5s ago) and ran for 0ms
    last completion reported error: no blueprint

sunshowers

This generally looks good to me! This will need some updates as part of #6399, so I'll base that on top of this.

sunshowers · 2024-08-27T02:02:45Z

dev-tools/omdb/src/bin/omdb/nexus.rs

+            Err(error) => eprintln!(
+                "warning: failed to interpret task details: {:?}: {:?}",
+                error, details
+            ),


Wondering if there's a way we can incorporate a check that this deserializes correctly. I guess to do that we'd need a test which kicks off a blueprint execution and ensures that it deserializes correctly.

We could definitely do more to reify the interface between tasks' status messages and omdb. In an ideal world they'd show up in the OpenAPI spec and you wouldn't have to do this at all. When we added background tasks the obvious way to do that was with an enum with variants for each task, but that seemed unwieldy on a bunch of levels. But yeah, the more we add here, the more annoying this is.

Oh yeah I think the common layer should be serde_json::Value, not an enum. But the last and first things we do can be to work with this more specific type.

I had to do something similar for the update engine—in there, I implemented a scheme where each event report carries the name of the spec it is associated with. That worked reasonably well. (It's similar in spirit to storing a type ID in dynamic objects that you can downcast to.)

(If this is too hard feel free to defer this, I'm going to be making some changes in this area soon)

Now that you mention the idea of a separate name specifying the schema, I wonder if we could first-class that in the JsonSchema (i.e., "if this string property is X, then this other property is of type Y") so that on the other end Progenitor could deserialize to the specific type.

That's basically a tagged enum, right? Except we're modeling it as a type rather than an enum variant. (The difference is that a tagged enum is a closed universe and a "type ID" string is an open universe)

Yeah, and maybe the answer is to have the JsonSchema for this type expose it the same way it would a tagged enum?

I think I understand what you're saying, but I guess I'm not quite following what additional support Progenitor could provide over doing this by hand. (But let's discuss this separately.)

davepacheco added 3 commits August 26, 2024 13:23

omdb could interpret status of blueprint tasks

289efd0

Merge branch 'main' into dap/omdb-blueprint-status

a61d665

fix lint

8329a16

davepacheco requested a review from jgallagher August 26, 2024 20:47

davepacheco marked this pull request as ready for review August 26, 2024 20:48

davepacheco added 2 commits August 26, 2024 15:23

fix test

06f769d

Merge branch 'main' into dap/omdb-blueprint-status

d25d52b

sunshowers reviewed Aug 27, 2024

View reviewed changes

sunshowers approved these changes Aug 27, 2024

View reviewed changes

sunshowers mentioned this pull request Aug 27, 2024

[3/n] use update engine for reconfigurator execution #6399

Merged

davepacheco merged commit 758818a into main Aug 27, 2024
22 checks passed

davepacheco deleted the dap/omdb-blueprint-status branch August 27, 2024 15:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

omdb could interpret status of blueprint tasks #6440

omdb could interpret status of blueprint tasks #6440

davepacheco commented Aug 26, 2024

davepacheco commented Aug 26, 2024

sunshowers left a comment

sunshowers Aug 27, 2024

davepacheco Aug 27, 2024

sunshowers Aug 27, 2024 •

edited

Loading

sunshowers Aug 27, 2024

davepacheco Aug 27, 2024

sunshowers Aug 27, 2024 •

edited

Loading

davepacheco Aug 27, 2024

sunshowers Aug 27, 2024

omdb could interpret status of blueprint tasks #6440

omdb could interpret status of blueprint tasks #6440

Conversation

davepacheco commented Aug 26, 2024

davepacheco commented Aug 26, 2024

sunshowers left a comment

Choose a reason for hiding this comment

sunshowers Aug 27, 2024

Choose a reason for hiding this comment

davepacheco Aug 27, 2024

Choose a reason for hiding this comment

sunshowers Aug 27, 2024 • edited Loading

Choose a reason for hiding this comment

sunshowers Aug 27, 2024

Choose a reason for hiding this comment

davepacheco Aug 27, 2024

Choose a reason for hiding this comment

sunshowers Aug 27, 2024 • edited Loading

Choose a reason for hiding this comment

davepacheco Aug 27, 2024

Choose a reason for hiding this comment

sunshowers Aug 27, 2024

Choose a reason for hiding this comment

sunshowers Aug 27, 2024 •

edited

Loading

sunshowers Aug 27, 2024 •

edited

Loading