Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split instance state into Instance and VMM tables #4194

Merged
merged 59 commits into from
Oct 12, 2023
Merged
Changes from 1 commit
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
4f56437
Define vmm table & update instance table
gjcolombo Sep 25, 2023
ed4f8e0
update DB model types for instances and VMMs
gjcolombo Sep 25, 2023
75a2f6a
redefine internal instance/vmm runtime state types
gjcolombo Sep 25, 2023
27eb921
Update sled-agent params types
gjcolombo Sep 26, 2023
f1ada46
sled agent: rework instance state management
gjcolombo Sep 26, 2023
cae4922
Rework InstanceInner -> InstanceStates contract & termination logic
gjcolombo Sep 27, 2023
74257ae
Rework simulated instances
gjcolombo Sep 27, 2023
a5c3488
Fix up simulated sled agent collection tests
gjcolombo Sep 27, 2023
4bcced2
Plumb parameters from sled agent entry points
gjcolombo Sep 27, 2023
39b7e64
sled agent: clean up test transcription errors
gjcolombo Sep 27, 2023
70cc2fb
update sled agent OpenAPI spec
gjcolombo Sep 27, 2023
854c77f
sled agent: reimplement generated-type From impls
gjcolombo Sep 27, 2023
46560b2
nexus: adapt some db queries to instance/vmm split
gjcolombo Sep 27, 2023
df011d5
Draft CTE for updating an instance/VMM in a single statement
gjcolombo Sep 27, 2023
896ec96
nexus: fix network interface queries
gjcolombo Sep 28, 2023
f5241c3
clean up test code that assumes instances always have sleds
gjcolombo Sep 28, 2023
94021a0
nexus: return instance/vmm info tuples; fix up serial console APIs
gjcolombo Sep 29, 2023
38b3939
Fix handling of instance/vmm state changes from sled agent
gjcolombo Sep 29, 2023
7ec7f84
Update Nexus::handle_instance_put_result
gjcolombo Sep 29, 2023
8dfdfdd
Update Nexus::instance_request_state
gjcolombo Sep 29, 2023
6dfeae1
Remove Nexus::instance_sled
gjcolombo Sep 29, 2023
3952f90
Rework instance create saga
gjcolombo Sep 29, 2023
bcc2f3b
Update instance start saga
gjcolombo Sep 29, 2023
0edf89c
Update instance migration saga
gjcolombo Sep 29, 2023
d5ab2f2
Update instance delete saga
gjcolombo Sep 29, 2023
cafa141
Update disk snapshot saga
gjcolombo Sep 29, 2023
781503e
Fix build errors
gjcolombo Sep 29, 2023
28683b8
cleanup: use dendrite deletion helper from start saga undo
gjcolombo Sep 30, 2023
bac9309
Actually check reservoir space when allocating to sleds
gjcolombo Sep 30, 2023
b3df607
bugfix: reorder instance columns in Diesel table schema
gjcolombo Sep 30, 2023
fd30291
bugfix: only query ID in 'find' prongs of instance/vmm update CTE
gjcolombo Sep 30, 2023
902c0a0
bugfix: use a valid state name in instance state subquery
gjcolombo Sep 30, 2023
7ffd8ef
Update Nexus OpenAPI spec
gjcolombo Sep 30, 2023
db4fa9c
Re-enable sled agent -> Nexus updates; fix tests
gjcolombo Oct 1, 2023
6534c38
Merge fix: use connections instead of pool in new code
gjcolombo Oct 2, 2023
521163f
remove dead code
gjcolombo Oct 2, 2023
6114aec
Update omdb to handle vmm optionality
gjcolombo Oct 2, 2023
fe709ad
clippy
gjcolombo Oct 2, 2023
5285226
omdb: fix expected schema version in tests
gjcolombo Oct 2, 2023
b8709e7
Don't abort instance monitor task on RunningState drop
gjcolombo Oct 3, 2023
f635ba1
Report Propolis Stopped state to Nexus as Stopping
gjcolombo Oct 3, 2023
71dc6e9
Remove instances from instance manager if terminated before starting
gjcolombo Oct 3, 2023
9a93a78
fix doc build
gjcolombo Oct 3, 2023
20c0d39
Merge branch 'main' into gjcolombo/your-vmm-table-is-ready
gjcolombo Oct 6, 2023
98bd378
Remove immutable data from VmmRuntimeState API struct
gjcolombo Oct 6, 2023
f20a6d8
Remove useless parameter from db::model::Instance constructor
gjcolombo Oct 6, 2023
f845e3a
clean up instance update-and-check CTE
gjcolombo Oct 6, 2023
32358bf
use Running as the state for instances w/active VMMs
gjcolombo Oct 6, 2023
3f41b17
clarify remarks re safety of rude termination
gjcolombo Oct 6, 2023
e688552
break schema upgrade into multiple statements
gjcolombo Oct 6, 2023
78f72e5
Standardize on "propolis_id" name in most places
gjcolombo Oct 6, 2023
d2a0636
make Nexus sole owner of Instance records' state values
gjcolombo Oct 7, 2023
b8cbde5
amend schema update README to suggest named constraints
gjcolombo Oct 7, 2023
8923fb4
improve comment on instance_and_vmm_update_runtime
gjcolombo Oct 7, 2023
e5b2a54
add issue number to TODO
gjcolombo Oct 7, 2023
2933c4b
clarify doc comment
gjcolombo Oct 9, 2023
6b5dfd3
Merge branch 'main' into gjcolombo/your-vmm-table-is-ready
gjcolombo Oct 11, 2023
ec65227
Merge branch 'main' into gjcolombo/your-vmm-table-is-ready
gjcolombo Oct 12, 2023
acbaf43
pick up Diesel error changes in new code
gjcolombo Oct 12, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix build errors
Also add a helper function to get the sled ID from an InstanceAndActiveVmm,
since the `.vmm().as_ref().expect().sled_id` pattern cropped up in a few places.
  • Loading branch information
gjcolombo committed Oct 2, 2023
commit 781503e9ed386134e33b5fbab19251e0e4106d1c
4 changes: 4 additions & 0 deletions nexus/db-queries/src/db/datastore/instance.rs
Original file line number Diff line number Diff line change
@@ -57,6 +57,10 @@ impl InstanceAndActiveVmm {
pub fn vmm(&self) -> &Option<Vmm> {
&self.vmm
}

pub fn sled_id(&self) -> Option<Uuid> {
self.vmm.as_ref().map(|v| v.sled_id)
}
}

impl From<InstanceAndActiveVmm> for omicron_common::api::external::Instance {
1 change: 1 addition & 0 deletions nexus/src/app/sagas/instance_migrate.rs
Original file line number Diff line number Diff line change
@@ -653,6 +653,7 @@ mod tests {
.vmm()
.as_ref()
.expect("instance should have a vmm before migrating");

let dst_sled_id =
select_first_alternate_sled(old_vmm, &other_sleds);

19 changes: 7 additions & 12 deletions nexus/src/app/sagas/snapshot_create.rs
Original file line number Diff line number Diff line change
@@ -1960,11 +1960,10 @@ mod test {
.await
.unwrap();

let vmm = instance_state
.vmm()
.as_ref()
let sled_id = instance_state
.sled_id()
.expect("starting instance should have a sled");
let sa = nexus.sled_client(&vmm.sled_id).await.unwrap();
let sa = nexus.sled_client(&sled_id).await.unwrap();

sa.instance_finish_transition(instance.identity.id).await;
let instance_state = nexus
@@ -2083,10 +2082,8 @@ mod test {
.await;

let sled_id = state
.vmm()
.as_ref()
.expect("running instance should have a vmm")
.sled_id;
.sled_id()
.expect("running instance should have a vmm");

Some((state.instance().id(), sled_id))
} else {
@@ -2369,10 +2366,8 @@ mod test {
.await;

let sled_id = instance_state
.vmm()
.as_ref()
.expect("running instance should have a sled")
.sled_id;
.sled_id()
.expect("running instance should have a vmm");

// Rerun the saga
let params = new_test_params(
3 changes: 1 addition & 2 deletions nexus/src/app/test_interfaces.rs
Original file line number Diff line number Diff line change
@@ -92,8 +92,7 @@ impl TestInterfaces for super::Nexus {
.datastore()
.instance_fetch_with_vmm(&opctx, &authz_instance)
.await?
.vmm()
.map(|vmm| vmm.sled_id))
.sled_id())
}

async fn set_disk_as_faulted(&self, disk_id: &Uuid) -> Result<bool, Error> {
29 changes: 23 additions & 6 deletions nexus/tests/integration_tests/instances.rs
Original file line number Diff line number Diff line change
@@ -574,12 +574,20 @@ async fn test_instance_start_creates_networking_state(
let opctx =
OpContext::for_tests(cptestctx.logctx.log.new(o!()), datastore.clone());

let (.., authz_instance, db_instance) = LookupPath::new(&opctx, &datastore)
let (.., authz_instance) = LookupPath::new(&opctx, &datastore)
.instance_id(instance.identity.id)
.fetch()
.lookup_for(nexus_db_queries::authz::Action::Read)
.await
.unwrap();

let instance_state = datastore
.instance_fetch_with_vmm(&opctx, &authz_instance)
.await
.unwrap();

let sled_id =
instance_state.sled_id().expect("running instance should have a sled");

let guest_nics = datastore
.derive_guest_network_interface_info(&opctx, &authz_instance)
.await
@@ -589,7 +597,7 @@ async fn test_instance_start_creates_networking_state(
for agent in &sled_agents {
// TODO(#3107) Remove this bifurcation when Nexus programs all mappings
// itself.
if agent.id != db_instance.runtime().sled_id {
if agent.id != sled_id {
assert_sled_v2p_mappings(
agent,
&nics[0],
@@ -3644,16 +3652,25 @@ async fn test_instance_v2p_mappings(cptestctx: &ControlPlaneTestContext) {
let opctx =
OpContext::for_tests(cptestctx.logctx.log.new(o!()), datastore.clone());

let (.., authz_instance, db_instance) = LookupPath::new(&opctx, &datastore)
let (.., authz_instance) = LookupPath::new(&opctx, &datastore)
.instance_id(instance.identity.id)
.fetch()
.lookup_for(nexus_db_queries::authz::Action::Read)
.await
.unwrap();

let instance_state = datastore
.instance_fetch_with_vmm(&opctx, &authz_instance)
.await
.unwrap();

let sled_id =
instance_state.sled_id().expect("running instance should have a sled");

let guest_nics = datastore
.derive_guest_network_interface_info(&opctx, &authz_instance)
.await
.unwrap();

assert_eq!(guest_nics.len(), 1);

let mut sled_agents: Vec<&Arc<SledAgent>> =
@@ -3663,7 +3680,7 @@ async fn test_instance_v2p_mappings(cptestctx: &ControlPlaneTestContext) {
for sled_agent in &sled_agents {
// TODO(#3107) Remove this bifurcation when Nexus programs all mappings
// itself.
if sled_agent.id != db_instance.runtime().sled_id {
if sled_agent.id != sled_id {
assert_sled_v2p_mappings(
sled_agent,
&nics[0],