Ability to test a validation plan without an existing device

[sungo]

The API has a way of testing a validation plan, given a device report blob. It requires the device to exist, however, because it needs to look up the device's hardware information and what not.

I need a way of testing a validation plan against a device that doesn't exist. For instance, we might be testing a new piece of hardware in the lab and need to see if it will pass our current validations. setting up test racks and all that fun is cumbersome and laborious.

I figure we'll at least need, for now, an existing SKU in the database and provide that identifier when we call this endpoint. While that's not as flexible as I'd really like, I think it's a reasonable entry barrier, particularly down the road if we fully apply the hardware specification json schemas we've been discussing.

[karenetheridge]

As discussed, I think the easiest way of achieving this is to add the test device to the database inside a transaction that is then rolled back after the validation(s) are run. There are already interfaces in (the newly-refactored) Conch::ValidationSystem that allow running validations without saving results to the database.

Unfortunately, I don't think I can make the transactional parts happen without more of the (still-underway) validation refactoring getting merged -- presently in master (v2.22), Conch::Pg is still being used in the last mile to fetch the subject device row, which will use a separate database connection and hence not see the device created inside the transaction. I will need to push Conch::Pg+Conch::Model::Device all the way out of the system first. (Good news though, that's not far off now.)

[sungo]

Yeah, I expected this to be a long-term request.

[karenetheridge]

Necessary db work is now in, so this is no longer blocked.

[karenetheridge]

notes, to self if that's not obvious

• this requires that all device report processing currently done in the controller should use its own $schema object that is passed around, not one derived from $c, in order that we can start a transaction on that connection
• we will assume that the hypothetical device being validated does not have a location (unless it really exists in the db already, in which case we'll use that)

So it really should just be a matter of:

• get db connection, start a transaction on it
• process all the content from the device report (update disks, nics etc etc etc)
• pass off the device to validations and run them - using that same db connection the whole time (although as we stated in Pull important fields out of the device report and into the database #551, validations shouldn't be going to the database for anything device-specific)
• get the validation results, serialize them for the response
• roll back all the db work we just did

I can test this by actually killing the main db connection entirely, which will result in an exception if any code anywhere at all tries to use a db connection other than the one we created a transaction on.

[karenetheridge]

Do you want to select the hardware_product_id and/or validation_plan that the report runs against, or just use the current hardcoded logic for determining these from report content? If the former, we might want an endpoint that looks like POST /device/validation_plan/:id/hardware_product/:id. (ew. suggestions welcome.)

[perigrin]

The latter. Just use the current hardcoded logic because that's what we'd have when we tried posting it anyway.