Redesign nb-tester interface

[frankharkins]

Attempt to simplify the way we control nb-tester.

This PR includes the patch information in the config file. Like before, each notebook is part of a group. The new aspect is each group can define how to patch each notebook in different situations.

This PR also adds tests and documentation for the config part of the script. If you exclude the new README and tests, this PR reduces the script size by ~100 LOC.

Config example

Take the following config file as an example.

[my-group]
test-strategy.ci = { patch="qiskit_ibm_runtime", backend="test-eagle" }
test-strategy.hardware = {}
notebooks = ["..."]

If we run nb-tester against this file with --test-strategy=ci, each notebook in the group will be patched with the test-eagle device. If we run it with --test-strategy=hardware, the notebooks will be run with no patch. Running with a test strategy not defined for that group (e.g. --test-strategy=new-strat) means that group will be skipped.

CLI example

You can also run the script without a config file with the --patch arg. In this case, all files listed after the --patch argument are part of the same "group". You must include a patch for this group using TOML syntax. For example:

test-docs-notebooks --patch '{ patch="qiskit-fake-provider", num_qubits=6 }' path/to/file.ipynb

---

Closes #2401

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[frankharkins]

Switched to the built-in tomllib. I believe we kept tomli for a while for backwards-compatibility with older Python versions, but we now specify Python >= 3.11 in the pyproject.toml, so I think it's time to remove it here.

[Eric-Arellano]

I like this interface! It's a good design to only allow specifying one test strategy at a time. If you want to run with multiple test strategies, then run once per test strategy. That keeps the interface and code more understandable.

Nit in the PR title that this is not a "refactor" but a "revamp" or "redesign".

Consider splitting out the changes I proposed with scripts/ci as a prefactor. Fwict, they can be separated out from everything else. We've messed up CI config before, so it's worth having close review on them.

[emilkovacev]

Great PR! Thank you for implementing the file patch interface, I think it will be really powerful. I included a couple of suggestions for how to utilize patches, and a small formatting error.

[arnaucasau]

Amazing work Frank! I left a small typo fix suggestion and a comment about the args that I think it's cool to know. I like a lot the new interface and the code looks good, so I think we can merge it once @emilkovacev's suggestions are implemented :)

[emilkovacev]

New patching interface works really well! I tested this against a few notebooks locally, and was able to very quickly and easily get them to run.