Callable_from_QUA

[TheoLaudatQM]

Call Python functions from the QUA program directly using the decorator @run_local

[github-actions]

Unit Test Results

394 tests   391 ✔️  23s ⏱️
    1 suites      3 💤
    1 files        0 ❌

Results for commit 870f578.

♻️ This comment has been updated with latest results.

[TheoLaudatQM]

Two things are currently not working well:

1. Efficient way of transferring vectors from QUA to Python (useful for fitting resonance in Python and update the readout frequency in QUA for instance)
2. Transfer data from Python to QUA: now I can do it using io_values or input_streams but it is a bit ugly (cf. test_from_python_to_qua()_inputstreams.py) and it would be much nicer if the assign(a, IO1) or input_streams functions could be wrapped and automatically called when the run_local function returns something. Ideally the user should be able to simply write:

new_a, new_f = update_from_python(qm=qm, value=I, n=n)
update_frequency(element, new_f)
play(operation * amp(new_a), element)

3. It seems that all the variables used in the run_local functions can't be stored and incremented as in:

@run_local
def update_offset(QM, channel: str, signal: float):
    correction = 0 # Initialize the value only once
    target = 0.05  # Target voltage in V
    signal = -signal * 2**12 / readout_len
    correction += target - signal  # Correction to apply  # Update the value at each callback
    print(f"Set DC offset of channel {channel} to {correction} V (signal = {signal})")
    # Can be QDAC or whatever
    QM.set_output_dc_offset_by_element(channel, "single", correction)
    sleep(0.5)

[TheoLaudatQM]

Points 1. and 2. will be added as a second version and I will work on implementing point 3. using a class.

[nulinspiratie]

Following our earlier conversation with @TheoLaudatQM @yonatanrqm @yomach I'll try to implement the changes related to inheriting from Program and QuantumMachine.

The goal is that we no longer need to run the following:

with prog.local_run(qm):
    job = qm.execute(prog)

but can instead directly run

job = qm.execute(prog)

I think the following items should be done

• Inherit from QuantumMachine, add QuantumMachine.callables which get executed when calling qm.execute()
• Inherit from Program (or ProgramScope?), add something like Program.plugins which the local run manager can attach itself to
• Rename @local_run back to @callable_from_qua
• Rename LocalRunManager to CallableEventManager
• Move callable_from_qua away from qualang_tools/addons. @yomach where should it belong? qualang_tools/callable_from_qua?

Open questions

• Should the CallableEventManager be opened in the main thread or in a separate thread while the program is executing on the machine?
  I think the best is if it by default opens in a separate thread, but the user can specify to run it in the main thread, in which case it's blocking.
• How to pass additional callables?
  E.g. we can currently run with prog.local_run(qm, funcs=[live_plot]): to also add live plotting

[nulinspiratie]

This PR is ready for review by @yomach and @yonatanrqm.

Summary

We've developed the @callable_from_qua decorator to be able to easily call external functions.
By decorating a python function by @callable_from_qua, it can be called during a program's execution.
Parameters can be passed to the python function.

Basic code example

patch_callable_from_qua()  # Necessary patches

@callable_from_qua
def qua_print(*args):
    print(", ".join(args))

with program() as prog:
    n1 = declare(int)
    n2 = declare(int)
    qua_print("n1", n1, "n2", n2)

qm.execute(prog)

When the program is executed and reaches qua_print, it is paused, the parameters are streamed to the PC.
An event manager on the PC then passes these parameters to qua_print, and then resumes execution.

Considerations

Event manager

While the QUA program is being executed, the local PC is running a QuaCallableEventManager that keeps listening for any calls. This is currently not run in a separate thread, so it is blocking.

Addons

The event manager attaches itself as a ProgramAddon to program.addons: Dict[str, ProgramAddon]. These addons can perform ancillary actions when

• A program context manager is entered
• A program context manager is exited
• A program is executed

In the current implementation, the event manager addon is automatically added to the program if a @callable_from_qua is called within the program

Currently this functionality is added through patching (see next point), but if this PR is accepted, it should be incorporated into QUA in one way or another.

Patching

Currently the function patch_callable_from_qua needs to be called before any callbacks work. The patch modifies:

1. QuantumMachine.execute also runs any addons after starting the main program
2. _ProgramScope calls ProgramAddon.enter_program(program) and ProgramAddon.exit_program(program) to upon entering/exiting the Program context manager
3. Program.addons: Dict[str, ProgramAddon] is added

These changes should ideally be incorporated into QUA directly, making this patch obsolete. Once this is the case, the patch will simply print a deprecation warning.

Private QUA variables

Several private variables from qm.qua._dsl are called:

• _ResultSource
• _Variable
• _get_root_program_scope: If a @callable_from_qua command is called from within the Program, it needs to be able to access the program to be able to attach the QuaCallableEventManager addon.
  We should consider making these variables public

Follow-up features

To be implemented in subsequent PR:

• Execute callable without blocking
• Allow @callable_from_qua functions to also be called outside of a QUA program
• Return values back to QUA
• Automated testing

[nulinspiratie]

I should mention that this addon feature might also be useful in other cases, e.g. QuAM, execution standardization

[yomach]

Looks very good, few comments on the example folder:

1. Let's remove IPs, calibration database, and other specific information from there.
2. There are 5 usecases written in the readme, but 7 "test_" files. Let's narrow it down so it would match
3. At the same time, let's remove the test_ from their name, and give them a name which suits the example their are showing. Some are ok, so are not clear...
4. Add a short explanation to each example
5. Remove the capital C from the name of the folder

[nulinspiratie]

@yomach I rewrote the patch of qm.execute. It's not using super() because I opted not to replace the class. I think this makes more sense because we retain the original QuantumMachine class, what do you think?

[yomach]

@yomach I rewrote the patch of qm.execute. It's not using super() because I opted not to replace the class. I think this makes more sense because we retain the original QuantumMachine class, what do you think?

So we're patching the function and not the class?
I have no opinion here - @yonatanrqm ?

In any case, the previous comments I wrote are still valid.

[nulinspiratie]

@yomach I rewrote the patch of qm.execute. It's not using super() because I opted not to replace the class. I think this makes more sense because we retain the original QuantumMachine class, what do you think?

So we're patching the function and not the class? I have no opinion here - @yonatanrqm ?

In any case, the previous comments I wrote are still valid.

Yeah I think it's safer to patch the function rather than the class. There's a slight chance of unintended consequences, e.g. type(qm) == QuantumMachine could fail (you should of course do isinstance, but still)...

[TheoLaudatQM]

Hey, I fixed the examples folder and the readme.
To me, 3 features are missing:

• The possibility to run custom python functions at each callback (live plotting)
• The possibility to transfer QUA array to Python to enable fitting in Python and then send back the fitting values in QUA. It shouldn't be too hard using the length of a QUA array.
• The possibility to transfer variables from Python to QUA with a simple syntax like var1, var2 = callable_function(a, b, c) in QUA

[yomach]

Didn't do a full review again, but just looking at what I wrote previously, this looks super good and almost everything is done!
A bit minor, but can we remove the capital C from examples/Callable_from_qua?

[TheoLaudatQM]

Thanks Yoav, it's done now (I actually had to delete the folder and put it back so that it changes the name...)

[yomach]

Thanks Yoav, it's done now (I actually had to delete the folder and put it back so that it changes the name...)

I think it's a windows thing... Anyway, thanks!

Good from my side, @yonatanrqm @nulinspiratie - any more comments, or should we merge?

[nulinspiratie]

Looks good to me, thanks for cleaning up @TheoLaudatQM! I think it's ready to be merged