Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid pickling of cached values of cached_method decorated functions #6674

Closed
pavoljuhas opened this issue Jul 23, 2024 · 0 comments · Fixed by #6677 or #6701
Closed

Avoid pickling of cached values of cached_method decorated functions #6674

pavoljuhas opened this issue Jul 23, 2024 · 0 comments · Fixed by #6677 or #6701
Assignees
Labels
kind/bug-report Something doesn't seem to work. triage/accepted A consensus emerged that this bug report, feature request, or other action should be worked on

Comments

@pavoljuhas
Copy link
Collaborator

Description of the issue

For some gates, e.g., cirq.H, the hash(gate) value is stored with cached_method decorator which breaks dictionary lookup if such gate is passed to multiprocessing worker. This seems to be the root cause for dictionary lookup problem in #6667 (comment).

How to reproduce the issue

import multiprocessing
import cirq

def compare_hash(gate):
    print(f"{hash(gate)=}  {hash(cirq.H)=}")

if __name__ == "__main__":
    ctx = multiprocessing.get_context("spawn")
    p = ctx.Process(target=compare_hash, args=(cirq.H,))
    p.start()
    p.join()

Output

hash(gate)=4057262618035729319  hash(cirq.H)=-7715581409246145106

Proposed solution

Do not pickle cached values from cached_method calls. Arguments to multiprocessing workers are passed as pickles so the results of cached methods (and of hash(cirq.H)) would be computed on the worker side.

Similar to #3777 and #5738

Cirq version
You can get the cirq version by printing cirq.__version__. From the command line:

1.5.0.dev at 8d09a32

@pavoljuhas pavoljuhas added the kind/bug-report Something doesn't seem to work. label Jul 23, 2024
@pavoljuhas pavoljuhas self-assigned this Jul 23, 2024
@dstrain115 dstrain115 added the triage/accepted A consensus emerged that this bug report, feature request, or other action should be worked on label Jul 24, 2024
pavoljuhas added a commit to pavoljuhas/Cirq that referenced this issue Jul 25, 2024
@pavoljuhas pavoljuhas reopened this Aug 7, 2024
pavoljuhas added a commit to pavoljuhas/Cirq that referenced this issue Aug 8, 2024
ProjectorSum stores data in a mutable LinearDict which is not hashable.
LinearDict also does not pickle, because by default it has a
lambda-function attribute.

This excludes ProjectorSum from the `test_hash_from_pickle` test.

Resolves quantumlib#6674
pavoljuhas added a commit that referenced this issue Aug 8, 2024
ProjectorSum stores data in a mutable LinearDict which is not hashable.
LinearDict also does not pickle, because by default it has a
lambda-function attribute.

This excludes ProjectorSum from the `test_hash_from_pickle` test.

Resolves #6674
harry-phasecraft pushed a commit to PhaseCraft/Cirq that referenced this issue Oct 31, 2024
…b#6677)

Ensure cirq objects passed as pickles in multiprocessing calls work
consistently in dictionaries just as the in-process created objects.

Partially resolves quantumlib#6674
harry-phasecraft pushed a commit to PhaseCraft/Cirq that referenced this issue Oct 31, 2024
ProjectorSum stores data in a mutable LinearDict which is not hashable.
LinearDict also does not pickle, because by default it has a
lambda-function attribute.

This excludes ProjectorSum from the `test_hash_from_pickle` test.

Resolves quantumlib#6674
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug-report Something doesn't seem to work. triage/accepted A consensus emerged that this bug report, feature request, or other action should be worked on
Projects
None yet
2 participants