Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

_random_seed not retained when using ax_client.save_to_json_file() and AxClient.load_from_json_file() #2166

Open
sgbaird opened this issue Feb 1, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@sgbaird
Copy link
Contributor

sgbaird commented Feb 1, 2024

I'm not sure if this is intended or not, but here is a Colab reproducer: https://colab.research.google.com/drive/12_9U4ikUu9q9hIwz9w_8krPzuDOx9JWc?usp=sharing

For provenance, here is the reproducer code:

import numpy as np
from ax.service.ax_client import AxClient, ObjectiveProperties


obj1_name = "branin"


def branin(x1, x2):
    y = float(
        (x2 - 5.1 / (4 * np.pi**2) * x1**2 + 5.0 / np.pi * x1 - 6.0) ** 2
        + 10 * (1 - 1.0 / (8 * np.pi)) * np.cos(x1)
        + 10
    )

    return y


ax_client = AxClient(random_seed=42)
ax_client.create_experiment(
    parameters=[
        {"name": "x1", "type": "range", "bounds": [-5.0, 10.0]},
        {"name": "x2", "type": "range", "bounds": [0.0, 10.0]},
    ],
    objectives={
        obj1_name: ObjectiveProperties(minimize=True),
    },
)


for _ in range(10):
    parameters, trial_index = ax_client.get_next_trial()
    results = branin(
        parameters["x1"],
        parameters["x2"],
    )
    ax_client.complete_trial(trial_index=trial_index, raw_data=results)


best_parameters, metrics = ax_client.get_best_parameters()

ax_client.save_to_json_file()

ax_client_restored = AxClient.load_from_json_file()
print(ax_client_restored._random_seed == 42)
# False
@bernardbeckerman
Copy link
Contributor

Hi @sgbaird, thanks for the crystal clear issue and repro! Taking a look now.

@bernardbeckerman bernardbeckerman self-assigned this Feb 1, 2024
@bernardbeckerman bernardbeckerman added the bug Something isn't working label Feb 1, 2024
@bernardbeckerman
Copy link
Contributor

This seems like an oversight on our part, but unfortunately I won't be able to implement the fix before I leave (am going out of town for a few weeks). I'll follow up internally to find a new owner, and if that's not possible I'll need to get to it once I'm back. Of course, please feel free to submit a fix if you have the bandwidth, otherwise I'll be happy to get to it when I'm back. Thanks again for surfacing this!

@bernardbeckerman bernardbeckerman self-assigned this Jul 10, 2024
@bernardbeckerman bernardbeckerman added the wishlist Long-term wishlist feature requests label Jul 10, 2024
@bernardbeckerman bernardbeckerman removed their assignment Jul 10, 2024
@lena-kashtelyan lena-kashtelyan removed the wishlist Long-term wishlist feature requests label Jul 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants