X.random_evaluate() broken with the neldermead example notebook.

[ChristopherMayes]

In scipy/neldermead.ipynb:

X.random_evaluate()

AssertionError                            Traceback (most recent call last)
Cell In[5], line 1
----> 1 X.random_evaluate()

File [~/Code/GitHub/Xopt/xopt/base.py:440](http://localhost:8890/~/Code/GitHub/Xopt/xopt/base.py#line=439), in Xopt.random_evaluate(self, n_samples, seed, custom_bounds)
    414 """
    415 Convenience method to generate random inputs using VOCs and evaluate them.
    416 
   (...)
    435 
    436 """
    437 random_inputs = self.vocs.random_inputs(
    438     n_samples, seed=seed, custom_bounds=custom_bounds, include_constants=True
    439 )
--> 440 result = self.evaluate_data(random_inputs)
    441 return result

File [~/Code/GitHub/Xopt/xopt/base.py:338](http://localhost:8890/~/Code/GitHub/Xopt/xopt/base.py#line=337), in Xopt.evaluate_data(self, input_data)
    335 # explode any list like results if all the output names exist
    336 output_data = explode_all_columns(output_data)
--> 338 self.add_data(output_data)
    340 # dump data to file if specified
    341 if self.dump_file is not None:

File [~/Code/GitHub/Xopt/xopt/base.py:369](http://localhost:8890/~/Code/GitHub/Xopt/xopt/base.py#line=368), in Xopt.add_data(self, new_data)
    367         new_data.index = new_data.index.astype(np.int64)
    368     self.data = new_data
--> 369 self.generator.add_data(new_data)

File [~/Code/GitHub/Xopt/xopt/generators/scipy/neldermead.py:137](http://localhost:8890/~/Code/GitHub/Xopt/xopt/generators/scipy/neldermead.py#line=136), in NelderMeadGenerator.add_data(self, new_data)
    134     return
    135 else:
    136     # Must have made at least 1 step, require future_state
--> 137     assert self.future_state is not None
    139     # new data -> advance state machine 1 step
    140     assert ndata == self.future_state.ngen == ngen + 1

AssertionError:

[ChristopherMayes]

@nikitakuklev it looks like this assertion comes from you.

[ChristopherMayes]

MWE:

from xopt import Xopt

YAML = """
generator:
  name: neldermead
  initial_point: {x0: -1, x1: -1}
evaluator:
  function: xopt.resources.test_functions.rosenbrock.evaluate_rosenbrock
vocs:
  variables:
    x0: [-5, 5]
    x1: [-5, 5]
  objectives: {y: MINIMIZE}
"""
X = Xopt(YAML)
X.random_evaluate()

[nikitakuklev]

Thanks, I'll try take a look this week. My instinct is that is the correct behavior - it makes no sense to do a random evaluation on simplex, since it must maintain state from one step to next. You can't add/remove/modify its data once started. Maybe a better error message is needed.

[roussel-ryan]

This is also breaking the current workflow from badger ATM. It would be great if this generator could treat the last datapoint in X.data as the starting point and then run from there (locking the dataframe in the process?)

[nikitakuklev]

The problem is that it is not just 'last point' but 'last simplex' + 'stage' (contraction, etc.). The implementation before current one did exactly this - it took last ndim+1 points and restarted with that as initial simplex. Because of that it was not reproducible on reload, whereas current implementation is (see here). This is fundamental difference from BO methods, which makes simplex a pain to handle with BO-like interface.

Maybe some override flags for old behavior are in order. I'll prototype a bit.

[roussel-ryan]

Thanks for looking into this Nikita, I think using multiple points may be helpful but are then causing the reproducibility issue you mentioned. If we just started simplex from a single point that would be ok as well if it fixes the reproducibility issues

[nikitakuklev]

Update: didn't have time implement a full solution yet, ETA over the weekend.