LR scheduler str versus float

[bw4sz]

We will need to find a way to reproduce this, but 11 epochs and 6 hours into a run, I got a strange error that I have not seen before.

COMET WARNING: To get all data logged automatically, import comet_ml before the following modules: torch.
Traceback (most recent call last):
  File "/home/b.weinstein/BOEM/USGS_backbone.py", line 61, in <module>
    m.trainer.fit(m)
  File "/blue/ewhite/b.weinstein/miniconda3/envs/BOEM/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 538, in fit
    call._call_and_handle_interrupt(
  File "/blue/ewhite/b.weinstein/miniconda3/envs/BOEM/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py", line 47, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
  File "/blue/ewhite/b.weinstein/miniconda3/envs/BOEM/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 574, in _fit_impl
    self._run(model, ckpt_path=ckpt_path)
  File "/blue/ewhite/b.weinstein/miniconda3/envs/BOEM/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 981, in _run
    results = self._run_stage()
  File "/blue/ewhite/b.weinstein/miniconda3/envs/BOEM/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1025, in _run_stage
    self.fit_loop.run()
  File "/blue/ewhite/b.weinstein/miniconda3/envs/BOEM/lib/python3.10/site-packages/pytorch_lightning/loops/fit_loop.py", line 206, in run
    self.on_advance_end()
  File "/blue/ewhite/b.weinstein/miniconda3/envs/BOEM/lib/python3.10/site-packages/pytorch_lightning/loops/fit_loop.py", line 386, in on_advance_end
    self.epoch_loop.update_lr_schedulers("epoch", update_plateau_schedulers=not self.restarting)
  File "/blue/ewhite/b.weinstein/miniconda3/envs/BOEM/lib/python3.10/site-packages/pytorch_lightning/loops/training_epoch_loop.py", line 349, in update_lr_schedulers
    self._update_learning_rates(interval=interval, update_plateau_schedulers=update_plateau_schedulers)
  File "/blue/ewhite/b.weinstein/miniconda3/envs/BOEM/lib/python3.10/site-packages/pytorch_lightning/loops/training_epoch_loop.py", line 400, in _update_learning_rates
    call._call_lightning_module_hook(
  File "/blue/ewhite/b.weinstein/miniconda3/envs/BOEM/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py", line 167, in _call_lightning_module_hook
    output = fn(*args, **kwargs)
  File "/blue/ewhite/b.weinstein/miniconda3/envs/BOEM/lib/python3.10/site-packages/pytorch_lightning/core/module.py", line 1273, in lr_scheduler_step
    scheduler.step(metric)
  File "/blue/ewhite/b.weinstein/miniconda3/envs/BOEM/lib/python3.10/site-packages/torch/optim/lr_scheduler.py", line 1364, in step
    self._reduce_lr(epoch)
  File "/blue/ewhite/b.weinstein/miniconda3/envs/BOEM/lib/python3.10/site-packages/torch/optim/lr_scheduler.py", line 1374, in _reduce_lr
    if old_lr - new_lr > self.eps:
TypeError: '>' not supported between instances of 'float' and 'str'

[bw4sz]

We should allow learning rate scheduler to be None, quick check of source doesn't allow this.

DeepForest/src/deepforest/main.py

Line 896 in 2bf8a9a

scheduler_config = self.config["train"]["scheduler"]

[bw4sz]

Initial thought is eps: 1e-08
is being read as scientific notation.