change config file format to leverage toml tables

[NickleDave]

won't do until 0.5.0 but making a note for now

currently we do not leverage toml tables and instead basically use an INI like format with sections.
This imposes a certain flatness on the format and makes it difficult to e.g. combine with different libraries that also use a pyproject.toml file

we should change to use a poetry-ike format. I need to plan out the schema better but it will look something like this

[vak.prep]
data_dir = './tests/data_for_tests/source/audio_cbin_annot_notmat/gy6or6/032312'
output_dir = './tests/data_for_tests/generated/prep/learncurve/audio_cbin_annot_notmat'
audio_format = 'cbin'
annot_format = 'notmat'
spect_output_dir = "./tests/data_for_tests/generated/prep/learncurve/audio_cbin_annot_notmat"
labelset = 'iabcdefghjk'
train_dur = 50
val_dur = 15
test_dur = 30

[vak.prep.spect_params]
fft_size = 512
step_size = 64
freq_cutoffs = [ 500, 10000 ]
thresh = 6.25
transform_type = 'log_spect'

[vak.learncurve]
models = 'TeenyTweetyNet'
normalize_spectrograms = true
batch_size = 4
num_epochs = 2
val_step = 50
ckpt_step = 200
patience = 3
num_workers = 4
train_set_durs = [ 4, 6 ]
num_replicates = 2
device = "cuda"
root_results_dir = './tests/data_for_tests/generated/results/learncurve/audio_cbin_annot_notmat/teenytweetynet'

[vak.learncurve.dataloader]
window_size = 44

[vak.model.TeenyTweetyNet]
optimizer = {'lr' = 0.001}
network = {'hidden_size' = 16}

[NickleDave]

would be nice to have annot_format optionally be a table so one can pass in kwargs, e.g.

[vak.prep.annot_format.simple-csv]
columns_map = {onsets = 'start', offsets = 'stop', labels = }

[NickleDave]

Consider using a schema? As in #307 (referencing so I can close that issue)

Or using pydantic which is basically writing a schema as Python code anyway

edit 2023-02-05:

• thinking about this again as I need to update config/valid.toml
• thought I recently saw a project that was validating TOML with jsonschema but now I don't find it.

Some searching brings up:

• https://json-schema-everywhere.github.io/toml
• https://marketplace.visualstudio.com/items?itemName=tamasfe.even-better-toml#completion-and-validation-with-json-schema
• and this discussion that has gone on for some time: Proposal: TOML Schema toml-lang/toml#792

[NickleDave]

Thinking about this again.
One schema that might work is to mirror the structure of the package after refactor, where we have the high-level commands and then we group model families below that. So that family-specific config options are naturally grouped.

E.g.

[vak.prep.frame_classification]
labelset = "abcdefg"

[vak.train.frame_classification]
window_size = 1000
stride = 750

[vak.train.frame_classification.model]
name = "TweetyNet"

[vak.train.frame_classification.model.network]
hidden_size = 256

[NickleDave]

Another annoyance about the current config is having to put a dummy table with the model name, as in #734 (comment), that matches the key-value pair model = "name

This would also be obviated by the more hierarchical format above (see #345 (comment)) -- just commenting here to underscore why we should make this change

[NickleDave]

Fixed by #750