Include field descriptors in autogenerated config file

[derollins]

Is your feature request related to a problem? Please describe.
When using the --create-config-file flag and also in the config file generated at the ends of runs there are none of the comments with the field descriptors that are present in the default config file. I think it would be helpful, especially for new users, to have these comments in the automatically generated file.

Describe the solution you'd like
Include the comments in default config file https://github.com/AFM-SPM/TopoStats/blob/fe3d5c1386bd96f8d9dfc27880b8efd75192b519/topostats/default_config.yaml in autogenerated config files.
Describe alternatives you've considered
Alternative could be clearer documentation.

Additional context
Add any other context or screenshots about the feature request here.

[ns-rse]

This may be facilitated using the ruamel.yaml package which supports literate YAML files. The package is already used for loading and writing YAML files by TopoStats.

[SylviaWhittle]

Would it not be acceptable to copy and paste default_config.yaml instead of parsing and dumping it when running --create-config-file? This could solve this issue and reduce complexity? I'm probably overlooking something here

[Jean-Du]

@SylviaWhittle I am guessing people might not always know where the TopoStats package is, especially if it's pip installed?

[SylviaWhittle]

@Jean-Du I mean we could make the --create-config-file flag run something like...

import shutil

src_file = './topostats/default_config.yaml'
dest_folder = '/path/to/destination/folder/'

shutil.copy(src_file, dst_folder)

So that the whole file is copied with the comments

[ns-rse]

That is certainly possible, although if any modifiers were specified on the command line they wouldn't be included in the output.

For example you can currently specify modifiers which are applied to the configuration after its loaded but before it is written out.

(topostats) ❱ run_topostats --create-config-file plain.yaml                        
[Mon, 22 May 2023 09:45:14] [INFO    ] [topostats] The YAML configuration file is valid.
[Mon, 22 May 2023 09:45:14] [INFO    ] [topostats] A sample configuration has been written to : ./plain.yaml
[Mon, 22 May 2023 09:45:14] [INFO    ] [topostats] Please refer to the documentation on how to use the configuration file : 

https://afm-spm.github.io/TopoStats/usage.html#configuring-topostats
https://afm-spm.github.io/TopoStats/configuration.html
 09:45:14 AM  0  neil  ~/work/projects/topostats  
(topostats) ❱ run_topostats --create-config-file modified.yaml --output_dir ~/tmp/test
[Mon, 22 May 2023 09:45:21] [INFO    ] [topostats] Updated config config[output_dir] : ./output > /home/neil/tmp/test 
[Mon, 22 May 2023 09:45:21] [INFO    ] [topostats] The YAML configuration file is valid.
[Mon, 22 May 2023 09:45:21] [INFO    ] [topostats] A sample configuration has been written to : ./modified.yaml
[Mon, 22 May 2023 09:45:21] [INFO    ] [topostats] Please refer to the documentation on how to use the configuration file : 

https://afm-spm.github.io/TopoStats/usage.html#configuring-topostats
https://afm-spm.github.io/TopoStats/configuration.html
 09:45:21 AM  0  neil  ~/work/projects/topostats  
(topostats) ❱ diff plain.yaml modified.yaml 
1c1
< # Sample configuration file auto-generated : 2023-05-22 09:45:14
---
> # Sample configuration file auto-generated : 2023-05-22 09:45:21
4c4
< output_dir: output
---
> output_dir: /home/neil/tmp/test

I think if this is required at all, and I'm of the opinion that its unnecessary as documentations is where people should look to find this level of information, otherwise why do we write it? then this should be addressed as part of #517.

[alicepyne]

I do agree that it’s useful to have it there in the config file if possible

On Mon, 22 May 2023 at 09:48, Neil Shephard ***@***.***> wrote: That is certainly possible, although if any modifiers were specified on the command line they wouldn't be included in the output. For example you can currently specify modifiers which are applied to the configuration after its loaded but before it is written out. (topostats) ❱ run_topostats --create-config-file plain.yaml [Mon, 22 May 2023 09:45:14] [INFO ] [topostats] The YAML configuration file is valid. [Mon, 22 May 2023 09:45:14] [INFO ] [topostats] A sample configuration has been written to : ./plain.yaml [Mon, 22 May 2023 09:45:14] [INFO ] [topostats] Please refer to the documentation on how to use the configuration file : https://afm-spm.github.io/TopoStats/usage.html#configuring-topostatshttps://afm-spm.github.io/TopoStats/configuration.html  09:45:14 AM  0  neil  ~/work/projects/topostats   (topostats) ❱ run_topostats --create-config-file modified.yaml --output_dir ~/tmp/test [Mon, 22 May 2023 09:45:21] [INFO ] [topostats] Updated config config[output_dir] : ./output > /home/neil/tmp/test [Mon, 22 May 2023 09:45:21] [INFO ] [topostats] The YAML configuration file is valid. [Mon, 22 May 2023 09:45:21] [INFO ] [topostats] A sample configuration has been written to : ./modified.yaml [Mon, 22 May 2023 09:45:21] [INFO ] [topostats] Please refer to the documentation on how to use the configuration file : https://afm-spm.github.io/TopoStats/usage.html#configuring-topostatshttps://afm-spm.github.io/TopoStats/configuration.html  09:45:21 AM  0  neil  ~/work/projects/topostats   (topostats) ❱ diff plain.yaml modified.yaml 1c1 < # Sample configuration file auto-generated : 2023-05-22 09:45:14 --- > # Sample configuration file auto-generated : 2023-05-22 09:45:21 4c4 < output_dir: output --- > output_dir: /home/neil/tmp/test I think if this is required at all, and I'm of the opinion that its unnecessary as documentations is where people should look to find this level of information, otherwise why do we write it? then this should be addressed as part of #517 <#517>. — Reply to this email directly, view it on GitHub <#536 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADJSFMOIAZXDODTDTSODUPTXHMR6ZANCNFSM6AAAAAAXFO6U7A> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

-- Dr Alice Pyne UKRI Future Leaders Fellow & Senior Lecturer Department of Materials Science and Engineering University of Sheffield Email: ***@***.*** Website: www.pyne-lab.uk

[SylviaWhittle]

I have tried to do this through packages ruamel and pyaml, however to no success. It seems to have been confirmed that both packages do not yet / officially support loading of comments.

I have asked around and the users are, as far as I can tell, in unanimous agreement that they would prefer being able to see the comments in generated configs, over being able to edit parameters in the configs through the command-line arguments when running --create-config-file.