58: update readme

[aharjati]

update readme:

• Change focus to use Poetry
• Add more details on POETRY steps (installation, developments and tests)
• Add more details on VScode development
• Add contact/help information

[github-actions]

Coverage report

The coverage rate went from 93.6% to 93.97% ⬆️
The branch rate is 82%.

0% of new lines are covered.

Diff Coverage details (click to unfold)

src/validator/main.py

0% of new lines are covered (0% of the complete file).
Missing lines: 8, 25

[aharjati]

the README preview can be viewed on this branch: https://github.com/cfpb/regtech-data-validator/tree/features/update_README

[hkeeler]

@aharjati, thanks for putting all of this together. I've taken a first pass, and put up my initial thoughts. Time permitting, I will try to contribute a bit as well. Some of the re-org bits are difficult to describe via PR review, and may just be easier if I tweak it.

[hkeeler]

Let's make this a link to pyproject.toml.

[hkeeler]

Also, I think the poetry install step should be included here.

[hkeeler]

Suggested change

	- Poetry is used as the package management tool.
	- (Optional) Visual Studio Code for development.
	- (Optional) Docker is needed when using Visual Studio Code / Dev Container.
	The following software packages are pre-requisites to installing this software.
	- [Python](https://www.python.org/downloads/) version 3.10 or greater.
	- [Poetry](https://python-poetry.org/docs/#installation) for Python package management.

I think most users coming to this repo for the first time just want to see it run, not necessarily develop anything. To simplify things for them, I think we should move the following down into a separate Development section.

• (Optional) Visual Studio Code for development.
• (Optional) Docker is needed when using Visual Studio Code / Dev Container.

Now, we it would be nice to have a Dockerfile dedicated to just running the CLI. That's an even easier setup for those who just want to see it run, and don't want to have to know anything about Python, etc. The one in .devcontainer is close, but I don't think it's quite what we'd want. Perhaps that'd be a good follow-up PR.

[hkeeler]

I think we should separate the test data from the tests themselves documentation-wise. When users first come here, they're probably going to just run the CLI and see what happens. Having the test data there is a nice thing for them to try first before they try their own data.

Test data

This repo includes 2 test datasets, one with all valid data, and one where each line represents a different failed validation, or different permutation of the same failed validation.

• sbl-validations-pass.csv
• sbl-validations-fail.csv

We use these test files in for automated test, but can also be passed in via the CLI for manual testing.

Similarly, I think all detailed testing-related bits should be moved under Development.

• The repo includes unit tests that can be executed using pytest or in Visual Studio Code. These tests can be located under src/tests.

[hkeeler]

Hmmm. We should probably tweak the CLI so it doesn't print the DataFrame by default. I think that's going to be confusing to first-time users unfamiliar with Pandas.

[aharjati]

I changed main.py to remove the DF print and also use pprint to display json content better.

[hkeeler]

👍 to removing the DF print.

Unfortunately, what we're currently printing isn't actually JSON. It's a string representation of the a Python dict.

I think what'd work better would be to have run_validation_on_df return a list of objects that could be rendered how the user chooses. We can start by defaulting to JSON, but it could be useful to render it as some tabular form as well. That's what I was going for on:

• https://github.com/cfpb/regtech-data-validator/blob/structure-output/src/validator/main.py

Let's not worry about that for this PR, though. I have some other tweaks I'd like to make to the code, and we can do that then.

[hkeeler]

We should pretty-print the JSON. If it's too long, we can always truncate it with a ....

And there's also this sweet trick to allow expandable areas with GitHub Markdown:

• https://gist.github.com/pierrejoubert73/902cc94d79424356a8d20be2b382e1ab#how-to-add-a-collapsible-section-in-markdown

[aharjati]

thanks. I used html tags to put all terminal output examples into an expandable block

[hkeeler]

I think it'd be nice to add the badge up near the top of the README.

[hkeeler]

I think this is buried too far down the page. I don't think most people are going to scroll this far. Perhaps we should move it to just before the Development section?

[hkeeler]

We should switch to the less verbose mode, or truncate the sample test output.

[hkeeler]

This section should go up above Development.

[hkeeler]

I think this is a good first step. I have some follow-up ideas on both the docs and how to deal with the output of the CLI, but we don't need to do that as part of this PR.

[hkeeler]

We don't need to prefix it with anything. It's fairly standard for repos to just have a set of badges near the top of the README.

Suggested change

	Current overall coverage: [](https://github.com/cfpb/regtech-data-validator/tree/python-coverage-comment-action-data)
	[](https://github.com/cfpb/regtech-data-validator/tree/python-coverage-comment-action-data)

[hkeeler]

👍 to removing the DF print.

Unfortunately, what we're currently printing isn't actually JSON. It's a string representation of the a Python dict.

I think what'd work better would be to have run_validation_on_df return a list of objects that could be rendered how the user chooses. We can start by defaulting to JSON, but it could be useful to render it as some tabular form as well. That's what I was going for on:

• https://github.com/cfpb/regtech-data-validator/blob/structure-output/src/validator/main.py

Let's not worry about that for this PR, though. I have some other tweaks I'd like to make to the code, and we can do that then.