Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include inter-class and inter-collection relationship graphs in schema documentation #2198

Closed
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
3d4820b
WIP: Generate `refgraph` visualizations via GHA workflow
eecavanna Sep 27, 2024
9852362
Constrain Python version and regenerate `poetry.lock` file
eecavanna Sep 27, 2024
5ecdcd5
WIP: Inject `refgraph` pages into documentation preview site
eecavanna Sep 27, 2024
ea3794b
Install `refscan` via `pipx` instead of via `poetry`
eecavanna Sep 27, 2024
27754b1
Regenerate `poetry.lock` file (by running `$ poetry lock --no-update`)
eecavanna Sep 27, 2024
9c22098
Use correct directory name
eecavanna Sep 27, 2024
aa4843c
Remove comment to simplify PR diff
eecavanna Sep 27, 2024
80d509d
Remove unnecessary commands from GHA workflow
eecavanna Sep 27, 2024
d8ed9b3
Use `pipx run` to avoid installing anything
eecavanna Sep 27, 2024
4e753b9
Revert unnecessary changes to `poetry.lock` file
eecavanna Sep 27, 2024
9b750e1
WIP: Update site navigation bar so it contains links to graphs
eecavanna Sep 27, 2024
8915b2b
Restrict which `refscan` version will be used
eecavanna Sep 27, 2024
53a7875
Remove comment unrelated to other changes on branch
eecavanna Sep 27, 2024
168a3cc
Standardize comments between doc-related GHA workflows
eecavanna Sep 27, 2024
4da13d4
WIP: Populate project version number before generating derivative files
eecavanna Sep 27, 2024
472270e
Install `refscan` and update `gendoc` Makefile target to run `refgraph`
eecavanna Oct 15, 2024
8acd4c3
Remove `refgraph` step from GHA workflows (is handled by `make gendoc`)
eecavanna Oct 15, 2024
904ff38
Clarify comment
eecavanna Oct 15, 2024
9911c4c
Update sidebar link text to be more descriptive
eecavanna Oct 15, 2024
e5943a9
Introduce "Visualizations" page as an intermediate destination
eecavanna Oct 26, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 29 additions & 5 deletions .github/workflows/deploy-docs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,34 @@ jobs:
- name: Install dependencies.
run: poetry install -E docs

- name: Build and deploy documentation.
# Build the docs and deploy them to GitHub Pages.
- name: Derive files from sources
# Note: First, we replace the "0.0.0" placeholder version number in `pyproject.toml`.
# Reference: https://github.com/mtkennerly/poetry-dynamic-versioning/blob/master/README.md#command-line-mode
run: |
poetry self add "poetry-dynamic-versioning[plugin]"
poetry dynamic-versioning
make squeaky-clean all

- name: Generate web-based documentation
run: |
mkdir -p docs
touch docs/.nojekyll
make gendoc

- name: Generate reference graphs
# Use `refgraph` (part of `refscan`) to generate graphs (i.e. network diagrams)
# depicting the relationships between collections and between classes.
#
# Note: We generate the graphs (i.e. the HTML files) within the directory that
# will eventually be deployed to GitHub Pages.
#
# Reference: https://github.com/microbiomedata/refscan
#
run: |
pipx run --spec 'refscan==0.1.20' refgraph --schema nmdc_schema/nmdc_materialized_patterns.yaml --subject collection --graph docs/collection-graph.html
pipx run --spec 'refscan==0.1.20' refgraph --schema nmdc_schema/nmdc_materialized_patterns.yaml --subject class --graph docs/class-graph.html

# Deploy the docs to GitHub Pages.
#
# Note: The `make mkd-gh-deploy` command below uses the `mkd-%` target
# defined in `Makefile`. That target's action (i.e. recipe)
Expand All @@ -34,8 +60,6 @@ jobs:
#
# Reference: https://www.mkdocs.org/user-guide/deploying-your-docs/
#
- name: Deploy web-based documentation to GitHub Pages
run: |
mkdir -p docs
touch docs/.nojekyll
make gendoc
Comment on lines -38 to -40
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI: These 3 lines were moved to an earlier step (i.e. to lines 36-38) so that a documentation website file tree exists by the time we try to inject the graphs into it (i.e. on lines 50-51).

make mkd-gh-deploy
27 changes: 24 additions & 3 deletions .github/workflows/test_pages_build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,34 @@ jobs:
- name: Install dependencies
run: poetry install -E docs

- name: Build documentation
- name: Derive files from sources
# Note: First, we replace the "0.0.0" placeholder version number in `pyproject.toml`.
# Reference: https://github.com/mtkennerly/poetry-dynamic-versioning/blob/master/README.md#command-line-mode
run: |
mkdir -p site
touch site/.nojekyll
poetry self add "poetry-dynamic-versioning[plugin]"
poetry dynamic-versioning
make squeaky-clean all

- name: Generate web-based documentation
run: |
mkdir -p docs
touch docs/.nojekyll
make gendoc
poetry run mkdocs build -d site
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that this command is present in this test_pages_build.yaml file, but not in the deploy-docs.yaml file. I don't know why it's necessary in one situation, but not the other. Maybe it's because the author wanted the documentation website's file tree to be generated in a directory named site (instead of—or in addition to—the directory it gets generated in by default).


- name: Generate reference graphs
# Use `refgraph` (part of `refscan`) to generate graphs (i.e. network diagrams)
# depicting the relationships between collections and between classes.
#
# Note: We generate the graphs (i.e. the HTML files) within the directory that
# will eventually be deployed to GitHub Pages.
#
# Reference: https://github.com/microbiomedata/refscan
#
run: |
pipx run --spec 'refscan==0.1.20' refgraph --schema nmdc_schema/nmdc_materialized_patterns.yaml --subject collection --graph site/collection-graph.html
pipx run --spec 'refscan==0.1.20' refgraph --schema nmdc_schema/nmdc_materialized_patterns.yaml --subject class --graph site/class-graph.html

- name: Deploy preview
uses: rossjrw/pr-preview-action@v1
with:
Expand Down
3 changes: 3 additions & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ extra_javascript:
- https://unpkg.com/[email protected]/dist/tablesort.min.js
- javascripts/tablesort.js
nav:
# Reference: https://www.mkdocs.org/user-guide/configuration/#nav
- NMDC Schema: index.md
- About: about.md
- How to run a collaborative data modeling project: https://linkml.io/linkml/howtos/collaborative-development.html
Expand All @@ -30,6 +31,8 @@ nav:
- NMDC Schema Contributors How-to from Schema Hackathon 2023-12-10: https://docs.google.com/presentation/d/1ZH41QAoESUwAkdHyUxlrmSKS5M-bT0TOulBgX4rBx2A/edit#slide=id.g26390794265_0_693
- NMDC Schema Validation: schema-validation.md
- Schema element deprecation guide: schema_element_deprecation_guide.md
- Class graph: class-graph.html
- Collection graph: collection-graph.html


site_url: https://microbiomedata.github.io/nmdc-schema
Expand Down
Loading