DM-46556 Add consdb documentation #51
Conversation
Vebop
force-pushed
the
tickets/DM-46556-deployment
branch
from
e9117f0 to
4feb271
Compare
JeremyMcCormick
changed the title
doc/user-guide/schemas.rst
Outdated
|
|
||
| Think about the utility of these versions in terms of interaction with the ConsDB APIs, migrations, etc. | ||
|
|
||
| Do sdm_schemas versions appear in the db? |
There was a problem hiding this comment.
The versions are not currently stored in the database, but I plan to add this capability.
See DM-47367
doc/user-guide/schemas.rst
Outdated
| Currently the schemas are tagged and versioned as a set, at least w.r.t. the Science Platform. | ||
| So once ConsDB is available in TAP, it should be part of that set. | ||
|
|
||
| What do users see, how does TAP play into this, do the schema need this type of micro versioning? |
There was a problem hiding this comment.
We do not currently have any capability to provide individual schema versions via TAP, as TAP_SCHEMA does not include a column providing this information. Users on the Science Platform do not really see any schema versions at all, currently, because the tag on sdm_schemas is only specified in Phalanx and not propagated to the web UI either.
There was a problem hiding this comment.
This section was added just to capture the conversation that was happening in reference to schema versioning the other week. Is there still clarification I need to add here, or maybe we need to speak about this at a weekly meeting? Did this happen on 11.11, when I missed?
There was a problem hiding this comment.
Unless the procedures change, ConsDB schemas will be deployed to the Science Platform as part of an sdm_schemas release. This version will not be visible to users. This will determine what tables and columns are visible in TAP. We can cut new releases of sdm_schemas to incorporate changes to ConsDB as needed. (The sdm_schemas semver releases used on the Science Platform are made ad hoc/as needed and not on a fixed schedule.)
We will have the capability to deploy different releases of sdm_schemas to different environments if this is required (usdf-rsp, usdf-rsp-dev, etc.). But we cannot currently deploy different versions of specific schemas individually. They are versioned as a set.
doc/user-guide/schemas.rst
Outdated
|
|
||
| - or there can be ones that are not neutral. | ||
|
|
||
| Think about the utility of these versions in terms of interaction with the ConsDB APIs, migrations, etc. |
There was a problem hiding this comment.
I would think that a release of the ConsDB API should be connected to a set of compatible schema versions. Would a tag of sdm_schemas be good enough? We don't currently have a good way of retrieving schemas by their internal versions, so I don't know how useful it would be to have them listed individually. This sdm_schemas tag could be included as part of the release notes.
I am not sure how this information could be tracked automatically though. Maybe it is just something that needs to be tracked and included manually.
|
As a general comment, the RST formatting needs work. See cheat sheet here. |
Vebop
force-pushed
the
tickets/DM-46556-deployment
branch
2 times, most recently
from
da8576b to
f4b66f1
Compare
There was a problem hiding this comment.
I left some specific suggestions for changes.
Overall, I'm wondering why all the additional content is being added that doesn't have to do with the deployment process. I would think going forward that it would be better to stick more closely to the issue a ticket is intended to solve in updating the documentation, rather than including a bunch of different updates that are not closely related. (I'm not requesting that these changes be removed this time, though.)
alembic-autogenerate.py
Outdated
| # python alembic-autogenerate.py this is my revision message "\n" \ | ||
| # the message can span multiple lines "\n" \ | ||
| # if desired | ||
| # python alembic-autogenerate.py DM-12345 |
There was a problem hiding this comment.
I don't know that we have "officially" determined that ticket names will be used in migrations. Maybe it needs to be discussed in the meeting first?
There was a problem hiding this comment.
Did we officially decide something? it wasn't super clear to me from the recording
|
@Vebop Additional note: If you want to make various changes/updates across the whole documentation site, I do not feel that this needs to be connected to a ticket. I think it would be reasonable to do this on user branches. Tickets can be used for more focused/specific work. Unless you want to create and use a generic ticket like "Write consdb documentation site." :) |
|
|
||
| #consolidated-database | ||
|
|
||
| Overall complaints: |
There was a problem hiding this comment.
This seems somewhat terse and could use more explanation. Is the idea that someone would open a ticket and assign it to a certain individual based on the specific sub-component?
There was a problem hiding this comment.
I copied this from the linked runbook, so I'm actually not sure.
https://rubinobs.atlassian.net/wiki/spaces/LSSTOps/pages/45665320/Consolidated+Database+ConsDB+Runbook+draft+incomplete
|
@Vebop Another general comment... There is a mix of URLs and linking styles in this PR, some of which do not work. Links should be consistently formatted like this in RST: We should avoid raw URLs and use the above format instead. |
Vebop
changed the title
Vebop
force-pushed
the
tickets/DM-46556-deployment
branch
from
828d909 to
d6c9f8f
Compare
Vebop
force-pushed
the
tickets/DM-46556-deployment
branch
from
828aac0 to
059952b
Compare
Added Deployment process documentation to Schema Migration Process in the operator guide.
Added other small documentation, like schema versioning conversation, and monitoring notes.
Most recent build of docs: https://consdb.lsst.io/builds/105