Auto-populating default/required TAP_SCHEMA rows

[jwfraustro]

After our discussions at ADASS/IVOA Interop, I've been having a great time experimenting with felis, and trying out moving our workflows to use it.

As you know, the TAP_SCHEMA tables are self-describing, with their own columns/tables/schema defined as rows within themselves (TAP_SCHEMA.columns.column_name, *.datatype, *.xtype, etc.) and required to be defined.

When creating/loading a new TAP database with init-tap and load-tap, I noticed that these rows are not auto-populated in the db. I saw that there is a definition of them in the package: https://github.com/lsst/felis/blob/main/python/felis/schemas/tap_schema_std.yaml

Is there a way to have the rows auto-populate through the CLI?

[JeremyMcCormick]

This is currently done with an external SQL file, but I agree it would be nice functionality to have in Felis itself.

I will make a ticket in our Jira and leave this issue open as well on GitHub until this is resolved.

BTW, it is probably not 100% clear but the load-tap-schema command is an intended replacement of load-tap and we are in a deprecation period. They should result in identical output according to my testing. (If not, that would something I would like to know about should you try out the new command.)

[jwfraustro]

Thanks for looking at this, that would be an awesome feature. It's surprising how often we've had a catalog fail because of a simple mistake like forgetting one of these rows in the schema.

[gpdf]

I strongly support this step; we had discussed it internally previously, but it just hadn't made it to the top of the stack yet.

[gpdf]

By the way, you mentioned xtype above, @jwfraustro. We have a longer-term plan to start doing things in Felis to support xtype directly. For instance, there's an open ticket to propagate the timestamp type from Felis to labeling the resulting column in a VOTable with the corresponding xtype.

If you have specific existing uses of xtype in your archives that's you'd like Felis to support, I'd love to hear about them as another issue. I can't make any guarantees about prioritization, but also once you see our implementation for timestamp it might give you enough context for you to be able to upstream something in this area that we could accept.

[JeremyMcCormick]

I've created DM-48167 for tracking this request.