You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Building on some of the research done for #4997, we want to improve catalog creation performance by selecting only the needed relation information, instead of information for every relation in a schema.
Acceptance criteria
In dbt's base adapter implementation (base/impl.py), the _get_one_catalog method has been modified to accept an additional new parameter, relations_by_schema: Optional[Dict[str, Optional[List[BaseRelation]]]] = None, mapping schema names to the list of relations in that schema for which catalog information should be retrieved.
When the new parameter defaults to None, behavior should be the same as before.
When _get_one_catalog invokes the get_catalog macro, it should include the new parameter.
Adapters which do not yet implement the new get_catalog macro parameter should continue to work as before. Therefore we need to evaluate the macro without generating an error and we also need to continue post-filtering the results.
When the value of the relations parameter is None, the _get_one_catalog method, and get_catalog macros should continue to retrieve all relation information for every schema listed in the schema parameter, matching historical behavior.
Impact to Other Teams
The adapter teams will need to implement changes to the get_catalog macro in each adapter implementation, in order to take advantage of the new parameter:
Short description
Building on some of the research done for #4997, we want to improve catalog creation performance by selecting only the needed relation information, instead of information for every relation in a schema.
Acceptance criteria
_get_one_catalog
method has been modified to accept an additional new parameter,relations_by_schema: Optional[Dict[str, Optional[List[BaseRelation]]]] = None
, mapping schema names to the list of relations in that schema for which catalog information should be retrieved._get_one_catalog
invokes theget_catalog
macro, it should include the new parameter.relations
parameter isNone
, the_get_one_catalog
method, andget_catalog
macros should continue to retrieve all relation information for every schema listed in theschema
parameter, matching historical behavior.Impact to Other Teams
The adapter teams will need to implement changes to the get_catalog macro in each adapter implementation, in order to take advantage of the new parameter:
The text was updated successfully, but these errors were encountered: