Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] adapter.get_columns_in_relation does not work with materialized views #19

Closed
2 tasks done
rubenhelsloot opened this issue Feb 17, 2024 · 2 comments
Closed
2 tasks done
Labels
bug Something isn't working duplicate This issue or pull request already exists materialized_views

Comments

@rubenhelsloot
Copy link

Is this a new bug?

  • I believe this is a new bug
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

When I use dbt_utils.star() to list the columns of a materialized view, it renders /* no columns returned from star() macro */ instead. I looked into it and found that adapter.get_columns_in_relation returns an empty list.

I believe this is because the relation is a materialized view, and adapter.get_columns_in_relation only returns the columns of tables.

Expected Behavior

I would expect the construct to work for materialized views as well.

Steps To Reproduce

  1. Create a postgres project;
  2. Create a materialized view with some columns;
  3. In a new model, try to get the columns of the materialized view.

Relevant log output

No response

Environment

- OS: MacOS
- Python: 3.11
- dbt-postgres: 1.7.7

Additional Context

No response

@rubenhelsloot rubenhelsloot added bug Something isn't working triage labels Feb 17, 2024
@rubenhelsloot
Copy link
Author

As a temporary workaround, I've overwritten postgres__get_columns_in_relation like this:

-- Override get_columns_in_relation to work with materialized views

{% macro postgres__get_columns_in_relation(relation) -%}
  {% call statement('get_columns_in_relation', fetch_result=True) %}
      with from_table AS (
          select
              column_name,
              data_type,
              character_maximum_length,
              numeric_precision,
              numeric_scale

          from {{ relation.information_schema('columns') }}
          where table_name = '{{ relation.identifier }}'
              {% if relation.schema %}
              and table_schema = '{{ relation.schema }}'
              {% endif %}
          ORDER BY ordinal_position
      ),

      from_matview AS (
          select
              attname AS column_name,
              pg_catalog.format_type(
                pg_attribute.atttypid,
                pg_attribute.atttypmod
              ) AS data_type,
              NULL::int AS character_maximum_length,
              38 AS numeric_precision,
              9 AS numeric_scale

          FROM pg_attribute
            JOIN pg_class on pg_attribute.attrelid = pg_class.oid
            JOIN pg_namespace on pg_class.relnamespace = pg_namespace.oid
          WHERE pg_attribute.attnum > 0
              AND NOT pg_attribute.attisdropped
              AND pg_class.relname = '{{ relation.identifier }}'
              {% if relation.schema %}
              AND pg_namespace.nspname = '{{ relation.schema }}'
              {% endif %}
          ORDER BY pg_attribute.attnum
      )

      select *
      FROM from_table
      UNION ALL (
        SELECT *
        FROM from_matview
      )

  {% endcall %}
  {% set table = load_result('get_columns_in_relation').table %}
  {{ return(sql_convert_columns_in_relation(table)) }}
{% endmacro %}

@dbeatty10
Copy link
Contributor

Thanks for reporting this @rubenhelsloot !

This looks the same as dbt-labs/dbt-core#9419, so I'm going to close this as a duplicate.

Your workaround might work, but I'd recommend this instead for full-backwards compatibility: dbt-labs/dbt-core#9419 (comment).

@dbeatty10 dbeatty10 closed this as not planned Won't fix, can't repro, duplicate, stale Feb 19, 2024
@dbeatty10 dbeatty10 added duplicate This issue or pull request already exists and removed triage labels Feb 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working duplicate This issue or pull request already exists materialized_views
Projects
None yet
Development

No branches or pull requests

2 participants