Denesting - is there a way to not denest primitive arrays?

[MattMacGillivray]

Denesting makes sense for complex objects, but if my postgres tap pulls out an integer[] (integer array), the target always denests it into a separate table. Is there a way for the target to insert it as an integer array?

Below is my partial config relating to the integer[] column.

         {
            "breadcrumb": [
               "properties",
               "label_ids"
            ],
            "metadata": {
               "sql-datatype": "integer[]",
               "inclusion": "available",
               "selected-by-default": true,
               "selected": true
            }
         }

            "label_ids": {
               "type": "array",
               "items": {
                  "type": "integer"
               }
            }

[awm33]

@qmnonic We typically follow what Stitch does and they denest basic types in a similar way. We also have shared logic in denesting that works for other SQL-like targets (Redshift, Snowflake) and try to avoid target specific typing. Stitch does this for the same reason, avoiding lots of target specific code and added complexity.

The target also doesn't see the metadata, only the schema, so if this was added the logic would be only on the schema. It would also have to be optional.

Another option we've discussed is a config option to turn off denesting completely or potentially for a list of stream names. Everything that is nested would be a JSONB type.

[dmcquay]

Any updates on this since 2019?