Allow for local compile of incremental models.

[0xRobin]

This PR:

Enhances the default is_incremental logic to allow an override with a command line parameter.
The default value is set as False.
Usage:
dbt compile --vars '{force-incremental: True}'

The base logic was taken from here: is_incremental.sql

[0xRobin]

1. I'm not sure why but the pre-commit doesn't pick up the configs in macros/macros_schema.yml
2. This will cause a full-refresh of all incremental models (which is not really necessary)
   We could manually run a general incremental update after merging (without full refresh) to prevent this.

[0xRobin]

@dot2dotseurat @jeff-dude @soispoke
What do you think? :)

[dot2dotseurat]

Would we get the same thing from the proposed change in mentioned in this issue?

[0xRobin]

@dot2dotseurat I don't think so, for me they don't really seem related.
This PR just allows us to locally get the compiled sql for an incremantal model (as if the is_incremental would return true)
It's really helpful when developing complex models and testing them out on Dune.

[0xRobin]

Thanks!
@cryptofreedman we should probably add a note of this in the docs somewhere.
I would add it next to "make sure to compile your models and test them out on Dune.com" but it seems we don't really suggest that anywhere currently. 🤔

Not sure where the best place would be

[cryptofreedman]

@0xRobin Noted, will do, ty!

[aranke]

Hi @0xRobin,

I'm Kshitij from the dbt-core team and I'm a fan of Dune.
I'm currently working on interactive compile (dbt-labs/dbt-core#6358) which will display the compiled SQL for a single model in the terminal.
Let me know if this will solve your use case, thanks!

[0xRobin]

Let me know if this will solve your use case, thanks!

@aranke thanks for reaching out, the proposed features certainly look useful!
But I don't think the current proposed setup would be able to replace our workaround here.
We're looking for the compiled SQL of an incremental model (as if is_incremental() would return true)

[0xRobin]

@dot2dotseurat the --no-allow-introspection option that would be introduced could be a workaround for this other issue we're having with long compile times?

[dot2dotseurat]

@0xRobin I think it could but I'd want to do a little due diligence first on if this affects the manifest file or not. Specifically, I'd want us to double check how this affects finding state:modified models.