Upstream migrations not run on preview

[danielrbradley]

We currently run the upstream migrations on the Diff, Apply and Refresh methods within the tfshim provider2.go. On Pulumi's side this results in the migrations being run during Diff, Create and Update. This results in state migrations not being applied to output properties during preview time. This currently breaks users migrating to AzureAD v6 when previewing due to the upstream provider expecting that dependent resources recieve the newly migrated versions of property values (post migration) from other resources.

For the migration (as part of the plan resource change diff) to be applied during preview in Pulumi programs, the Diff needs to be called during the Pulumi Check method which is the only opportunity on Pulumi's side to modify state during the preview.

Upstream migrations should be idempotent, so it should not matter than migrations are then run in both Check and subseqent methods.

See also:

• Upgrade causes breaking changes in azuread.ServicePrincipalPassword pulumi-azuread#1551
• Consider moving planning to be performed in Check #2281

[t0yv0]

Thank you for this issue, this looks like a perfect example of what we have been looking for - some practical examples that demonstrate 2281 is really needed.

[t0yv0]

Taking some notes from a quick call.

azuread.ServicePrincipal wants to migrate the ID format.
azuread.ServicePrincipalPassword fails when receiving ServicePrincipal in the old format.

In detail, at preview:

ServicePrincipal Check returns the old ID
ServicePrincipal Diff computes the new ID but decides a NO CHANGES plan
ServicePrincpal Update(preview=True) is not run because of the NO CHANGES plan
Engine assumes that the old ID is the planned value and pipes it to ServicePrincipalPassword, which fails

[t0yv0]

Curiously, since the program works at pulumi --skip-preview time, I'm wondering what the call sequence is there. I suspect that the sequence passes the correct migrated ID, which likely happens by Diff returning an Update plan, and then Update running to recompute the new ID. If this hunch is correct, we may have a more surgical way forward to fix the issue, namely understand why in preview Diff decides no-changes instead of Update plan, and fixing that may possibly resolve the problem as well.

Moving planning to Check is still interesting as suggested and likely would solve the problem but involves an engine protocol change and possibly more repercussions.