You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The current implementation of the DestinationStreamStateModel assumes a one-to-one relationship between a Source name and a Destination name. This is fine in principle, under the assumption that the Connection only has one instance of the Source type. However, this is not always the case, lets imagine we have multiple cases of a Google Sheets Source, for which we want to keep its state tracked via state_cache when writing to a destination. How can we achieve this?
In Airbyte, we achieve this by aliasing the Sources and Destinations:
By separating the ORM model to write based on the aliased Source and Destination/Cache names, we can support multiple types of Sources and Destinations and still hold cache for them.
The text was updated successfully, but these errors were encountered:
The current implementation of the DestinationStreamStateModel assumes a one-to-one relationship between a Source name and a Destination name. This is fine in principle, under the assumption that the Connection only has one instance of the Source type. However, this is not always the case, lets imagine we have multiple cases of a Google Sheets Source, for which we want to keep its state tracked via
state_cache
when writing to a destination. How can we achieve this?In Airbyte, we achieve this by aliasing the Sources and Destinations:
By separating the ORM model to write based on the aliased Source and Destination/Cache names, we can support multiple types of Sources and Destinations and still hold cache for them.
The text was updated successfully, but these errors were encountered: