Mechanism to track provenance of output records

[ryzhyk]

No description provided.

[ryzhyk]

One way to do this is to add a provenance field to Value, but make it invisible to comparison operators, so that different derivations of the same Value look the same to differential. This way provenance won't change the semantics of DDlog, but we will only observe one derivation of each value.

[namibj]

I just want to mention the existence of prior art: https://github.com/frankmcsherry/explanation
It is only for an ancient version of Differential Dataflow, but lessons learnt should be valuable

[sdifv]

One way to do this is to add a provenance field to Value, but make it invisible to comparison operators, so that different derivations of the same Value look the same to differential. This way provenance won't change the semantics of DDlog, but we will only observe one derivation of each value.

Indeed, this is a very intuitive method, but as you said, making this field invisible will actually make the provenance information partly missing if there existing multi-derivation rule.

I found that another datalog dialect, souffle’, already supports the provenance feature to help users with interactive debugging. Could you please share me whether there is any development plan for DDlog in this aspect? Therefore, users do not need to directly modify the DDlog code logic to get the support of provenance.

Thank you!

[mihaibudiu]

This is certainly something we'd like to do, but at this point no one is working on this topic.

[Kixiron]

Soufflé has done some other relevant work on this recently