Observability vs privacy

[nferch]

Problem statement

We need a flexible approach to User identification that enables telemetry insights, troubleshooting, and performance and reliability analysis that provides privacy transparency and control for end-users.

Context

• We want to collect telemetry events/page loads, metrics, and error logs to give us product insights, aid in troubleshooting, and better understand the performance and reliability our platform and apps.
• In particular, metrics and error logs are helpful for understanding issues that are difficult to reproduce in synthetic tests.
• We use SaaS services to help us collect and process this data.
• The platform does not explicitly collect direct PII such as Name, Address, or Email address.
• The platform can collect the public key from the user's HALO, which corresponds to an identity but is not currently coo-relatable to PII.
• The platform can collect information about the user's rough geographic location, device type, browser, etc.

Current state

• End users are notified that their data is collected when the app and CLI starts up.
• In browser-based applications, they can opt-out using the settings.
• In CLI-based applications, they can opt-out by setting an environment variable.
• This preference is saved to local storage.

Ideal state

• Users have full control of whether their observability data is collected, and the ability for the data collected to be disassociated with any PII or Personal Data (e.g. "the right to be forgotten")
• There is a low friction way for the DXOS team to opt-in to full observability collection in a manner that enables easy troubleshooting (e.g. identity, device, and space keys are exposed)

Telemetry identifiers

• Identifier should be generated client-side with no attachment to identity.
• Identifier can then be reset by user.
• Identifier can be synced via HALO to allow for correlation of events bewteen devices.
• Identifier should also be stored outside of HALO such that it is available immediately on startup.

Prior art

• Endless OS’s privacy-preserving metrics system