Disable/remove Easy Telemetry for all maintained releases

[toco-cam]

Acceptance Criteria (Required on creation)

• It is ensured that for all maintained releases, a patch is available that ensures users can no longer switch on Easy Telemetry (ET).

7.22

• Backend & docs: Remove Easy Telemetry #4483
• Frontend: In Admin, remove telemetry configuration options #4484

7.20 & 7.19

• Disable telemetry (backport of Deactivate easy telemetry #4167).

Hints

• Easy telemetry has been disabled by default since 7.21 Deactivate easy telemetry #4167 (comment)
• Ensure diagnostics endpoint still works.
• Consider removing code related to Telemetry.

Links

Breakdown

Pull Requests

Beta Give feedback

1. Update announcement.md camunda-docs-static#462
   +0
2. Remove Easy Telemetry #4483
   4 of 4
   scope:core-api type:subtask version:7.22.0 version:7.22.0-alpha5 +4
   
3. https://github.com/camunda/camunda-bpm-platform-maintenance/pull/1252
4. https://github.com/camunda/camunda-bpm-platform-maintenance/pull/1253
5. [Backport 7.19] chore(telemetry): document reporting telemetry is disabled by default camunda-docs-manual#1663
   +0
   
6. [Backport 7.20] chore(telemetry): document reporting telemetry is disabled by default camunda-docs-manual#1662
   +0
   
7. https://github.com/camunda/camunda-bpm-platform-maintenance/pull/1272
8. https://github.com/camunda/camunda-bpm-platform-maintenance/pull/1273

Dev2QA handover

• Does this ticket need a QA test and the testing goals are not clear from the description? Add a Dev2QA handover comment

[yanavasileva]

We want to do define phase to estimate effort for code removal. If it higher we might only disable the ET in maintained releases.

Considerations: backwards compatibility, rolling upgrade.

[yanavasileva]

Prototype: #4465

WIP dump

• ProcessEngineConfigurationImpl - not needed, remove:
  • initializeTelemetry
  • telemetryEndpoint
  • telemetryRequestRetries
  • telemetryRequestTimeout
  • telemetryReportingPeriod
  • telemetryHttpConnector
  • what is some customers have custom values Tomcat, Spring Boot

we need the reported active
scheduling timer task

• TelemetryReporter - not needed, remove:
  • telemetryEndpoint
  • telemetryRequestRetries
  • telemetryRequestTimeout
  • telemetryReportingPeriod
  • telemetryHttpConnector

timer task but it will be used only for updateAndSendData (false, false)
does this mean we don't need a timer task, can we change it?

• TelemetrySendingTask - not needed, remove:
  • telemetryEndpoint
  • telemetryRequestRetries
  • telemetryRequestTimeout
  • telemetryReportingPeriod
  • telemetryHttpConnector
  • nothing when #run

Loggers - remove logging that is not used any longer

GetTelemetryDataCmd - unchanged
IsTelemetryEnabledCmd - always return false or remove entirely? rest api?
TelemetryConfigureCmd - reschedules the timer tasks, we don't need it anymore though, rest api?

remove connect dependency - not needed since we won't send requests
check if the shading works as expected

tests getTelemtryData needs to be still tested
TelemetryReporterTest - rewrite to have coverage for the different parts of the result - webapps
cfg files - remove endpoint

wiremock

rest api
distros
IT

Docs
migration guide - feature removed, process engine conf options should be cleaned up
https://docs.camunda.org/manual/7.21/introduction/telemetry/ - leave only collected data that can be used for the diagnostics

differences between the versions - will backport be higher effort, when toucing the pom.xml, most probably auto backport won't work.

[yanavasileva]

Kickoff

Current state

• 7.22 and 7.21 - Telemetry is disabled by default for new setups. In updated instances where telemetry was already enabled, telemetry will continue to be enabled
• >= 7.20 - telemetry (initializeTelemetry = null) is configured first time opening the Admin webapp.
• For all Camunda 7 versions we have the following telemetry public APIs, they might be used by users:
  • isTelemetryEnabled
  • toggleTelemetry
  • getTelemetryConfiguration
  • configureTelemetry
  • all of the other implementation, reporter, sending task, configuration properties is all private API.
• Functionality that we want to keep:
  • Java and REST API: getTelemetryData()
  • Webapps: Diagnostics page
  • In backend, that means we want to keep the telemetry reporter.

Proposal

1. Engine
   • ProcessEngineConfigurationImpl
     • 7.22 - Remove configuration properties and related code that are not needed anymore.
     • 7.21 and below
       • Disable telemetry (backport of Deactivate easy telemetry #4167).
       • Keep the properties but without the functionality. Otherwise if users have them with custom values the process engine won't start up after an update
     • for updated instance, the database property/ies will be left there and they won't be used. Do we want to remove it?
   • TelemetryReporter and TelemetrySendingTask
     • Remove sending functionality. Keep general code so it can be used for fetching the diagnostic data without the interval to send HTTP requests.
   • IsTelemetryEnabledCmd
     • deprecate Java and REST API, so whoever uses it knows that is no longer function and can remove it but their code won't brake
     • empty out command, should always return false
   • TelemetryConfigureCmd
     • deprecate Java and REST API, so whoever uses it knows that is no longer function and can remove it but their code won't brake
     • empty out command, should do nothing
   • connect dependency - remove it and check if shading is working as expected
   • Loggers - remove all logging that is not used any longer
2. REST API
   • deprecate and empty out
   • remove tests
3. Distros, javaee, connect plugin
   • remove connect process engine plugin used for telemetryHttpConnector (sending HTTP requests for telemetry) and the a
   • exclude connect dependency
     • if users use the dependency for other reasons, they can add the dependency themselves (we can consider adding this to the migration guide)
4. Webapps
   • Remove Telemetry Settings page
   • Remove initial popup when initializeTelemetry = null
   • Adjust tests
5. Tests (CE and EE)
   • tests configuration files - remove telemetry endpoint property
   • tests - remove sending telemetry tests but leave and ensure diagnostic data API is tested
   • wiremock - remove where not needed anymore
6. Docs
   • Configuration pages - remove telemetry properties: process engine, quarkus, spring boot
   • Telemetry page - leave only collected data that can be used for the diagnostics, maybe rename/move the page. For transparency we can add that feature has been removed.
   • Migration page - document any breaking changes
7. Optional
   • TelemetryReporter and TelemetrySendingTask
     • rename the classes to move away from telemetry as they will be used only for fetching diagnostic data
     • remove the TimerTask as it is not needed anymore

Estimation

There are differences in code base (of the files that needs to be adjusted) between current development (7.22) and maintenance versions. Potentially, we want to do different changes on current and maintenance versions.

1. Backend - 6 days total
   • 7.22 - 2 days
   • backports and buffer - 4 days
2. Frontend - 2 days total
   • 7.22 - 0.5-1 day
   • backports and buffer - 1.5 days
3. Docs - 0.5 days total

[yanavasileva]

• Discussion

  • If users have enabled telemetry and endpoint is discontinued, that might can cause performance issues in sending data, and default retries. There might some CPU consumption and log entries in the server output.
  • leave connect dependency if users already have it for backports
  • Very few users might have enabled it, so maybe we can wait for user feedback. If no customer is asking or complain, we reduce effort by not doing the backports. If customers approach us about it, we advise them to disabled it.
  • PM will check if ET is still working and there's traffic for C7. What's the end of life? Is it only demo data sent at the moment?

• Decision

  • Only for 7.22 - Remove all configuration, empty public API and mark it deprecated and add note in migration guide.
  • 7.21 and below - we confirm about ET about the status of the endpoint but in general wait for customer feedback to schedule the backport.

[yanavasileva]

Next: create breakdown. - done, ticket's description has been updated.

[yanavasileva]

Need to resolve failures on CI 7.19, 7.20?

[yanavasileva]

Reopen for a manual test.

[yanavasileva]

Dev2QA handover.

Task report test

How will it impact the users

• 7.22 - telemetry feature has been removed.
  • There should be no popup for configuring telemetry on new installation.
  • Admin webapp should work as until now. Diagnostics page should be available and working as well.
• 7.20-7.19 - telemetry feature is disabled by default.
  • There should be no popup for configuring telemetry on new installation.
  • On Telemetry settings page in Admin, the configuration should be disabled.
  • Diagnostics page should be available and working as well.

[gbetances089]

Verified on:

• camunda-bpm-run-ee-7.19.15-20240827.143027-16
• camunda-bpm-run-ee-7.20.8-20240827.124425-13
• camunda-bpm-run-ee-7.21.4-20240827.132709-19

[yanavasileva]

As of today, telemetry requests are still accepted by ET => if there are users sending data, they won't receive errors currently.