[Pitch] Create Metrics Dashboard

[Amzani]

Problem

We aim to provide a dashboard for monitoring our Developer Experience (DX) metrics, which should be collected in asyncapi/community#879

Solution

The dashboard must have the capability to showcase Developer Experience Working group metrics and enable other working groups to add their respective metrics.

• Time to first API Design
• System errors
• Validation errors
• Time to fix a validation error
• Number of created AsyncAPI files
• AsyncAPI 3.0x Adoption

Look & Feel

Link to Figma

Where do we get metrics:

This initiative focuses on the CLI metrics

Definition of metrics

Time to first API Design

It's the time between the creation of a new AsyncAPI file and generation (doc, code...) for a specific user.

 asyncapi generate - asyncapi new file

System errors

Refer to any issue or malfunction of our tools that are not generated by the user (eg. validation errors).
Example: asyncapi/cli#1137

Validation errors

Errors from asyncapi validate

Time to fix a validation error

The delta between a failing validation and successful validation.

 asyncapi validate (success) - asyncapi validate (error)

Number of created AsyncAPI File

From asyncapi new file

AsyncAPI 3.0x Adoption

We can parse the version when a user triggers asyncapi validate

Tech stack

• React/NextJS
• Library: https://www.tremor.so/ or NewRelic Widgets

Note: This is a suggestion! anything else relevant with a valid argument can work as well.

Rabbit holes

1. Assurance in the Metrics Collection:

   • Due to the pending merge of Measure AsyncAPI Adoption community#879, uncertainty surrounds whether we'll have the required metrics in time. To proactively address this risk, let's proceed without being blocked and develop using mocked data.

2. Rate Limit Consideration for New Relic API:

   • Should we opt to construct our own component rather than relying on NewRelic widgets, it's imperative that the backend incorporates a caching mechanism. Refer to [this discussion]((Measure AsyncAPI Adoption community#879 (comment)) for more details.

Scope

• Metrics readiness (Do we have what we need)
• Tech stack decision - Tremor or NewRelic Widgets
• Metric dashboard Implementation

Out of bounds

• We show only Developer Experience (DX) metrics

Success criteria

• The dashboard should be accessible to the public either through the website or a designated URL such as metrics.asyncapi.com.
• Maintainers need to have a clear understanding of the metrics presented on the dashboard.

[shapeit-bot]

Thanks for creating a new pitch 🥳. You can now create or link existing scopes.
You can create new scopes in two different ways:

Option 1

1. Edit the Pitch or Bet issue
2. Add your scope under the scope section

See this example

Option 2

1. Create a new issue
2. Add this keywork in the description related to #ISSUE_BET_NUMBER

See this example