Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable step function tracing at forwarder level #831

Merged
merged 15 commits into from
Aug 1, 2024

Conversation

avedmala
Copy link
Contributor

@avedmala avedmala commented Jul 30, 2024

What does this PR do?

This PR adds logic to allow users to enable tracing for all their Step Functions via the dd_step_function_trace_enabled tag

This tag can be set via the DdStepFunctionTraceEnabled CloudFormation parameter which is used to set the DD_STEP_FUNCTION_TRACE_ENABLED env var

Link to log with the new tag and screenshot of the tag

Motivation

Testing Guidelines

Additional Notes

Types of changes

  • Bug fix
  • New feature
  • Breaking change
  • Misc (docs, refactoring, dependency upgrade, etc.)

Check all that apply

  • This PR's description is comprehensive
  • This PR contains breaking changes that are documented in the description
  • This PR introduces new APIs or parameters that are documented and unlikely to change in the foreseeable future
  • This PR impacts documentation, and it has been updated (or a ticket has been logged)
  • This PR's changes are covered by the automated tests
  • This PR collects user input/sensitive content into Datadog
  • This PR passes the integration tests (ask a Datadog member to run the tests)
  • This PR passes the unit tests
  • This PR passes the installation tests (ask a Datadog member to run the tests)

@github-actions github-actions bot added the aws label Jul 30, 2024
@avedmala avedmala marked this pull request as ready for review July 31, 2024 18:08
@avedmala avedmala requested a review from a team as a code owner July 31, 2024 18:08
@@ -32,6 +33,12 @@ def enrich(events, cache_layer):
extract_host_from_guardduty(event)
extract_host_from_route53(event)

# Set tracing behavior for all step functions
if DD_STEP_FUNCTION_TRACE_ENABLED:
event[DD_CUSTOM_TAGS] = ",".join(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Null check for event[DD_CUSTOM_TAGS] ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will be moving this logic as per Kimi's comment and adding a null check on the entry

SetDdStepFunctionTraceEnabled:
Fn::Equals:
- Ref: DdStepFunctionTraceEnabled
- true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What will be the default value in CloudFormation console (shown as below) when customer install a new DD forwarder?
It should be default false, but customers can set it to be true if they want.

Screenshot 2024-07-31 at 11 27 05 AM

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The console will use the default value in Parameters: section, so in this case it would be false.

https://github.com/DataDog/datadog-serverless-functions/pull/831/files#diff-7c5bda646bf6435d9285913888d0473f0ac98ed0fdd025ae5d9ea357ad9b3ffaR164-R166

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be the default behavior, here is the flow

  • Template has the default for the DdStepFunctionTraceEnabled to be false
  • SetDdStepFunctionTraceEnabled condition will then be false because DdStepFunctionTraceEnabled != true
  • DD_STEP_FUNCTION_TRACE_ENABLED will be AWS::NoValue, which means the env var won't be set

Copy link
Contributor

@kimi-p kimi-p left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry that I forgot to share the original PR with you before. Could you also add a test like this test from the previous PR?

https://github.com/DataDog/datadog-serverless-functions/pull/618/files#diff-caaaee34b52da49a4f9f45b1d6f59c91d60b24d7570e75aab8876df91f5324a7R874

aws/logs_monitoring/README.md Outdated Show resolved Hide resolved
if DD_STEP_FUNCTION_TRACE_ENABLED:
event[DD_CUSTOM_TAGS] = ",".join(
[event[DD_CUSTOM_TAGS]] + ["dd_step_function_trace_enabled:true"]
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should to add the step functions specific logic to handle_step_function_source().

We'd also want to do the empty value (e.g. "") check like this one:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved the logic to the handle_step_function_source() but I used a slightly different approach with the join so I don't think need the empty check

SetDdStepFunctionTraceEnabled:
Fn::Equals:
- Ref: DdStepFunctionTraceEnabled
- true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The console will use the default value in Parameters: section, so in this case it would be false.

https://github.com/DataDog/datadog-serverless-functions/pull/831/files#diff-7c5bda646bf6435d9285913888d0473f0ac98ed0fdd025ae5d9ea357ad9b3ffaR164-R166

@@ -184,6 +185,12 @@ def handle_step_function_source(self):
+ ",".join(formatted_stepfunctions_tags)
)

if DD_STEP_FUNCTION_TRACE_ENABLED:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@nine5two7
Copy link
Contributor

LGTM.
Nit: We typically use the term 'Step Functions' instead of 'Step Function' to be consistent with AWS naming conventions.

Copy link
Contributor

@nine5two7 nine5two7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@avedmala avedmala changed the title [SVLS-5044] Enable step function tracing at forwarder level Enable step function tracing at forwarder level Aug 1, 2024
@avedmala avedmala merged commit d507e14 into master Aug 1, 2024
13 checks passed
@avedmala avedmala deleted the avedmala/step-fn-tracing branch August 1, 2024 20:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants