Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add server timing to responses #164

Merged
merged 12 commits into from
Jan 14, 2025

Conversation

SgtPooki
Copy link
Member

@SgtPooki SgtPooki commented Dec 13, 2024

Title

feat: add server timing to responses

Description

This PR adds Server Timing headers to responses by wrapping all responses returned by fetch in a Response object that includes the server timing headers.

The headers are generated by wrapping async functions with a serverTiming utility function that records the time taken to execute the function, and then adding those headers to an array on the VerifiedFetch class.

Notes & open questions

  • We need to work out exactly what events we need to time
  • I'd also like to clean up the serverTiming utility function usage so that it's easier to add new timings without having to handle the error and success cases manually for each one.

Example of current server timings for http://bafybeie4vcqkutumw7s26ob2bwqwqi44m6lrssjmiirlhrzhs2akdqmkw4.ipfs.helia-sw-gateway.localhost/:

image

Example of current server timings for http://blog-ipfs-tech.ipns.helia-sw-gateway.localhost/:

image

Change checklist

  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation if necessary (this includes comments as well)
  • I have added tests that prove my fix is effective or that my feature works

@SgtPooki SgtPooki linked an issue Dec 13, 2024 that may be closed by this pull request
@SgtPooki SgtPooki marked this pull request as ready for review December 13, 2024 18:43
@SgtPooki SgtPooki requested a review from a team as a code owner December 13, 2024 18:43
@SgtPooki
Copy link
Member Author

SgtPooki commented Dec 13, 2024

Copy link
Member Author

@SgtPooki SgtPooki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self review

Copy link
Member Author

@SgtPooki SgtPooki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self review of added test

Copy link
Member

@2color 2color left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is pretty neat. Implementation looks good to me.

What do you think about making this optional per request? I can imagine that in node.js this isn't very useful and just makes the response object larger.

We need to work out exactly what events we need to time

At the very least we'd want:

  • IPNS/DNSLink resolution (if it's an ipns url)
  • content routing, i.e. finding providers.

@SgtPooki
Copy link
Member Author

SgtPooki commented Jan 7, 2025

What do you think about making this optional per request? I can imagine that in node.js this isn't very useful and just makes the response object larger.

We could look into making it an option and defaulting to true in sw-gateway and helia-http-gateway. FYI with helia-http-gateway, which runs on nodejs, it's still useful because it's returning the response directly from verified-fetch.


  • IPNS/DNSLink resolution (if it's an ipns url)
  • content routing, i.e. finding providers.

more granular IPNS/DNSLink resolution will require some deeper piping of the server timing. I will look into those, or see if I can finagle some determinism from onProgress events

@SgtPooki
Copy link
Member Author

SgtPooki commented Jan 9, 2025

@2color added the option to include server timing headers, defaulting to false on construction but overridable per request. Also added docs and updated readme

@@ -452,6 +460,26 @@ export class VerifiedFetch {
[identity.code]: this.handleRaw
}

private async handleServerTiming<T> (name: string, description: string, fn: () => Promise<T>): Promise<T> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we check this.includeServerTiming here to encapsulate most of the logic here and avoid the measuring and assigning of this.serverTimingHeaders?

That would also allow simplifying responseWithServerTiming and renaming to something a bit more generic, e.g. augmentResponse or handleResponse

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea let me peek at this

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so, it just so happened that switching the server timing handling to handleServerTiming required me to pipe things through a lot more, so I'm moving forward with this and adding the "IPNS/DNSLink resolution (if it's an ipns url)" you previously mentioned.

I didn't originally like it in handleServerTiming because it required plumbing the request-level "withServerTiming" into more places, but it makes more sense for multiple reasons.

it also prevents the extra work of wrapping the timed functions.. so it's a win. thanks for the push

@SgtPooki
Copy link
Member Author

@2color all requested changes are in, but the code is a little bit sloppier now because of how parse-resource & parse-url-string are currently written.

Copy link
Member

@2color 2color left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I still think it would be really useful to add a duration for finding providers from each router/block-broker.

Since that would probably require us to add progress hooks to the helia routers, we can handle this in a follow up PR.

Not directly related, but more insight into routing would help surface more nuanced errors: no providers vs. failure to connect to providers, to the user.

@SgtPooki
Copy link
Member Author

CI seems to be failing when attempting to install Kubo on windows.. i'm looking into it and will try clearing package-lock and updating Kubo

@SgtPooki
Copy link
Member Author

CI seems to be failing when attempting to install Kubo on windows.. i'm looking into it and will try clearing package-lock and updating Kubo

Nikolai is looking into the collab cluster

@SgtPooki SgtPooki merged commit 0701c71 into main Jan 14, 2025
20 checks passed
@SgtPooki SgtPooki deleted the 160-feat-add-server-timings-to-responses branch January 14, 2025 15:29
github-actions bot pushed a commit that referenced this pull request Jan 14, 2025
## [@helia/verified-fetch-v2.4.0](https://github.com/ipfs/helia-verified-fetch/compare/@helia/verified-fetch-2.3.1...@helia/verified-fetch-2.4.0) (2025-01-14)

### Features

* add server timing to responses ([#164](#164)) ([0701c71](0701c71))
Copy link

🎉 This PR is included in version @helia/verified-fetch-v2.4.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feat: add server timings to responses
2 participants