Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: supply system log to trusted plugins via FD #5391

Merged
merged 12 commits into from
Nov 20, 2023
Merged

Conversation

Skn0tt
Copy link
Contributor

@Skn0tt Skn0tt commented Nov 15, 2023

System logs are super useful for our internal observability. Giving system logs to (trusted) build plugins isn't straight-forward since plugins are run inside a child process. This PR is an attempt at bridging this, by sharing the system log file descriptor with the child process. This means build plugins will be able to write to system logs like this:

import { appendFileSync } from  "fs"

export const onBuild = function () {
  // ...
  appendFileSync("/dev/fd/4", "next.js customer using ISC, SSR and XYZ")
  // ...
}

/dev/fd/4 is a bit of a magic name - but it works.

There's still some tests failing, but I wanted to share the approach sooner than later, and get your feedback.

@Skn0tt Skn0tt self-assigned this Nov 15, 2023
@Skn0tt Skn0tt requested review from a team as code owners November 15, 2023 13:30
Copy link
Contributor

This pull request adds or modifies JavaScript (.js, .cjs, .mjs) files.
Consider converting them to TypeScript.

@Skn0tt
Copy link
Contributor Author

Skn0tt commented Nov 15, 2023

Looks like tests are failing on Windows. There's a fair chance that this FD magic doesn't work on Windows, i'll have to investigate that.

Copy link
Member

@eduardoboucas eduardoboucas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this capability is nice, but I'm not a big fan of exposing the raw file descriptor to plugins. This feels leaky, and if we ever want to change the mechanism we use for system logs (like the structured logs idea that @JGAntunes shared) we'll have a hard time with existing plugins.

Could we instead add a systemLog function to the properties we expose to build plugin hooks, which currently just wraps the logic for writing to the file description if available (and perhaps act as a no-op if not available), and further down the line we can swap the internals of that while maintaining the user-facing interface?

@Skn0tt
Copy link
Contributor Author

Skn0tt commented Nov 17, 2023

AFAICT, all properties we send to plugins are serialized because of the IPC bridge. If we send over a function, that would break during serialisation. If we had some sort of "bootstrap" code within the plugin, then we could set up said function in there. Thinking about it, we probably have something like that! I'll check that.

@eduardoboucas
Copy link
Member

We have things like failBuild() inside plugins, so we should be able to wire things up in a similar way?

@Skn0tt
Copy link
Contributor Author

Skn0tt commented Nov 17, 2023

Totally, yeah! Implemented in bbcad09

eduardoboucas
eduardoboucas previously approved these changes Nov 17, 2023
Copy link
Member

@eduardoboucas eduardoboucas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! 🚀

@@ -1,3 +1,4 @@
export const onBuild = function ({ featureFlags }) {
export const onBuild = function ({ featureFlags, systemLog }) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add this to the types.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done in 6fa00d8

eduardoboucas
eduardoboucas previously approved these changes Nov 20, 2023
@Skn0tt Skn0tt enabled auto-merge (squash) November 20, 2023 10:52
@Skn0tt
Copy link
Contributor Author

Skn0tt commented Nov 20, 2023

Tests failed on windows, because /dev/fd/4 isn't really a thing there. Makes sense. I won't put effort into making that work because we don't run CI in Windows - updated the tests instead.

eduardoboucas
eduardoboucas previously approved these changes Nov 20, 2023
packages/build/tests/plugins/tests.js Show resolved Hide resolved
@Skn0tt Skn0tt merged commit 8fee92b into main Nov 20, 2023
36 checks passed
@Skn0tt Skn0tt deleted the supply-system-log-as-fd branch November 20, 2023 13:59
This was referenced May 23, 2024
This was referenced Jun 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants