Error: subprocess terminated with status code exit status 127

[jadoonf]

Describe the bug

Scanning certain projects (e.g. tier, n8n) using lstn in returns an error code 127.

• lstn v0.0.2-8-g2e03d22+2e03d223.20230217
• macOS Monterey 12.0.1
• go version: go1.19.4 darwin/arm64

Steps to reproduce the behavior

1. Clone this repo
2. Run lstn in (inside project root)
3. See error:

Error: subprocess terminated with status code exit status 127

Expected vs actual behavior

Should provide verdicts. For context, scanning for the npm package itself (e.g. lstn to tier 5.1.2) works.

Logs

Error: subprocess terminated with status code exit status 127

[fntlnz]

@jadoonf do you have npm installed in your system?. Yes, you have it. When npm is not installed we give a specific error.

/lstn_0.0.2_linux_arm64/node-sdk # ../lstn 
Running without a configuration file
Error: couldn't find the npm executable in any way

[didof]

This issue has been analyzed in #62 already.

[leodido]

I've just tested this out against the main branch, and it works.

❯ npm --version
6.14.17

❯ ./lstn version -v
lstn v0.0.2+be8d6581.20230223

❯ ./lstn in node-sdk --json
Running without a configuration file
[{"name":"@ampproject/remapping","shasum":"56c133824780de3174aed5ab6834f3026790154d","verdicts":[],"version":"2.2.0"},{"name":"@babel/code-frame","shasum":"3b25d38c89600baa2dcc219edfa88a74eb2c427a","verdicts":[],"version":"7.18.6"},{"name":"@babel/compat-data","shasum":"c241dc454e5b5917e40d37e525e2f4530c399298","verdicts":[],"version":"7.21.0"},{"name":"@babel/core","shasum":"1341aefdcc14ccc7553fcc688dd8986a2daffc13","verdicts":[],"version":"7.21.0"},{"name":"@babel/generator","shasum":"951cc626057bc0af2c35cd23e9c64d384dea83dd","verdicts":[],"version":"7.21.1"},{"name":"@babel/helper-compilation-targets","shasum":"a6cd33e93629f5eb473b021aac05df62c4cd09bb","verdicts":[],"version":"7.20.7"},{"name":"@babel/helper-environment-visitor","shasum":"0c0cee9b35d2ca190478756865bb3528422f51be","verdicts":[],"version":"7.18.9"},{"name":"@babel/helper-function-name","shasum":"d552829b10ea9f120969304023cd0645fa00b1b4","verdicts":[],"version":"7.21.0"},{"name":"@babel/helper-hoist-variables","shasum":"d4d2c8fb4baeaa5c68b99cc8245c56554f926678","verdicts":[],"version":"7.18.6"},{"name":"@babel/helper-module-imports","shasum":"1e3ebdbbd08aad1437b428c50204db13c5a3ca6e","verdicts":[],"version":"7.18.6"}...

The problem is likely to depend on the different npm version.

[jadoonf]

Thank you! This is helpful. Resolved.

[fntlnz]

Just for future reference.

We could generate the package lock with arborist as npm does and forget about npm completely.

const Arborist = require('@npmcli/arborist')

const arb = new Arborist({
    // options object

    // where we're doing stuff.  defaults to cwd.
    path: '/path/to/the/folder/containing-package-lock-json',

    // url to the default registry.  defaults to npm's default registry
    registry: 'https://registry.npmjs.org/',
})

// Make the idealTree be the thing that's on disk
arb.reify({
    // write the lockfile(s) back to disk, and package.json with any updates
    // defaults to 'true'
    save: true,
}).then(() => {
    // node modules has been written to match the idealTree
})

[fntlnz]

I suggest to package the node_modules for using this in the command as a single js file using vercel's ncc https://github.com/vercel/ncc

[leodido]

I'm gonna investigate this path further as we discussed privately, thank you!

[leodido]

This issue inspired the work I've done at https://github.com/listendev/erborist.

What's erborist?

It's a CLI that lets you generate a package-lock.json file...

• without Node.js on your machine
• without npm on your machine
• without actually installing anything (bye node_modules/ see you soon)
• targeting a specific lock file version disregarding the npm version you may have or not
• even if the package.json scripts are broken

The plans are to integrate erborist in lstn for the supported OSes/architectures.

Should any OS/architecture tuple be not supported by erborist, then lstn will fallback to the current behavior (ie., looking for npm >=6 and generate a package-lock.json file with it).

What's missing?

• erborist lacks a fully functioning release mechanism
• which is propedeutic to get and execute its binary in lstn in place of npm

[jadoonf]

Syncing up on this:

The issue with tier was resolved by downgrading to npm version 6.14.17. The temp fix is using that node version in the Docker action.

However, the persisting issue is with n8n, where running lstn in returns Error: subprocess terminated with status code exit status 1

@leodido wondering if this going to be resolved through arborist or there is some other issue?

[leodido]

Yes. Will write a detailed write up later today or tomorrow.

[leodido]

@jadoonf the plan is to have a way to generate package-lock.json files even from broken package.json files.

Let's consider for example the package.json files containing scripts that use globally installed tools...

The low-hanging fruit would be making lstn execute npm install --package-lock-only --ignore-scripts ... (see #127). But this doesn't solve everything...

Since there could be many other ways that the above command doesn't work, the ideal solution would be to have a tool that behaves like npm install does but:

1. never downloads dependencies (some older npm versions do, disregarding the --package-lock-only flag)
2. generates a package-lock.json without the need of having npm
3. ignores everything that is not about creating the full dependencies tree

This would make lstn a tool to audit the npm dependencies with security in mind (no downloads, no scripts being run, etc.).

Also, this approach would make lstn work on platforms without Node.js being even installed, narrowing a lot of the attack surface.

That's why I'm working on https://github.com/listendev/erborist.

The plan going forward (after the low-hanging fruit fix) is to embed erborist in lstn for supported platforms.

Whether we will fall back to the current implementation (ie., using npm) for platforms that erborist doesn't support is still to decide.

[leodido]

Ok, PR #128 should fix the issue tierrun/node-sdk while using other npm versions too (not only 6.x).

The issues with n8n-io/n8n have a different nature.

I'm on it investigating if there's a low-hanging fruit here before reverting to work on erborist.