Builds for public project that use yarn 2 (or above) with workspaces are failing

[fulopkovacs]

Is your build failing?

Our builds our failing. We use yarn 3.2.0 with workspaces in a public repo.

This is the step where yarn fails:

5:10:09 PM: ➤ YN0001: │ Error: While cloning /opt/build/repo/theatre/node_modules/source-map -> /opt/build/repo/examples/r3f-cra/node_modules/react-scripts/node_modules/workbox-build/node_modules/@babel/core/node_modules/source-map ENOENT: no such file or directory, scandir '/opt/build/repo/theatre/node_modules/source-map'

Describe the bug

A bit of context: in yarn 1.x only private repos can have workspaces, but this limitation was removed in yarn 2:

Worktrees used to be required to be private (ie list "private": true in their package.json). This requirement got removed with the 2.0 release in order to help standalone projects to progressively adopt workspaces (for example by listing their documentation website as a separate workspace). (source: docs)

The problem is in the run-build-functions.sh file:

  # YARN_IGNORE_PATH will ignore the presence of a local yarn executable (i.e. yarn 2) and default
  # to using the global one (which, for now, is always yarn 1.x). See https://yarnpkg.com/configuration/yarnrc#ignorePath
  # we can actually use this command for npm workspaces as well
  workspace_output="$(YARN_IGNORE_PATH=1 yarn workspaces --json info 2>/dev/null)"
  workspace_exit_code=$?

As you can see it tries to output information about the workspaces with yarn 1.x. in the command substitution and redirects STDERR to /dev/null. After that it only checks if the process exited with 0 or an error code, but ignores the error message, which would tell us, that 'workspaces can only be enabled in private projects'. Here's an example for the problem from the theatre-js/theatre repo:

• when the errors are hidden:

 $ YARN_IGNORE_PATH=1 yarn workspaces --json info 2>/dev/null
{"type":"info","data":"Visit \u001b[1mhttps://yarnpkg.com/en/docs/cli/workspaces\u001b[22m for documentation about this command."}

• when the errors are visible:

$ YARN_IGNORE_PATH=1 yarn workspaces --json info
{"type":"error","data":"Workspaces can only be enabled in private projects."}
{"type":"info","data":"Visit \u001b[1mhttps://yarnpkg.com/en/docs/cli/workspaces\u001b[22m for documentation about this command."}

To Reproduce

Steps to reproduce the behavior:

1. Fork the repository at https://github.com/theatre-js/theatre
2. Create a Netlify site and link it to the forked repository
3. See that the Netlify build fails

Configuration

Right now we use the UI to configure our deploy previews, but I can give you the values we set there:

build:
    command: yarn workspace playground  build:static
     commandOrigin: ui
    environment:
       - NETLIFY_USE_YARN
      - REVIEW_ID
     publish: /opt/build/repo/packages/playground/build
     publishOrigin: ui

Expected behavior

In an ideal situation the workspaces should be detected, cached and restored when possible.

Build Output

Link to the logs: https://app.netlify.com/sites/theatrejs-playground/deploys/623de90057211c0008613a1a#L75

Full logs

5:08:32 PM: Build ready to start
5:08:34 PM: build-image version: 195fbe127e5c374d9c4758652cb62e3b8936a395 (focal)
5:08:34 PM: build-image tag: v4.6.0
5:08:34 PM: buildbot version: 43c70c62bfed821cfaaabbb50f00238dd4bdd2d6
5:08:34 PM: Fetching cached dependencies
5:08:34 PM: Starting to download cache of 249.6MB
5:08:38 PM: Finished downloading cache in 4.470642038s
5:08:38 PM: Starting to extract cache
5:08:45 PM: Finished extracting cache in 6.804082377s
5:08:45 PM: Finished fetching cache in 11.297846854s
5:08:45 PM: Starting to prepare the repo for build
5:08:45 PM: Preparing Git Reference pull/107/head
5:08:46 PM: Parsing package.json dependencies
5:08:47 PM: Starting build script
5:08:47 PM: Installing dependencies
5:08:47 PM: Python version set to 2.7
5:08:47 PM: Started restoring cached node version
5:08:48 PM: Finished restoring cached node version
5:08:48 PM: v16.14.2 is already installed.
5:08:48 PM: Now using node v16.14.2 (npm v8.5.0)
5:08:48 PM: Started restoring cached build plugins
5:08:48 PM: Finished restoring cached build plugins
5:08:49 PM: Attempting ruby version 2.7.2, read from environment
5:08:49 PM: Using ruby version 2.7.2
5:08:50 PM: Using PHP version 8.0
5:08:50 PM: Started restoring cached yarn cache
5:08:50 PM: Finished restoring cached yarn cache
5:08:51 PM: Found yarn version (3.2.0) that doesn't match expected (1.22.10)
5:08:51 PM: up to date, audited 1 package in 207ms
5:08:51 PM: found 0 vulnerabilities
5:08:52 PM: Installing yarn at version 1.22.10
5:08:52 PM: Installing Yarn!
5:08:52 PM: > Downloading tarball...
5:08:52 PM: [1/2]: https://yarnpkg.com/downloads/1.22.10/yarn-v1.22.10.tar.gz --> /tmp/yarn.tar.gz.uwMzQmiCuP
5:08:52 PM:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
5:08:52 PM:                                  Dload  Upload   Total   Spent    Left  Speed
5:08:52 PM:   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
5:08:52 PM: 100    81  100    81    0     0    723      0 --:--:-- --:--:-- --:--:--   723
5:08:52 PM: 100    94  100    94    0     0    522      0 --:--:-- --:--:-- --:--:--   522
5:08:52 PM: 100   657  100   657    0     0   2894      0 --:--:-- --:--:-- --:--:--  2894
5:08:52 PM:   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
5:08:52 PM: 100 1215k  100 1215k    0     0  3133k      0 --:--:-- --:--:-- --:--:-- 10.6M
5:08:52 PM: [2/2]: https://yarnpkg.com/downloads/1.22.10/yarn-v1.22.10.tar.gz.asc --> /tmp/yarn.tar.gz.uwMzQmiCuP.asc
5:08:52 PM:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
5:08:52 PM:                                  Dload  Upload   Total   Spent    Left  Speed
5:08:52 PM:   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
5:08:52 PM: 100    85  100    85    0     0   2931      0 --:--:-- --:--:-- --:--:--  2931
5:08:52 PM: 100    99  100    99    0     0   1903      0 --:--:-- --:--:-- --:--:--  1903
5:08:52 PM: 100   661  100   661    0     0  10169      0 --:--:-- --:--:-- --:--:-- 10169
5:08:52 PM: 100   832  100   832    0     0   9142      0 --:--:-- --:--:-- --:--:--  9142
5:08:52 PM: > Verifying integrity...
5:08:52 PM: gpg: Signature made Fri 02 Oct 2020 11:17:27 AM UTC
5:08:52 PM: gpg:                using RSA key 6D98490C6F1ACDDD448E45954F77679369475BAA
5:08:52 PM: gpg: Good signature from \"Yarn Packaging <[email protected]>\" [unknown]
5:08:52 PM: gpg: WARNING: This key is not certified with a trusted signature!
5:08:52 PM: gpg:          There is no indication that the signature belongs to the owner.
5:08:52 PM: Primary key fingerprint: 72EC F46A 56B4 AD39 C907  BBB7 1646 B01B 86E5 0310
5:08:52 PM:      Subkey fingerprint: 6D98 490C 6F1A CDDD 448E  4595 4F77 6793 6947 5BAA
5:08:52 PM: > GPG signature looks good
5:08:52 PM: > Extracting to ~/.yarn...
5:08:52 PM: > Adding to $PATH...
5:08:52 PM: > Successfully installed Yarn 1.22.10! Please open another terminal where the `yarn` command will now be available.
5:08:53 PM: No yarn workspaces detected
5:08:53 PM: Started restoring cached node modules
5:08:53 PM: Finished restoring cached node modules
5:08:53 PM: Installing NPM modules using Yarn version 3.2.0
5:08:54 PM: ➤ YN0050: The cache-folder option has been deprecated; use rc settings instead
5:08:55 PM: ➤ YN0000: ┌ Resolution step
5:08:55 PM: ➤ YN0002: │ @examples/dom-cra@workspace:examples/dom-cra doesn't provide @testing-library/dom (pd85ff), requested by @testing-library/user-event➤ YN0002: │ @examples/r3f-cra@workspace:examples/r3f-cra doesn't provide @testing-library/dom (p532ab), requested by @testing-library/user-event➤ YN0060: │ @examples/r3f-cra@workspace:examples/r3f-cra provides three (p78e7d) with version 0.130.1, which doesn't satisfy what @theatre/r3f and some of its descendants request➤ YN0002: │ @theatre/r3f@workspace:packages/r3f doesn't provide prop-types (p0a95f), requested by react-shadow➤ YN0002: │ @theatre/r3f@workspace:packages/r3f doesn't provide react-is (pddf13), requested by styled-components➤ YN0002: │ @theatre/r3f@workspace:packages/r3f [3278b] doesn't provide prop-types (p7ea79), requested by react-shadow➤ YN0002: │ @theatre/r3f@workspace:packages/r3f [3278b] doesn't provide react-is (pd816a), requested by styled-components➤ YN0002: │ @theatre/r3f@workspace:packages/r3f [b1ff4] doesn't provide prop-types (p7b534), requested by react-shadow➤ YN0002: │ @theatre/r3f@workspace:packages/r3f [b1ff4] doesn't provide react-is (pb0672), requested by styled-components➤ YN0002: │ @theatre/studio@workspace:theatre/studio doesn't provide react-is (pbc5b9), requested by styled-components➤ YN0002: │ @theatre/studio@workspace:theatre/studio [5df52] doesn't provide react-is (pd60bb), requested by styled-components➤ YN0002: │ @types/react-icons@npm:3.0.0 doesn't provide react (p5d791), requested by react-icons➤ YN0002: │ playground@workspace:packages/playground doesn't provide playwright (p332e1), requested by @percy/playwright➤ YN0002: │ playground@workspace:packages/playground doesn't provide react (pa1827), requested by @react-three/drei➤ YN0002: │ playground@workspace:packages/playground doesn't provide react (p4cb89), requested by @react-three/fiber➤ YN0002: │ playground@workspace:packages/playground doesn't provide react (pc981d), requested by @theatre/r3f➤ YN0002: │ playground@workspace:packages/playground doesn't provide react-dom (pca11e), requested by @theatre/r3f➤ YN0060: │ playground@workspace:packages/playground provides three (p72055) with version 0.130.1, which doesn't satisfy what @theatre/r3f and some of its descendants request➤ YN0002: │ reakit-warning@npm:0.6.1 [7cd90] doesn't provide react-dom (p1b609), requested by reakit-utils➤ YN0002: │ theatre@workspace:theatre doesn't provide webpack (peeb0b), requested by babel-loader➤ YN0002: │ theatre@workspace:theatre doesn't provide webpack (pfc26d), requested by circular-dependency-plugin➤ YN0002: │ theatre@workspace:theatre doesn't provide webpack (p3cea2), requested by esbuild-loader➤ YN0002: │ theatre@workspace:theatre doesn't provide webpack (pb4b7d), requested by exec-loader➤ YN0002: │ theatre@workspace:theatre doesn't provide webpack (p92128), requested by file-loader➤ YN0002: │ theatre@workspace:theatre doesn't provide webpack (p42777), requested by html-loader➤ YN0002: │ theatre@workspace:theatre doesn't provide webpack (p7820e), requested by null-loader➤ YN0002: │ theatre@workspace:theatre doesn't provide webpack (p2f3d3), requested by url-loader➤ YN0000: │ Some peer dependencies are incorrectly met; run yarn explain peer-requirements <hash> for details, where <hash> is the six-letter p-prefixed code
5:08:55 PM: ➤ YN0000: └ Completed in 0s 437ms
5:08:55 PM: ➤ YN0000: ┌ Fetch step
5:10:05 PM: ➤ YN0013: │ 20 packages were already cached, 2256 had to be fetched
5:10:05 PM: ➤ YN0000: └ Completed in 1m 10s
5:10:05 PM: ➤ YN0000: ┌ Link step
5:10:09 PM: ➤ YN0000: └ Completed in 4s 407ms
5:10:09 PM: ➤ YN0000: Failed with errors in 1m 15s
5:10:09 PM: Error during Yarn install
5:10:09 PM: Build was terminated: Build script returned non-zero exit code: 1
5:10:09 PM: Creating deploy upload records
5:10:10 PM: Failing build: Failed to build site
5:10:10 PM: Failed during stage 'building site': Build script returned non-zero exit code: 1 (https://ntl.fyi/exit-code-1)
5:10:10 PM: Finished processing build request in 1m36.221681106s

Additional context