Can't update fields of an already uploaded media using the S3 adapter

[fofoy]

Link to reproduction

No response

Payload Version

3.0.0-beta.56

Node Version

20.12.2

Next.js Version

15.0.0-rc.0

Describe the Bug

Whenever I want to update a field (e.g. Alt field), I get a 500 error when I try to submit the form.

Here is the log from the console

Reproduction Steps

Install Payload v3.0.0-beta.56 with the default Media Collection
Install @payloadcms/storage-s3 v3.0.0-beta.56
Set up S3 storage (I tried both on minIO and AWS S3 and I get the same behavior)
Upload a first Media
Update the Alt field and submit the form

Adapters and Plugins

@payloadcms/db-postgres v3.0.0-beta.56
@payloadcms/storage-s3 v3.0.0-beta.56

[fofoy]

Any news on this one ? Is there anything I can help you with @PatrikKozak ?

[PatrikKozak]

Hey @fofoy - we'll get right on this!

[PatrikKozak]

Hi @fofoy - I did some testing on my end and was not able to reproduce the above issue.

It may have something to do with your S3 storage setup - do you have a reverse proxy setup?

Is this purely local?

Just want to get some additional information before I can dig deeper into this - thank you!

[fofoy]

Hi @PatrikKozak I have the same issue in local (using a dockerized MinIO instance) and on my staging environment using an AWS S3 bucket so it doesn't seem to be a setup issue.

[PatrikKozak]

@fofoy Thanks for the info - hmm unfortunately there isn't enough here to fully understand what could be causing this issue.

A couple things though, could you try and log out some of the info in that function where the error is coming from (in your node_modules) - this may be able to give us some more information

Also, could you update your payload packages versions to latest (just in case) - I'd like to get to the bottom of this but just will need more info to continue

[fofoy]

@PatrikKozak Thanks, I had the time to update my project to latest beta version 91 and the problem remains, I'll dig further to see where it's going wrong and I'll let you know.

[fofoy]

Hi @PatrikKozak here is the result of logging the Promise getExternalFile on line 41 of generateFileData.js file:

Promise {
  <pending>,
  [Symbol(async_id_symbol)]: 652801,
  [Symbol(trigger_async_id_symbol)]: 652790,
  [Symbol(kResourceStore)]: {
    url: {
      pathname: '/api/media/c26034b9-71ab-4e88-8ff1-2dfc5a980261',
      search: '?depth=0&fallback-locale=null'
    },
    headers: [Getter],
    cookies: [Getter],
    mutableCookies: [Getter],
    draftMode: [Getter],
    reactLoadableManifest: {},
    assetPrefix: '',
    afterContext: undefined,
    isHmrRefresh: undefined,
    serverComponentsHmrCache: undefined
  },
  [Symbol(kResourceStore)]: {
    isStaticGeneration: false,
    page: '/(payload)/api/[...slug]/route',
    fallbackRouteParams: null,
    route: '/api/[...slug]',
    incrementalCache: IncrementalCache {
      locks: Map(0) {},
      unlocks: Map(0) {},
      hasCustomCacheHandler: false,
      dev: true,
      disableForTestmode: false,
      minimalMode: false,
      requestHeaders: [Object],
      requestProtocol: 'https',
      allowedRevalidateHeaderKeys: undefined,
      prerenderManifest: [Object],
      revalidateTimings: [SharedRevalidateTimings],
      fetchCacheKeyPrefix: '',
      cacheHandler: [FileSystemCache]
    },
    isRevalidate: false,
    isPrerendering: undefined,
    fetchCache: undefined,
    isOnDemandRevalidate: undefined,
    isDraftMode: undefined,
    requestEndedState: undefined,
    revalidate: 0,
    tags: [
      '_N_T_/layout',
      '_N_T_/(payload)/layout',
      '_N_T_/(payload)/api/layout',
      '_N_T_/(payload)/api/[...slug]/layout',
      '_N_T_/(payload)/api/[...slug]/route',
      '_N_T_/api/media/c26034b9-71ab-4e88-8ff1-2dfc5a980261'
    ],
    nextFetchId: 2
  },
  [Symbol(kResourceStore)]: undefined,
  [Symbol(kResourceStore)]: undefined,
  [Symbol(kResourceStore)]: undefined,
  [Symbol(kResourceStore)]: { isAppRoute: true, isAction: false }
}

and here is the actual error when trying to fetch the file :

TypeError: fetch failed
     at node:internal/deps/undici/undici:12618:11
     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
     at async getExternalFile (webpack-internal:///(rsc)/../node_modules/.pnpm/[email protected][email protected][email protected][email protected]
 a48-202_bgnc5eiwwsaz2mukappcno4s3m/node_modules/payload/dist/uploads/getExternalFile.js:18:21) {
   cause: Error: connect ECONNREFUSED 127.0.0.1:443
       at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1605:16)
       at TCPConnectWrap.callbackTrampoline (node:internal/async_hooks:130:17) {
     errno: -111,
     code: 'ECONNREFUSED',
     syscall: 'connect',
     address: '127.0.0.1',
     port: 443
   }
 }
  ⨯ unhandledRejection: TypeError: fetch failed
     at node:internal/deps/undici/undici:12618:11
     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
     at async getExternalFile (webpack-internal:///(rsc)/../node_modules/.pnpm/[email protected][email protected][email protected][email protected]
 a48-202_bgnc5eiwwsaz2mukappcno4s3m/node_modules/payload/dist/uploads/getExternalFile.js:18:21) {
   cause: Error: connect ECONNREFUSED 127.0.0.1:443
       at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1605:16)
       at TCPConnectWrap.callbackTrampoline (node:internal/async_hooks:130:17) {
     errno: -111,
     code: 'ECONNREFUSED',
     syscall: 'connect',
     address: '127.0.0.1',
     port: 443
   }
 }
  ⨯ unhandledRejection: TypeError: fetch failed
     at node:internal/deps/undici/undici:12618:11
     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
     at async getExternalFile (webpack-internal:///(rsc)/../node_modules/.pnpm/[email protected][email protected][email protected][email protected]
 a48-202_bgnc5eiwwsaz2mukappcno4s3m/node_modules/payload/dist/uploads/getExternalFile.js:18:21) {
   cause: Error: connect ECONNREFUSED 127.0.0.1:443
       at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1605:16)
       at TCPConnectWrap.callbackTrampoline (node:internal/async_hooks:130:17) {
     errno: -111,
     code: 'ECONNREFUSED',
     syscall: 'connect',
     address: '127.0.0.1',
     port: 443
   }
 }

So it seems like it's something to do with the network. I'm running everything in Docker with several containers for the postgres db, the S3 instance (minIO) and the payload admin but it also does the same on our dev cluster (kubernetes) with an AWS S3 instance connected to it.

[nickejoha]

Hi @PatrikKozak and @fofoy

I'm having the same problem as well. I can't edit any fields of an already uploaded file, and I got the same error as described above.

We also run our environments in kubernetes and we use Google cloud storage for storing files. I have however noticed that when I'm running payload locally (and still using google cloud storage for storing files) it works. When I'm running payload in docker I get the same error we got in our kubernetes environments.

@fofoy Have you found any temporary workaround for this problem?

[fofoy]

Hi @nickejoha, I didn't have the time to dig any further than this yet as I need to make progress on my current project but that's definitely something that needs to be sorted, I'll try to invest more time in a bit.

[denolfe]

I also attempted to recreate this issue, but I was unable to. I'm not sure there's much more we can do on our end to troubleshoot this.

Please try and recreate this in the latest beta to ensure it is still an issue.

[fofoy]

Hi @denolfe, I updated to the latest version of beta v122 and the same problem remains, let me know if I can help you in any way to help you debug this.

[github-actions]

This issue has been marked as stale due to lack of activity.

To keep this issue open, please indicate that it is still relevant in a comment below.

[github-actions]

This issue was automatically closed due to lack of activity.

[nickejoha]

@denolfe @fofoy
I think we have found the issue here and we have implemented a workaround.

In short, the problem in our case seems to be connected our firewall in Google cloud plattform and locally that we use a different external port in our docker build.

The problem here seems to be that when we edit a field of media file that has been uploaded already, payload tries to fetch the file but the fetch fails. The problem in our case was that when payload tries to fetch the file, it uses the external url which works fine when used from a frontend client or when running a docker config that doesn't alter the ports, but because the request is executed from the container itself the url is invalid. In our local docker config the external url is localhost:3077, but inside the container the app runs on port 3000. So when we access file from outside the container localhost:3077 is valid, but when trying to fetch the same file from inside the container the url is invalid because the application runs on port 3000. The problem in our Google cloud plattform was due to the same fact that the external url isn't valid from inside the cluster.

So with that figured out we implemented a bit of a hack as a workaround. By temporarily changing the files url to a valid url (from the containers perspective) with a hook, and the error is gone and editing the file works :)

import { CollectionBeforeOperationHook, CollectionBeforeReadHook } from 'payload'

const fixMediaUrl = (req: any) => {
  if (req.data?.url) {
    const prefix = process.env.CMS_INTERNAL_URL || ''
    if (req.data.url.startsWith('/')) {
      req.data.url = prefix + req.data.url
    }
    if (req.data?.sizes) {
      for (const key in req.data.sizes) {
        if (req.data.sizes[key].url?.startsWith('/')) {
          req.data.sizes[key].url = prefix + req.data.sizes[key].url
        }
      }
    }
  }
}

export const fixMediaUrlOperation: CollectionBeforeOperationHook = async ({ req }) =>
  fixMediaUrl(req)

export const fixMediaUrlRead: CollectionBeforeReadHook = async ({ req }) => fixMediaUrl(req)