Sync failed from docker hub

[BlueCase]

Version
pulpcore: 3.68.0
pulp_container: 2.22.0
pulp-oci-images

Describe the bug
One of my sync from docker hub stopped working and always failed with following error:

"error": {
  "traceback": "  File \"/usr/local/lib/python3.9/site-packages/pulpcore/tasking/tasks.py\", line 68, in _execute_task\n    result = func(*args, **kwargs)\n  File \"/usr/local/lib/python3.9/site-packages/pulp_container/app/tasks/synchronize.py\", line 41, in synchronize\n    return dv.create()\n  File \"/usr/local/lib/python3.9/site-packages/pulpcore/plugin/stages/declarative_version.py\", line 161, in create\n    loop.run_until_complete(pipeline)\n  File \"/usr/lib64/python3.9/asyncio/base_events.py\", line 647, in run_until_complete\n    return future.result()\n  File \"/usr/local/lib/python3.9/site-packages/pulpcore/plugin/stages/api.py\", line 220, in create_pipeline\n    await asyncio.gather(*futures)\n  File \"/usr/local/lib/python3.9/site-packages/pulpcore/plugin/stages/api.py\", line 41, in __call__\n    await self.run()\n  File \"/usr/local/lib/python3.9/site-packages/pulp_container/app/tasks/sync_stages.py\", line 254, in run\n    await self.resolve_flush()\n  File \"/usr/local/lib/python3.9/site-packages/pulp_container/app/tasks/sync_stages.py\", line 283, in resolve_flush\n    manifest_dc.content.init_image_nature()\n  File \"/usr/local/lib/python3.9/site-packages/pulp_container/app/models.py\", line 177, in init_image_nature\n    return self.init_manifest_nature()\n  File \"/usr/local/lib/python3.9/site-packages/pulp_container/app/models.py\", line 208, in init_manifest_nature\n    elif self.is_helm_chart():\n  File \"/usr/local/lib/python3.9/site-packages/pulp_container/app/models.py\", line 285, in is_helm_chart\n    return self.json_manifest[\"config\"][\"mediaType\"] == MEDIA_TYPE.CONFIG_BLOB_HELM\n  File \"/usr/local/lib/python3.9/site-packages/pulp_container/app/models.py\", line 263, in json_manifest\n    self._json_manifest = json.loads(self.data)\n  File \"/usr/lib64/python3.9/json/__init__.py\", line 339, in loads\n    raise TypeError(f'the JSON object must be str, bytes or bytearray, '\n",
  "description": "the JSON object must be str, bytes or bytearray, not NoneType"
}

[
  {
    "pulp_href": "/pulp/api/v3/remotes/container/container/018c65b4-62cc-79e4-a767-6ea2c781799d/",
    "prn": "prn:container.containerremote:018c65b4-62cc-79e4-a767-6ea2c781799d",
    "pulp_created": "2023-12-14T00:22:49.804518Z",
    "pulp_last_updated": "2024-11-20T18:21:33.598916Z",
    "name": "gitea",
    "url": "https://registry-1.docker.io",
    "ca_cert": null,
    "client_cert": null,
    "tls_validation": true,
    "proxy_url": null,
    "pulp_labels": {},
    "download_concurrency": 2,
    "max_retries": null,
    "policy": "immediate",
    "total_timeout": null,
    "connect_timeout": null,
    "sock_connect_timeout": null,
    "sock_read_timeout": null,
    "headers": null,
    "rate_limit": 2,
    "hidden_fields": [
      {
        "name": "client_key",
        "is_set": false
      },
      {
        "name": "proxy_username",
        "is_set": false
      },
      {
        "name": "proxy_password",
        "is_set": false
      },
      {
        "name": "username",
        "is_set": false
      },
      {
        "name": "password",
        "is_set": false
      }
    ],
    "upstream_name": "gitea/gitea",
    "include_tags": [
      "1.21-rootless",
      "latest-rootless"
    ],
    "exclude_tags": null,
    "sigstore": null
  }
]

Other sync form dockerhub still working. I don't know whats special with this repository .

To Reproduce
Create remote and try to sync

Expected behavior
Sync without error

Additional context

[git-hyagi]

Hi @BlueCase,

From the error provided, I believe this is happening because of an upgrade in pulp-container plugin without the migration of the manifests files to the database. More info here: https://discourse.pulpproject.org/t/starting-the-migration-countdown-with-a-new-release-of-the-pulp-container-plugin/1217
If so, would you mind trying the following command

pulpcore-manager container-handle-image-data

and see if you can sync without errors after that?

[BlueCase]

@git-hyagi thanks for the help. Resolved my issue. But would be interesting why only this repository was a problem.

[git-hyagi]

Glad to know it is fixed!

My guess is that this was the only repository synced when you were running pulp-container 2.19 (and maybe the others were removed or not synced again in this version).
From versions >= 2.20 all manifests pushed/synced to pulp-container were already stored in the database, so the

self._json_manifest = json.loads(self.data)

instruction would be able to find the self.data and continue the code execution without errors.