From 5d791137be5adb0baca9b46ec2945bb0cbb5a95a Mon Sep 17 00:00:00 2001 From: Craig Date: Fri, 10 Jan 2025 13:25:52 -0800 Subject: [PATCH] hub: usage refresh Signed-off-by: Craig --- content/manuals/desktop/setup/sign-in.md | 2 +- content/manuals/docker-hub/_index.md | 4 +- .../manuals/docker-hub/download-rate-limit.md | 381 ------------------ .../docker-hub/image-library/mirror.md | 2 +- content/manuals/docker-hub/release-notes.md | 2 +- content/manuals/docker-hub/usage/_index.md | 56 +++ content/manuals/docker-hub/usage/manage.md | 49 +++ content/manuals/docker-hub/usage/pulls.md | 218 ++++++++++ content/manuals/docker-hub/usage/storage.md | 27 ++ content/manuals/subscription/details.md | 4 +- content/reference/api/hub/latest.yaml | 2 +- 11 files changed, 358 insertions(+), 389 deletions(-) delete mode 100644 content/manuals/docker-hub/download-rate-limit.md create mode 100644 content/manuals/docker-hub/usage/_index.md create mode 100644 content/manuals/docker-hub/usage/manage.md create mode 100644 content/manuals/docker-hub/usage/pulls.md create mode 100644 content/manuals/docker-hub/usage/storage.md diff --git a/content/manuals/desktop/setup/sign-in.md b/content/manuals/desktop/setup/sign-in.md index 0fcac9e92a5..72ac850af5c 100644 --- a/content/manuals/desktop/setup/sign-in.md +++ b/content/manuals/desktop/setup/sign-in.md @@ -42,7 +42,7 @@ In large enterprises where admin access is restricted, administrators can [enfor - You can access your Docker Hub repositories directly from Docker Desktop. -- Authenticated users also get a higher pull rate limit compared to anonymous users. For example, if you are authenticated, you get 200 pulls per 6 hour period, compared to 100 pulls per 6 hour period per IP address for anonymous users. For more information, see [Download rate limit](/manuals/docker-hub/download-rate-limit.md). +- Authenticated users also get a higher pull rate limit compared to anonymous users. For more information, see [Usage and limits](/manuals/docker-hub/usage/_index.md). - Improve your organization’s security posture for containerized development by taking advantage of [Hardened Desktop](/manuals/security/for-admins/hardened-desktop/_index.md). diff --git a/content/manuals/docker-hub/_index.md b/content/manuals/docker-hub/_index.md index c47f97f76dc..f1da3b3f5b7 100644 --- a/content/manuals/docker-hub/_index.md +++ b/content/manuals/docker-hub/_index.md @@ -24,10 +24,10 @@ grid: description: Learn about organization administration. icon: store link: /admin/ -- title: Usage +- title: Usage and limits description: Explore usage limits and how to better utilize Docker Hub. icon: leaderboard - link: /docker-hub/download-rate-limit/ + link: /docker-hub/usage/ - title: Release notes description: Find out about new features, improvements, and bug fixes. icon: note_add diff --git a/content/manuals/docker-hub/download-rate-limit.md b/content/manuals/docker-hub/download-rate-limit.md deleted file mode 100644 index 477d40625f4..00000000000 --- a/content/manuals/docker-hub/download-rate-limit.md +++ /dev/null @@ -1,381 +0,0 @@ ---- -description: Learn about usage and rate limits for Docker Hub. -keywords: Docker Hub, pulls, download, limit, usage, storage -title: Docker Hub usage and rate limits -linkTitle: Usage and rate limits -weight: 30 ---- - -Docker may impose usage and rate limits for Docker Hub to ensure fair resource -consumption and maintain service quality. Understanding your usage helps you -manage your and your organization's usage effectively. - -## Usage - -Usage refers to both: -- Pulls: The amount of data transferred from Docker Hub -- Storage: The amount of data stored on Docker Hub - -### Fair use - -When utilizing the Docker Platform, users should be aware that excessive data -transfer, pull rates, or data storage can lead to throttling, or additional -charges. To ensure fair resource usage and maintain service quality, we reserve -the right to impose restrictions or apply additional charges to accounts -exhibiting excessive data and storage consumption. - -### View Docker Hub usage - -1. Sign in to [Docker Hub](https://hub.docker.com). - - If you want to download usage for all members of an organization, you must - sign in to an account that is an owner for that organization. Otherwise, - you can only view your own personal usage. - -2. In Docker Hub, select **Usage** from the top-level navigation menu. -3. In the drop-down, select whether you want to view your personal or organization data. -4. The usage page displays **Pulls** and **Storage** usage. -5. Select a usage type and use the available filters to view usage. - -### Download Docker Hub pulls usage - -You can download a CSV file of your or your organization's Docker Hub pulls usage. You can't download a CSV file for storage usage. - -To download the pulls usage file: - -1. Sign in to [Docker Hub](https://hub.docker.com). - - If you want to download usage for all members of an organization, you must - sign in to an account that is an owner for that organization. Otherwise, - you can only view your own personal usage. - -2. In Docker Hub, select **Usage** from the top-level navigation menu. -3. In the drop-down, select whether you want to view your personal or organization data. -4. Optional. Use the **Filter by privacy** drop-down to select **Public** or **Private** pulls. -5. Use the **From** and **To** filters to select a date range for the data. -6. Select **Send report to email** to have Docker email you a link to the data - file. Note that email processing time may vary. - -The file contains the following comma separated values: - -| CSV column | Definition | Usage guidance | -|----------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `datehour` | The date and hour (`yyyy/mm/dd/hh`) of the pull that resulted in the data transfer. | This helps in identifying peak usage times and patterns. | -| `user_name` | The Docker ID of the user that pulled the image | This lets organization owners track data consumption per user and manage resources effectively. | -| `repository` | The name of the repository of the image that was pulled. | This lets you identify which repositories are most frequently accessed and consume most of the data transfer. | -| `access_token_name` | Name of the access token that was used for authentication with Docker CLI. `generated` tokens are automatically generated by the Docker client when a user signs in. | Personal access tokens are usually used to authenticate automated tools (Docker Desktop, CI/CD tools, etc.). This is useful for identifying which automated system issued the pull. | -| `ips` | The IP address that was used to pull the image. This field is aggregated, so more than one IP address may appear, representing all the IPs used to pull an image within the same date and hour. | This helps you understand the origin of the data transfer, which is useful for diagnosing and identifying patterns in automated or manual pulls. | -| `repository_privacy` | The privacy state of the image repository that was pulled. This can either be `public` or `private`. | This distinguishes between public and private repositories to identify which data transfer threshold the pull impacts. | -| `tag` | The tag for the image. The tag is only available if the pull included a tag. | This helps in identifying the image. Tags are often used to identify specific versions or variants of an image. | -| `digest` | The unique image digest for the image. | This helps in identifying the image. | -| `version_checks` | The number of version checks accumulated for the date and hour of each image repository. Depending on the client, a pull can do a version check to verify the existence of an image or tag without downloading it. | This helps identify the frequency of version checks, which you can use to analyze usage trends and potential unexpected behaviors. | -| `pulls` | The number of pulls accumulated for the date and hour of each image repository. | This helps identify the frequency of repository pulls, which you can use to analyze usage trends and potential unexpected behaviors. | - -### Best practices for managing Docker Hub usage - -Use the following steps to help optimize and manage your Docker Hub usage for -both individuals and organizations: - -1. [View your Docker Hub usage](#view-docker-hub-usage). - -2. Use the Docker Hub usage data to identify which accounts consume the most - data, determine peak usage times, and identify which images are related to - the most data usage. In addition, look for usage trends, such as the - following: - - - Inefficient pull behavior: Identify frequently accessed repositories to - assess whether you can optimize caching practices or consolidate usage to - reduce pulls. - - Inefficient automated systems: Check which automated tools, such as CI/CD - pipelines, may be causing higher pull rates, and configure them to avoid - unnecessary image pulls. - -3. Optimize image pulls by: - - - Use caching: Implement local image caching via - [mirroring](/docker-hub/mirror/) or within your CI/CD pipelines to reduce - redundant pulls. - - Automate manual workflows: Avoid unnecessary pulls by configuring automated - systems to pull only when a new version of an image is available. - -4. Optimize your storage by: - - - Regularly audit and remove repositories with untagged, unused, or outdated images. - - Look for private repositories in Hub storage that exceed your plan's limits. - -5. Increase your limits by upgrading or purchasing additional consumption. For - details, see [Scale your subscription](../subscription/scale.md). - -6. For organizations, monitor and enforce organizational policies by doing the - following: - - - Routinely [view Docker Hub usage](#view-docker-hub-usage) to monitor usage. - - [Enforce sign-in](/security/for-admins/enforce-sign-in/) to ensure that you - can monitor the usage of your users and users receive higher usage limits. - - Look for duplicate user accounts in Docker and remove accounts from your organization - as needed. - -## Storage and repository limits - -{{< include "hub-limits.md" >}} - -The following storage and repository limits apply based on your subscription, subject to fair use: - -| Plan | Public repositories | Public repository storage | Private repositories | Private repository storage | -|----------|---------------------|---------------------------|----------------------------|----------------------------| -| Personal | Unlimited | Unlimited | Up to 1 private repository | Up to 2 GB | -| Pro | Unlimited | Unlimited | Unlimited | Up to 5 GB | -| Team | Unlimited | Unlimited | Unlimited | Up to 50 GB | -| Business | Unlimited | Unlimited | Unlimited | Up to 500 GB | - - -Private repository storage is calculated on a monthly basis based on the average -storage used throughout the month per organization. Docker measures your storage -usage in the amount of Bytes stored per hour, which are accumulated throughout -the month to determine your monthly storage. If a repository is private at any -point within an hour, it is counted as private for the full hour. The total -hours are calculated based on the actual number of days in the month. Any -storage usage beyond the included amounts in each paid subscription tier will be -charged at an on-demand rate. You can [scale your -limit](../subscription/scale.md) or [upgrade](../subscription/change.md) to get -a higher limit. - -For more information on how Docker Hub storage pricing is calculated, see the [Docker Hub storage pricing](/manuals/billing/docker-hub-pricing.md) guide. - -## Pull limit and rate limit - -A pull is defined as the following: - - - A Docker pull includes both a version check and any download that - occurs as a result of the pull. Depending on the client, a `docker pull` can - verify the existence of an image or tag without downloading it by performing - a version check. - - Version checks do not count towards usage pricing. - - A pull for a normal image makes one pull for a [single - manifest](https://github.com/opencontainers/image-spec/blob/main/manifest.md). - - A pull for a multi-arch image will count as one pull for each - different architecture. - -### Pull attribution - -Pulls can be attributed to either a personal or organization [namespace](https://docs.docker.com/contribute/style/terminology/#namespace). - -#### Private pulls - -Pulls for private repositories are attributed to the repository's namespace owner. - -#### Public pulls - -When pulling images from a public repository, attribution is determined based on domain affiliation and organization membership. - -#### Verified domain ownership - -When pulling an image from an account linked to a verified domain, the attribution is set to be the owner of that [domain](https://docs.docker.com/security/faqs/single-sign-on/domain-faqs/) - -#### Single organization membership - -- If the owner of the verified domain is a company and the user is part of only one organization within that [company](https://docs.docker.com/admin/faqs/company-faqs/#what-features-are-supported-at-the-company-level), the pull is attributed to that specific organization. -- If the user is part of only one organization, the pull is attributed to that specific organization. - -#### Multiple organization memberships - -If the user is part of multiple organizations under the company, the pull is attributed to the user's personal namespace. - -### Pull limit - -Rate limits apply to pull usage. A user's rate limit is equal to the highest entitlement of their personal account or any organization they belong to. To take advantage of this, you must sign in to [Docker Hub](https://hub.docker.com/) as an authenticated user. For -more information, see [How do I authenticate pull -requests](#how-do-i-authenticate-pulls). Unauthenticated (anonymous) -users will have the limits enforced via IP. - -Pull limit refers to the total number of image pulls allowed within a specific time frame. For example, a Business plan has a total pull limit of 1M pulls per month. - -The pull limit is calculated on a per month basis and only applies to -Docker Pro, Docker Team, and Docker Business users. The limit automatically -scales on-demand, but on-demand rates do apply. You can [scale your -limit](../subscription/scale.md) or [upgrade](../subscription/change.md) to get -a higher limit. - - -| User type | Pull count per month | -|--------------------------|----------------------| -| Business (authenticated) | 1M | -| Team (authenticated) | 100K | -| Pro (authenticated) | 25K | -| Personal (authenticated) | Not applicable | -| Unauthenticated users | Not applicable | - -### Pull rate limit - -Pull rate limit refers to the frequency of image pulls per unit of time, specifying how quickly you can pull images within a specific time. - -The pull rate limit is calculated on a per hour basis. There is no -pull rate limit for users or automated systems with a paid subscription. -Unauthenticated and Docker Personal users using Docker Hub will experience rate -limits on image pulls. - -The following table describes the pull rate limit per hour -for each subscription tier, subject to fair use: - -| User type | Pull rate limit per hour | -|--------------------------|--------------------------| -| Business (authenticated) | No limit | -| Team (authenticated) | No limit | -| Pro (authenticated) | No limit | -| Personal (authenticated) | 40 | -| Unauthenticated users | 10 per IP address | - -> [!TIP] -> -> Always sign in to Docker. Authenticated Docker Personal users receive -> increased rate limits, while authenticated Docker Pro, Team, and Business -> users are not rate limited. For more information, see [How do I authenticate -> pulls](#how-do-i-authenticate-pulls). - -#### How do I know my pulls are being limited? - -When you issue a pull and you are over the limit, Docker Hub returns a -`429` response code with the following body when the manifest is requested: - -```text -You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limits -``` - -This error message appears in the Docker CLI or in the Docker Engine logs. - -#### How can I check my current rate? - -Valid API requests to Hub usually include the following rate limit headers in -the response: - -```text -ratelimit-limit -ratelimit-remaining -docker-ratelimit-source -``` - -These headers are returned on both GET and HEAD requests. - -> [!NOTE] -> -> Using GET emulates a real pull and counts towards the limit. Using HEAD won't. -> To check your limits, you need `curl`, `grep`, and `jq` installed. - -To get a token anonymously, if you are pulling anonymously: - -```console -$ TOKEN=$(curl "https://auth.docker.io/token?service=registry.docker.io&scope=repository:ratelimitpreview/test:pull" | jq -r .token) -``` - -To get a token with a user account, if you are authenticated (insert your -username and password in the following command): - -```console -$ TOKEN=$(curl --user 'username:password' "https://auth.docker.io/token?service=registry.docker.io&scope=repository:ratelimitpreview/test:pull" | jq -r .token) -``` - -Then to get the headers showing your limits, run the following: - -```console -$ curl --head -H "Authorization: Bearer $TOKEN" https://registry-1.docker.io/v2/ratelimitpreview/test/manifests/latest -``` - -Which should return the following headers: - -```http -ratelimit-limit: 100;w=21600 -ratelimit-remaining: 76;w=21600 -docker-ratelimit-source: 192.0.2.1 -``` - -In the previous example, the pull limit is 100 pulls per 21600 seconds (6 -hours), and there are 76 pulls remaining. - -If you don't see any RateLimit header, it could be because the image or your IP -is unlimited in partnership with a publisher, provider, or an open source -organization. It could also mean that the user you are pulling as is part of a -paid Docker plan. Pulling that image won’t count toward pull rate limits if you -don't see these headers. Note that users with a paid subscription have a monthly -pull limit that can be viewed in the Docker Hub [usage dashboard](#view-docker-hub-usage). - -#### I'm being limited to a lower rate even though I have a paid Docker subscription - -To take advantage of the unlimited limits included in a paid Docker subscription, -you must [authenticate pulls](#how-do-i-authenticate-pulls) with your -user account. - -A Pro, Team, or a Business tier doesn't increase limits on your images for other -users. See Docker's [Open -Source](https://www.docker.com/blog/expanded-support-for-open-source-software-projects/), -[Publisher](https://www.docker.com/partners/programs), or [Large -Organization](https://www.docker.com/pricing) offerings. - -#### Other limits - -Docker Hub also has an abuse rate limit to protect the application and -infrastructure. This limit applies to all requests to Hub properties including -web pages, APIs, and image pulls. The limit is applied per-IP, and while the -limit changes over time depending on load and other factors, it's in the order -of thousands of requests per minute. The abuse limit applies to all users -equally regardless of account level. - -You can differentiate between these limits by looking at the error code. The -"abuse limit" returns a simple `429 Too Many Requests` response. The pull -limit returns a longer error message that includes a link to this page. - -### How do I authenticate pulls? - -The following section contains information on how to sign in to Docker Hub to -authenticate pulls. - -#### Docker Desktop - -If you are using Docker Desktop, you can sign in to Docker Hub from the Docker -Desktop menu. - -Select **Sign in / Create Docker ID** from the Docker Desktop menu and follow -the on-screen instructions to complete the sign-in process. - -#### Docker Engine - -If you're using a standalone version of Docker Engine, run the `docker login` -command from a terminal to authenticate with Docker Hub. For information on how -to use the command, see [docker login](/reference/cli/docker/login.md). - -#### Docker Swarm - -If you're running Docker Swarm, you must use the `--with-registry-auth` flag to -authenticate with Docker Hub. For more information, see [Create a -service](/reference/cli/docker/service/create.md#with-registry-auth). If you -are using a Docker Compose file to deploy an application stack, see [docker -stack deploy](/reference/cli/docker/stack/deploy.md). - -#### GitHub Actions - -If you're using GitHub Actions to build and push Docker images to Docker Hub, -see [login action](https://github.com/docker/login-action#dockerhub). If you are -using another Action, you must add your username and access token in a similar -way for authentication. - -#### Kubernetes - -If you're running Kubernetes, follow the instructions in [Pull an Image from a -Private -Registry](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/) -for information on authentication. - -#### Third-party platforms - -If you're using any third-party platforms, follow your provider’s instructions on using registry authentication. - -- [Artifactory](https://www.jfrog.com/confluence/display/JFROG/Advanced+Settings#AdvancedSettings-RemoteCredentials) -- [AWS CodeBuild](https://aws.amazon.com/blogs/devops/how-to-use-docker-images-from-a-private-registry-in-aws-codebuild-for-your-build-environment/) -- [AWS ECS/Fargate](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/private-auth.html) -- [Azure Pipelines](https://docs.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints?view=azure-devops&tabs=yaml#sep-docreg) -- [Chipper CI](https://docs.chipperci.com/builds/docker/#rate-limit-auth) -- [CircleCI](https://circleci.com/docs/2.0/private-images/) -- [Codefresh](https://codefresh.io/docs/docs/docker-registries/external-docker-registries/docker-hub/) -- [Drone.io](https://docs.drone.io/pipeline/docker/syntax/images/#pulling-private-images) -- [GitLab](https://docs.gitlab.com/ee/user/packages/container_registry/#authenticate-with-the-container-registry) -- [LayerCI](https://layerci.com/docs/advanced-workflows#logging-in-to-docker) -- [TeamCity](https://www.jetbrains.com/help/teamcity/integrating-teamcity-with-docker.html#Conforming+with+Docker+download+rate+limits) diff --git a/content/manuals/docker-hub/image-library/mirror.md b/content/manuals/docker-hub/image-library/mirror.md index 7b993055cc9..94a87e866b3 100644 --- a/content/manuals/docker-hub/image-library/mirror.md +++ b/content/manuals/docker-hub/image-library/mirror.md @@ -38,7 +38,7 @@ Hub can be mirrored. > [!NOTE] > -> Mirrors of Docker Hub are still subject to Docker's [fair use policy](/manuals/docker-hub/download-rate-limit.md#fair-use). +> Mirrors of Docker Hub are still subject to Docker's [fair use policy](/manuals/docker-hub/usage/_index.md#fair-use). ### Solution diff --git a/content/manuals/docker-hub/release-notes.md b/content/manuals/docker-hub/release-notes.md index 69dc8b48d2a..18efecb5d33 100644 --- a/content/manuals/docker-hub/release-notes.md +++ b/content/manuals/docker-hub/release-notes.md @@ -199,7 +199,7 @@ Docker introduces Hub Vulnerability Scanning which enables you to automatically * Docker has announced a new, per-seat pricing model to accelerate developer workflows for cloud-native development. The previous private repository/concurrent autobuild-based plans have been replaced with new **Pro** and **Team** plans that include unlimited private repositories. For more information, see [Docker subscription](../subscription/_index.md). -* Docker has enabled download rate limits for downloads and pull requests on Docker Hub. This caps the number of objects that users can download within a specified timeframe. For more information, see [Download rate limit](download-rate-limit.md). +* Docker has enabled download rate limits for downloads and pull requests on Docker Hub. This caps the number of objects that users can download within a specified timeframe. For more information, see [Usage and limits](/manuals/docker-hub/usage/_index.md). ## 2019-11-04 diff --git a/content/manuals/docker-hub/usage/_index.md b/content/manuals/docker-hub/usage/_index.md new file mode 100644 index 00000000000..f7311adf01c --- /dev/null +++ b/content/manuals/docker-hub/usage/_index.md @@ -0,0 +1,56 @@ +--- +description: Learn about usage and limits for Docker Hub. +keywords: Docker Hub, limit, usage +title: Docker Hub usage and limits +linkTitle: Usage and limits +weight: 30 +aliases: + /docker-hub/download-rate-limit/ +--- + +{{< include "hub-limits.md" >}} + +When using Docker Hub, unauthenticated and Docker Personal users are subject to +strict limits. In contrast, Docker Pro, Team, and Business users benefit from a +consumption-based model with a base amount of included usage. This included +usage is not a hard limit; users can scale or upgrade their subscriptions to +receive additional usage or use on-demand usage. + +The following table provides an overview of the included usage and limits for each +user type, subject to fair use: + + +| User type | Pulls per month | Pull rate limit per hour | Public repositories | Public repository storage | Private repositories | Private repository storage | +|--------------------------|-----------------|--------------------------|---------------------|---------------------------|----------------------|----------------------------| +| Business (authenticated) | 1M | Unlimited | Unlimited | Unlimited | Unlimited | Up to 500 GB | +| Team (authenticated) | 100K | Unlimited | Unlimited | Unlimited | Unlimited | Up to 50 GB | +| Pro (authenticated) | 25K | Unlimited | Unlimited | Unlimited | Unlimited | Up to 5 GB | +| Personal (authenticated) | Not applicable | 40 | Unlimited | Unlimited | Up to 1 | Up to 2 GB | +| Unauthenticated users | Not applicable | 10 per IP address | Not applicable | Not applicable | Not applicable | Not applicable | + +For more details, see the following: + +- [Pull usage and limits](./pulls.md) +- [Storage usage and limits](./storage.md) + +## Fair use + +When utilizing the Docker Platform, users should be aware that excessive data +transfer, pull rates, or data storage can lead to throttling, or additional +charges. To ensure fair resource usage and maintain service quality, we reserve +the right to impose restrictions or apply additional charges to accounts +exhibiting excessive data and storage consumption. + +### Abuse rate limit + +Docker Hub has an abuse rate limit to protect the application and +infrastructure. This limit applies to all requests to Hub properties including +web pages, APIs, and image pulls. The limit is applied per-IP, and while the +limit changes over time depending on load and other factors, it's in the order +of thousands of requests per minute. The abuse limit applies to all users +equally regardless of account level. + +You can differentiate between the pull rate limit and abuse rate limit by +looking at the error code. The abuse limit returns a simple `429 Too Many +Requests` response. The pull limit returns a longer error message that includes +a link to documentation. diff --git a/content/manuals/docker-hub/usage/manage.md b/content/manuals/docker-hub/usage/manage.md new file mode 100644 index 00000000000..dfe81db7c55 --- /dev/null +++ b/content/manuals/docker-hub/usage/manage.md @@ -0,0 +1,49 @@ +--- +description: Learn how to optimize and manage your Docker Hub usage. +keywords: Docker Hub, limit, usage +title: Best practices for optimizing Docker Hub usage +linkTitle: Optimize usage +weight: 40 +--- + +Use the following steps to help optimize and manage your Docker Hub usage for +both individuals and organizations: + +1. [View your Docker Hub usage](https://hub.docker.com/usage). + +2. Use the Docker Hub usage data to identify which accounts consume the most + data, determine peak usage times, and identify which images are related to + the most data usage. In addition, look for usage trends, such as the + following: + + - Inefficient pull behavior: Identify frequently accessed repositories to + assess whether you can optimize caching practices or consolidate usage to + reduce pulls. + - Inefficient automated systems: Check which automated tools, such as CI/CD + pipelines, may be causing higher pull rates, and configure them to avoid + unnecessary image pulls. + +3. Optimize image pulls by: + + - Use caching: Implement local image caching via + [mirroring](/docker-hub/mirror/) or within your CI/CD pipelines to reduce + redundant pulls. + - Automate manual workflows: Avoid unnecessary pulls by configuring automated + systems to pull only when a new version of an image is available. + +4. Optimize your storage by: + + - Regularly audit and remove repositories with untagged, unused, or outdated images. + - Look for private repositories in Hub storage that exceed your plan's limits. + +5. Increase your limits by upgrading or purchasing additional consumption. For + details, see [Scale your subscription](/manuals/subscription/scale.md). + +6. For organizations, monitor and enforce organizational policies by doing the + following: + + - Routinely [view Docker Hub usage](https://hub.docker.com/usage) to monitor usage. + - [Enforce sign-in](/security/for-admins/enforce-sign-in/) to ensure that you + can monitor the usage of your users and users receive higher usage limits. + - Look for duplicate user accounts in Docker and remove accounts from your organization + as needed. \ No newline at end of file diff --git a/content/manuals/docker-hub/usage/pulls.md b/content/manuals/docker-hub/usage/pulls.md new file mode 100644 index 00000000000..7ca07a6b560 --- /dev/null +++ b/content/manuals/docker-hub/usage/pulls.md @@ -0,0 +1,218 @@ +--- +description: Learn about pull usage and limits for Docker Hub. +keywords: Docker Hub, pulls, usage, limit +title: Docker Hub pull usage and limits +linkTitle: Pulls +weight: 10 +--- + +{{< include "hub-limits.md" >}} + +Unauthenticated and Docker Personal users are subject to hourly pull rate limits +on Docker Hub. In contrast, Docker Pro, Team, and Business users benefit from a +base number of included pulls per month without hourly rate restrictions. This +included usage is flexible, allowing you to scale or upgrade your subscription +to accommodate additional pulls or utilize on-demand pulls as needed. + +Any pulls exceeding the included amounts in each subscription tier will be +charged at an on-demand rate. To increase your monthly pull allowance and avoid +on-demand charges, you can [scale](/manuals/subscription/scale.md) or +[upgrade](/manuals/subscription/change.md) your subscription. + +The following pull usage and limits apply based on your subscription, subject to +fair use: + + +| User type | Pulls per month | Pull rate limit per hour | +|--------------------------|-----------------|--------------------------| +| Business (authenticated) | 1M | Unlimited | +| Team (authenticated) | 100K | Unlimited | +| Pro (authenticated) | 25K | Unlimited | +| Personal (authenticated) | Not applicable | 40 | +| Unauthenticated Users | Not applicable | 10 per IP address | + +## Pull definition + +A pull is defined as the following: + + - A Docker pull includes both a version check and any download that + occurs as a result of the pull. Depending on the client, a `docker pull` can + verify the existence of an image or tag without downloading it by performing + a version check. + - Version checks do not count towards usage pricing. + - A pull for a normal image makes one pull for a [single + manifest](https://github.com/opencontainers/image-spec/blob/main/manifest.md). + - A pull for a multi-arch image will count as one pull for each + different architecture. + +## Pull attribution + +Pulls from authenticated users can be attributed to either a personal or an +organization +[namespace](/reference/glossary/#organization-name). + +Attribution is based on the following: + +- Private pulls: Pulls for private repositories are attributed to the + repository's namespace owner. +- Public pulls: When pulling images from a public repository, attribution is + determined based on domain affiliation and organization membership. +- Verified domain ownership: When pulling an image from an account linked to a + verified domain, the attribution is set to be the owner of that + [domain](/manuals/security/faqs/single-sign-on/domain-faqs.md). +- Single organization membership: + - If the owner of the verified domain is a company and the user is part of + only one organization within that + [company](/manuals/admin/faqs/company-faqs.md/#what-features-are-supported-at-the-company-level), + the pull is attributed to that specific organization. + - If the user is part of only one organization, the pull is attributed to + that specific organization. +- Multiple organization memberships: If the user is part of multiple + organizations under the company, the pull is attributed to the user's personal + namespace. + +When pulling Docker Verified Publisher images, attribution towards rate limiting +is not applied. For more details, see [Docker Verified Publisher +Program](/manuals/docker-hub/repos/manage/trusted-content/dvp-program.md). + +### Authentication + +To ensure correct attribution of your pulls, you must authenticate with Docker +Hub. The following sections provide information on how to sign in to Docker Hub +to authenticate your pulls. + +#### Docker Desktop + +If you are using Docker Desktop, you can sign in to Docker Hub from the Docker +Desktop menu. + +Select **Sign in / Create Docker ID** from the Docker Desktop menu and follow +the on-screen instructions to complete the sign-in process. + +#### Docker Engine + +If you're using a standalone version of Docker Engine, run the `docker login` +command from a terminal to authenticate with Docker Hub. For information on how +to use the command, see [docker login](/reference/cli/docker/login.md). + +#### Docker Swarm + +If you're running Docker Swarm, you must use the `--with-registry-auth` flag to +authenticate with Docker Hub. For more information, see [Create a +service](/reference/cli/docker/service/create.md#with-registry-auth). If you +are using a Docker Compose file to deploy an application stack, see [docker +stack deploy](/reference/cli/docker/stack/deploy.md). + +#### GitHub Actions + +If you're using GitHub Actions to build and push Docker images to Docker Hub, +see [login action](https://github.com/docker/login-action#dockerhub). If you are +using another Action, you must add your username and access token in a similar +way for authentication. + +#### Kubernetes + +If you're running Kubernetes, follow the instructions in [Pull an Image from a +Private +Registry](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/) +for information on authentication. + +#### Third-party platforms + +If you're using any third-party platforms, follow your provider’s instructions on using registry authentication. + +- [Artifactory](https://www.jfrog.com/confluence/display/JFROG/Advanced+Settings#AdvancedSettings-RemoteCredentials) +- [AWS CodeBuild](https://aws.amazon.com/blogs/devops/how-to-use-docker-images-from-a-private-registry-in-aws-codebuild-for-your-build-environment/) +- [AWS ECS/Fargate](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/private-auth.html) +- [Azure Pipelines](https://docs.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints?view=azure-devops&tabs=yaml#sep-docreg) +- [Chipper CI](https://docs.chipperci.com/builds/docker/#rate-limit-auth) +- [CircleCI](https://circleci.com/docs/2.0/private-images/) +- [Codefresh](https://codefresh.io/docs/docs/docker-registries/external-docker-registries/docker-hub/) +- [Drone.io](https://docs.drone.io/pipeline/docker/syntax/images/#pulling-private-images) +- [GitLab](https://docs.gitlab.com/ee/user/packages/container_registry/#authenticate-with-the-container-registry) +- [LayerCI](https://layerci.com/docs/advanced-workflows#logging-in-to-docker) +- [TeamCity](https://www.jetbrains.com/help/teamcity/integrating-teamcity-with-docker.html#Conforming+with+Docker+download+rate+limits) + +## View monthly pulls and included usage + +You can view your monthly pulls on the [Usage page](https://hub.docker.com/usage/pulls) in Docker Hub. + +On that page, you can also send a report to your email that contains a comma +separated file with the following detailed information. + +| CSV column | Definition | Usage guidance | +|----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `datehour` | The date and hour (`yyyy/mm/dd/hh`) of the pull that resulted in the data transfer. | This helps in identifying peak usage times and patterns. | +| `user_name` | The Docker ID of the user that pulled the image | This lets organization owners track data consumption per user and manage resources effectively. | +| `repository` | The name of the repository of the image that was pulled. | This lets you identify which repositories are most frequently accessed and consume most of the data transfer. | +| `access_token_name` | Name of the access token that was used for authentication with Docker CLI. `generated` tokens are automatically generated by the Docker client when a user signs in. | Personal access tokens are usually used to authenticate automated tools (Docker Desktop, CI/CD tools, etc.). This is useful for identifying which automated system issued the pull. | +| `ips` | The IP address that was used to pull the image. This field is aggregated, so more than one IP address may appear, representing all the IPs used to pull an image within the same date and hour. | This helps you understand the origin of the data transfer, which is useful for diagnosing and identifying patterns in automated or manual pulls. | +| `repository_privacy` | The privacy state of the image repository that was pulled. This can either be `public` or `private`. | This distinguishes between public and private repositories to identify which data transfer threshold the pull impacts. | +| `tag` | The tag for the image. The tag is only available if the pull included a tag. | This helps in identifying the image. Tags are often used to identify specific versions or variants of an image. | +| `digest` | The unique image digest for the image. | This helps in identifying the image. | +| `version_checks` | The number of version checks accumulated for the date and hour of each image repository. Depending on the client, a pull can do a version check to verify the existence of an image or tag without downloading it. | This helps identify the frequency of version checks, which you can use to analyze usage trends and potential unexpected behaviors. | +| `pulls` | The number of pulls accumulated for the date and hour of each image repository. | This helps identify the frequency of repository pulls, which you can use to analyze usage trends and potential unexpected behaviors. | + + +## View hourly pull rate and limit + +The pull rate limit is calculated on a per hour basis. There is no pull rate +limit for users or automated systems with a paid subscription. Unauthenticated +and Docker Personal users using Docker Hub will experience rate limits on image +pulls. + +When you issue a pull and you are over the limit, Docker Hub returns a +`429` response code with the following body when the manifest is requested: + +```text +You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limits +``` + +This error message appears in the Docker CLI or in the Docker Engine logs. + +To view your current pull rate and limit: + +> [!NOTE] +> +> To check your limits, you need `curl`, `grep`, and `jq` installed. + +1. Get a token. + + - To get a token anonymously, if you are pulling anonymously: + + ```console + $ TOKEN=$(curl "https://auth.docker.io/token?service=registry.docker.io&scope=repository:ratelimitpreview/test:pull" | jq -r .token) + ``` + + - To get a token with a user account, if you are authenticated (insert your + username and password in the following command): + + ```console + $ TOKEN=$(curl --user 'username:password' "https://auth.docker.io/token?service=registry.docker.io&scope=repository:ratelimitpreview/test:pull" | jq -r .token) + ``` + +2. Get the headers that contain your limits. These headers are returned on both + GET and HEAD requests. Using GET emulates a real pull and counts towards the + limit. Using HEAD won't. + + + ```console + $ curl --head -H "Authorization: Bearer $TOKEN" https://registry-1.docker.io/v2/ratelimitpreview/test/manifests/latest + ``` + +3. Examine the headers. You should see the following headers. + + ```text + ratelimit-limit: 100;w=21600 + ratelimit-remaining: 76;w=21600 + docker-ratelimit-source: 192.0.2.1 + ``` + + In the previous example, the pull limit is 100 pulls per 21600 seconds (6 + hours), and there are 76 pulls remaining. + + If you don't see any `ratelimit` header, it could be because the image or your IP + is unlimited in partnership with a publisher, provider, or an open source + organization. It could also mean that the user you are pulling as is part of a + paid Docker plan. Pulling that image won't count toward pull rate limits if you + don't see these headers. \ No newline at end of file diff --git a/content/manuals/docker-hub/usage/storage.md b/content/manuals/docker-hub/usage/storage.md new file mode 100644 index 00000000000..0c8c7a4c65a --- /dev/null +++ b/content/manuals/docker-hub/usage/storage.md @@ -0,0 +1,27 @@ +--- +description: Learn about storage usage limits for Docker Hub. +keywords: Docker Hub, usage, storage, repository +title: Docker Hub storage usage and limits +linkTitle: Storage +weight: 20 +--- + +{{< include "hub-limits.md" >}} + +The following storage and repository limits apply based on your subscription, subject to fair use: + +| Plan | Public repositories | Public repository storage | Private repositories | Private repository storage | +|----------|---------------------|---------------------------|----------------------------|----------------------------| +| Personal | Unlimited | Unlimited | Up to 1 private repository | Up to 2 GB | +| Pro | Unlimited | Unlimited | Unlimited | Up to 5 GB | +| Team | Unlimited | Unlimited | Unlimited | Up to 50 GB | +| Business | Unlimited | Unlimited | Unlimited | Up to 500 GB | + +Any storage usage beyond the included amounts in each paid subscription tier +will be charged at an on-demand rate. For more details about storage +calcultations and billing, see [Docker Hub storage +pricing](/manuals/billing/docker-hub-pricing.md). + +## View storage usage and repositories + +You can view your storage usage on the [Usage page](https://hub.docker.com/usage/storage) in Docker Hub. diff --git a/content/manuals/subscription/details.md b/content/manuals/subscription/details.md index 27952d45c13..b4158c92bf0 100644 --- a/content/manuals/subscription/details.md +++ b/content/manuals/subscription/details.md @@ -183,7 +183,7 @@ Legacy Docker Pro includes: - Unlimited [collaborators](/docker-hub/repos/manage/access/#collaborators) for public repositories at no cost per month. - Access to [Legacy Docker Scout Free](#legacy-docker-scout-free) to get started with software supply chain security. - Unlimited private repositories -- 5000 image [pulls per day](/docker-hub/download-rate-limit/) +- 5000 image [pulls per day](/manuals/docker-hub/usage/pulls.md) - [Auto Builds](/docker-hub/builds/) with 5 concurrent builds - 300 [Vulnerability Scans](/docker-hub/vulnerability-scanning/) @@ -215,7 +215,7 @@ Legacy Docker Team includes: - Unlimited teams - [Auto Builds](/docker-hub/builds/) with 15 concurrent builds - Unlimited [Vulnerability Scanning](/docker-hub/vulnerability-scanning/) -- 5000 image [pulls per day](/docker-hub/download-rate-limit/) for each team member +- 5000 image [pulls per day](/manuals/docker-hub/usage/pulls.md) for each team member There are also advanced collaboration and management tools, including organization and team management with [Role Based Access Control (RBAC)](/security/for-admins/roles-and-permissions/), [activity logs](/admin/organization/activity-logs/), and more. diff --git a/content/reference/api/hub/latest.yaml b/content/reference/api/hub/latest.yaml index 7d0099ab8a6..86d2f3b30dc 100644 --- a/content/reference/api/hub/latest.yaml +++ b/content/reference/api/hub/latest.yaml @@ -39,7 +39,7 @@ tags: The `X-Retry-After` header is a unix timestamp of when you can call the API again. **Note**: These rate limits are separate from anti-abuse and Docker Hub download, or pull rate limiting. - To learn more about Docker Hub pull rate limiting, see [Docker Hub download rate limit](https://docs.docker.com/docker-hub/download-rate-limit/). + To learn more about Docker Hub pull rate limiting, see [Usage and limits](https://docs.docker.com/docker-hub/usage/). - name: authentication x-displayName: Authentication description: |