Skip to content

Latest commit

 

History

History
177 lines (146 loc) · 6.87 KB

pagination.adoc

File metadata and controls

177 lines (146 loc) · 6.87 KB
Raw

REST Design - Pagination

{MUST} support pagination

Access to lists of data items must support pagination to protect the service against overload as well as for best client side iteration and batch processing experience. This holds true for all lists that are (potentially) larger than just a few hundred entries.

There are two well known page iteration techniques:

The technical conception of pagination should also consider user experience related issues. As mentioned in this article, jumping to a specific page is far less used than navigation via {next}/{prev} page links (See {SHOULD} use pagination links where applicable). This favours cursor-based over offset-based pagination.

Note: To provide a consistent look and feel of pagination patterns, you must stick to the common query parameter names defined in [137].

{SHOULD} prefer cursor-based pagination, avoid offset-based pagination

Cursor-based pagination is usually better and more efficient when compared to offset-based pagination. Especially when it comes to high-data volumes and/or storage in NoSQL databases.

Before choosing cursor-based pagination, consider the following trade-offs:

  • Usability/framework support:

    • Offset-based pagination is more widely known than cursor-based pagination, so it has more framework support and is easier to use for API clients

  • Use case - jump to a certain page:

    • If jumping to a particular page in a range (e.g., 51 of 100) is really a required use case, cursor-based navigation is not feasible.

  • Data changes may lead to anomalies in result pages:

    • Offset-based pagination may create duplicates or lead to missing entries if rows are inserted or deleted between two subsequent paging requests.

    • If implemented incorrectly, cursor-based pagination may fail when the cursor entry has been deleted before fetching the pages.

  • Performance considerations - efficient server-side processing using offset-based pagination is hardly feasible for:

    • Very big data sets, especially if they cannot reside in the main memory of the database.

    • Sharded or NoSQL databases.

  • Cursor-based navigation may not work if you need the total count of results.

The {cursor} used for pagination is an opaque pointer to a page, that must never be inspected or constructed by clients. It usually encodes (encrypts) the page position, i.e. the identifier of the first or last page element, the pagination direction, and the applied query filters - or a hash over these - to safely recreate the collection (see also [cursor-based-pagination]).

{SHOULD} use pagination response page object

For iterating over collections (result sets) we propose to either use cursors (see {SHOULD} prefer cursor-based pagination, avoid offset-based pagination) or simple hypertext controls (see {SHOULD} use pagination links where applicable). To implement these in a consistent way, we have defined a response page object pattern with the following field semantics:

  • {self}:the link or cursor in a pagination response or object pointing to the same collection object or page.

  • {first}: the link or cursor in a pagination response or object pointing to the first collection object or page.

  • {prev}: the link or cursor in a pagination response or object pointing to the previous collection object or page.

  • {next}: the link or cursor in a pagination response or object pointing to the next collection object or page.

  • {last}: the link or cursor in a pagination response or object pointing to the last collection object or page.

Pagination responses should contain the following additional array field to transport the page content:

  • {items}: array of resources, holding all the items of the current page ({items} may be replaced by a resource name).

To simplify user experience, the applied query filters may be returned using the following field (see also {GET-with-body}):

  • {query}: object containing the query filters applied in the search request to filter the collection resource.

As Result, the standard response page using cursors or pagination links may be defined as follows:

ResponsePage:
  type: object
  required:
    - items
    - next
  properties:
    self:
      description: Pagination link|cursor pointing to the current page.
      type: string
      format: uri|cursor
    first:
      description: Pagination link|cursor pointing to the first page.
      type: string
      format: uri|cursor
    prev:
      description: Pagination link|cursor pointing to the previous page.
      type: string
      format: uri|cursor
    next:
      description: Pagination link|cursor pointing to the next page.
      type: string
      format: uri|cursor
    last:
      description: Pagination link|cursor pointing to the last page.
      type: string
      format: uri|cursor

    query:
      description: >
        Object containing the query filters applied to the collection resource.
      type: object
      properties: ...

    items:
      description: Array of collection items.
      type: array
      required: false
      items:
        type: ...

Note: While you may support cursors for {next}, {prev}, {first}, {last}, and {self}, it is best practice to replace these with links in favor of {SHOULD} use pagination links where applicable.

{SHOULD} use pagination links where applicable

To simplify client design, APIs should support simplified hypertext controls for paginating over collections whenever applicable as follows (see also [pagination-fields] for details):

{
  "self": "http://my-service.zalandoapis.com/resources?cursor=<self-position>",
  "first": "http://my-service.zalandoapis.com/resources?cursor=<first-position>",
  "prev": "http://my-service.zalandoapis.com/resources?cursor=<previous-position>",
  "next": "http://my-service.zalandoapis.com/resources?cursor=<next-position>",
  "last": "http://my-service.zalandoapis.com/resources?cursor=<last-position>",
  "query": {
    "query-param-<1>": ...,
    "query-param-<n>": ...
  },
  "items": [...]
}

Remark: You should avoid providing a total count unless there is a clear need to do so. Very often, there are significant system and performance implications when supporting full counts. Especially, if the data set grows and requests become complex queries and filters drive full scans. While this is an implementation detail relative to the API, it is important to consider the ability to support serving counts over the life of a service.