Remove the experimental name space from cookie parameters

CHANGES.rst

@@ -1,12 +1,31 @@
 Changes
 =======
 
-TBR
----
+0.13.0 (unreleased)
+-------------------
+
+* Cookie support is no longer experimental:
+
+  * If the ``COOKIES_ENABLED`` setting is ``True`` (default), automatic request
+    parameter mapping now sets ``responseCookies`` to ``True`` and maps request
+    cookies to ``requestCookies``.
+
+  * The ``ZYTE_API_EXPERIMENTAL_COOKIES_ENABLED`` setting is now deprecated.
+    When enabled, however, the ``experimental`` name space is still used for
+    automatic request parameter mapping.
+
+  * If you use ``requestCookies``, ``responseCookies``, or ``cookieManagement``
+    within the ``experimental`` name space in request parameters, a deprecation
+    warning is now logged.
+
+  * The ``responseCookies`` response parameter is now handled the same as
+    ``experimental.responseCookies``; the latter still works but is deprecated.
 
 * Zyte API Request IDs are now included in the error logs.
+
 * Bump the zyte-api dependency: 0.4.7 → 0.4.8.
 
+
 0.12.2 (2023-10-19)
 -------------------
 
@@ -42,7 +61,7 @@ TBR
   Experimental is treated as a namespace, and its parameters are the ones
   counted, i.e. there is no ``scrapy-zyte-api/request_args/experimental`` stat,
   but there are stats like
-  ``scrapy-zyte-api/request_args/experimental.responseCookies``.
+  ``scrapy-zyte-api/request_args/experimental.foo``.
 
 
 0.11.1 (2023-08-25)

README.rst

@@ -323,10 +323,10 @@ possible:
 
 -   ``statusCode`` becomes ``response.status``.
 
--   ``httpResponseHeaders`` and ``experimental.responseCookies`` become
+-   ``httpResponseHeaders`` and ``responseCookies`` become
     ``response.headers``.
 
--   ``experimental.responseCookies`` is also mapped into the request cookiejar.
+-   ``responseCookies`` is also mapped into the request cookiejar.
 
 -   ``browserHtml`` and ``httpResponseBody`` are mapped into both
     ``response.text`` (``str``) and ``response.body`` (``bytes``).
@@ -412,17 +412,15 @@ parameters are chosen as follows by default:
 
 -   ``Request.body`` becomes ``httpRequestBody``.
 
--   If the ``ZYTE_API_EXPERIMENTAL_COOKIES_ENABLED`` Scrapy setting is
-    ``True``, the COOKIES_ENABLED_ Scrapy setting is ``True`` (default), and
-    provided request metadata does not set dont_merge_cookies_ to ``True``:
+-   If the COOKIES_ENABLED_ Scrapy setting is ``True`` (default), and provided
+    request metadata does not set dont_merge_cookies_ to ``True``:
 
     .. _COOKIES_ENABLED: https://docs.scrapy.org/en/latest/topics/downloader-middleware.html#std-setting-COOKIES_ENABLED
     .. _dont_merge_cookies: https://docs.scrapy.org/en/latest/topics/request-response.html#std-reqmeta-dont_merge_cookies
 
-    -   ``experimental.responseCookies`` is set to ``True``.
+    -   ``responseCookies`` is set to ``True``.
 
-    -   Cookies from the request `cookie jar`_ become
-        ``experimental.requestCookies``.
+    -   Cookies from the request `cookie jar`_ become ``requestCookies``.
 
         .. _cookie jar: https://docs.scrapy.org/en/latest/topics/downloader-middleware.html#std-reqmeta-cookiejar
 
@@ -434,10 +432,10 @@ parameters are chosen as follows by default:
         If the cookies to be set exceed the limit defined in the
         ``ZYTE_API_MAX_COOKIES`` setting (100 by default), a warning is logged,
         and only as many cookies as the limit allows are set for the target
-        request. To silence this warning, set ``experimental.requestCookies``
-        manually, e.g. to an empty dict. Alternatively, if Zyte API starts
-        supporting more than 100 request cookies, update the
-        ``ZYTE_API_MAX_COOKIES`` setting accordingly.
+        request. To silence this warning, set ``requestCookies`` manually, e.g.
+        to an empty dict. Alternatively, if Zyte API starts supporting more
+        than 100 request cookies, update the ``ZYTE_API_MAX_COOKIES`` setting
+        accordingly.
 
         If you are using a custom downloader middleware to handle request
         cookiejars, you can point the ``ZYTE_API_COOKIE_MIDDLEWARE`` setting to
@@ -511,20 +509,18 @@ following parameters:
                 "value": "application/json"
             }
         ],
-        "experimental": {
-            "requestCookies": [
-                {
-                    "name": "a",
-                    "value": "b",
-                    "domain": ""
-                }
-            ],
-            "responseCookies": true
-        },
         "httpResponseBody": true,
         "httpResponseHeaders": true,
         "httpRequestBody": "eyJmb28iOiAiYmFyIn0=",
         "httpRequestMethod": "POST",
+        "requestCookies": [
+            {
+                "name": "a",
+                "value": "b",
+                "domain": ""
+            }
+        ],
+        "responseCookies": true,
         "url": "https://httpbin.org/anything"
     }
 
@@ -553,10 +549,8 @@ following parameters:
 
     {
         "browserHtml": true,
-        "experimental": {
-            "responseCookies": true
-        },
         "requestHeaders": {"referer": "https://example.com/"},
+        "responseCookies": true,
         "url": "https://quotes.toscrape.com"
     }
 
@@ -815,7 +809,7 @@ The following Zyte API parameters are *not* taken into account for request
 fingerprinting:
 
 -   Request header parameters (``customHttpRequestHeaders``,
-    ``requestHeaders``)
+    ``requestHeaders``, ``requestCookies``)
 
 -   Metadata parameters (``echoData``, ``jobId``)
 

scrapy_zyte_api/_cookies.py

@@ -30,7 +30,8 @@ def _process_cookies(
 ):
     if not cookie_jars:
         return
-    response_cookies = api_response.get("experimental", {}).get("responseCookies")
+    old_response_cookies = api_response.get("experimental", {}).get("responseCookies")
+    response_cookies = api_response.get("responseCookies", old_response_cookies)
     if not response_cookies:
         return
     cookie_jar = _get_cookie_jar(request, cookie_jars)