Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid duplicate requests #105

Commits on Jul 3, 2023

  1. Configuration menu
    Copy the full SHA
    5908b6c View commit details
    Browse the repository at this point in the history
  2. Run the provider tests on CI.

    wRAR committed Jul 3, 2023
    Configuration menu
    Copy the full SHA
    eaa3409 View commit details
    Browse the repository at this point in the history

Commits on Jul 4, 2023

  1. Merge pull request scrapy-plugins#98 from scrapy-plugins/scrapy-poet-…

    …optional
    
    Move the provider deps into an optional feature.
    kmike authored Jul 4, 2023
    Configuration menu
    Copy the full SHA
    a3f3e15 View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2023

  1. Configuration menu
    Copy the full SHA
    5ad4d79 View commit details
    Browse the repository at this point in the history
  2. pre-commit fixes

    kmike committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    914d599 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    3c81296 View commit details
    Browse the repository at this point in the history
  4. cleanup tools configuration

    * move to pyproject.toml when possible
    * use pre-commit in tox, don't duplicate configuration
    * don't run mypy in pre-commit
    * update flake8, mypy and isort
    * update github actions
    kmike committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    be25b79 View commit details
    Browse the repository at this point in the history
  5. fix black configuration

    kmike committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    edd2ed2 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    44c3a5f View commit details
    Browse the repository at this point in the history
  7. proper mypy config

    kmike committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    8fd2138 View commit details
    Browse the repository at this point in the history
  8. Update README.rst

    Co-authored-by: Adrián Chaves <[email protected]>
    kmike and Gallaecio authored Jul 12, 2023
    Configuration menu
    Copy the full SHA
    7ac7f12 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    aeca861 View commit details
    Browse the repository at this point in the history
  10. Merge pull request scrapy-plugins#101 from scrapy-plugins/cleanup-ci

    Cleanup tools configuration
    kmike authored Jul 12, 2023
    Configuration menu
    Copy the full SHA
    fd91fbf View commit details
    Browse the repository at this point in the history
  11. Merge pull request scrapy-plugins#100 from scrapy-plugins/more-data-t…

    …ypes
    
    Support more data types in the provider
    kmike authored Jul 12, 2023
    Configuration menu
    Copy the full SHA
    bd4bd6b View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    6d549b7 View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    226620c View commit details
    Browse the repository at this point in the history
  14. fix rst syntax in readme

    kmike committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    139f55e View commit details
    Browse the repository at this point in the history
  15. Merge pull request scrapy-plugins#104 from scrapy-plugins/quickstart

    add a quick start section to readme
    kmike authored Jul 12, 2023
    Configuration menu
    Copy the full SHA
    267d689 View commit details
    Browse the repository at this point in the history

Commits on Jul 13, 2023

  1. WIP

    Gallaecio committed Jul 13, 2023
    Configuration menu
    Copy the full SHA
    23c4940 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    1c7db87 View commit details
    Browse the repository at this point in the history
  3. Add release notes for 0.10.0.

    wRAR committed Jul 13, 2023
    Configuration menu
    Copy the full SHA
    4e4c829 View commit details
    Browse the repository at this point in the history

Commits on Jul 14, 2023

  1. Merge pull request scrapy-plugins#109 from scrapy-plugins/relnotes-0.…

    …10.0
    
    Add release notes for 0.10.0.
    wRAR authored Jul 14, 2023
    Configuration menu
    Copy the full SHA
    0c8b648 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    46ae308 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    24a4914 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    4361ca7 View commit details
    Browse the repository at this point in the history
  5. Merge pull request scrapy-plugins#106 from Gallaecio/trust-env

    Support driving Zyte API requests through a proxy
    kmike authored Jul 14, 2023
    Configuration menu
    Copy the full SHA
    28927c0 View commit details
    Browse the repository at this point in the history
  6. Bump version: 0.9.0 → 0.10.0

    wRAR committed Jul 14, 2023
    Configuration menu
    Copy the full SHA
    23762b6 View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2023

  1. Configuration menu
    Copy the full SHA
    febe99f View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    5a8ce24 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    561d9ff View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2023

  1. Configuration menu
    Copy the full SHA
    276c7f4 View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2023

  1. Configuration menu
    Copy the full SHA
    660cdb2 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    63a9d44 View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2023

  1. Styling improvements

    Co-authored-by: Adrián Chaves <[email protected]>
    BurnzZ and Gallaecio authored Aug 3, 2023
    Configuration menu
    Copy the full SHA
    2820d56 View commit details
    Browse the repository at this point in the history
  2. Merge pull request scrapy-plugins#114 from scrapy-plugins/max-zapi-re…

    …quests
    
    Introduce ZYTE_API_MAX_REQUESTS
    kmike authored Aug 3, 2023
    Configuration menu
    Copy the full SHA
    3c09dfc View commit details
    Browse the repository at this point in the history

Commits on Aug 4, 2023

  1. CHANGES.rst: Cover 0.11.0

    Gallaecio committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    8438007 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    1097650 View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2023

  1. Merge pull request scrapy-plugins#117 from Gallaecio/release-notes

    CHANGES.rst: Cover 0.11.0
    kmike authored Aug 7, 2023
    Configuration menu
    Copy the full SHA
    6da9a39 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    b2d251d View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    654aaba View commit details
    Browse the repository at this point in the history
  4. Fix version bumping

    Gallaecio committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    e11265e View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    30c91d0 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    d3e7115 View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2023

  1. Configuration menu
    Copy the full SHA
    5708e0e View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    75808e3 View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2023

  1. Configuration menu
    Copy the full SHA
    a330b4b View commit details
    Browse the repository at this point in the history
  2. Merge pull request scrapy-plugins#123 from scrapy-plugins/stat-status…

    …-codes
    
    Clarify how status codes are reflected in stats
    kmike authored Aug 10, 2023
    Configuration menu
    Copy the full SHA
    51823c9 View commit details
    Browse the repository at this point in the history

Commits on Aug 21, 2023

  1. Configuration menu
    Copy the full SHA
    8b1411b View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2023

  1. Merge pull request scrapy-plugins#126 from scrapy-plugins/dont-warn-o…

    …n-default-headers
    
    Do not warn about dropping the default Accept-Encoding header
    kmike authored Aug 23, 2023
    Configuration menu
    Copy the full SHA
    b8a2fa4 View commit details
    Browse the repository at this point in the history

Commits on Aug 24, 2023

  1. Configuration menu
    Copy the full SHA
    a82ccbb View commit details
    Browse the repository at this point in the history
  2. Fix a typo

    Gallaecio authored Aug 24, 2023
    Configuration menu
    Copy the full SHA
    deb1826 View commit details
    Browse the repository at this point in the history
  3. Merge pull request scrapy-plugins#127 from Gallaecio/release-notes

    Cover 0.11.1 in the release notes
    kmike authored Aug 24, 2023
    Configuration menu
    Copy the full SHA
    20046c1 View commit details
    Browse the repository at this point in the history

Commits on Aug 25, 2023

  1. Configuration menu
    Copy the full SHA
    6fd143b View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    cd47399 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    b9eb5d1 View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2023

  1. Configuration menu
    Copy the full SHA
    21926eb View commit details
    Browse the repository at this point in the history

Commits on Aug 31, 2023

  1. Merge pull request scrapy-plugins#128 from Gallaecio/request-arg-stats

    Track top-level request args in stats
    kmike authored Aug 31, 2023
    Configuration menu
    Copy the full SHA
    d61cb68 View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2023

  1. Configuration menu
    Copy the full SHA
    0282e00 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    af1e223 View commit details
    Browse the repository at this point in the history
  3. Merge pull request scrapy-plugins#133 from Gallaecio/provider-geoloca…

    …tion
    
    Implement ZYTE_API_PROVIDER_PARAMS
    kmike authored Sep 26, 2023
    Configuration menu
    Copy the full SHA
    c5f02e0 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    f2dac4e View commit details
    Browse the repository at this point in the history
  5. Merge pull request scrapy-plugins#134 from Gallaecio/release-notes

    Add release notes for 0.12.0
    kmike authored Sep 26, 2023
    Configuration menu
    Copy the full SHA
    207fed4 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    c60c48a View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    0918370 View commit details
    Browse the repository at this point in the history

Commits on Sep 28, 2023

  1. change ellipsis to 3 dots

    BurnzZ committed Sep 28, 2023
    Configuration menu
    Copy the full SHA
    b4ce6e0 View commit details
    Browse the repository at this point in the history

Commits on Sep 29, 2023

  1. Identify scrapy-zyte-api usage via custom user-agent (scrapy-plugins#130

    )
    
    * send new user_agent with api_params
    
    * add _user_agent() to utils
    
    * remove _user_agent() from handler
    
    * add user_agent to AsyncClient
    
    * extract custom ua from settings and send to client
    
    * move package name to function
    
    * add test for _user_agent()
    
    * formatting
    
    * remove old test for user_agent
    
    * add test for user_agent (for _build_client)
    
    * se USER_AGENT as constant
    
    * send user_agent to client
    
    * change order for user agent in test/formatting
    
    * change order and delimeter for user agent in client
    
    * rename _USER_AGENT
    
    * _ZYTE_API_USER_AGENT rewrites any other user-agent
    
    * add __version__.py
    
    * fetching version from __version__.py
    
    * formatting
    
    * formatting
    
    * fix changing version in bump
    
    * formatting
    
    * set version to USER_AGENT from __version__.py
    
    * Set new version
    
    Co-authored-by: Adrián Chaves <[email protected]>
    
    * fix test to use full user_agent
    
    * construct user-agent at once
    
    * using USER_AGENT as default user agent
    
    * bump zyte-api version to the latest with user-agent
    
    ---------
    
    Co-authored-by: Adrián Chaves <[email protected]>
    PyExplorer and Gallaecio authored Sep 29, 2023
    Configuration menu
    Copy the full SHA
    d50b14d View commit details
    Browse the repository at this point in the history
  2. updates for 0.12.1 release (scrapy-plugins#137)

    * updates for 0.12.1 release
    
    * Update CHANGES.rst
    
    Co-authored-by: Adrián Chaves <[email protected]>
    
    ---------
    
    Co-authored-by: Adrián Chaves <[email protected]>
    PyExplorer and Gallaecio authored Sep 29, 2023
    Configuration menu
    Copy the full SHA
    6de738a View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    1bd2e25 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    b110833 View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2023

  1. Merge pull request scrapy-plugins#136 from scrapy-plugins/ellipsis

    change ellipsis to 3 dots for ZYTE_API_LOG_REQUESTS text truncation
    kmike authored Oct 4, 2023
    Configuration menu
    Copy the full SHA
    3db30f1 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    60e2265 View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2023

  1. Configuration menu
    Copy the full SHA
    2cabbe0 View commit details
    Browse the repository at this point in the history

Commits on Oct 13, 2023

  1. Configuration menu
    Copy the full SHA
    ab34b93 View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2023

  1. Forbid multiple extractFrom.

    wRAR committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    ecc42d4 View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2023

  1. document the new behavior

    kmike committed Oct 18, 2023
    Configuration menu
    Copy the full SHA
    7f15a9c View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2023

  1. Update README.rst

    Co-authored-by: Adrián Chaves <[email protected]>
    kmike and Gallaecio authored Oct 19, 2023
    Configuration menu
    Copy the full SHA
    8ee4e24 View commit details
    Browse the repository at this point in the history
  2. Merge pull request scrapy-plugins#138 from scrapy-plugins/remove-extr…

    …action-options
    
    Remove unused extration options
    kmike authored Oct 19, 2023
    Configuration menu
    Copy the full SHA
    ad39749 View commit details
    Browse the repository at this point in the history
  3. 0.12.2 release notes

    kmike committed Oct 19, 2023
    Configuration menu
    Copy the full SHA
    bb54a28 View commit details
    Browse the repository at this point in the history
  4. Bump version: 0.12.1 → 0.12.2

    kmike committed Oct 19, 2023
    Configuration menu
    Copy the full SHA
    1370505 View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2023

  1. Configuration menu
    Copy the full SHA
    fbf37f0 View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2023

  1. Configuration menu
    Copy the full SHA
    600a571 View commit details
    Browse the repository at this point in the history

Commits on Nov 2, 2023

  1. Configuration menu
    Copy the full SHA
    80ae0a0 View commit details
    Browse the repository at this point in the history

Commits on Nov 3, 2023

  1. Use repr when logging request-id

    Co-authored-by: Adrián Chaves <[email protected]>
    BurnzZ and Gallaecio authored Nov 3, 2023
    Configuration menu
    Copy the full SHA
    63228ec View commit details
    Browse the repository at this point in the history
  2. Merge pull request scrapy-plugins#142 from scrapy-plugins/log-request-id

    log Zyte API request id on errors
    BurnzZ authored Nov 3, 2023
    Configuration menu
    Copy the full SHA
    5e205ef View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2023

  1. Configuration menu
    Copy the full SHA
    0b58307 View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2023

  1. Configuration menu
    Copy the full SHA
    438df61 View commit details
    Browse the repository at this point in the history
  2. Update README.rst

    Gallaecio authored Nov 7, 2023
    Configuration menu
    Copy the full SHA
    b73bfc2 View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2023

  1. Merge pull request scrapy-plugins#146 from Gallaecio/max-request-docs

    Clarify the max requests docs
    kmike authored Nov 8, 2023
    Configuration menu
    Copy the full SHA
    b395711 View commit details
    Browse the repository at this point in the history

Commits on Nov 14, 2023

  1. Configuration menu
    Copy the full SHA
    e55e99c View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    87bd23a View commit details
    Browse the repository at this point in the history
  3. Fix removing *Options.

    wRAR committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    4b5c088 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    aa3f77b View commit details
    Browse the repository at this point in the history
  5. Fixes and improvements.

    wRAR committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    b63ad66 View commit details
    Browse the repository at this point in the history
  6. Fix CI issues.

    wRAR committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    15fc06e View commit details
    Browse the repository at this point in the history
  7. More fixes.

    wRAR committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    08a5f48 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    e0ba154 View commit details
    Browse the repository at this point in the history
  9. Merge pull request scrapy-plugins#148 from Gallaecio/bad-auth

    Set a close reason for bad key and suspended account scenarios
    kmike authored Nov 14, 2023
    Configuration menu
    Copy the full SHA
    37e3517 View commit details
    Browse the repository at this point in the history

Commits on Nov 15, 2023

  1. Fix an old typing issue.

    wRAR committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    2d936fd View commit details
    Browse the repository at this point in the history

Commits on Nov 20, 2023

  1. Configuration menu
    Copy the full SHA
    b892453 View commit details
    Browse the repository at this point in the history

Commits on Nov 24, 2023

  1. Configuration menu
    Copy the full SHA
    493a48c View commit details
    Browse the repository at this point in the history

Commits on Nov 28, 2023

  1. Configuration menu
    Copy the full SHA
    dc09ac3 View commit details
    Browse the repository at this point in the history

Commits on Nov 30, 2023

  1. Configuration menu
    Copy the full SHA
    702cc63 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    d701f48 View commit details
    Browse the repository at this point in the history

Commits on Dec 12, 2023

  1. Configuration menu
    Copy the full SHA
    cad9c3a View commit details
    Browse the repository at this point in the history
  2. Bump the andi version.

    wRAR committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    e9be1a3 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    3ee24f0 View commit details
    Browse the repository at this point in the history
  4. Roll back capture_exceptions.

    wRAR committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    2b0d659 View commit details
    Browse the repository at this point in the history
  5. Bump the scrapy-poet version.

    wRAR committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    de01fc2 View commit details
    Browse the repository at this point in the history
  6. Bump the web-poet version.

    wRAR committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    412475e View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    5305b46 View commit details
    Browse the repository at this point in the history
  8. Add docs for ExtractFrom.

    wRAR committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    f5bb894 View commit details
    Browse the repository at this point in the history
  9. Merge pull request scrapy-plugins#141 from scrapy-plugins/annotated-s…

    …upport
    
    Initial support for typing.Annotated, support for extractFrom
    wRAR authored Dec 12, 2023
    Configuration menu
    Copy the full SHA
    11a9b03 View commit details
    Browse the repository at this point in the history

Commits on Dec 13, 2023

  1. Release notes for 0.13.0.

    wRAR committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    a80fc0b View commit details
    Browse the repository at this point in the history
  2. Merge pull request scrapy-plugins#153 from scrapy-plugins/relnotes-0.…

    …13.0
    
    Release notes for 0.13.0.
    wRAR authored Dec 13, 2023
    Configuration menu
    Copy the full SHA
    fab5e58 View commit details
    Browse the repository at this point in the history
  3. Bump version: 0.12.2 → 0.13.0

    wRAR committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    0536f42 View commit details
    Browse the repository at this point in the history

Commits on Dec 27, 2023

  1. Configuration menu
    Copy the full SHA
    a5f909c View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    857b51c View commit details
    Browse the repository at this point in the history