Code

feat(RFC): Adds altair.datasets#3631

Code

Draft

dangotbanned wants to merge 164 commits intomainvega/altair:mainfrom vega-datasetsvega/altair:vega-datasetsCopy head branch name to clipboard

+4,044-15

Commits

Commits on Oct 2, 2024

• wip

  dangotbanned

  committed
  7933771

Commits on Oct 4, 2024

• feat(DRAFT): Minimal reimplementation

  dangotbanned

  committed
  b30081e

• refactor: Make version accessible via data.source_tag

  Show description for 279586b
  dangotbanned

  committed
  279586b

• refactor: ext_fn -> Dataset.read_fn

  dangotbanned

  committed
  32150ad

• docs: Add trailing docs to long literals

  dangotbanned

  committed
  f1d18a2

• docs: Add module-level doc

  dangotbanned

  committed
  4d3c550

• Merge branch 'main' into vega-datasets

  dangotbanned

  authored
  7e65841

Commits on Oct 5, 2024

• Merge branch 'main' into vega-datasets

  dangotbanned

  authored
  05773af

Commits on Oct 6, 2024

• Merge branch 'main' into vega-datasets

  dangotbanned

  authored
  4fff80a

Commits on Oct 7, 2024

• feat: Adds .arrow support

  Show description for 3a284a5
  dangotbanned

  committed
  3a284a5

• feat: Add support for caching metadata

  dangotbanned

  committed
  22a5039

• feat: Support env var VEGA_GITHUB_TOKEN

  Show description for a618ffc
  dangotbanned

  committed
  a618ffc

• feat: Add support for multi-version metadata

  Show description for 1792340
  dangotbanned

  committed
  1792340

Commits on Oct 8, 2024

• refactor: Renaming, docs, reorganize

  dangotbanned

  committed
  fa2c9e7

• feat: Support collecting release tags

  Show description for 24cd7d7
  dangotbanned

  committed
  24cd7d7

• feat: Adds refresh_tags

  Show description for 7dd461f
  dangotbanned

  committed
  7dd461f

• feat(DRAFT): Adds url_from

  Show description for 9768495
  dangotbanned

  committed
  9768495

• fix: Wrap all requests with auth

  dangotbanned

  committed
  c38c235

Commits on Oct 9, 2024

• chore: Remove DATASET_NAMES_USED

  dangotbanned

  committed
  a22cc8a

Commits on Oct 11, 2024

• feat: Major GitHub rewrite, handle rate limiting

  Show description for 1181860
  dangotbanned

  committed
  1181860

• feat(DRAFT): Partial implement data("name")

  dangotbanned

  committed
  31eeb20

• fix(typing): Resolve some mypy errors

  dangotbanned

  committed
  511a845

Commits on Oct 12, 2024

• Merge branch 'main' into vega-datasets

  dangotbanned

  authored
  c76cfd4

Commits on Oct 13, 2024

• Merge branch 'main' into vega-datasets

  dangotbanned

  authored
  d3f0497

Commits on Oct 24, 2024

• Merge branch 'main' into vega-datasets

  dangotbanned

  authored
  1b3390b

• fix(ruff): Apply 3.8 fixes

  Show description for a770ba9
  dangotbanned

  committed
  a770ba9

• docs(typing): Add WorkInProgress marker to data(...)

  Show description for 686a485
  dangotbanned

  committed
  686a485

Commits on Oct 25, 2024

• Merge branch 'main' into vega-datasets

  dangotbanned

  authored
  ba4491d

Commits on Oct 29, 2024

• Merge branch 'main' into vega-datasets

  dangotbanned

  authored
  1a4e107

Commits on Nov 5, 2024

• Merge remote-tracking branch 'upstream/main' into vega-datasets

  dangotbanned

  committed
  989b9b7

• feat(DRAFT): Add a source for available npm versions

  dangotbanned

  committed
  0bbf2e9

Commits on Nov 6, 2024

• refactor: Bake "v" prefix into tags_npm

  dangotbanned

  committed
  9c386e2

• refactor: Move _npm_metadata into a class

  dangotbanned

  committed
  1937f2b

• chore: Remove unused, add todo

  dangotbanned

  committed
  66fa6d1

• Merge remote-tracking branch 'upstream/main' into vega-datasets

  dangotbanned

  committed
  937aa01

• feat: Adds app context for github<->npm

  dangotbanned

  committed
  21b2edd

• fix: Invalidate old trees

  dangotbanned

  committed
  6527305

• chore: Remove early test files#

  dangotbanned

  committed
  336eeca

• refactor: Rename metadata_full -> metadata

  Show description for 225be0a
  dangotbanned

  committed
  225be0a

• refactor: tools.vendor_datasets -> tools.datasets package

  Show description for e91baab
  dangotbanned

  committed
  e91baab

• refactor: Move TypedDict, NamedTuple(s) -> datasets.models

  dangotbanned

  committed
  7782925

• refactor: Move, rename semver-related tools

  dangotbanned

  committed
  bc86ca1

• refactor: Remove write_schema from _Npm, _GitHub

  Show description for a6f5645
  dangotbanned

  committed
  a6f5645

• refactor: Rename, split _Npm, _GitHub into own modules

  Show description for 07a8342
  dangotbanned

  committed
  07a8342

• refactor: Move DataLoader.__call__ -> DataLoader.url()

  Show description for b89e6dc
  dangotbanned

  committed
  b89e6dc

• feat(typing): Generate annotations based on known datasets

  dangotbanned

  committed
  7b0fe29

• refactor(typing): Utilize datasets._typing

  dangotbanned

  committed
  572d069

• feat: Adds Npm.dataset for remote reading]

  dangotbanned

  committed
  07dcc0b

• refactor: Remove dead code

  dangotbanned

  committed
  d8f3791

Commits on Nov 7, 2024

• refactor: Replace name_js, name_py with dataset_name

  Show description for 4642a23
  dangotbanned

  committed
  4642a23

• fix: Remove invalid semver.sort op

  Show description for 65f87fc
  dangotbanned

  committed
  65f87fc

• fix: Add missing init path for refresh_trees

  dangotbanned

  committed
  6349b0f

• refactor: Move public interface to _io

  Show description for f1d610c
  dangotbanned

  committed
  f1d610c

• refactor(perf): Don't recreate path mapping on every attribute access

  dangotbanned

  committed
  c4ef112

• refactor: Split Reader._url_from into url, _query

  Show description for eb876eb
  dangotbanned

  committed
  eb876eb

• feat(DRAFT): Adds GitHubUrl.BLOBS

  Show description for 661a385
  dangotbanned

  committed
  661a385

• feat: Store sha instead of github_url

  Show description for 22dcb17
  dangotbanned

  committed
  22dcb17

• feat(perf): Adds caching to ALTAIR_DATASETS_DIR

  dangotbanned

  committed
  669df02

• feat(DRAFT): Adds initial generic backends

  dangotbanned

  committed
  2051410

Commits on Nov 8, 2024

• feat: Generate and move Metadata (TypedDict) to datasets._typing

  dangotbanned

  committed
  0ea4e21

• feat: Adds optional backends, polars[pyarrow], with_backend

  dangotbanned

  committed
  a2e9baa

• feat: Adds pyarrow backend

  dangotbanned

  committed
  c8a1429

• docs: Update .with_backend()

  dangotbanned

  committed
  279fea9

• chore: Remove duckdb comment

  Show description for 7d6c7ca
  dangotbanned

  committed
  7d6c7ca

• ci(typing): Add pyarrow-stubs to dev dependencies

  Show description for 0bb4210
  dangotbanned

  committed
  0bb4210

• refactor: generate_datasets_typing -> Application.generate_typing

  dangotbanned

  committed
  8984425

• refactor: Split datasets into public/private packages

  Show description for 9d062c8
  dangotbanned

  committed
  9d062c8

• refactor: Provide npm url to GitHub(...)

  dangotbanned

  committed
  a17d674

• refactor: Rename ext -> suffix

  dangotbanned

  committed
  69a619c

• refactor: Remove unimplemented tag="latest"

  Show description for a259b10
  dangotbanned

  committed
  a259b10

Commits on Nov 9, 2024

• feat: Rename _datasets_dir, make configurable, add docs

  Show description for 88968c8
  dangotbanned

  committed
  88968c8

• docs: Adds examples to Loader.with_backend

  dangotbanned

  committed
  b987308

• refactor: Clean up requirements -> imports

  dangotbanned

  committed
  4a2a2e0

• docs: Add basic example to Loader class

  Show description for e6dd27e
  dangotbanned

  committed
  e6dd27e

• refactor: Reorder alt.datasets module

  dangotbanned

  committed
  2a7bc4f

• docs: Fill out Loader.url

  dangotbanned

  committed
  c572180

• feat: Adds _Reader._read_metadata

  dangotbanned

  committed
  9ab9463

• refactor: Rename (reader|scanner_from() -> (read|scan)_fn()

  dangotbanned

  committed
  dd3edd6

• refactor(typing): Replace some explicit casts

  dangotbanned

  committed
  146cb50

Commits on Nov 10, 2024

• refactor: Shorten and document request delays

  dangotbanned

  committed
  94ad0d1

• feat(DRAFT): Make [tag] a pl.Enum

  dangotbanned

  committed
  4093383

• fix: Handle pyarrow scalars conversion

  dangotbanned

  committed
  76cdd45

• test: Adds test_datasets

  Show description for bb7bc17
  dangotbanned

  committed
  bb7bc17

• fix(DRAFT): hotfix pyarrow read

  dangotbanned

  committed
  ebc1bfa

• fix(DRAFT): Treat polars as exception, invalidate cache

  Show description for fe0ae88
  dangotbanned

  committed
  fe0ae88

• test: Skip pyarrow tests on 3.9

  Show description for 7089f2a
  dangotbanned

  committed
  7089f2a

Commits on Nov 11, 2024

• refactor: Tidy up changes from last 4 commits

  Show description for e1290d4
  dangotbanned

  committed
  e1290d4

• refactor: Rework _readers.py

  Show description for 9d88e1b
  dangotbanned

  committed
  9d88e1b

• test: Adds tests for missing dependencies

  dangotbanned

  committed
  60d39f5

• test: Adds test_dataset_not_found

  dangotbanned

  committed
  d6f0e45

• test: Adds test_reader_cache

  dangotbanned

  committed
  b7d57a0

• Merge remote-tracking branch 'upstream/main' into vega-datasets

  dangotbanned

  committed
  5c2e581

• docs: Finish _Reader, fill parameters of Loader.__call__

  Show description for b70aef8
  dangotbanned

  committed
  b70aef8

• refactor: Rename backend -> backend_name, get_backend -> backend

  Show description for 403b787
  dangotbanned

  committed
  403b787

Commits on Nov 12, 2024

• fix(DRAFT): Add multiple fallbacks for pyarrow JSON

  dangotbanned

  committed
  3fbc759

• test: Remove pandas fallback for pyarrow

  Show description for 4f5b4de
  dangotbanned

  committed
  4f5b4de

• test: Adds test_all_datasets

  Show description for 69a72b6
  dangotbanned

  committed
  69a72b6

• refactor: Remove _Reader._response

  Show description for 08101cc
  dangotbanned

  committed
  08101cc

• fix: Correctly handle no remote connection

  Show description for 90428a6
  dangotbanned

  committed
  90428a6

• docs: Align _typing.Metadata and Loader.(url|__call__) descriptions

  Show description for 8ad78c1
  dangotbanned

  committed
  8ad78c1

• feat: Update to v2.10.0, fix tag inconsistency

  Show description for e650454
  dangotbanned

  committed
  e650454

• refactor: Tidying up tools.datasets

  dangotbanned

  committed
  72296b0

• revert: Remove tags schema files

  dangotbanned

  committed
  ca1b500

• ci: Introduce datasets refresh to generate_schema_wrapper

  Show description for 5bd70d1
  dangotbanned

  committed
  5bd70d1

• docs: Add tools.datasets.Application doc

  dangotbanned

  committed
  012f98b

• Merge remote-tracking branch 'upstream/main' into vega-datasets

  dangotbanned

  committed
  bc0f42c

• revert: Remove comment

  dangotbanned

  committed
  5e677c0

Commits on Nov 13, 2024

• docs: Add a table preview to Metadata

  dangotbanned

  committed
  a99d2c9

• docs: Add examples for Loader.__call__

  dangotbanned

  committed
  7e6da39

• refactor: Rename DatasetName -> Dataset, VersionTag -> Version

  dangotbanned

  committed
  b49e679

• fix: Ensure latest [tag] appears first

  Show description for 7a14394
  dangotbanned

  committed
  7a14394

• refactor: Misc models.py updates

  Show description for 99f823e
  dangotbanned

  committed
  99f823e

• docs: Update tools.datasets.__init__.py

  dangotbanned

  committed
  dcef1d9

• test: Fix @datasets_debug selection

  Show description for 173f3d6
  dangotbanned

  committed
  173f3d6

• test: Add support for overrides in test_all_datasets

  Show description for 3f5a805
  dangotbanned

  committed
  3f5a805

• test: Adds test_metadata_columns

  dangotbanned

  committed
  4fc8446

• Merge remote-tracking branch 'upstream/main' into vega-datasets

  dangotbanned

  committed
  882af33

• fix: Warn instead of raise for hit rate limit

  Show description for 9e9deeb
  dangotbanned

  committed
  9e9deeb

• Merge remote-tracking branch 'upstream/main' into vega-datasets

  dangotbanned

  committed
  88d4491

Commits on Nov 15, 2024

• Merge branch 'main' into vega-datasets

  dangotbanned

  authored
  ebc8dec

• Merge branch 'main' into vega-datasets

  dangotbanned

  authored
  f2823b4

Commits on Nov 16, 2024

• feat: Update for v2.11.0

  Show description for fa5bea8
  dangotbanned

  committed
  fa5bea8

• feat: Always use pl.read_csv(try_parse_dates=True)

  Show description for 95582df
  dangotbanned

  committed
  95582df

• feat: Adds _pl_read_json_roundtrip

  Show description for dc4a230
  dangotbanned

  committed
  dc4a230

Commits on Nov 17, 2024

• feat(DRAFT): Adds infer-based altair.datasets.load

  Show description for 7ddb2a8
  dangotbanned

  committed
  7ddb2a8

Commits on Nov 18, 2024

• refactor: Rename Loader.with_backend -> Loader.from_backend

  Show description for 9544d9b
  dangotbanned

  committed
  9544d9b

• feat(DRAFT): Add optional backend parameter for load(...)

  Show description for 7b3a89e
  dangotbanned

  committed
  7b3a89e

Commits on Nov 20, 2024

• feat(DRAFT): Adds altair.datasets.url

  Show description for c835c13
  dangotbanned

  committed
  c835c13

• feat: Support url(...) without dependencies

  Show description for 0817ff8
  dangotbanned

  committed
  0817ff8

• fix(DRAFT): Don't generate csv on refresh

  Show description for e01fdd7
  dangotbanned

  committed
  e01fdd7

• test: Replace rogue NotImplementedError

  Show description for 0c5195e
  dangotbanned

  committed
  0c5195e

Commits on Nov 21, 2024

• fix: Omit .gz last modification time header

  Show description for 5595d90
  dangotbanned

  committed
  5595d90

• docs: Add doc for Application.write_csv_gzip

  dangotbanned

  committed
  9f62151

• revert: Remove `"polars[pyarrow]" backend

  Show description for 1bd4552
  dangotbanned

  committed
  1bd4552

• test: Add a complex xfail for test_load_call

  Show description for 11da9c8
  dangotbanned

  committed
  11da9c8

Commits on Nov 22, 2024

• refactor: Renaming/recomposing _readers.py

  Show description for 694ada0
  dangotbanned

  committed
  694ada0

• build: Generate VERSION_LATEST

  Show description for 6f41c7e
  dangotbanned

  committed
  6f41c7e

• feat: Adds _cache.py for UrlCache, DatasetCache

  Show description for 88d06a6
  dangotbanned

  committed
  88d06a6

• Merge remote-tracking branch 'upstream/main' into vega-datasets

  dangotbanned

  committed
  a0d2df4

• ci(ruff): Ignore 0.8.0 violations

  Show description for f21b52b
  dangotbanned

  committed
  f21b52b

• Merge remote-tracking branch 'upstream/main' into vega-datasets

  dangotbanned

  committed
  de03046

• fix: Use stable narwhals imports

  Show description for e7974d9
  dangotbanned

  committed
  e7974d9

Commits on Nov 23, 2024

• Merge branch 'main' into vega-datasets

  dangotbanned

  authored
  8ba48a9

Commits on Nov 24, 2024

• Merge branch 'main' into vega-datasets

  dangotbanned

  authored
  9d97096

• Merge remote-tracking branch 'upstream/main' into vega-datasets

  dangotbanned

  committed
  a698de9

• revert(ruff): Ignore 0.8.0 violations

  Show description for c907dc5
  dangotbanned

  committed
  c907dc5

• revert: Remove _readers._filter

  Show description for a3b38c4
  dangotbanned

  committed
  a3b38c4

• feat: Adds example and tests for disabling caching

  dangotbanned

  committed
  a6c5096

• refactor: Tidy up DatasetCache

  dangotbanned

  committed
  71423ea

• docs: Finish Loader.cache

  Show description for 7dd9c18
  dangotbanned

  committed
  7dd9c18

• refactor(typing): Use Mapping instead of dict

  Show description for a982759
  dangotbanned

  committed
  a982759

Commits on Nov 30, 2024

• perf: Use to_list() for all backends

  Show description for d20e9c1
  dangotbanned

  committed
  d20e9c1

Commits on Dec 2, 2024

• feat(DRAFT): Utilize datapackage schemas in pandas backends

  Show description for 909e7d0
  dangotbanned

  committed
  909e7d0

• Merge remote-tracking branch 'upstream/main' into vega-datasets

  dangotbanned

  committed
  d93fda1

• refactor(ruff): Apply TC006 fixes in new code

  Show description for 9274284
  dangotbanned

  committed
  9274284

• docs(DRAFT): Add notes on datapackage.features_typing

  dangotbanned

  committed
  8e232b8

• docs: Update Loader.from_backend example w/ dtypes

  Show description for 9330895
  dangotbanned

  committed
  9330895

• feat: Use _pl_read_json_roundtrip instead of pl.read_json for pyarrow

  Show description for caf534d
  dangotbanned

  committed
  caf534d

• docs: Replace example dataset

  Show description for 75bf2ba
  dangotbanned

  committed
  75bf2ba

Commits on Dec 20, 2024

• Merge branch 'main' into vega-datasets

  dangotbanned

  authored
  9e1fd09

• fix(ruff): resolve RUF043 warnings

  Show description for d4930e7
  dangotbanned

  committed
  d4930e7

• build: run generate-schema-wrapper

  Show description for 5a31333
  dangotbanned

  committed
  5a31333

Commits on Dec 21, 2024

• chore: update schemas

  Show description for 6080116
  dangotbanned

  committed
  6080116

Commits on Dec 22, 2024

• feat(typing): Update frictionless model hierarchy

  Show description for 897e8f9
  dangotbanned

  committed
  897e8f9