🎉 (entity selector) sort by external indicators

[sophiamersmann]

Cycle 2024.2: Entity selector | Designs

Summary

Offers to sort by "Population" and "GDP per capita", even if the chart doesn't include population or GDP per capita indicators.

Details

• If the chart has a Population or GDP per capita indicator, then we re-use that data
• If we need to download additional indicators, then we do that on demand, i.e. selecting "Population" or "GDP per capita" triggers their download
• Indicator IDs for population and GDP per capita are hard-coded (but the data team might come up with a better solution)
• "Population" and "GDP per capita" are only offered for selection if entities are detected to include countries or regions
  • This is done by checking whether any of the available entities are listed in the regions.json file
  • Testing the available entities against the regions.json file is not perfect since the default population and GDP per capita indicators that we are using have data for a few entities that are not listed in regions.json (see details below)
  • However, we only need a single matching entity to trigger sorting by population or GDP per capita, so in practice this works well
  • If we wanted to be more correct here, we could also download population and GDP per capita metadata when the entity selector is opened and then check the actual population/GDP per capita entities against the entities that are available for the chart

Entities of the population or GDP per capita indicator that are not included in the `regions.json` file

• For the population indicator:
  - Africa (UN)
  - Asia (UN)
  - Europe (UN)
  - High-income countries
  - Latin America and the Caribbean (UN)
  - Low-income countries
  - Lower-middle-income countries
  - Northern America (UN)
  - Oceania (UN)
  - Upper-middle-income countries
• For the GDP per capita indicator:
  - East Asia and Pacific (WB)
  - Europe and Central Asia (WB)
  - High-income countries
  - Latin America and Caribbean (WB)
  - Low-income countries
  - Lower-middle-income countries
  - Middle East and North Africa (WB)
  - Middle-income countries
  - North America (WB)
  - South Asia (WB)
  - Sub-Saharan Africa (WB)
  - Upper-middle-income countries

SVG tester

The SVG tester fails due to the changes in #3373

[sophiamersmann]

• 🎉 (entity selector) sort by external indicators #3466 : 2 dependent PRs (#3443 , #3490 ) 👈
• 🎉 (entity selector) highlight user location #3452
• 🎉 (entity selector) sort by name or value #3447
• ✨ (entity selector) redesign #3436
• 🔨 (grapher) refactor modals #3435
• 🔨 (grapher) add CloseButton component #3434
• ✨ (entity selector) render into panel, drawer or modal #3373
• ✨ (entity selector) remove slide-in settings drawer #3353 : 2 other dependent PRs (#3356 , #3363 )
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @sophiamersmann and the rest of your teammates on Graphite

[marcelgerber]

Nice, this works well!

[marcelgerber]

Hm, this regex will also match "Population density", "Female population", "Share of population in poverty", etc.

[sophiamersmann]

yeah... but I can't think of a better way to detect population/GDP per capita indicators. Alternatively, we could rely on isPopulationVariableId only and ask d&r for a similar list of GDP per capita indicators. Do you reckon that would be better?

[sophiamersmann]

For population, I think it is best to rely on isPopulationVariableId only and not use a regex since we'd get many spurious matches (Datasette)

[marcelgerber]

Yes, agreed.
Or otherwise, match only /Population( \(historical estimates\))?/, which currently finds exactly these four.

[marcelgerber]

Similar here: This'll also match any variable that contains % of GDP per capita, for example.

[sophiamersmann]

GDP per capita is different because we have lots of GDP per capita variations like GDP per capita (constant 2015 US$) or Real GDP per capita (2011$) that are often simply referred to as "GDP per capita" in charts (Datasette; example chart). In such cases, using a second, slightly different GDP per capita version in the entity selector would be confusing. So, I think matching the name makes sense here. But I updated the regex to make sure that we're not matching "GDP per capita" if it's in parens, like Government expenditure per student, primary (% of GDP per capita).

There are some indicators that we'll match incorrectly (like Growth rate of GDP per capita or Affordability of cigarettes: percentage of GDP per capita required to purchase 2000 cigarettes of the most sold brand), which means that we'll fail to offer "GDP per capita" as a sorting option, which I think is acceptable.

[marcelgerber]

Yes, agreed. Good on you for pointing out that the outcome of a false positive is simply that we're not offering GDP per capita for sorting in that case, which is totally accepting.

[marcelgerber]

What I don't quite like here is how set-in-stone these two variables are: There are many methods that have "population" or "gdp" in their name, and all of that makes it quite hard to add any other such indicators in the future.

More ideally, we would have a more configurable set; something like

const INDICATORS_TO_LOAD = [{
    key: "population",
    name: "Population",
    indicatorId: POPULATION_INDICATOR_ID_USED_IN_ENTITY_SELECTOR,
    columnNameRegex: /\bpopulation\b/i
}, // ...
]

could work.

[sophiamersmann]

I had refactored the code at some point to allow for a list of external indicators like that 👆🏻, but it made the code harder to read, and at the same time, I think it's quite unlikely that we'll keep adding external indicators. I don't have a strong opinion though – happy to refactor if you want me to!

[marcelgerber]

Earlier today, I was thinking that we will probably unify the explorer entity selector with the grapher entity selector at some point.
It's a bit weird at the moment that the explorer entity selector is shown on the left, that it is also visible on the map and table tabs, etc.

Explorers currently have their own options for sorting, which can be set in the spreadsheet. Basically, they can be any column of any csv file that's being used in the explorer, including ones that haven't yet been fetched.

I'm not sure what the future of explorers entails, exactly - I think we want to move away a bit from csv's as a data source.
But still, supporting the current explorer feature set would be easier if it was easier to set/extend the set of sort indicators.

[marcelgerber]

Not something for now, though, just something to keep in mind for the future.

[sophiamersmann]

Good point! That would def make it easier :)

[owidbot]

Quick links (staging server):

Site	Admin	Wizard

Login: ssh owid@staging-site-entity-selector-external-sort

• Site-screenshots: https://github.com/owid/site-screenshots/compare/entity-selector-external-sort
• SVG tester: https://github.com/owid/owid-grapher-svgs/compare/entity-selector-external-sort

SVG tester:

Number of differences (default views): 1237

Number of differences (all views): 389

Edited: 2024-05-02 14:45:35 UTC
Execution time: 1.18 seconds

[sophiamersmann]

Merge activity

• May 3, 4:04 AM EDT: @sophiamersmann started a stack merge that includes this pull request via Graphite.
• May 3, 4:18 AM EDT: Graphite rebased this pull request as part of a merge.
• May 3, 4:22 AM EDT: @sophiamersmann merged this pull request with Graphite.