🐛 fix script that refreshes pageviews

[sophiamersmann]

• The script that runs on make refresh.pageviews currently doesn't work. It incorrectly filters out all rows, and then inserts 0 rows into the database.
• This statement threw out all entries (maybe because additional columns were recently added to the Datsette table?):

  owid-grapher/db/refreshPageviewsFromDatasette.ts

  Lines 30 to 32 in 8fbf4ab

  	const onlyValidRows = [...parsedData.data].filter(
  	(row) => Object.keys(row as any).length === 5
  	) as any[]

• To make this a bit more robust, I grab all valid pageview fields from each row of the downloaded data. A row is then declared "valid" if it has a day and a URL
  • Note that we're also removing rows where the URL contains an emoji since MySQL couldn't insert those. We currently have one single entry where the URL contains an emoji (Datasette)

---

Also, knex gives me the following warning when running this script:

Is this something we should care about?

[danyx23]

Thanks a lot for fixing this! The complaint was because TRUNCATE is a DDL statement (i.e. in the category of schema modifying operations) and they have their own implicit transaction so should not be run in a data modifying transaction. Good to go now!