Skip to content

Commit

Permalink
fix(baking): bake graphers in parallel, using multiple db transactions
Browse files Browse the repository at this point in the history
  • Loading branch information
marcelgerber committed Apr 27, 2024
1 parent 56d5b05 commit 8843ead
Showing 1 changed file with 8 additions and 3 deletions.
11 changes: 8 additions & 3 deletions baker/GrapherBaker.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -523,9 +523,14 @@ export const bakeAllChangedGrapherPagesVariablesPngSvgAndDeleteRemovedGraphers =
await pMap(
jobs,
async (job) => {
// TODO: not sure if the shared transaction will be an issue - I think it should be fine but just to put a flag here
// that this could be causing issues
await bakeSingleGrapherChart(job, knex)
// We want to run this code on multiple threads, so we need to
// be able to use multiple transactions so that we can use
// multiple connections to the database.
// Read-write consistency is not a concern here, thankfully.
await db.knexReadWriteTransaction(
async (knex) => await bakeSingleGrapherChart(job, knex),
db.TransactionCloseMode.KeepOpen
)
progressBar.tick({ name: `slug ${job.slug}` })
},
{ concurrency: 10 }
Expand Down

0 comments on commit 8843ead

Please sign in to comment.