feat: auto-expand api lists (#104)

When Stripe sends webhooks with nested lists, Stripe only sends the first 10 items. For example, if you get a webhook for an invoice and you have more than 10 line items, Stripe won't send the exhaustive list, but only the first 10 items. This leads to missing data in the synced database.

PR introduces a new AUTO_EXPAND_LISTS (opt-in) environment variable to automatically expand API lists that are not exhaustive. Supported lists:

-invoice.lines
-charges.refunds
-subscription.items

When the nested lists are not exhaustive (hasMore: true), we query the Stripe API to fetch the full list before persisting. the entry.

Misc

- Update Github actions to latest majors
- Update local Supabase Postgres image to latest
- Cleaned up duplicate imports from stripe

.env.sample

@@ -13,3 +13,11 @@ PORT=8080
 
 # optional
 STRIPE_API_VERSION="2020-08-27"
+
+# optional, needed if you want to automatically fetch related lists that extend 10 entries (Stripe limits the number of entries returned to 10)
+# STRIPE_SECRET_KEY needs to be set in order for this to work, as the Stripe API is queried
+# Supported lists so far:
+## - invoice.lines
+## - charges.refunds
+## - subscription.items
+AUTO_EXPAND_LISTS=false

.github/workflows/ci.yml

@@ -20,7 +20,7 @@ jobs:
 
     steps:
       - uses: actions/checkout@v4
-      - uses: actions/cache@v3
+      - uses: actions/cache@v4
         with:
           path: ~/.npm
           key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}

.github/workflows/release.yml

@@ -21,7 +21,7 @@ jobs:
       new-release-version: ${{ steps.semantic-release.outputs.new_release_version }}
 
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - name: Set up Node
         uses: actions/setup-node@v4
@@ -34,7 +34,7 @@ jobs:
 
       - name: Run semantic-release
         id: semantic-release
-        uses: cycjimmy/semantic-release-action@v3
+        uses: cycjimmy/semantic-release-action@v4
         with:
           semantic_version: 19
         env: