Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mint: table locks #566

Merged
merged 62 commits into from
Jul 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
6b408ea
clean up db
callebtc Jun 30, 2024
b7d611d
Merge branch 'main' into db-lock-quotes
callebtc Jun 30, 2024
da0013d
db: table lock
callebtc Jun 30, 2024
ca97342
db.table_with_schema
callebtc Jun 30, 2024
55b2a62
fix encrypt.py
callebtc Jun 30, 2024
fbba4f3
postgres nowait
callebtc Jul 1, 2024
0b849a2
add timeout to lock
callebtc Jul 1, 2024
19a2bd2
melt quote state in db
callebtc Jul 2, 2024
af2d400
kinda working
callebtc Jul 3, 2024
5647c07
kinda working with postgres
callebtc Jul 3, 2024
dc581f5
remove dispose
callebtc Jul 3, 2024
f21d2e7
Merge branch 'main' into db-lock-quotes
callebtc Jul 3, 2024
0fb2f73
getting there
callebtc Jul 4, 2024
ce66c22
porperly clean up db for tests
callebtc Jul 4, 2024
d4f041d
faster tests
callebtc Jul 4, 2024
7f81671
configure connection pooling
callebtc Jul 4, 2024
0bf30b9
try github with connection pool
callebtc Jul 4, 2024
baf2b0c
invoice dispatcher does not lock db
callebtc Jul 4, 2024
6f8c21a
fakewallet: pay_if_regtest waits
callebtc Jul 5, 2024
effbf85
pay fakewallet invoices
callebtc Jul 5, 2024
c2b3fd2
add more
callebtc Jul 5, 2024
f712c23
faster
callebtc Jul 5, 2024
73b456e
slower
callebtc Jul 5, 2024
03f90c5
pay_if_regtest async
callebtc Jul 5, 2024
de68c28
do not lock the invoice dispatcher
callebtc Jul 5, 2024
5916ad0
test: do I get disk I/O errors if we disable the invoice_callback_dis…
callebtc Jul 5, 2024
e134b6d
fix fake so it workss without a callback dispatchert
callebtc Jul 6, 2024
cc89d3e
test on github
callebtc Jul 6, 2024
deaa3e5
readd tasks
callebtc Jul 6, 2024
20099a4
refactor
callebtc Jul 6, 2024
aa90f9f
increase time for lock invoice disatcher
callebtc Jul 6, 2024
a0453f5
try avoiding a race
callebtc Jul 6, 2024
3b5a43a
remove task
callebtc Jul 6, 2024
f93dc60
github actions: test regtest with postgres
callebtc Jul 6, 2024
e3daf21
mint per module
callebtc Jul 6, 2024
29efd20
no connection pool for testing
callebtc Jul 6, 2024
97354b9
enable pool
callebtc Jul 6, 2024
45fe907
do not resend paid event
callebtc Jul 6, 2024
915c2d1
reuse connection
callebtc Jul 6, 2024
bb1168b
close db connections
callebtc Jul 7, 2024
656205e
sessions
callebtc Jul 7, 2024
f2ba89c
enable debug
callebtc Jul 7, 2024
cad97f4
dispose engine
callebtc Jul 7, 2024
5628cf3
disable connection pool for tests
callebtc Jul 8, 2024
a4d5df6
enable connection pool for postgres only
callebtc Jul 8, 2024
96b9e15
clean up shutdown routine
callebtc Jul 8, 2024
c80faa4
remove wait for lightning fakewallet lightning invoice
callebtc Jul 8, 2024
118ded6
cancel invoice listener tasks on shutdown
callebtc Jul 8, 2024
9643eb8
fakewallet conftest: decrease outgoing delay
callebtc Jul 8, 2024
f6b7256
delay payment and set postgres only if needed
callebtc Jul 8, 2024
dd3434d
disable fail fast for regtest
callebtc Jul 8, 2024
fd094d9
clean up regtest.yml
callebtc Jul 8, 2024
68f21e6
change order of tests_db.py
callebtc Jul 8, 2024
614e182
row-specific mint_quote locking
callebtc Jul 8, 2024
80861c3
refactor
callebtc Jul 8, 2024
7dc16e2
fix lock statement
callebtc Jul 8, 2024
1b0eda0
refactor swap
callebtc Jul 8, 2024
5a70509
refactor
callebtc Jul 8, 2024
e209a6f
remove psycopg2
callebtc Jul 8, 2024
1fb9e0f
add connection string example to .env.example
callebtc Jul 8, 2024
e9112e7
remove unnecessary pay
callebtc Jul 8, 2024
33326e2
shorter sleep in test_wallet_subscription_swap
callebtc Jul 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,10 @@ MINT_DERIVATION_PATH="m/0'/0'/0'"
# In this example, we have 2 keysets for sat, 1 for msat and 1 for usd
# MINT_DERIVATION_PATH_LIST=["m/0'/0'/0'", "m/0'/0'/1'", "m/0'/1'/0'", "m/0'/2'/0'"]

# To use SQLite, choose a directory to store the database
MINT_DATABASE=data/mint
# To use PostgreSQL, set the connection string
# MINT_DATABASE=postgres://cashu:cashu@localhost:5432/cashu

# Funding source backends
# Supported: FakeWallet, LndRestWallet, CLNRestWallet, BlinkWallet, LNbitsWallet, StrikeWallet, CoreLightningRestWallet (deprecated)
Expand Down
2 changes: 1 addition & 1 deletion .github/actions/prepare/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,5 +23,5 @@ runs:
cache: "poetry"
- name: Install dependencies
run: |
poetry install --extras pgsql
poetry install
shell: bash
5 changes: 3 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,13 +28,14 @@ jobs:
regtest:
uses: ./.github/workflows/regtest.yml
strategy:
fail-fast: false
matrix:
python-version: ["3.10"]
poetry-version: ["1.7.1"]
backend-wallet-class:
["LndRestWallet", "CLNRestWallet", "CoreLightningRestWallet", "LNbitsWallet"]
# mint-database: ["./test_data/test_mint", "postgres://cashu:cashu@localhost:5432/cashu"]
mint-database: ["./test_data/test_mint"]
mint-database: ["./test_data/test_mint", "postgres://cashu:cashu@localhost:5432/cashu"]
# mint-database: ["./test_data/test_mint"]
with:
python-version: ${{ matrix.python-version }}
backend-wallet-class: ${{ matrix.backend-wallet-class }}
Expand Down
22 changes: 7 additions & 15 deletions .github/workflows/regtest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,21 +23,13 @@ jobs:
regtest:
runs-on: ${{ inputs.os-version }}
timeout-minutes: 10
services:
postgres:
image: postgres:latest
env:
POSTGRES_USER: cashu
POSTGRES_PASSWORD: cashu
POSTGRES_DB: cashu
ports:
- 5432:5432
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- name: Start PostgreSQL service
if: contains(inputs.mint-database, 'postgres')
run: |
docker run -d --name postgres -e POSTGRES_USER=cashu -e POSTGRES_PASSWORD=cashu -e POSTGRES_DB=cashu -p 5432:5432 postgres:latest
until docker exec postgres pg_isready; do sleep 1; done

- uses: actions/checkout@v3

- uses: ./.github/actions/prepare
Expand Down Expand Up @@ -78,7 +70,7 @@ jobs:
MINT_CLNREST_URL: https://localhost:3010
MINT_CLNREST_RUNE: ./regtest/data/clightning-2/rune
MINT_CLNREST_CERT: ./regtest/data/clightning-2/regtest/ca.pem
MINT_CLNENABLE_MPP: false
MINT_CLNREST_ENABLE_MPP: false
run: |
sudo chmod -R 777 .
make test
Expand Down
19 changes: 5 additions & 14 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,21 +23,12 @@ jobs:
poetry:
name: Run (db ${{ inputs.mint-database }}, deprecated api ${{ inputs.mint-only-deprecated }})
runs-on: ${{ inputs.os }}
services:
postgres:
image: postgres:latest
env:
POSTGRES_USER: cashu
POSTGRES_PASSWORD: cashu
POSTGRES_DB: cashu
ports:
- 5432:5432
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- name: Start PostgreSQL service
if: contains(inputs.mint-database, 'postgres')
run: |
docker run -d --name postgres -e POSTGRES_USER=cashu -e POSTGRES_PASSWORD=cashu -e POSTGRES_DB=cashu -p 5432:5432 postgres:latest
until docker exec postgres pg_isready; do sleep 1; done
- name: Checkout repository and submodules
uses: actions/checkout@v2
- uses: ./.github/actions/prepare
Expand Down
2 changes: 0 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,8 +83,6 @@ pyenv local 3.10.4
poetry install
```

If you would like to use PostgreSQL as the mint database, use the command `poetry install --extras pgsql`.

#### Poetry: Update Cashu
To update Cashu to the newest version enter
```bash
Expand Down
18 changes: 11 additions & 7 deletions cashu/core/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -344,11 +344,13 @@ def kind(self) -> JSONRPCSubscriptionKinds:
def __setattr__(self, name, value):
# an unpaid quote can only be set to pending or paid
if name == "state" and self.state == MeltQuoteState.unpaid:
if value != MeltQuoteState.pending and value != MeltQuoteState.paid:
raise Exception("Cannot change state of an unpaid quote.")
if value not in [MeltQuoteState.pending, MeltQuoteState.paid]:
raise Exception(
f"Cannot change state of an unpaid melt quote to {value}."
)
# a paid quote can not be changed
if name == "state" and self.state == MeltQuoteState.paid:
raise Exception("Cannot change state of a paid quote.")
raise Exception("Cannot change state of a paid melt quote.")
super().__setattr__(name, value)


Expand Down Expand Up @@ -415,18 +417,20 @@ def __setattr__(self, name, value):
# un unpaid quote can only be set to paid
if name == "state" and self.state == MintQuoteState.unpaid:
if value != MintQuoteState.paid:
raise Exception("Cannot change state of an unpaid quote.")
raise Exception(
f"Cannot change state of an unpaid mint quote to {value}."
)
# a paid quote can only be set to pending or issued
if name == "state" and self.state == MintQuoteState.paid:
if value != MintQuoteState.pending and value != MintQuoteState.issued:
raise Exception(f"Cannot change state of a paid quote to {value}.")
raise Exception(f"Cannot change state of a paid mint quote to {value}.")
# a pending quote can only be set to paid or issued
if name == "state" and self.state == MintQuoteState.pending:
if value not in [MintQuoteState.paid, MintQuoteState.issued]:
raise Exception("Cannot change state of a pending quote.")
raise Exception("Cannot change state of a pending mint quote.")
# an issued quote cannot be changed
if name == "state" and self.state == MintQuoteState.issued:
raise Exception("Cannot change state of an issued quote.")
raise Exception("Cannot change state of an issued mint quote.")
super().__setattr__(name, value)


Expand Down
Loading
Loading