Skip to content

Commit

Permalink
feat: MFA (#162)
Browse files Browse the repository at this point in the history
* changes storage layer to take json instead of config file path

* adds new functions skeleton

* adds checks for conflicting configs for user pools

* changes to tests to make them pass

* adds skeleton for multi tenancy functions

* fixes bug

* adds connection pool ID function

* changes as per interface change

* adds one test for multi tenany storage layer

* adds more tests

* fixes bugs

* adds more tests and changes config parsing to prioritise connection uri input

* fixes a few config parsing bugs

* adds more tests

* modifies testing to clear multiple user pools after each test

* makes initlogging idempotent

* fixes all tests

* fixes tests

* adds more placeholder functions

* removes use of quiteprogramexception

* small change

* adds new function skeleton

* adds more skeleton functions

* updates exception import

* adds skeleton for tenantIdentifier for emailpassword and useridmapping recipes

* changes to incorporate tenantIndetifier for key value storage

* changes to session receipe to add tenantIdentifier

* introduces the concept of appIdentifier vs tenantIdentifier

* fixes test compilation issues

* changes as per plugin change

* modifes user roles functions to add tenantidentifier and appidentifiers

* modifies emailpassword functions

* changes to a few functions

* adds appidentifier to email verfication

* makes tests pass

* adds tenant identifier to third party

* adds tenantidentifier to passwordless

* function name changes

* fix: Multitenancy schema updates (#59)

* fix: few schema changes and multitenancy impl

* fix: handling pkey constraint

* fix: pr comments

* fix: pr comments

* fix: pr comments

* fix: pr comments

* fix: typo and logical mistakes

* fix: null handling and new exceptions

* fix: refactored provider SQLs

* fix: refactored select all

* fix: fix for concurrent test

* fix: cleanup

* fix: cleanup and handle null boolean

* fix: Multitenant emailpassword recipe changes (#60)

* fix: emailpassword schema

* fix: ep, ev and pless schema

* fix: prepare for ep review

* fix: app_id_to_user_id table

* fix: ep recipe impl

* fix: removed todo

* fix: updated as per plugin interface

* fix: fixed index

* fix: pr comments

* fix: removed backward compatibility

* fix: minor fix (#62)

* fix: Multitenant schema changes (#64)

* fix: ev and pless impl

* fix: ev fixes

* fix: pless and tp changes

* fix: revert delete user

* fix: pless impl

* fix: cleanup and fixed deleteUser

* fix: simplified queries and added fkey checks in ep

* fix: fkey checks for pless

* fix: fkey checks for thirdparty

* fix: fkey checks for emailverification

* fix: fixed test

* fix: updated to join query for ep

* fix: updated join queries

* fix: constraints

* fix: test fix

* fix: pr comments

* fix: to support PR comments on core (#65)

* fix: from core pr comments

* fix: updated tenant identifier conversion

* fix: Multitenant userroles (#69)

* fix: user roles impl

* fix: handling fkey

* fix: transaction fix

* fix: transaction fix

* fix: Multitenant usermetadata (#70)

* fix: user roles impl

* fix: handling fkey

* fix: usermetadata impl

* fix: transaction fix

* fix: transaction fix

* fix: ep storage (#71)

* fix: thirdparty storage (#74)

* fix: Multitenant thirdparty changes for update email (#75)

* fix: thirdparty storage

* fix: thirdparty changes

* fix: pr comments

* fix: Multitenant emailverification storage (#76)

* fix: thirdparty storage

* fix: emailverification storage

* fix: tokens tenant specific (#77)

* fix: Multitenant session (#78)

* fix: session changes

* fix: session changes

* fix: session changes

* comment modification

* fix: Multitenant session changes (#80)

* fix: key value changes

* fix: pr comments

* fix: adding tenant or app not found exceptions

* Multi tenant merging with latest (#79)

* merges with latest

* fixes test compilation issue

* increases threshold of deadlock retries

* adds simple test for loading 50 storages

* many fixes

* fix: jwt changes (#82)

* fix: Multitenant General Queries (#84)

* fix: updated general queries

* fix: fixed queries

* fix: Multitenant dashboard (#85)

* fix: updated general queries

* fix: fixed queries

* fix: dashboard queries

* fix: added fk contstraint

* fix: fixed index

* fix: Multitenant totp (#86)

* fix: totp queries

* fix: handling fk

* fix: pr comment

* merges (#87)

* adds new config

* fix: multitenancy changes (#88)

* fix: multitenancy changes

* fix: multitenant queries

* fix: add userid to tenant

* fix: saas test

* fix: remove DeletionInProgressException

* fix: pr comments

* fix: recipe id in appid_to_userid table

* fix: pr comment

* fix: query fixes

* fix: fixed validation

* fix: added comments

* fix: Misc changes (#89)

* fix: session expiry index

* fix: active users

* feat: Introduce MFA recipe in postgresql plugin

* chores: Mention MFA recipe support in CHANGELOG

* fix: Tenantid in userobjects (#90)

* fix: adding tenant ids to user objects

* fix: create user type

* fix: test fixes

* fix: transaction

* fix: refactored ep and tp

* fix: refactor pless

* fix: pr comment

* fix: pr comment

* fix: test fix (#92)

* fix: Startup log (#93)

* fix: removed log to console

* fix: tenant id in loadConfig

* fix: Userpool test (#94)

* fix: userpool test

* fix: added test with server restart

* fix: delete non auth user (#95)

* fix: Delete nonauth user (#96)

* fix: nonAuthRecipeuserData to take tenantIdentifier

* fix: pr comments

* feat: Add active user stat queries for MFA

* fix: Update user_id length in mfa_user_factors table

* Set factor_id VARCHAR length to 16

* fix: config validation (#97)

* fix: config per tenant, per app annotations (#98)

* feat: Consider multitenancy when getting MFA stats

* test: Fix mistake in MFA table create query

* feat: Add query to delete user from a tenant

* fix: config annotation (#102)

* fix: config annotation

* fix: removed comments

* Update src/main/java/io/supertokens/storage/postgresql/config/PostgreSQLConfig.java

* Update src/main/java/io/supertokens/storage/postgresql/config/PostgreSQLConfig.java

---------

Co-authored-by: Rishabh Poddar <[email protected]>

* fix: added setLogLevels (#103)

* fix: merge issue

* Overload deleteMfaInfoForUser and set factor column size to 64

* fix: fkey names (#104)

* fix: fixed fkey names on user tables

* fix: catching fkey constraints

* fix: added comments

* fix: Postgres migration (#105)

* fix: fixed fkey names on user tables

* fix: catching fkey constraints

* fix: added comments

* fix: changelog

* fix: changelog

* fix: pr comment

* fix: Fkey indexes (#109)

* fix: fkey indexes

* fix: fixes

* fix: active users storage stuff

* fix: active users storage stuff

* fix: fixed index name

* fix: updated migration script

* fix: Revert irrelevant changes

* refactor: Replace TotpNotEnabledError with UnknownUserIdTotpError (#133)

* refactor: Replace totp not enabled error with unknown device error

* Replace TotpNotEnabledError with UnknownUserIdTotpError

* chores: Update CHANGELOG

* fix: build

* fix: totp queries

---------

Co-authored-by: Sattvik Chakravarthy <[email protected]>

* fix: queries

* fix: changes as per plugin interface (#163)

* fix: mfa cleanup (#164)

* fix: mfa cleanup

* fix: pr comments

* Mfa multitenancy (#167)

* fix: mfa multitenancy queries

* fix: mfa cleanup

* fix: mfa config storage

* fix: mfa

* fix: tests

* fix: default values

* fix: pr comments

* fix: pr comments

* fix: minor fix

* fix: pr comments

* fix: set

* fix: pr comment

* fix: constraint

* fix: created_at in totp devices (#169)

* fix: created_at in totp devices

* fix: add createdat to totp device

* fix: mfa stats (#170)

* fix: mfa stats

* fix: pr comments

* fix: index name

* fix: mfa changes (#177)

* feat: make refresh update the signing key type of sessions (#180)

* fix: Merge latest (#199)

* fix: remove db password from logs (#181)

* fix: remove db password from logs

* fix: Update version

* fix: mask db password

* fix: Add tests

* fix: Add more tests

* fix: PR changes

* fix: PR changes

* fix: Connection pool issue (#182)

* fix: test connection pool

* fix: changelog

* fix: test for downtime during connection pool change

* fix: assert that there should be down time

* fix: cleanup

* adding dev-v5.0.7 tag to this commit to ensure building

* fix: cicd tests (#185)

* fix: logging test (#187)

* adding dev-v5.0.7 tag to this commit to ensure building

* fix: flaky test (#188)

* adding dev-v5.0.7 tag to this commit to ensure building

* fix: adds idle timeout and minimum idle configs (#184)

* fix: adds idle timeout and minimum idle configs

* fix: protected props

* fix: changelog

* fix: test protected config

* adding dev-v5.0.7 tag to this commit to ensure building

* fix: cicd (#189)

* fix: cicd

* fix: test

* adding dev-v5.0.7 tag to this commit to ensure building

* fixes tests

* adding dev-v5.0.7 tag to this commit to ensure building

* fix: vulnerability fix (#192)

* fix: vulnerability fix

* fix: vulnerability fix

* adding dev-v5.0.8 tag to this commit to ensure building

* fix: dependencies (#195)

* adding dev-v5.0.8 tag to this commit to ensure building

* fix: version update (#198)

* adding dev-v5.0.8 tag to this commit to ensure building

---------

Co-authored-by: Ankit Tiwari <[email protected]>
Co-authored-by: rishabhpoddar <[email protected]>

* merge latest (#204)

* fix: remove db password from logs (#181)

* fix: remove db password from logs

* fix: Update version

* fix: mask db password

* fix: Add tests

* fix: Add more tests

* fix: PR changes

* fix: PR changes

* fix: Connection pool issue (#182)

* fix: test connection pool

* fix: changelog

* fix: test for downtime during connection pool change

* fix: assert that there should be down time

* fix: cleanup

* adding dev-v5.0.7 tag to this commit to ensure building

* fix: cicd tests (#185)

* fix: logging test (#187)

* adding dev-v5.0.7 tag to this commit to ensure building

* fix: flaky test (#188)

* adding dev-v5.0.7 tag to this commit to ensure building

* fix: adds idle timeout and minimum idle configs (#184)

* fix: adds idle timeout and minimum idle configs

* fix: protected props

* fix: changelog

* fix: test protected config

* adding dev-v5.0.7 tag to this commit to ensure building

* fix: cicd (#189)

* fix: cicd

* fix: test

* adding dev-v5.0.7 tag to this commit to ensure building

* fixes tests

* adding dev-v5.0.7 tag to this commit to ensure building

* fix: vulnerability fix (#192)

* fix: vulnerability fix

* fix: vulnerability fix

* adding dev-v5.0.8 tag to this commit to ensure building

* fix: dependencies (#195)

* adding dev-v5.0.8 tag to this commit to ensure building

* fix: version update (#198)

* adding dev-v5.0.8 tag to this commit to ensure building

* fix: fixes storage handling for non-auth recipes (#203)

* fix: tests

* fix: user role table constraint

* fix: pr comments

* fix: according to updated interface

* fix: user roles

* fix: version and changelog

* fix: plugin interface version

* adding dev-v6.0.0 tag to this commit to ensure building

---------

Co-authored-by: Ankit Tiwari <[email protected]>
Co-authored-by: rishabhpoddar <[email protected]>

* fix: One million users test (#196)

* test: one million users first version

* fix: user data

* fix: update test

* fix: update cicd

* fix: wip

* fix: measurements

* fix: test

* fix: adding memory tests

* fix: memory limit

* fix: pass appId to getUserIdMappingForSuperTokensIds

* fix: one million users test

* fix: versions

* fix: versions

* Remaining changes (#206)

* fix: pass appId to getUserIdMappingForSuperTokensIds

* fix: one million users test

* fix: versions

* fix: versions

* fix: one million users

---------

Co-authored-by: rishabhpoddar <[email protected]>
Co-authored-by: KShivendu <[email protected]>
Co-authored-by: KShivendu <[email protected]>
Co-authored-by: Mihály Lengyel <[email protected]>
Co-authored-by: Ankit Tiwari <[email protected]>
  • Loading branch information
6 people authored Mar 13, 2024
1 parent 7a9cc50 commit 9f5d928
Show file tree
Hide file tree
Showing 30 changed files with 1,664 additions and 162 deletions.
103 changes: 102 additions & 1 deletion .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,88 @@ jobs:
name: running tests
command: (cd .circleci/ && ./doTests.sh)
- slack/status
test-onemillionusers:
docker:
- image: rishabhpoddar/supertokens_postgresql_plugin_test
resource_class: large
steps:
- add_ssh_keys:
fingerprints:
- "14:68:18:82:73:00:e4:fc:9e:f3:6f:ce:1d:5c:6d:c4"
- checkout
- run:
name: update postgresql max_connections
command: |
sed -i 's/^#*\s*max_connections\s*=.*/max_connections = 10000/' /etc/postgresql/9.5/main/postgresql.conf
- run:
name: starting postgresql
command: |
(cd / && ./runPostgreSQL.sh)
- run:
name: create databases
command: |
psql -c "create database st0;"
psql -c "create database st1;"
psql -c "create database st2;"
psql -c "create database st3;"
psql -c "create database st4;"
psql -c "create database st5;"
psql -c "create database st6;"
psql -c "create database st7;"
psql -c "create database st8;"
psql -c "create database st9;"
psql -c "create database st10;"
psql -c "create database st11;"
psql -c "create database st12;"
psql -c "create database st13;"
psql -c "create database st14;"
psql -c "create database st15;"
psql -c "create database st16;"
psql -c "create database st17;"
psql -c "create database st18;"
psql -c "create database st19;"
psql -c "create database st20;"
psql -c "create database st21;"
psql -c "create database st22;"
psql -c "create database st23;"
psql -c "create database st24;"
psql -c "create database st25;"
psql -c "create database st26;"
psql -c "create database st27;"
psql -c "create database st28;"
psql -c "create database st29;"
psql -c "create database st30;"
psql -c "create database st31;"
psql -c "create database st32;"
psql -c "create database st33;"
psql -c "create database st34;"
psql -c "create database st35;"
psql -c "create database st36;"
psql -c "create database st37;"
psql -c "create database st38;"
psql -c "create database st39;"
psql -c "create database st40;"
psql -c "create database st41;"
psql -c "create database st42;"
psql -c "create database st43;"
psql -c "create database st44;"
psql -c "create database st45;"
psql -c "create database st46;"
psql -c "create database st47;"
psql -c "create database st48;"
psql -c "create database st49;"
psql -c "create database st50;"
- run:
name: running tests
command: (cd .circleci/ && ./doOneMillionUsersTests.sh)
- slack/status
mark-passed:
docker:
- image: rishabhpoddar/supertokens_postgresql_plugin_test
steps:
- checkout
- run: (cd .circleci && ./markPassed.sh)
- slack/status

workflows:
version: 2
Expand All @@ -89,4 +171,23 @@ workflows:
tags:
only: /dev-v[0-9]+(\.[0-9]+)*/
branches:
ignore: /.*/
ignore: /.*/
- test-onemillionusers:
context:
- slack-notification
filters:
tags:
only: /dev-v[0-9]+(\.[0-9]+)*/
branches:
ignore: /.*/
- mark-passed:
context:
- slack-notification
filters:
tags:
only: /dev-v[0-9]+(\.[0-9]+)*/
branches:
ignore: /.*/
requires:
- test
- test-onemillionusers
135 changes: 135 additions & 0 deletions .circleci/doOneMillionUsersTests.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
function cleanup {
if test -f "pluginInterfaceExactVersionsOutput"; then
rm pluginInterfaceExactVersionsOutput
fi
}

trap cleanup EXIT
cleanup

pluginInterfaceJson=`cat ../pluginInterfaceSupported.json`
pluginInterfaceLength=`echo $pluginInterfaceJson | jq ".versions | length"`
pluginInterfaceArray=`echo $pluginInterfaceJson | jq ".versions"`
echo "got plugin interface relations"

./getPluginInterfaceExactVersions.sh $pluginInterfaceLength "$pluginInterfaceArray"

if [[ $? -ne 0 ]]
then
echo "all plugin interfaces found... failed. exiting!"
exit 1
else
echo "all plugin interfaces found..."
fi

# get plugin version
pluginVersion=`cat ../build.gradle | grep -e "version =" -e "version="`
while IFS='"' read -ra ADDR; do
counter=0
for i in "${ADDR[@]}"; do
if [ $counter == 1 ]
then
pluginVersion=$i
fi
counter=$(($counter+1))
done
done <<< "$pluginVersion"

responseStatus=`curl -s -o /dev/null -w "%{http_code}" -X PUT \
https://api.supertokens.io/0/plugin \
-H 'Content-Type: application/json' \
-H 'api-version: 0' \
-d "{
\"password\": \"$SUPERTOKENS_API_KEY\",
\"planType\":\"FREE\",
\"version\":\"$pluginVersion\",
\"pluginInterfaces\": $pluginInterfaceArray,
\"name\": \"postgresql\"
}"`
if [ $responseStatus -ne "200" ]
then
echo "failed plugin PUT API status code: $responseStatus. Exiting!"
exit 1
fi

someTestsRan=false
while read -u 10 line
do
if [[ $line = "" ]]; then
continue
fi
i=0
currTag=`echo $line | jq .tag`
currTag=`echo $currTag | tr -d '"'`

currVersion=`echo $line | jq .version`
currVersion=`echo $currVersion | tr -d '"'`
piX=$(cut -d'.' -f1 <<<"$currVersion")
piY=$(cut -d'.' -f2 <<<"$currVersion")
piVersion="$piX.$piY"

someTestsRan=true

response=`curl -s -X GET \
"https://api.supertokens.io/0/plugin-interface/dependency/core/latest?password=$SUPERTOKENS_API_KEY&planType=FREE&mode=DEV&version=$piVersion" \
-H 'api-version: 0'`
if [[ `echo $response | jq .core` == "null" ]]
then
echo "fetching latest X.Y version for core given plugin-interface X.Y version: $piVersion gave response: $response"
exit 1
fi
coreVersionX2=$(echo $response | jq .core | tr -d '"')

response=`curl -s -X GET \
"https://api.supertokens.io/0/core/latest?password=$SUPERTOKENS_API_KEY&planType=FREE&mode=DEV&version=$coreVersionX2" \
-H 'api-version: 0'`
if [[ `echo $response | jq .tag` == "null" ]]
then
echo "fetching latest X.Y.Z version for core X.Y version: $coreVersionX2 gave response: $response"
exit 1
fi
coreVersionTag=$(echo $response | jq .tag | tr -d '"')

cd ../../
git clone [email protected]:supertokens/supertokens-root.git
cd supertokens-root

update-alternatives --install "/usr/bin/java" "java" "/usr/java/jdk-15.0.1/bin/java" 2
update-alternatives --install "/usr/bin/javac" "javac" "/usr/java/jdk-15.0.1/bin/javac" 2

pluginX=$(cut -d'.' -f1 <<<"$pluginVersion")
pluginY=$(cut -d'.' -f2 <<<"$pluginVersion")
echo -e "core,$coreVersionX2\nplugin-interface,$piVersion\npostgresql-plugin,$pluginX.$pluginY" > modules.txt
./loadModules
cd supertokens-core
git checkout $coreVersionTag
cd ../supertokens-plugin-interface
git checkout $currTag
cd ../supertokens-postgresql-plugin
git checkout dev-v$pluginVersion
cd ../
echo $SUPERTOKENS_API_KEY > apiPassword
export ONE_MILLION_USERS_TEST=1
./utils/setupTestEnv --cicd
./gradlew :supertokens-postgresql-plugin:test --tests io.supertokens.storage.postgresql.test.OneMillionUsersTest

if [[ $? -ne 0 ]]
then
cat logs/*
cd ../project/
echo "test failed... exiting!"
exit 1
fi
cd ../
rm -rf supertokens-root
cd project/.circleci
done 10<pluginInterfaceExactVersionsOutput

if [[ $someTestsRan = "true" ]]
then
echo "all tests ran"
exit 0
else
echo "no test ran"
exit 1
fi
19 changes: 2 additions & 17 deletions .circleci/doTests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -125,23 +125,8 @@ done 10<pluginInterfaceExactVersionsOutput

if [[ $someTestsRan = "true" ]]
then
echo "calling /core PATCH to make testing passed"
responseStatus=`curl -s -o /dev/null -w "%{http_code}" -X PATCH \
https://api.supertokens.io/0/plugin \
-H 'Content-Type: application/json' \
-H 'api-version: 0' \
-d "{
\"password\": \"$SUPERTOKENS_API_KEY\",
\"planType\":\"FREE\",
\"name\":\"postgresql\",
\"version\":\"$pluginVersion\",
\"testPassed\": true
}"`
if [ $responseStatus -ne "200" ]
then
echo "patch api failed"
exit 1
fi
echo "all tests ran"
exit 0
else
echo "no test ran"
exit 1
Expand Down
29 changes: 29 additions & 0 deletions .circleci/markPassed.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
pluginVersion=`cat ../build.gradle | grep -e "version =" -e "version="`
while IFS='"' read -ra ADDR; do
counter=0
for i in "${ADDR[@]}"; do
if [ $counter == 1 ]
then
pluginVersion=$i
fi
counter=$(($counter+1))
done
done <<< "$pluginVersion"

echo "calling /core PATCH to make testing passed"
responseStatus=`curl -s -o /dev/null -w "%{http_code}" -X PATCH \
https://api.supertokens.io/0/plugin \
-H 'Content-Type: application/json' \
-H 'api-version: 0' \
-d "{
\"password\": \"$SUPERTOKENS_API_KEY\",
\"planType\":\"FREE\",
\"name\":\"postgresql\",
\"version\":\"$pluginVersion\",
\"testPassed\": true
}"`
if [ $responseStatus -ne "200" ]
then
echo "patch api failed"
exit 1
fi
2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
- [ ] Issue this PR against the latest non released version branch.
- To know which one it is, run find the latest released tag (`git tag`) in the format `vX.Y.Z`, and then find the latest branch (`git branch --all`) whose `X.Y` is greater than the latest released tag.
- If no such branch exists, then create one from the latest released branch.

- [ ] When adding new recipes, ensure that its performance is being measured in the `OneMillionUsersTest`
## Remaining TODOs for this PR
- [ ] Item1
- [ ] Item2
9 changes: 8 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,14 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

## [7.0.0] - 2024-03-13

- Replace `TotpNotEnabledError` with `UnknownUserIdTotpError`.
- Support for MFA recipe
- Adds a new `useStaticKey` param to `updateSessionInfo_Transaction`
- This enables smooth switching between `useDynamicAccessTokenSigningKey` settings by allowing refresh calls to
change the signing key type of a session

## [6.0.0] - 2024-03-05

- Implements `deleteAllUserRoleAssociationsForRole`
Expand Down Expand Up @@ -151,7 +159,6 @@ CREATE INDEX IF NOT EXISTS app_id_to_user_id_primary_user_id_index ON app_id_to_
```
4. Run the new instance(s) of the core (version 7.0.0)


## [4.0.2]

- Fixes null pointer issue when user belongs to no tenant.
Expand Down
2 changes: 1 addition & 1 deletion build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ plugins {
id 'java-library'
}

version = "6.0.0"
version = "7.0.0"

repositories {
mavenCentral()
Expand Down
Binary file added jar/postgresql-plugin-7.0.0.jar
Binary file not shown.
2 changes: 1 addition & 1 deletion pluginInterfaceSupported.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"_comment": "contains a list of plugin interfaces branch names that this core supports",
"versions": [
"5.0"
"6.0"
]
}
Loading

0 comments on commit 9f5d928

Please sign in to comment.