Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactoring UserIdentity logics #1097

Draft
wants to merge 174 commits into
base: syw-UID2-4159-token-gen-code-renaming
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
174 commits
Select commit Hold shift + click to select a range
e057a84
testing debug config value
Ian-Nara Oct 4, 2024
a70141b
testing debug config value
Ian-Nara Oct 4, 2024
ec27533
[CI Pipeline] Released Snapshot version: 5.40.49-alpha-24-SNAPSHOT
Oct 4, 2024
b24fbf8
don't use logback conf in debug mode
Ian-Nara Oct 4, 2024
3f1eb09
Merge remote-tracking branch 'origin/ian-UID2-4235-enclave-debug-thro…
Ian-Nara Oct 4, 2024
59d9dd7
add more log information
Ian-Nara Oct 4, 2024
b4aa9c4
[CI Pipeline] Released Snapshot version: 5.40.50-alpha-25-SNAPSHOT
Oct 4, 2024
da0e058
fix if statement
Ian-Nara Oct 4, 2024
7589983
Merge remote-tracking branch 'origin/ian-UID2-4235-enclave-debug-thro…
Ian-Nara Oct 4, 2024
7ec499f
[CI Pipeline] Released Snapshot version: 5.40.51-alpha-27-SNAPSHOT
Oct 4, 2024
3a377d5
add debug logback conf
Ian-Nara Oct 7, 2024
ea6d9fd
Merge remote-tracking branch 'origin/ian-UID2-4235-enclave-debug-thro…
Ian-Nara Oct 7, 2024
ba116b3
[CI Pipeline] Released Snapshot version: 5.40.52-alpha-28-SNAPSHOT
Oct 7, 2024
4803b7f
operator debug
Ian-Nara Oct 7, 2024
0de3185
[CI Pipeline] Released Snapshot version: 5.40.53-alpha-29-SNAPSHOT
Oct 7, 2024
7772761
Adding vsock and dante logging
thomasm-ttd Oct 7, 2024
4823823
[CI Pipeline] Released Snapshot version: 5.40.54-alpha-30-SNAPSHOT
Oct 7, 2024
407e548
Temp change the CID
thomasm-ttd Oct 8, 2024
3b65da6
Revert CID change
thomasm-ttd Oct 8, 2024
b7dd254
Adding debug port to eks pod
cody-constine-ttd Oct 8, 2024
efbc1ac
[CI Pipeline] Released Snapshot version: 5.40.55-alpha-31-SNAPSHOT
Oct 8, 2024
579fdc1
Remove assertions used to check argument validity in public methods
asloobq Oct 10, 2024
44093af
Exclude code and assertion from source needed only for tests
asloobq Oct 10, 2024
b91591b
replace assert for refresh token length with exception
asloobq Oct 10, 2024
0a82473
Replace assert with NPE in statsCollector public methods
asloobq Oct 10, 2024
21555c0
Update the handler for shutdown
thomasm-ttd Oct 11, 2024
d45f4cf
Turn of the exit on error
thomasm-ttd Oct 11, 2024
912a6a9
[CI Pipeline] Released Snapshot version: 5.40.56-alpha-34-SNAPSHOT
Oct 11, 2024
bd6438a
Convert assert in default switch case to exception. Refactor methods
asloobq Oct 11, 2024
6e964c6
Refactor error messages to static string
asloobq Oct 11, 2024
9c24935
refactor redundant code to call function
asloobq Oct 11, 2024
7e5ad5d
1. Refactor EncryptedTokenEncoder#encodeIntoIdentityResponse
sunnywu Oct 14, 2024
131d203
Use EncryptedTokenEncoder directly inside UIDOperatorService class in…
sunnywu Oct 14, 2024
d9845b7
Merge branch 'syw-UID2-4159-token-gen-code-renaming' into syw-UID2-41…
sunnywu Oct 14, 2024
b2fc597
Update version of shared
thomasm-ttd Oct 14, 2024
9a0d39e
[CI Pipeline] Released Snapshot version: 5.40.87-alpha-108-SNAPSHOT
Oct 14, 2024
5a0bb89
Merge branch 'syw-UID2-4159-token-gen-code-renaming' into syw-UID2-41…
sunnywu Oct 14, 2024
0e621d3
[CI Pipeline] Released Snapshot version: 5.40.88-alpha-109-SNAPSHOT
Oct 14, 2024
b507d8e
Fixed UserIdentity member variables as final and added constructor
sunnywu Oct 15, 2024
f5877b7
1. Addressed code review feedbacks
sunnywu Oct 15, 2024
ccc639f
Made UserIdentity abstract first
sunnywu Oct 15, 2024
dabff48
Fixed a few id variable to rawUid/firstLevelHash to make it clear. Fi…
sunnywu Oct 15, 2024
44ed5d3
Merge remote-tracking branch 'origin/syw-UID2-4159-token-gen-code-ren…
sunnywu Oct 15, 2024
60abeed
Convert asserts to exceptions in OptOut code
asloobq Oct 15, 2024
ce6bf60
[CI Pipeline] Released Snapshot version: 5.40.87-alpha-110-SNAPSHOT
Oct 15, 2024
19c6c7a
Fix partition timestamp check
asloobq Oct 16, 2024
6031d14
Merge remote-tracking branch 'origin/syw-UID2-4159-token-gen-code-ren…
sunnywu Oct 16, 2024
3b1718c
1. Removed the UserIdentity's refreshedAt field as that's redundant and
sunnywu Oct 16, 2024
0ab0a79
1. Removed privacyBits and establishedAt from UserIdentity/HashedDiiI…
sunnywu Oct 16, 2024
7538bfb
added comments
sunnywu Oct 16, 2024
249c25d
fixed variable name
sunnywu Oct 16, 2024
3624f15
Merge remote-tracking branch 'origin/syw-UID2-4159-token-gen-code-ren…
sunnywu Oct 16, 2024
a38af09
Merge remote-tracking branch 'origin/syw-UID2-4159-token-gen-code-ref…
sunnywu Oct 16, 2024
a890446
Changed assertion to warn
asloobq Oct 16, 2024
dff8dae
remove sleep and reset version
Ian-Nara Oct 16, 2024
29ae2c5
safe debugging enclave
Ian-Nara Oct 17, 2024
3c28951
safe debugging enclave
Ian-Nara Oct 17, 2024
58ca0e2
Merge branch 'main' into ian-UID2-4235-enclave-debug-without-debugger…
Ian-Nara Oct 17, 2024
e8916e5
[CI Pipeline] Released Snapshot version: 5.40.87-alpha-44-SNAPSHOT
Oct 17, 2024
78e61d0
Add comment for warning
asloobq Oct 17, 2024
aff43f2
fix if
Ian-Nara Oct 17, 2024
3ce1b21
[CI Pipeline] Released Snapshot version: 5.40.88-alpha-45-SNAPSHOT
Oct 17, 2024
1519f45
syslog
Ian-Nara Oct 18, 2024
3b50392
Merge remote-tracking branch 'origin/ian-UID2-4235-enclave-debug-with…
Ian-Nara Oct 18, 2024
21b407e
[CI Pipeline] Released Snapshot version: 5.40.89-alpha-46-SNAPSHOT
Oct 18, 2024
3cb8e47
syslog
Ian-Nara Oct 18, 2024
ef23487
Merge remote-tracking branch 'origin/ian-UID2-4235-enclave-debug-with…
Ian-Nara Oct 18, 2024
7eb0424
[CI Pipeline] Released Snapshot version: 5.40.90-alpha-47-SNAPSHOT
Oct 18, 2024
1d701e4
syslog
Ian-Nara Oct 18, 2024
f71550d
Merge remote-tracking branch 'origin/ian-UID2-4235-enclave-debug-with…
Ian-Nara Oct 18, 2024
b5ac102
[CI Pipeline] Released Snapshot version: 5.40.91-alpha-48-SNAPSHOT
Oct 18, 2024
4772242
syslog
Ian-Nara Oct 18, 2024
87a578b
Merge remote-tracking branch 'origin/ian-UID2-4235-enclave-debug-with…
Ian-Nara Oct 18, 2024
beacc48
[CI Pipeline] Released Snapshot version: 5.40.92-alpha-49-SNAPSHOT
Oct 18, 2024
5aa2004
syslog
Ian-Nara Oct 18, 2024
72fc624
Merge remote-tracking branch 'origin/ian-UID2-4235-enclave-debug-with…
Ian-Nara Oct 18, 2024
95a826c
syslog
Ian-Nara Oct 18, 2024
0e63e5a
debug
Ian-Nara Oct 18, 2024
aa8c292
[CI Pipeline] Released Snapshot version: 5.40.93-alpha-50-SNAPSHOT
Oct 18, 2024
c241870
debug
Ian-Nara Oct 18, 2024
f2ee9f5
Merge pull request #1099 from IABTechLab/ian-UID2-4235-enclave-debug-…
Ian-Nara Oct 18, 2024
3fc5dc3
[CI Pipeline] Released Patch version: 5.40.106
Oct 18, 2024
e08e05e
Merge pull request #1108 from IABTechLab/ci-f2ee9f5939369ecb70f2b8eab…
github-actions[bot] Oct 18, 2024
24ab08e
Remove unwanted assertions. Address review comments
asloobq Oct 18, 2024
029a25b
Merge main
thomasm-ttd Oct 20, 2024
9457a94
[CI Pipeline] Released Snapshot version: 5.40.107-alpha-111-SNAPSHOT
Oct 20, 2024
1afe2ee
Update version of shared
thomasm-ttd Oct 21, 2024
2f87dae
Merge branch 'tjm-UID2-4246-only-shutdown-on-401' of github.com:IABTe…
thomasm-ttd Oct 21, 2024
566120d
Merge pull request #1082 from IABTechLab/tjm-UID2-4246-only-shutdown-…
thomasm-ttd Oct 21, 2024
05e6377
[CI Pipeline] Released Minor version: 5.41.0
Oct 21, 2024
6303b64
Merge pull request #1110 from IABTechLab/ci-566120df6b8cee126e14b45d7…
github-actions[bot] Oct 21, 2024
4c1de75
Moved privacyBits out of FirstLevelHashIdentity and treat it as a sep…
sunnywu Oct 21, 2024
062f908
Added a new IdentityRequest constructor to set default values for pri…
sunnywu Oct 21, 2024
ce1ab8b
Merge pull request #1085 from IABTechLab/aaq-UID2-3393-remove-assertions
asloobq Oct 22, 2024
76a1345
Update src/main/java/com/uid2/operator/model/IdentityRequest.java
sunnywu Oct 28, 2024
5db6e4f
Replace privacy bits type to PrivacyBits class instance
sunnywu Oct 28, 2024
741237d
Replace privacy bits type to PrivacyBits class instance
sunnywu Oct 28, 2024
3900017
Replace privacy bits type to PrivacyBits class instance
sunnywu Oct 28, 2024
e5b104d
missing a newline
sunnywu Oct 28, 2024
e25cd19
1. Added PrivacyBitsTest/IdentityResponseTest/RawUidResponseTest classes
sunnywu Oct 28, 2024
986cf2e
Created verifyFirstLevelHashIdentityAndEstablishedAt method for verif…
sunnywu Oct 28, 2024
f445180
Created verifyFirstLevelHashIdentityAndEstablishedAt method for verif…
sunnywu Oct 28, 2024
e4e7d2d
Merge remote-tracking branch 'origin/main' into syw-UID2-4159-token-g…
sunnywu Oct 28, 2024
fa1c0f1
Checking privacy bits in more tests in UIDOperatorVerticleTest
sunnywu Oct 28, 2024
d745cc7
Change optout loading exception to a warning temporarily
asloobq Oct 28, 2024
20d695c
[CI Pipeline] Released Snapshot version: 5.41.1-alpha-116-SNAPSHOT
Oct 29, 2024
81646aa
Merge pull request #1125 from IABTechLab/aaq-UID2-3393-remove-asserti…
asloobq Oct 29, 2024
c950c6d
revered version in pom.xml
sunnywu Oct 30, 2024
a908e1f
added some comments and renamed refreshIdentity method param to input…
sunnywu Oct 30, 2024
04dfc14
fixed unit tests checking establishedAt between advertisingTokenInput…
sunnywu Oct 30, 2024
2edcb05
Refactored unit test codes to standardise the advertisingTokenInput/r…
sunnywu Oct 30, 2024
ca44945
Refactored unit test codes to standardise the advertisingTokenInput/r…
sunnywu Oct 30, 2024
da549b0
Code review feedback
sunnywu Nov 4, 2024
c6586a5
renamed IUIDOperatorService#mapIdentity to mapHashedDiiIdentity
sunnywu Nov 4, 2024
d76bceb
renamed to
sunnywu Nov 4, 2024
23a6f50
Code review feedback and renamed AdvertisingTOkenInput/RefreshTokenIn…
sunnywu Nov 4, 2024
bfde65c
add back use metrics for the operator-served SDKs
Ian-Nara Nov 5, 2024
3002d0d
Merge pull request #1139 from IABTechLab/ian-UID2-4408-add-old-js-sdk…
Ian-Nara Nov 5, 2024
7149e81
[CI Pipeline] Released Patch version: 5.41.6
Nov 5, 2024
bf42c0b
Merge pull request #1140 from IABTechLab/ci-3002d0d91bde6f24469f9ec64…
github-actions[bot] Nov 5, 2024
b880e15
[CI Pipeline] Released Patch version: 5.41.8
Nov 6, 2024
6efb434
Merge pull request #1141 from IABTechLab/ci-bf42c0b4e7ddc4ffc37984b70…
github-actions[bot] Nov 6, 2024
5c81dc8
[CI Pipeline] Released Patch version: 5.41.8
Nov 6, 2024
645ba1c
Merge pull request #1143 from IABTechLab/ci-bf42c0b4e7ddc4ffc37984b70…
github-actions[bot] Nov 6, 2024
facd5a6
Added counter for tcf
cody-constine-ttd Nov 6, 2024
de1569a
fixed bad config value:
cody-constine-ttd Nov 6, 2024
d58eb09
find participants on old sdks
Ian-Nara Nov 7, 2024
015091b
update workflow refs
Ian-Nara Nov 7, 2024
13e033e
Fixed extra tag
cody-constine-ttd Nov 7, 2024
8126001
support phone number
ssundahlTTD Nov 7, 2024
aae3f73
update shared version
Ian-Nara Nov 7, 2024
d99c64d
Merge pull request #1145 from IABTechLab/ian-find-participants-on-old…
Ian-Nara Nov 7, 2024
4976107
[CI Pipeline] Released Patch version: 5.41.15
Nov 7, 2024
9483d25
Merge remote-tracking branch 'origin/main' into sas-UID2-4272-support…
ssundahlTTD Nov 7, 2024
8b86c2c
UID2-4429 More efficient disk space utilisation when building operato…
atarassov-ttd Oct 28, 2024
2aba623
Merge pull request #1148 from IABTechLab/ant-4429-build-disk-space-cl…
atarassov-ttd Nov 8, 2024
62421ce
UID2-4430 Automatic allocation of cores to vsockpx and operator vertx
atarassov-ttd Oct 29, 2024
a543932
Merge pull request #1149 from IABTechLab/ant-UID2-4430-vsockpx-thread…
atarassov-ttd Nov 8, 2024
557754b
Merge pull request #1146 from IABTechLab/sas-UID2-4272-support-euid-p…
ssundahlTTD Nov 8, 2024
8886f91
Merge pull request #1144 from IABTechLab/cbc-add-metrics-to-tcf
cody-constine-ttd Nov 8, 2024
3dd9ba5
[CI Pipeline] Released Minor version: 5.42.0
Nov 8, 2024
4e36633
Merge pull request #1150 from IABTechLab/ci-8886f91bfed809f42591982c8…
github-actions[bot] Nov 8, 2024
564b274
Ignore CVE-2024-47535 and CVE-2024-7254 (#1153)
thomasm-ttd Nov 15, 2024
75dfd6d
[CI Pipeline] Released Patch version: 5.42.3
Nov 18, 2024
b83fa90
[CI Pipeline] Released Patch version: 5.42.4
Nov 18, 2024
0a51642
[CI Pipeline] Released Patch version: 5.42.5
Nov 18, 2024
1385f2f
[CI Pipeline] Released Patch version: 5.42.6
Nov 18, 2024
75a9c29
Added site name as contact for CSTG
cody-constine-ttd Nov 18, 2024
c20cc48
Added site ID
cody-constine-ttd Nov 18, 2024
b2e30d9
Removing API contact
cody-constine-ttd Nov 19, 2024
079bfe5
Merge pull request #1160 from IABTechLab/cbc-UID2-CSTG-log-api-contact
cody-constine-ttd Nov 19, 2024
4359b3d
Upgrading vertx version
cody-constine-ttd Nov 22, 2024
b23dcdc
[CI Pipeline] Released Snapshot version: 5.42.7-alpha-139-SNAPSHOT
Nov 22, 2024
3768f82
Adding new shared version
cody-constine-ttd Nov 22, 2024
0b1e00a
Merge branch 'cbc-UID2-4461-upgrade-vertx-version' of github.com:IABT…
cody-constine-ttd Nov 22, 2024
c687ccf
Merge pull request #1176 from IABTechLab/cbc-UID2-4461-upgrade-vertx-…
cody-constine-ttd Nov 25, 2024
3954046
Remove siteIdsUsingV4Tokens (#1178)
cYKatherine Nov 26, 2024
9f0f28b
UID2-3804 Remove `advertising_token_v4_percentage` env var (#1180)
cYKatherine Dec 3, 2024
7d2ad65
[CI Pipeline] Released Patch version: 5.42.17
Dec 3, 2024
eed0b69
Merge pull request #1182 from IABTechLab/ci-9f0f28b58612bc74206fb667e…
github-actions[bot] Dec 3, 2024
a002ad1
Code review feedback,
sunnywu Dec 5, 2024
242b08d
xuy-UID2-4341-change-log-level Refine response logging (#1183)
clarkxuyang Dec 5, 2024
4a1af7d
Xuy UI d2 2578 update share lib to add site name to metrics (#1179)
clarkxuyang Dec 5, 2024
06d49c8
[CI Pipeline] Released Minor version: 5.43.0
Dec 5, 2024
e148266
Merge pull request #1184 from IABTechLab/ci-4a1af7de1ed658dc0ea6f76ba…
github-actions[bot] Dec 5, 2024
ea0b247
Renamed IdentityRequest/IdentityResponse to TokenGenerateRequest/Resp…
sunnywu Dec 9, 2024
4476ee1
- Renamed RawUidResponse to IdentityMapResponseItem, MapRequest to Id…
sunnywu Dec 9, 2024
6bb1fb9
1. Renamed FirstLevelHashIdentity/HashedDiiIdentity/RawUidIdentity to…
sunnywu Dec 9, 2024
a8f0915
1. Renamed IdentityType to DiiType
sunnywu Dec 9, 2024
e5dea0e
Increase max form buffer
vishalegbert-ttd Dec 10, 2024
382facf
Merge pull request #1195 from IABTechLab/vse-UID2-4593-increase-max-f…
vishalegbert-ttd Dec 10, 2024
1b72e55
[CI Pipeline] Released Patch version: 5.43.4
Dec 10, 2024
b792232
Use correct raw UID version for UID2 UIDOperatorVerticleTest and rena…
sunnywu Dec 10, 2024
7875d98
Merge remote-tracking branch 'origin/main' into syw-UID2-4159-token-g…
sunnywu Dec 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/build-and-test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ on: [pull_request, push, workflow_dispatch]

jobs:
build:
uses: IABTechLab/uid2-shared-actions/.github/workflows/shared-build-and-test.yaml@v2
uses: IABTechLab/uid2-shared-actions/.github/workflows/shared-build-and-test.yaml@v3
with:
java_version: 21
secrets: inherit
2 changes: 1 addition & 1 deletion .github/workflows/publish-all-operators.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ jobs:
fetch-depth: 0

- name: Scan vulnerabilities
uses: IABTechLab/uid2-shared-actions/actions/vulnerability_scan_filesystem@v2
uses: IABTechLab/uid2-shared-actions/actions/vulnerability_scan_filesystem@v3
with:
scan_severity: HIGH,CRITICAL
failure_severity: CRITICAL
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/validate-image.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,15 +19,15 @@ on:

jobs:
build-publish-docker-default:
uses: IABTechLab/uid2-shared-actions/.github/workflows/shared-validate-image.yaml@v2
uses: IABTechLab/uid2-shared-actions/.github/workflows/shared-validate-image.yaml@v3
with:
failure_severity: ${{ inputs.failure_severity || 'CRITICAL,HIGH' }}
fail_on_error: ${{ inputs.fail_on_error || true }}
cloud_provider: 'default'
java_version: 21
secrets: inherit
build-publish-docker-aws:
uses: IABTechLab/uid2-shared-actions/.github/workflows/shared-validate-image.yaml@v2
uses: IABTechLab/uid2-shared-actions/.github/workflows/shared-validate-image.yaml@v3
with:
failure_severity: ${{ inputs.failure_severity || 'CRITICAL,HIGH' }}
fail_on_error: ${{ inputs.fail_on_error || true }}
Expand All @@ -36,7 +36,7 @@ jobs:
secrets: inherit
needs: [build-publish-docker-default]
build-publish-docker-gcp:
uses: IABTechLab/uid2-shared-actions/.github/workflows/shared-validate-image.yaml@v2
uses: IABTechLab/uid2-shared-actions/.github/workflows/shared-validate-image.yaml@v3
with:
failure_severity: ${{ inputs.failure_severity || 'CRITICAL,HIGH' }}
fail_on_error: ${{ inputs.fail_on_error || true }}
Expand All @@ -45,7 +45,7 @@ jobs:
secrets: inherit
needs: [build-publish-docker-aws]
build-publish-docker-azure:
uses: IABTechLab/uid2-shared-actions/.github/workflows/shared-validate-image.yaml@v2
uses: IABTechLab/uid2-shared-actions/.github/workflows/shared-validate-image.yaml@v3
with:
failure_severity: ${{ inputs.failure_severity || 'CRITICAL,HIGH' }}
fail_on_error: ${{ inputs.fail_on_error || true }}
Expand Down
4 changes: 3 additions & 1 deletion .trivyignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,6 @@
# See https://aquasecurity.github.io/trivy/v0.35/docs/vulnerability/examples/filter/
# for more details
# e.g.
# CVE-2022-3996

# https://thetradedesk.atlassian.net/browse/UID2-4460
CVE-2024-47535
9 changes: 6 additions & 3 deletions Makefile.eif
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@ all: build_eif
build_eif: uid2operator.eif euidoperator.eif

uid2operator.eif: build_artifacts build_configs build/proxies.nitro.yaml build/syslog-ng-client.conf build/syslog-ng-core_4.6.0-1_amd64.deb build/syslog-ng-ose-pub.asc build/entrypoint.sh build/vsockpx build/Dockerfile build/load_config.py build/make_config.py
cd build; docker build -t uid2operator . --build-arg JAR_VERSION=`cat package.version` --build-arg IMAGE_VERSION=`cat package.version`-`git show --format="%h" --no-patch`; docker save -o ./uid2operator.tar uid2operator; docker cp ./uid2operator.tar amazonlinux:/uid2operator.tar
cd build; docker build -t uid2operator . --build-arg JAR_VERSION=`cat package.version` --build-arg IMAGE_VERSION=`cat package.version`-`git show --format="%h" --no-patch`; docker save -o ./uid2operator.tar uid2operator; docker cp ./uid2operator.tar amazonlinux:/uid2operator.tar; rm -f ./uid2operator.tar
docker exec amazonlinux bash aws_nitro_eif.sh uid2operator

euidoperator.eif: build_artifacts build_configs build/proxies.nitro.yaml build/syslog-ng-client.conf build/syslog-ng-core_4.6.0-1_amd64.deb build/syslog-ng-ose-pub.asc build/entrypoint.sh build/vsockpx build/Dockerfile build/load_config.py build/make_config.py
cd build; docker build -t euidoperator . --build-arg IDENTITY_SCOPE='EUID' --build-arg JAR_VERSION=`cat package.version` --build-arg IMAGE_VERSION=`cat package.version`-`git show --format="%h" --no-patch`; docker save -o ./euidoperator.tar euidoperator; docker cp ./euidoperator.tar amazonlinux:/euidoperator.tar
cd build; docker build -t euidoperator . --build-arg IDENTITY_SCOPE='EUID' --build-arg JAR_VERSION=`cat package.version` --build-arg IMAGE_VERSION=`cat package.version`-`git show --format="%h" --no-patch`; docker save -o ./euidoperator.tar euidoperator; docker cp ./euidoperator.tar amazonlinux:/euidoperator.tar; rm -f ./euidoperator.tar
docker exec amazonlinux bash aws_nitro_eif.sh euidoperator

##################################################################################################################################################################
Expand All @@ -37,7 +37,7 @@ build/make_config.py: ./scripts/aws/make_config.py

.PHONY: build_configs

build_configs: build/conf/default-config.json build/conf/prod-uid2-config.json build/conf/integ-uid2-config.json build/conf/prod-euid-config.json build/conf/integ-euid-config.json build/conf/logback.xml
build_configs: build/conf/default-config.json build/conf/prod-uid2-config.json build/conf/integ-uid2-config.json build/conf/prod-euid-config.json build/conf/integ-euid-config.json build/conf/logback.xml build/conf/logback-debug.xml

build/conf/default-config.json: build_artifacts ./scripts/aws/conf/default-config.json
cp ./scripts/aws/conf/default-config.json ./build/conf/
Expand All @@ -57,6 +57,9 @@ build/conf/integ-euid-config.json: build_artifacts ./scripts/aws/conf/integ-euid
build/conf/logback.xml: build_artifacts ./scripts/aws/conf/logback.xml
cp ./scripts/aws/conf/logback.xml ./build/conf/

build/conf/logback-debug.xml: build_artifacts ./scripts/aws/conf/logback-debug.xml
cp ./scripts/aws/conf/logback-debug.xml ./build/conf/

build/Dockerfile: build_artifacts ./scripts/aws/Dockerfile
cp ./scripts/aws/Dockerfile ./build/

Expand Down
1 change: 0 additions & 1 deletion conf/docker-config.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
"storage_mock": true,
"refresh_token_expires_after_seconds": 86400,
"refresh_identity_token_after_seconds": 900,
"advertising_token_v3": false,
"refresh_token_v3": false,
"identity_v3": false,
"identity_scope": "uid2",
Expand Down
3 changes: 0 additions & 3 deletions conf/local-config.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,6 @@
"identity_token_expires_after_seconds": 3600,
"refresh_token_expires_after_seconds": 86400,
"refresh_identity_token_after_seconds": 900,
"advertising_token_v3": false,
"advertising_token_v4_percentage": 0,
"site_ids_using_v4_tokens": "",
"refresh_token_v3": false,
"identity_v3": false,
"identity_scope": "uid2",
Expand Down
1 change: 0 additions & 1 deletion conf/local-e2e-docker-private-config.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@
"identity_token_expires_after_seconds": 3600,
"refresh_token_expires_after_seconds": 86400,
"refresh_identity_token_after_seconds": 900,
"advertising_token_v3": false,
"refresh_token_v3": true,
"identity_v3": false,
"identity_scope": "uid2",
Expand Down
1 change: 0 additions & 1 deletion conf/local-e2e-docker-public-config.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@
"identity_token_expires_after_seconds": 3600,
"refresh_token_expires_after_seconds": 86400,
"refresh_identity_token_after_seconds": 900,
"advertising_token_v3": false,
"refresh_token_v3": true,
"identity_v3": false,
"identity_scope": "uid2",
Expand Down
1 change: 0 additions & 1 deletion conf/local-e2e-private-config.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@
"identity_token_expires_after_seconds": 3600,
"refresh_token_expires_after_seconds": 86400,
"refresh_identity_token_after_seconds": 900,
"advertising_token_v3": false,
"refresh_token_v3": true,
"identity_v3": false,
"identity_scope": "uid2",
Expand Down
1 change: 0 additions & 1 deletion conf/local-e2e-public-config.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@
"identity_token_expires_after_seconds": 3600,
"refresh_token_expires_after_seconds": 86400,
"refresh_identity_token_after_seconds": 900,
"advertising_token_v3": false,
"refresh_token_v3": true,
"identity_v3": false,
"identity_scope": "uid2",
Expand Down
1 change: 0 additions & 1 deletion conf/validator-latest-e2e-docker-public-config.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@
"identity_token_expires_after_seconds": 3600,
"refresh_token_expires_after_seconds": 86400,
"refresh_identity_token_after_seconds": 900,
"advertising_token_v3": false,
"refresh_token_v3": true,
"identity_v3": false,
"identity_scope": "uid2",
Expand Down
6 changes: 3 additions & 3 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@

<groupId>com.uid2</groupId>
<artifactId>uid2-operator</artifactId>
<version>5.40.86</version>
<version>5.43.4</version>

<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<vertx.version>4.5.3</vertx.version>
<vertx.version>4.5.11</vertx.version>
<vertx-maven-plugin.version>1.0.22</vertx-maven-plugin.version>
<junit-jupiter.version>5.11.2</junit-jupiter.version>
<junit-vintage.version>5.11.2</junit-vintage.version>
Expand All @@ -22,7 +22,7 @@
<enclave-aws.version>2.1.0</enclave-aws.version>
<enclave-azure.version>2.1.0</enclave-azure.version>
<enclave-gcp.version>2.1.0</enclave-gcp.version>
<uid2-shared.version>7.19.0</uid2-shared.version>
<uid2-shared.version>8.0.9</uid2-shared.version>
<image.version>${project.version}</image.version>
<maven.compiler.source>21</maven.compiler.source>
<maven.compiler.target>21</maven.compiler.target>
Expand Down
2 changes: 0 additions & 2 deletions scripts/aws/conf/default-config.json
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,5 @@
"failure_shutdown_wait_hours": 120,
"sharing_token_expiry_seconds": 2592000,
"validate_service_links": false,
"advertising_token_v4_percentage": 100,
"site_ids_using_v4_tokens": "",
"operator_type": "private"
}
15 changes: 15 additions & 0 deletions scripts/aws/conf/logback-debug.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
<configuration>
<statusListener class="ch.qos.logback.core.status.OnConsoleStatusListener" />

<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="net.logstash.logback.encoder.LogstashEncoder">
<jsonGeneratorDecorator class="net.logstash.logback.mask.MaskingJsonGeneratorDecorator">
<defaultMask>REDACTED - S3</defaultMask>
<value>\S+s3\.amazonaws\.com\/\S*X-Amz-Security-Token=\S+</value>
</jsonGeneratorDecorator>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="STDOUT" />
</root>
</configuration>
3 changes: 1 addition & 2 deletions scripts/aws/conf/prod-euid-config.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,8 @@
"refresh_identity_token_after_seconds": 3600,
"allow_legacy_api": false,
"identity_scope": "euid",
"advertising_token_v3": true,
"refresh_token_v3": true,
"enable_phone_support": false,
"enable_phone_support": true,
"enable_v1_phone_support": false,
"enable_v2_encryption": true
}
15 changes: 12 additions & 3 deletions scripts/aws/eks-pod/entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ CID=42
EIF_PATH=/home/uid2operator.eif
MEMORY_MB=24576
CPU_COUNT=6
DEBUG_MODE="false"

set -x

Expand All @@ -26,7 +27,7 @@ function setup_vsockproxy() {
echo "setup_vsockproxy"
VSOCK_PROXY=${VSOCK_PROXY:-/home/vsockpx}
VSOCK_CONFIG=${VSOCK_CONFIG:-/home/proxies.host.yaml}
VSOCK_THREADS=${VSOCK_THREADS:-$(( $(nproc) * 2 )) }
VSOCK_THREADS=${VSOCK_THREADS:-$(( ( $(nproc) + 1 ) / 2 )) }
VSOCK_LOG_LEVEL=${VSOCK_LOG_LEVEL:-3}
echo "starting vsock proxy at $VSOCK_PROXY with $VSOCK_THREADS worker threads..."
$VSOCK_PROXY -c $VSOCK_CONFIG --workers $VSOCK_THREADS --log-level $VSOCK_LOG_LEVEL --daemon
Expand Down Expand Up @@ -87,12 +88,20 @@ function update_config() {
{ set +x; } 2>/dev/null; { CPU_COUNT=$(echo $IDENTITY_SERVICE_CONFIG | jq -r '.enclave_cpu_count'); set -x; }
{ set +x; } 2>/dev/null; { MEMORY_MB=$(echo $IDENTITY_SERVICE_CONFIG | jq -r '.enclave_memory_mb'); set -x; }
fi

{ set +x; } 2>/dev/null; { DEBUG_MODE=$(echo $IDENTITY_SERVICE_CONFIG | jq -r '.debug_mode'); set -x; }

shopt -u nocasematch
}

function run_enclave() {
echo "starting enclave... --cpu-count $CPU_COUNT --memory $MEMORY_MB --eif-path $EIF_PATH --enclave-cid $CID"
nitro-cli run-enclave --cpu-count $CPU_COUNT --memory $MEMORY_MB --eif-path $EIF_PATH --enclave-cid $CID --enclave-name uid2-operator
if [ "$DEBUG_MODE" == "true" ]; then
echo "starting enclave... --cpu-count $CPU_COUNT --memory $MEMORY_MB --eif-path $EIF_PATH --enclave-cid $CID --debug-mode --attach-console"
nitro-cli run-enclave --cpu-count $CPU_COUNT --memory $MEMORY_MB --eif-path $EIF_PATH --enclave-cid $CID --enclave-name uid2-operator --debug-mode --attach-console
else
echo "starting enclave... --cpu-count $CPU_COUNT --memory $MEMORY_MB --eif-path $EIF_PATH --enclave-cid $CID"
nitro-cli run-enclave --cpu-count $CPU_COUNT --memory $MEMORY_MB --eif-path $EIF_PATH --enclave-cid $CID --enclave-name uid2-operator
fi
}

echo "starting ..."
Expand Down
22 changes: 14 additions & 8 deletions scripts/aws/entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,7 @@
LOG_FILE="/home/start.txt"

set -x
exec > $LOG_FILE
exec 2>&1
exec &> >(tee -a "$LOG_FILE")

set -o pipefail
ulimit -n 65536
Expand All @@ -17,11 +16,7 @@ ifconfig lo 127.0.0.1

# -- start vsock proxy
echo "Starting vsock proxy..."
/app/vsockpx --config /app/proxies.nitro.yaml --daemon --workers $(( $(nproc) * 2 )) --log-level 3

# -- setup syslog-ng
echo "Starting syslog-ng..."
/usr/sbin/syslog-ng --verbose
/app/vsockpx --config /app/proxies.nitro.yaml --daemon --workers $(( ( $(nproc) + 3 ) / 4 )) --log-level 3

# -- load config from identity service
echo "Loading config from identity service via proxy..."
Expand All @@ -42,6 +37,17 @@ do
sleep 2
done

DEBUG_MODE=$(jq -r ".debug_mode" < "${OVERRIDES_CONFIG}")

if [[ "$DEBUG_MODE" == "true" ]]; then
LOGBACK_CONF="./conf/logback-debug.xml"
else
LOGBACK_CONF="./conf/logback.xml"
# -- setup syslog-ng
echo "Starting syslog-ng..."
/usr/sbin/syslog-ng --verbose
fi

# check the config is valid. Querying for a known missing element (empty) makes jq parse the file, but does not echo the results
if jq empty "${OVERRIDES_CONFIG}"; then
echo "Identity service returned valid config"
Expand Down Expand Up @@ -101,6 +107,6 @@ java \
-Djava.library.path=/app/lib \
-Dvertx-config-path="${FINAL_CONFIG}" \
-Dvertx.logger-delegate-factory-class-name=io.vertx.core.logging.SLF4JLogDelegateFactory \
-Dlogback.configurationFile=./conf/logback.xml \
-Dlogback.configurationFile=${LOGBACK_CONF} \
-Dhttp_proxy=socks5://127.0.0.1:3305 \
-jar /app/"${JAR_NAME}"-"${JAR_VERSION}".jar
2 changes: 1 addition & 1 deletion scripts/aws/make_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ def apply_override(config, overrides, key, type):
config['optout_api_token'] = overrides['api_token']

# number of threads
config['service_instances'] = thread_count
config['service_instances'] = int((thread_count + 1) * 2 / 3)

# environment
if overrides.get('environment') == 'integ':
Expand Down
11 changes: 7 additions & 4 deletions scripts/aws/pipeline/amazonlinux2023.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,9 @@ FROM amazonlinux:2023
RUN dnf update -y
# systemd is not a hard requirement for Amazon ECS Anywhere, but the installation script currently only supports systemd to run.
# Amazon ECS Anywhere can be used without systemd, if you set up your nodes and register them into your ECS cluster **without** the installation script.
RUN dnf -y groupinstall "Development Tools"
RUN dnf -y install systemd vim-common wget git tar libstdc++-static.x86_64 cmake cmake3 aws-nitro-enclaves-cli aws-nitro-enclaves-cli-devel
RUN dnf -y groupinstall "Development Tools" \
&& dnf -y install systemd vim-common wget git tar libstdc++-static.x86_64 cmake cmake3 aws-nitro-enclaves-cli aws-nitro-enclaves-cli-devel \
&& dnf clean all

RUN systemctl enable docker

Expand All @@ -14,12 +15,14 @@ RUN wget https://www.inet.no/dante/files/dante-1.4.3.tar.gz \
&& sha256sum --check dante_checksum \
&& tar -xf dante-1.4.3.tar.gz \
&& cd dante-1.4.3; ./configure; make; cd .. \
&& cp dante-1.4.3/sockd/sockd ./
&& cp dante-1.4.3/sockd/sockd ./ \
&& rm -rf dante-1.4.3 dante-1.4.3.tar.gz

RUN git clone https://github.com/IABTechLab/uid2-aws-enclave-vsockproxy.git \
&& mkdir uid2-aws-enclave-vsockproxy/build \
&& cd uid2-aws-enclave-vsockproxy/build; cmake .. -DCMAKE_BUILD_TYPE=RelWithDebInfo; make; cd ../.. \
&& cp uid2-aws-enclave-vsockproxy/build/vsock-bridge/src/vsock-bridge ./vsockpx
&& cp uid2-aws-enclave-vsockproxy/build/vsock-bridge/src/vsock-bridge ./vsockpx \
&& rm -rf uid2-aws-enclave-vsockproxy

COPY ./scripts/aws/pipeline/aws_nitro_eif.sh /aws_nitro_eif.sh

Expand Down
1 change: 1 addition & 0 deletions scripts/aws/pipeline/aws_nitro_eif.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,6 @@ while (! docker stats --no-stream >/dev/null 2>&1); do
sleep 1
done
docker load -i $1.tar
rm -f $1.tar
nitro-cli build-enclave --docker-uri $1 --output-file $1.eif
nitro-cli describe-eif --eif-path $1.eif | jq -r '.Measurements.PCR0' | xxd -r -p | base64 > pcr0.txt
2 changes: 1 addition & 1 deletion scripts/aws/start.sh
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ function update_allocation() {
function setup_vsockproxy() {
VSOCK_PROXY=${VSOCK_PROXY:-/usr/bin/vsockpx}
VSOCK_CONFIG=${VSOCK_CONFIG:-/etc/uid2operator/proxy.yaml}
VSOCK_THREADS=${VSOCK_THREADS:-$(( $(nproc) * 2 )) }
VSOCK_THREADS=${VSOCK_THREADS:-$(( ( $(nproc) + 1 ) / 2 )) }
VSOCK_LOG_LEVEL=${VSOCK_LOG_LEVEL:-3}
echo "starting vsock proxy at $VSOCK_PROXY with $VSOCK_THREADS worker threads..."
$VSOCK_PROXY -c $VSOCK_CONFIG --workers $VSOCK_THREADS --log-level $VSOCK_LOG_LEVEL --daemon
Expand Down
2 changes: 0 additions & 2 deletions scripts/azure-cc/conf/default-config.json
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,5 @@
"failure_shutdown_wait_hours": 120,
"sharing_token_expiry_seconds": 2592000,
"validate_service_links": false,
"advertising_token_v4_percentage": 100,
"site_ids_using_v4_tokens": "",
"operator_type": "private"
}
2 changes: 0 additions & 2 deletions scripts/gcp-oidc/conf/default-config.json
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,5 @@
"failure_shutdown_wait_hours": 120,
"sharing_token_expiry_seconds": 2592000,
"validate_service_links": false,
"advertising_token_v4_percentage": 100,
"site_ids_using_v4_tokens": "",
"operator_type": "private"
}
6 changes: 3 additions & 3 deletions src/main/java/com/uid2/operator/Main.java
Original file line number Diff line number Diff line change
Expand Up @@ -414,7 +414,7 @@ private static Vertx createVertx() {
}

private static void setupMetrics(MicrometerMetricsOptions metricOptions) {
BackendRegistries.setupBackend(metricOptions);
BackendRegistries.setupBackend(metricOptions, null);

MeterRegistry backendRegistry = BackendRegistries.getDefaultNow();
if (backendRegistry instanceof PrometheusMeterRegistry) {
Expand Down Expand Up @@ -467,14 +467,14 @@ public DistributionStatisticConfig configure(Meter.Id id, DistributionStatisticC
.register(globalRegistry);
}

private Map.Entry<UidCoreClient, UidOptOutClient> createUidClients(Vertx vertx, String attestationUrl, String clientApiToken, Handler<Pair<Integer, String>> responseWatcher) throws Exception {
private Map.Entry<UidCoreClient, UidOptOutClient> createUidClients(Vertx vertx, String attestationUrl, String clientApiToken, Handler<Pair<AttestationResponseCode, String>> responseWatcher) throws Exception {
AttestationResponseHandler attestationResponseHandler = getAttestationTokenRetriever(vertx, attestationUrl, clientApiToken, responseWatcher);
UidCoreClient coreClient = new UidCoreClient(clientApiToken, CloudUtils.defaultProxy, attestationResponseHandler);
UidOptOutClient optOutClient = new UidOptOutClient(clientApiToken, CloudUtils.defaultProxy, attestationResponseHandler);
return new AbstractMap.SimpleEntry<>(coreClient, optOutClient);
}

private AttestationResponseHandler getAttestationTokenRetriever(Vertx vertx, String attestationUrl, String clientApiToken, Handler<Pair<Integer, String>> responseWatcher) throws Exception {
private AttestationResponseHandler getAttestationTokenRetriever(Vertx vertx, String attestationUrl, String clientApiToken, Handler<Pair<AttestationResponseCode, String>> responseWatcher) throws Exception {
String enclavePlatform = this.config.getString(Const.Config.EnclavePlatformProp);
String operatorType = this.config.getString(Const.Config.OperatorTypeProp, "");

Expand Down
Loading
Loading