Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

switch idp for regression test to login-gov #504

Merged
merged 4 commits into from
Aug 19, 2024

Conversation

TylerGuerrero-va
Copy link
Contributor

@TylerGuerrero-va TylerGuerrero-va commented Aug 15, 2024

Draft PR for changing IDP for the regression tests for SAML-Proxy to Login-Gov for better stability

  1. Two of the test's 'ICN Errors' and 'modify' are currently failing right now
  • With The 'ICN Errors' test its failing right now in the sandbox env but not the dev env. In sandbox its still able to get an access token and redirect the url even when using the user: [email protected] with the faulty ICN number, I did a manually test with Postman with that user in the dev env and did get the correct output, the expected Match Error page but in sandbox I am able to get the access token and it executes the redirect, so not sure why there is different behavior depending on the environment, Going to do further poking around to see what's going on with mpi

  • The 'modify' test I am still trying to figure out why its failing

  1. Link to sh file with export values to run the regression test locally: keybase://team/lighthouse_pivot/configs/saml-proxy/regression-test-envs.sh

  2. Added the new env vars to the param store to align with the regression tests, kept the old idme ones for history but added the new login-gov ones: SAML-proxy param store

Copy link
Collaborator

@crolarlibertyva crolarlibertyva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works as well as the previous ID me tests used to work. Maybe will try to recover the old 043 user for login.gov. I think that account still exists.

@crolarlibertyva crolarlibertyva self-requested a review August 16, 2024 10:55
@crolarlibertyva crolarlibertyva dismissed their stale review August 16, 2024 10:55

Still items to finish

@crolarlibertyva
Copy link
Collaborator

Before you can move this from draft, you need to align this with the new environment variables.

@TylerGuerrero-va TylerGuerrero-va marked this pull request as ready for review August 16, 2024 17:23
@TylerGuerrero-va TylerGuerrero-va requested a review from a team as a code owner August 16, 2024 17:23
Copy link
Collaborator

@crolarlibertyva crolarlibertyva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Makefile needs updating too.

  • regression:
    @:$(call check_defined, IMAGE, IMAGE variable should be saml-proxy-tests)
    docker run \
    -v "/var/run/docker.sock:/var/run/docker.sock" \
    --rm $(REPOSITORY)/$(NAMESPACE)/$(IMAGE):$(TAG) \
    --saml-proxy-url=$(SAML_PROXY_URL) \
    --client-id=$(CLIENT_ID) \
    --idp=$(IDP) \
    --authorization-url=$(AUTHORIZATION_URL) \
    --user-password=$(USER_PASSWORD) \
    --valid-user=$(VALID_USER_EMAIL) \
    --icn-error-user=$(ICN_ERROR_USER_EMAIL)
    --regression-test-timeout=$(REGRESSION_TEST_TIMEOUT)

Copy link
Collaborator

@crolarlibertyva crolarlibertyva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The entrypoint_test.sh needs adjusting as well.

@crolarlibertyva crolarlibertyva dismissed their stale review August 16, 2024 20:58

Changes incorporated

@mwise-va
Copy link
Contributor

mwise-va commented Aug 19, 2024

  • Tested locally, all test passing but modify test
      ✓ Happy Path (12799 ms)
      ✓ ICN Error (11394 ms)
      ✓ Replay (12419 ms)
      ✓ Empty SSO (2738 ms)
      ✓ Empty Assertion Response (2708 ms)
      ✓ 404 (2731 ms)
      ✕ modify (71101 ms)
  • Double checked Parameter Store values and they look good

@crolarlibertyva crolarlibertyva merged commit 81f0740 into master Aug 19, 2024
3 checks passed
@crolarlibertyva crolarlibertyva deleted the API-38828-regression-tests-login-gov branch August 19, 2024 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants