From ace6651e8bbe24a248bf285f755702ca15d7c3a8 Mon Sep 17 00:00:00 2001 From: Gavin Date: Mon, 12 Feb 2024 16:34:34 -0800 Subject: [PATCH 1/4] Add documentation re setting up environments Also delete the extremely crufty jetty-config.md and some build.gradle cruft --- README.md | 55 ++++++++++----- build.gradle | 4 -- documentation/Environments.md | 63 +++++++++++++++++ ...st_account_due_to_email_provider_change.md | 2 +- jetty-config.md | 67 ------------------- 5 files changed, 101 insertions(+), 90 deletions(-) create mode 100644 documentation/Environments.md delete mode 100644 jetty-config.md diff --git a/README.md b/README.md index a322429d..301e673c 100644 --- a/README.md +++ b/README.md @@ -203,7 +203,9 @@ the user. `DELETE /testmode/api/V2/testmodeonly/clear` Removes all test mode data from the system. -## Admin notes +## Administration + +### Notes * It is expected that this server always runs behind a reverse proxy (such as nginx) that enforces https / TLS and as such the auth server is configured to @@ -224,6 +226,11 @@ Removes all test mode data from the system. * Get Google OAuth2 creds [here](https://console.developers.google.com/apis) * Get OrcID creds [here](https://orcid.org/content/register-client-application-0) * Note that only the public API has been tested with the auth server. + +#### Migration notes + +##### 0.6.0 + * In version 0.6.0, the canonicalization algorithm for user display names changed and the database needs to be updated. * See the `--recanonicalize-display-names` option for the `manage_auth` script @@ -233,39 +240,46 @@ Removes all test mode data from the system. used to remove flags set on database objects to avoid reprocessing if the recanonicalize process does not complete. -## Requirements +### Requirements -Java 8 (OpenJDK OK) -MongoDB 2.6+ (https://www.mongodb.com/) -Jetty 9.3+ (http://www.eclipse.org/jetty/download.html) - (see jetty-config.md for version used for testing) -This repo (git clone https://github.com/kbase/auth2) +* Java 8 (OpenJDK OK) +* MongoDB 2.6+ (https://www.mongodb.com/) +* Jetty 9.3+ (http://www.eclipse.org/jetty/download.html) +* This repo (git clone https://github.com/kbase/auth2) -## To start server +### Starting the server -Either use `docker-compose --build -d`, which is easier and starts the server in test mode -(which can be configured in the docker-compose file), or: +#### Docker -start mongodb -if using mongo auth, create a mongo user -cd into the auth2 repo +The provided `Dockerfile` can be used to build and run an image. See the deployment template +in `deployment/conf/.templates` for the environment variables available to configure the +service - the `deploy.cfg.example` file provides documentation for these variables. -``` +`docker-compose --build -d` can be used to start a MongoDB instance and the auth server in +test mode (which can be configured via environment variables in the compose file). + +#### Manually + +* Start mongodb +* If using mongo auth, create a mongo user +* `cd` into the auth2 repo + +```shell ./gradlew war mkdir -p jettybase/webapps cp build/libs/auth2.war jettybase/webapps/ROOT.war cp templates jettybase/templates ``` -copy `deploy.cfg.example` to `deploy.cfg` and fill in appropriately +* copy `deploy.cfg.example` to `deploy.cfg` and fill in appropriately -``` +```shell export KB_DEPLOYMENT_CONFIG= cd jettybase ./jettybase$ java -jar -Djetty.port= /start.jar ``` -## Administer the server +### Perform initial setup Create the administration script: @@ -274,11 +288,16 @@ Create the administration script: Set a root password: `build/manage_auth -d -r` +* Note that the `deploy.cfg` file only needs accurate MongoDB connection information for use + with the auth CLI. + Login to a local account as `***ROOT***` with the password you set. Create a local account and assign it the create administrator role. That account can then be used to create further administrators (including itself) without needing to login as root. The root account can then be disabled. +To set up alternate login / link environments, see [Environments](documentation/Environments.md). + ### Revoking tokens in an emergency The simple HTML only test UI included with the server supports most administration functions, @@ -300,7 +319,7 @@ curl -X POST --cookie "kbase_session=" http:///admin/revokeal If the `token-cookie-name` deployment configuration value is not `kbase_session` change the request to match. -## Start & stop server w/o a pid +### Start & stop server w/o a pid `./jettybase$ java -DSTOP.PORT=8079 -DSTOP.KEY=foo -jar ~/jetty/jetty-distribution-9.3.11.v20160721/start.jar` `./jettybase$ java -DSTOP.PORT=8079 -DSTOP.KEY=foo -jar ~/jetty/jetty-distribution-9.3.11.v20160721/start.jar --stop` diff --git a/build.gradle b/build.gradle index 64258ed4..fad8346c 100644 --- a/build.gradle +++ b/build.gradle @@ -1,9 +1,5 @@ /* * This file was generated by the Gradle 'init' task. - * - * This generated file contains a sample Java application project to get you started. - * For more details take a look at the 'Building Java & JVM projects' chapter in the Gradle - * User Manual available at https://docs.gradle.org/7.4.2/userguide/building_java_projects.html */ plugins { diff --git a/documentation/Environments.md b/documentation/Environments.md new file mode 100644 index 00000000..46366e97 --- /dev/null +++ b/documentation/Environments.md @@ -0,0 +1,63 @@ +# Authentication service environments + +The auth service can support multiple environments, allowing it to service multiple hosts. +Alternate environments only affect account login and linking - all the environments share +tokens and are invisible from the perspective of a service contacting the auth server to look up +users or validate tokens. + +## Setup + +There are several steps to setting up an environment. The following steps assume the the default +or standard environment is `host1.org` and is properly configured in the auth service and +3rd party identity providers (IDPs). The new environment is assumed to be served at `host2.org`. +The environment names, respectively, will be `host1` and `host2`. + +* The login and link redirect URLs for `host2` must be registered with each IDP. + * They are usually going to be the same as the redirect URLs for `host1` with the replacement + of `host1.org` with `host2.org`. However, if URLs are rewritten by the remote proxy in + a different manner between `host1` and `host2` that must be taken into account. +* The login and link redirect URLs must be added to the `deploy.cfg` file as described in + `deploy.cfg.example`. If using the Docker image, they need to be added to the environment + in the `env1` and `auth_base_url_env1` keys or the `additional_config` key. + * For example: + +``` +identity-provider-Google-env-host2-login-redirect-url=https://host2.org/auth/login/complete/google +identity-provider-Google-env-host2-link-redirect-url=https://host2.org/auth/link/complete/google +``` + +* The new environment must be activated in the `deploy.cfg` file, either directly or via + environment variable if using Docker: + +``` +identity-provider-envs=, host2 +``` + +* On login or linking requests, the auth server must know which environment to use. There are two + options for providing this information: + * Send a header to the auth server defining the environment. The name of the header is + defined in the `deploy.cfg` `environment-header` key. For example, if the environment + header was set to `X-AUTH-ENV` the auth service would receive requests with the header + `X-AUTH-ENV: host2` in order to set the environment to `host2`. This is the simplest + and preferred approach, as it generally only needs a one line addition to the reverse + proxy configuration for the host, at which point all requests to the auth service get the + environment header. + * The UI code can specify the environment when contacting the auth2 service to start a login or + link flow via the `environment` form parameter. This requires the UI code to be + environment aware which is generally held to be undesired. +* Once the previous steps have been completed the service can be restarted with the updated + configuration. +* The final step requires an admin to go to the `/admin/config` endpoint of the auth server. + The new environment should now be visible in the configuration. While the IDP relevant + configuration is all specified in the `deploy.cfg` file to avoid passing secrets over the API + and to locate cohesive configuration elements together, the host side configuration is + updateable on the fly via the API. The redirect URLs for login and linking must be updated + appropriately for the host. In many cases, this will be simply replacing `host1.org` with + `host2.org` but see the previously mentioned caveats regarding updating the URLs. + +## Notes + +* When using multiple environments, it may be wise to clear the default environment and use + alternate environments for all the environments. This makes it less likely that a + misconfiguration could cause an alternate environment to use settings from the default + environment mistakenly. \ No newline at end of file diff --git a/documentation/recovering_lost_account_due_to_email_provider_change.md b/documentation/recovering_lost_account_due_to_email_provider_change.md index 54325eb2..e63dff47 100644 --- a/documentation/recovering_lost_account_due_to_email_provider_change.md +++ b/documentation/recovering_lost_account_due_to_email_provider_change.md @@ -101,7 +101,7 @@ should be impossible. 1. Add an option to the Auth command line tool to add identities to users to avoid incorrect DB updates. The CLI could, when possible, query the identity provider to ensure the information - provided command line is correct or pull additional information from the provider. + provided on the command line is correct or pull additional information from the provider. 2. Develop procedures for verifying a user's possession of an account and transferring trusted information needed to restore access to an account. 3. Develop and deploy user documentation recommending that users always have at least one diff --git a/jetty-config.md b/jetty-config.md deleted file mode 100644 index e0c5a5bf..00000000 --- a/jetty-config.md +++ /dev/null @@ -1,67 +0,0 @@ - -Java Environment: ------------------ - java.home = /usr/lib/jvm/java-8-openjdk-amd64/jre - java.vm.vendor = Oracle Corporation - java.vm.version = 25.91-b14 - java.vm.name = OpenJDK 64-Bit Server VM - java.vm.info = mixed mode - java.runtime.name = OpenJDK Runtime Environment - java.runtime.version = 1.8.0_91-8u91-b14-0ubuntu4~14.04-b14 - java.io.tmpdir = /tmp - user.dir = *snip*/jettybase - user.language = en - user.country = US - -Jetty Environment: ------------------ - jetty.version = 9.3.11.v20160721 - jetty.tag.version = master - jetty.home = *snip*/jetty/jetty-distribution-9.3.11.v20160721 - jetty.base = *snip*/jettybase - -Config Search Order: --------------------- - - ${jetty.base} -> *snip*/jettybase - ${jetty.home} -> *snip*/jetty/jetty-distribution-9.3.11.v20160721 - - -JVM Arguments: --------------- - (no jvm args specified) - -System Properties: ------------------- - (no system properties specified) - -Properties: ------------ - java.version = 1.8.0_91 - java.version.major = 1 - java.version.minor = 8 - java.version.revision = 0 - java.version.update = 91 - -Jetty Server Classpath: ------------------------ -Version Information on 11 entries in the classpath. -Note: order presented here is how they would appear on the classpath. - changes to the --module=name command line options will be reflected here. - 0: 3.1.0 | ${jetty.home}/lib/servlet-api-3.1.jar - 1: 3.1.0.M0 | ${jetty.home}/lib/jetty-schemas-3.1.jar - 2: 9.3.11.v20160721 | ${jetty.home}/lib/jetty-http-9.3.11.v20160721.jar - 3: 9.3.11.v20160721 | ${jetty.home}/lib/jetty-server-9.3.11.v20160721.jar - 4: 9.3.11.v20160721 | ${jetty.home}/lib/jetty-xml-9.3.11.v20160721.jar - 5: 9.3.11.v20160721 | ${jetty.home}/lib/jetty-util-9.3.11.v20160721.jar - 6: 9.3.11.v20160721 | ${jetty.home}/lib/jetty-io-9.3.11.v20160721.jar - 7: 9.3.11.v20160721 | ${jetty.home}/lib/jetty-security-9.3.11.v20160721.jar - 8: 9.3.11.v20160721 | ${jetty.home}/lib/jetty-servlet-9.3.11.v20160721.jar - 9: 9.3.11.v20160721 | ${jetty.home}/lib/jetty-webapp-9.3.11.v20160721.jar -10: 9.3.11.v20160721 | ${jetty.home}/lib/jetty-deploy-9.3.11.v20160721.jar - -Jetty Active XMLs: ------------------- - ${jetty.home}/etc/jetty.xml - ${jetty.home}/etc/jetty-http.xml - ${jetty.home}/etc/jetty-deploy.xml From 833e4aed898acc512e1264a4652ca224cd8aa036 Mon Sep 17 00:00:00 2001 From: Gavin Date: Sat, 17 Feb 2024 12:24:58 -0800 Subject: [PATCH 2/4] Remove unnecessary resources specification Now that they're separated out no need to specify file types --- build.gradle | 9 --------- 1 file changed, 9 deletions(-) diff --git a/build.gradle b/build.gradle index fad8346c..4fff21ec 100644 --- a/build.gradle +++ b/build.gradle @@ -65,15 +65,6 @@ jacocoTestReport { } } -// Custom java project layout -sourceSets { - test { - resources { - include "**/*.testdata" - } - } -} - javadoc { options { links "https://mongodb.github.io/mongo-java-driver/4.11/apidocs/mongodb-driver-sync/" From 0115527fd981725a134702def21c6de35a7af861 Mon Sep 17 00:00:00 2001 From: Gavin Date: Sun, 18 Feb 2024 12:17:03 -0800 Subject: [PATCH 3/4] Clarifications & expansions re environment docs --- README.md | 3 +++ documentation/Environments.md | 25 ++++++++++++++----------- 2 files changed, 17 insertions(+), 11 deletions(-) diff --git a/README.md b/README.md index 301e673c..ec606e80 100644 --- a/README.md +++ b/README.md @@ -217,6 +217,9 @@ Removes all test mode data from the system. proxy_pass http://localhost:20002/; proxy_cookie_path /login /auth/login; proxy_cookie_path /link /auth/link; + + # If using alternate environments (see below) + add_header X-AUTH-ENV "environment_name"; } * Get Globus creds [here](https://developers.globus.org) diff --git a/documentation/Environments.md b/documentation/Environments.md index 46366e97..3d536679 100644 --- a/documentation/Environments.md +++ b/documentation/Environments.md @@ -1,9 +1,19 @@ # Authentication service environments The auth service can support multiple environments, allowing it to service multiple hosts. -Alternate environments only affect account login and linking - all the environments share -tokens and are invisible from the perspective of a service contacting the auth server to look up -users or validate tokens. +Alternate environments only affect the account login and linking flows and are otherwise +not relevant to the auth system once a login or account link is complete. For example, +a service contacting the auth service to validate a login token does not need to concern +itself about which environment was used to create the token, and the auth service itself +does not record any information about which environment was used to create a login +token or link an account. + +## Recommendations + +* When using multiple environments, it may be wise to clear the default environment and use + alternate environments for all the environments. This makes it less likely that a + misconfiguration could cause an alternate environment to use settings from the default + environment mistakenly. ## Setup @@ -14,7 +24,7 @@ The environment names, respectively, will be `host1` and `host2`. * The login and link redirect URLs for `host2` must be registered with each IDP. * They are usually going to be the same as the redirect URLs for `host1` with the replacement - of `host1.org` with `host2.org`. However, if URLs are rewritten by the remote proxy in + of `host1.org` with `host2.org`. However, if URLs are rewritten by the reverse proxy in a different manner between `host1` and `host2` that must be taken into account. * The login and link redirect URLs must be added to the `deploy.cfg` file as described in `deploy.cfg.example`. If using the Docker image, they need to be added to the environment @@ -54,10 +64,3 @@ identity-provider-envs=, host2 updateable on the fly via the API. The redirect URLs for login and linking must be updated appropriately for the host. In many cases, this will be simply replacing `host1.org` with `host2.org` but see the previously mentioned caveats regarding updating the URLs. - -## Notes - -* When using multiple environments, it may be wise to clear the default environment and use - alternate environments for all the environments. This makes it less likely that a - misconfiguration could cause an alternate environment to use settings from the default - environment mistakenly. \ No newline at end of file From a9d1a4a882b21f8ada53dbce3fb7a010406b1496 Mon Sep 17 00:00:00 2001 From: Gavin Date: Wed, 21 Feb 2024 16:35:20 -0800 Subject: [PATCH 4/4] Clarify what is meant by hosts in environments doc --- documentation/Environments.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/documentation/Environments.md b/documentation/Environments.md index 3d536679..62ad9326 100644 --- a/documentation/Environments.md +++ b/documentation/Environments.md @@ -1,6 +1,10 @@ # Authentication service environments -The auth service can support multiple environments, allowing it to service multiple hosts. +The auth service can support multiple environments, allowing it to service multiple domains - +for a KBase example, `narrative.kbase.us`, `appdev.kbase.us`, and `narrative-dev.kbase.us` +are all served by the same auth service. This allows sharing user accounts and tokens between +environments and deploying alternate versions of UIs running on the same backend. + Alternate environments only affect the account login and linking flows and are otherwise not relevant to the auth system once a login or account link is complete. For example, a service contacting the auth service to validate a login token does not need to concern