Authentication failed on high load and multiple replicas of jans auth server

[aguskwee]

Hi,
I encountered an issue on Janssen 1.1.0 during high load and used multiple replicas of jans auth server. For your info, single replica works OK. I am using redis to store sessionId to speed up authentication process. I also disable useLocalCache properties. By doing this, I assume, auth server will always read from the Redis to get the information on the session attributes.

However, sometimes, it seems that the session attributes are not synced between 2 different pods. Let says the user completes step 1 authentication and the request is handled inside pod1 auth server. Then, on the step 2 authentication, the request is handled by pod2. Then when getting the session attributes, some of the attributes are missing causing the auth server thinks that it has not passed step 1 authentication, hence, it returns error.

Has anyone encountered this issue?

Regards.

[mzico]

Hello @aguskwee,

What type of high load we are talking about? Any number?
How many replicas we are talking about? Just any number more than 1?

[aguskwee]

Hi @mzico,

I tried to use k6 to do the load testing, with 200 vus with 5 minutes duration. I used 2-step authentication. First step is username + password, second step is phone OTP. I also used 3 replicas for auth server.

The issue is if first step (username + password) goes to replica 1 and the 2nd step goes to replica 2 (for example), then, sometimes the replica 2 does not have the info that this request has already passed step 1, therefore throwing the error.

Now, I try to use sticky session to make sure that the same request will always go to the same replica. So far, I don't encounter this anymore. But, more testing is needed to be done.

Is this the correct approach? Or is there any better approach?

[nynymike]

Are you using an Agama Script for the multi-step authn flow, or a Person Authn Interception script?

[aguskwee]

I am using Person Authn interception scripts.

[nynymike]

It's possible that under high load the redis cache is not updated in time for your step 2 request to hit pod 2. Session affinity is a good workaround. I wouldn't look for a more complex solution if this workaround works. We're dropping support for Couchbase, which had worked well for high concurrency deployments. But the reality is that most customers prefer RDBMS and Redis.