Scaling on external API that reports queue length

[chinery]

Hello, apologies if this is a silly question, but I'm trying to work out if it's possible to get KEDA to scale using an external API that reports queue length – much like the way e.g. the RabbitMQ scaler works, but querying an external API like the Metrics API scaler. Either a ScaledObject or ScaledJob would be fine.

The Metrics API scaler requires a target value, and will scale up if the external API returns a value higher than your target. So it seems this won't work if the API returns a queue length, the number of pods that should be running (including ones that are in progress), once it starts increasing it won't stop.

Perhaps it's possible to set the target value to be a "target average value" instead in the HPA? So the metric API value divided by the number of workers. But on the Scaling Jobs page, it says "It is defined on the scaler side. e.g. queueLength on Azure Storage Queue scaler." – and this doesn't seem to be present on the Metrics API scaler.

Please correct me if I've misunderstood something. Thanks!

[chinery]

After spending a long time with premature confusion, I finally got around to actually setting up and testing a ScaledJob with the Metrics API, and everything seems to be working as I hoped it would after all. So I'm closing the topic.

To document my confusion for anyone else who might find this, there were a few factors. Firstly, a colleague had explained how they'd tried to use the Metrics API trigger for a queue length API, and they said it would always over-scale their deployment, constantly increasing the number of replicas until it hit the maximum whenever the queue length was >1, following the logic that you'd expect for a metric like CPU load (always scale up if api value>target). They asked me how I got my RabbitMQ scaler to work on a queue length, and I couldn't come up with a good answer other than the mode attribute of the RabbitMQ trigger which didn't exist on the Metric API trigger. So I went into this with extra trepidation.

(Now that I have a better understanding, I have asked whether they tried the AverageValue metricType, and they thought they had, and it seems like it should be the default anyway, but it was a while ago so they couldn't remember exactly. Ultimately I can't explain their experience!)

After comparing the code for RabbitMQ vs Metrics API I couldn't see anything the scaler was doing differently. The CREATE-NEW-SCALER.md readme specifies that GetMetricSpecForScaling should use TargetValue or TargetAverageValue. But neither the RabbitMQ nor Metrics API code used those, instead they just used Target. I'm still not sure I understand the discrepancy here.

I realised it seemed to be set as metricType=Value vs AverageValue, but couldn't find where in the docs. Eventually I found the metricType attribute in the ScaledObject spec doc (I had been focussing on ScaledJob and the triggers themselves, so hadn't spotted it), where it says you can set the value in spec.triggers[x].metricType. The code seemed to confirm that this should work for a ScaledJob too, as they both use the same list of ScaleTriggers objects, but when I tried it in my yaml, it failed validation. Putting it in spec.triggers[0].metadata.metricType worked though – at least it applied successfully...I'm not sure if it is doing anything, since AverageValue appears to be the default anyway, but for now things are working!

[chinery]

To follow up my final point

spec.triggers[0].metadata.metricType is not validated, so it is indeed doing nothing

I found why the field doesn't work for ScaledJobs, it is thanks to this line in scaledjob_patch.yaml

## triggers are shared by ScaledObjects and ScaledJobs and therefore generated for both, including all properties.
## since the metricType property is only supported for ScaledObject, removing it from the generated ScaledJob CRD
- op: remove
  path: /spec/versions/0/schema/openAPIV3Schema/properties/spec/properties/triggers/items/properties/metricType

I'm not sure why metricType isn't supported for ScaledJob but at least that explains it.

[zroubalik]

MetricType is used by HPA, SJ is not utilizing HPAs.

[chinery]

Yes I see, I’m getting confused, since the HPA calculates the number of replicas as, essentially, scale up if currentMetric/targetMetric > 1, the AverageValue vs Value distinction is important, but obviously ScaledJob doesn’t use that formula.

Thanks for commenting on my mad ramblings, even though closed.