My Kafka Consumer and producer are in same deployment but it keep scale up and scale down again and again

[abhi-dwivedi]

HI Team

after deploy the Keda 1.5 because my k8s version is 1.15 and then I deploy the ScaledObject below is the configuration:

apiVersion: keda.k8s.io/v1alpha1
kind: ScaledObject
metadata:
  name: kafka-scaledobject
  labels:
    deploymentName: kafka-consumer # Required Name of the deployment we want to scale.
spec:
  scaleTargetRef:
    deploymentName: kafka-consumer # Required Name of the deployment we want to scale.
  pollingInterval: 5
  cooldownPeriod:  30
  minReplicaCount: 2   #Optional Default 0
  maxReplicaCount: 10  #Optional Default 100
  triggers:
  - type: kafka
    metadata:
      bootstrapServers: xyz:9092,abc:9092
      consumerGroup: kafka-consumer #ake sure that this consumer group name is the same one as the one that is consuming topics
      topic: kafka-consumer
      lagThreshold: "4" # Optional. How much the stream is lagging on the current consumer group

and then my deployment automatically keep doing scale up and scale down again nd again. Could you tell me why its doing of there is no work on worker

[tomkerkhove]

So your question is why it is scaling it up and down or what is your question exactly?

[abhi-dwivedi]

Any update?

[zroubalik]

Could you please point me to the place in the documentation where we are talking about running KEDA in parallel with an existing HPA? We should fix that.
KEDA is using HPA for the scaling, but that's completely different thing.

In KEDA 2.0 we have added an option to specify cpu/memory requests as a separate scaler, which are then mirrored to the HPA managed by KEDA. So you can use that. https://keda.sh/docs/2.0/scalers/cpu/

[tomkerkhove]

You can use our CPU scaler in the same scaledobject if it needs to be combined - https://keda.sh/docs/2.0/scalers/cpu/

[abhi-dwivedi]

Thanks for sharing the doument. Is there a way we can use both trigger kafka lagoffset and CPU and Memory also in the same ScaledObject?

[tomkerkhove]

You can just add multiple triggers to the same ScaledObject - https://keda.sh/docs/2.0/concepts/scaling-deployments/#scaledobject-spec