Not seeing any HPA after deploying scaledobject

[Savitha-Amaldev]

After deploying scaled objects, I'm not able to verify whether it started scaling or not. when I describe the scaled object, I'm not seeing any events triggered. Before I used to see events as scaled object is active or if there is any warning or not How to check whether something is missing in my environment or not. Any help for debugging the scaled objects are greatly appreciated

[Savitha-Amaldev]

my scaled object looks like this:

---

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: prometheus-keda-scaledobject
namespace: aa
spec:
scaleTargetRef:
name: fastload-lcache-mem-only
pollingInterval: 60 # Optional. Default: 30 seconds
minReplicaCount: 2 # Optional. Default: 0
maxReplicaCount: 20 # Optional. Default: 100
triggers:
- type: prometheus
metadata:
metricName: nginx_ingress_controller_requests
query: nginx_ingress_controller_requests{job="metrics-ingress",namespace="aa", service="xyz"}
serverAddress: "http://localhost:9090"
threshold: "50"

[tomkerkhove]

Can you please provide the KEDA logs?

[tomkerkhove]

And what is the output of kubectl get scaledobjects?

[Savitha-Amaldev]

Can you please provide the KEDA logs?

I couldn't find much logs regarding this.. I used "kubectl logs scaledobjects/scaledobjectname -n scaledobject_deployed_namespace -c keda-operator" command and got a one line error which I'm mentioning below
error: no kind "ScaledObject" is registered for version "keda.sh/v1alpha1" in scheme "k8s.io/kubectl/pkg/scheme/scheme.go:28"

[Savitha-Amaldev]

And what is the output of kubectl get scaledobjects?

I can see my scaledobject in the namespace

[tomkerkhove]

Our CRD doesn't provide any logs, you need to check the operator of KEDA itself, can you check the logs there and share the output please?

Also, can you run kubectl get scaledobject and share the output please?

[zroubalik]

And I would add kubectl get crd | grep keda, seems like your CRDs are not correctly installed

[Savitha-Amaldev]

Our CRD doesn't provide any logs, you need to check the operator of KEDA itself, can you check the logs there and share the output please?

Also, can you run kubectl get scaledobject and share the output please?

This is how it looks.. crds are installed correctly as i can see one hpa created for one scaledobject. But none of the others are getting created

[tomkerkhove]

Can you also share the logs of the KEDA operator please?

[Savitha-Amaldev]

This is what Im seeing in operator metrics log and operator logs are as follows :

2021-06-03T09:15:48.260Z ERROR prometheus_scaler error executing prometheus query {"error": "Get "http://localhost:9090/api/v1/query?query=nginx_ingress_controller_requests%7Bjob%3D%22metrics-ingress%22%2Cnamespace%3D%22cpqa5%22%2C+service%3D%22xyzxyzxyzxyzxyzxyzxyz%22%7D&time=2021-06-03T09:15:48Z\": dial tcp [::1]:9090: connect: connection refused"}
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).checkScaledObjectScalers
/workspace/pkg/scaling/scale_handler.go:216
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).checkScalers
/workspace/pkg/scaling/scale_handler.go:205
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).startScaleLoop
/workspace/pkg/scaling/scale_handler.go:146

[tomkerkhove]

Oh I have not noticed your server address which is pointing to localhost, you'll need to fix that since KEDA is unable to connect to it (hence logs above)

[Savitha-Amaldev]

I changed it to serverip by editing the scaledobject script.. But no luck.. It's not reflecting in logs as well and if i'm deploying new scaledobject with serverip instead of localhost, no hpa is getting created. Usually how long does it takes to get a hpa activated?

[tomkerkhove]

If it can connect to Prometheus, it will create the HPA if it has to do 1<->n scaling.

[Savitha-Amaldev]

Any idea why it's not happening here? No hpa is getting created for mssql trigger as well.. I'm not getting any logs for that either