Keda Jobs : expose pod information to containers

[subhamkhemka]

Hi

I am running a keda job and I am unable to expose pod information to containers as environment variables.

Yaml

apiVersion: v1
kind: Secret
metadata:
  name: keda-secrets
  namespace: default
data:
  AWS_ROLE_ARN: encoded_role_arn
---
apiVersion: keda.k8s.io/v1alpha1
kind: TriggerAuthentication
metadata:
  name: keda-trigger
  namespace: default
spec:
  secretTargetRef:
  - parameter: awsRoleArn
    name: keda-secrets
    key: AWS_ROLE_ARN
---
apiVersion: keda.k8s.io/v1alpha1
kind: ScaledObject
metadata:
  name: keda-scaler
  namespace: default
  labels:
    test: keda-scaler
spec:
  scaleType: job
  pollingInterval: 30  # Optional. Default: 30 seconds
  cooldownPeriod:  10 # Optional. Default: 300 seconds
  minReplicaCount: 1   # Optional. Default: 0
  maxReplicaCount: 100 # Optional. Default: 100
  jobTargetRef:
    parallelism: 1
    completions: 1
    activeDeadlineSeconds: 1800
    backoffLimit: 6
    template:
      spec:
        nodeSelector:
          node-type: "OnDemand"
        containers:
        - name: keda-scaler
          image: image:v1
          env:
          - name: REGION
            value: "us-east-1"
          - name: PYTHONUNBUFFERED
            value: "1"
          - name: NODE_NAME
            valueFrom:
              fieldRef:
                fieldPath: spec.nodeName
      restartPolicy: OnFailure
  triggers:
  - type: aws-sqs-queue
    authenticationRef:
      name: keda-trigger
    metadata:
      queueURL: aws_sqs_url
      awsRegion: "us-east-1"
      awsRoleArn: AWS_ROLE_ARN
      queueLength: "5"
---

Keda Operator Logs

{"level":"debug","ts":1610366225.5888655,"logger":"scalehandler","msg":"Handle Scale Job called","ScaledObject.Namespace":"default","ScaledObject.Name":"keda-scaler","ScaledObject.ScaleType":"job"}
{"level":"dpanic","ts":1610366225.5889187,"logger":"scalehandler","msg":"odd number of arguments passed as key-value pairs for logging","ScaledObject.Namespace":"default","ScaledObject.Name":"keda-scaler","ScaledObject.ScaleType":"job","ignored key":"NODE_NAME","stacktrace":"github.com/go-logr/zapr.handleFields\n\t/Users/zroubali/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:106\ngithub.com/go-logr/zapr.(*infoLogger).Info\n\t/Users/zroubali/go/pkg/mod/github.com/go-logr/[email protected]/zapr.go:70\ngithub.com/kedacore/keda/pkg/handler.(*ScaleHandler).resolveEnv\n\tkeda/pkg/handler/scale_handler.go:134\ngithub.com/kedacore/keda/pkg/handler.(*ScaleHandler).resolveJobEnv\n\tkeda/pkg/handler/scale_jobs.go:105\ngithub.com/kedacore/keda/pkg/handler.(*ScaleHandler).getJobScalers\n\tkeda/pkg/handler/scale_handler.go:246\ngithub.com/kedacore/keda/pkg/handler.(*ScaleHandler).handleScaleJob\n\tkeda/pkg/handler/scale_loop.go:53\ngithub.com/kedacore/keda/pkg/handler.(*ScaleHandler).handleScale\n\tkeda/pkg/handler/scale_loop.go:42\ngithub.com/kedacore/keda/pkg/handler.(*ScaleHandler).HandleScaleLoop\n\tkeda/pkg/handler/scale_loop.go:28"}
{"level":"debug","ts":1610366225.5889153,"logger":"scalehandler","msg":"cannot resolve env %s to a value. fieldRef and resourceFieldRef env are skipped","ScaledObject.Namespace":"default","ScaledObject.Name":"keda-scaler","ScaledObject.ScaleType":"job"}

Kindly help with this issue, thanks in advance.

[tomkerkhove]

It seems like we don't support the valueFrom but I might be wrong. @zroubalik @TsuyoshiUshio ?

[subhamkhemka]

Hi @tomkerkhove

Is there any other way by which I could expose pod information to the containers ?

[tomkerkhove]

Not that I'm aware of but maybe @zroubalik knows

[subhamkhemka]

Hi @zroubalik @TsuyoshiUshio

How can I expose pod information to the containers ?
Kindly help with this, Thanks!

[TsuyoshiUshio]

Hmm. I don't know much about the ScaleObject with type job, however, I'm sure ScaledJob v2 works with that expression.
This is the ScaledJob (v2) sample. It is just copy under the jobTargetRef. So that if it works with job, it works. I tested with Job, it works. :) Try ScaledJob v2. https://keda.sh/docs/2.1/concepts/scaling-jobs/

https://github.com/TsuyoshiUshio/KafkaScaledJobWithGo/blob/main/scaledjob.yaml
https://github.com/kedacore/keda/blob/main/pkg/scaling/executor/scale_jobs.go#L91