Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: proposal for exposing Sidecar configuration #467

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

chore: proposal for exposing Sidecar configuration #467

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions .github/issue-30.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Support for sidecar configuration

## Why?

For now only Sidecar image can be specified during configuration of Tailing Sidecar Operator.
This comes with one, but problematic requirement. In order to use custom Sidecar provider or custom configuration,
custom image has to be built and provided.

Example issues which could be resolved by exposing configuration:

- [Increase buffer size for fluent bit - lines are too long, skipping file](https://github.com/SumoLogic/tailing-sidecar/issues/226)
- [Allow tailing directory instead of specific file](https://github.com/SumoLogic/tailing-sidecar/issues/276)

## Implementation

### Configuration options

In order to resolve this issue and following the planned architecture ([Expose fluent-bit configurations](https://github.com/SumoLogic/tailing-sidecar/issues/30))
We would like to expose the following properties of Sidecar containers:

- `image` - this is already exposed
- `commands` - this is necessary if someone would like to modify arguments of the Sidecar provider
- `configuration` - we would like to expose configuration as Config Map name, so everyone could override it

There is one technical issue. We cannot mount configMap from different namespace than the application we are going to get logs from.
Possible solution is to allow Operator to create configmaps in other namespaces than it is installed.
It would require customer to create one configmap in Operator Namespace and Operator would take care of rest.

This should consists of the following suboptions:

- `configMapName` - name of Config Map where the configuration is
- `mountPath` - path to the directory where the configuration should be mounted

We may consider different options of providing configuration (Volumes, Secrets)

- `resources` - every Sidecar provider can require different resources to work correctly

### Providing Configuation

In order to provide default override values for operator I propose to add to Operator support for configuartion file.
In helm Chart it would be manged by configMap, and it would make it easier to handle complex configuartion structures
like `commands` and `resoures`.

### Extending TalingSidecarConfig Custom Resource Definition

I believe it should be possible to specify conifiguration per TailingSidecarConfig, but I would leave it for future.
Copy link
Contributor

@kasia-kujawa kasia-kujawa Mar 9, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Example TailingSidecarConfig with options to configure sidecar, example structure with sidecar key added:

apiVersion: tailing-sidecar.sumologic.com/v1
kind: TailingSidecarConfig
metadata:
  name: tailingsidecar-sample
spec:
  sidecar:
    config:
      # configuration which will be added to ConfigMap created by TailingSidecar Operator
    container:
      image: {}
      command: ""
      resources: {}
      # the rest of configuration options for Container Spec https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.22/#container-v1-core     
  annotationsPrefix: tailing-sidecar.sumologic.com
  podSelector:
    matchLabels:
      tailing-sidecar: "true"
  configs:
    sidecar-0:
      volumeMount:
        name: varlog
        mountPath: /var/log
      path: /var/log/example0.log
      annotations:
        sourceCategory: sourceCategory-0
        annotation-0: "true"
    sidecar-1:
      volumeMount:
        name: varlog
        mountPath: /var/log
      path: /var/log/example1.log
      annotations:
        sourceCategory: sourceCategory-1
        annotation-1: "true"
    sidecar-2:
      volumeMount:
        name: varlogconfig
        mountPath: /varconfig-new-dir/log
        readOnly: true
        mountPropagation: HostToContainer
      path: /varconfig-new-dir/log/example2.log
      annotations:
        sourceCategory: sourceCategory-2
        annotation-2: "true"


### Steps

The following steps are required to implement this functionality:

- [ ] Support configuration via config file

- [ ] in Operator
- [ ] in helm chart

- [ ] Add support for `configuration`

- [ ] in Operator
- [ ] in helm chart

- [ ] Add support for `commands`

- [ ] in Operator
- [ ] in helm chart

- [ ] Add support for `resources`

- [ ] in Operator
- [ ] in helm chart