Unable to use Filebeat service due to hard-coded default.

[fifthecho]

Describe the bug

The Filebeat service definition has a hardcoded default of an output to an elasticsearch instance on localhost. If I want to have a filebeat output to logstash, however, the Filebeat service only supports one output and with my configured output and the default, the service will not start.

Steps To Reproduce

Steps to reproduce the behavior:
A configuration of:

services.filebeat = {
    enable = true;
    settings = {
      output = {
        logstash = {
          hosts = ["logstash5044"];
        };
      };
    };
    inputs = {
      journald.id = "everything";
      log = {
        enabled = true;
        paths = [
          "/var/log/*.log"
        ];
      };
    };

Generates the following file:

{
  "filebeat": {
    "inputs": [
      {
        "id": "everything",
        "type": "journald"
      },
      {
        "enabled": true,
        "paths": [
          "/var/log/*.log"
        ],
        "type": "log"
      }
    ],
    "modules": []
  },
  "output": {
    "elasticsearch": {
      "hosts": [
        "127.0.0.1:9200"
      ]
    },
    "logstash": {
      "hosts": [
        "logstash:5044"
      ]
    }
  }
}

and a config of

services.filebeat = {
    enable = true;
    settings = {
      output = {
        logstash = {
          hosts = ["logstash:5044"];
        };
        elasticsearch = {
          hosts = [];
        }
      };
    };
    inputs = {
      journald.id = "everything";
      log = {
        enabled = true;
        paths = [
          "/var/log/*.log"
        ];
      };
    };
  };

generates a config of

{
  "filebeat": {
    "inputs": [
      {
        "id": "everything",
        "type": "journald"
      },
      {
        "enabled": true,
        "paths": [
          "/var/log/*.log"
        ],
        "type": "log"
      }
    ],
    "modules": []
  },
  "output": {
    "elasticsearch": {
      "hosts": []
    },
    "logstash": {
      "hosts": [
        "logstash:5044"
      ]
    }
  }
}

Neither is a config that Filebeats will accept with an error of
Exiting: error unpacking config data: more than one namespace configured accessing 'output' (source:'/var/lib/filebeat/filebeat.yml')

Expected behavior

There shouldn't be a default so that a valid config file can be generated with an output other than ElasticSearch

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.71, NixOS, 23.11 (Tapir), 23.11.2774.3dc440faeee9`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.1`
 - channels(root): `"nixos-23.11"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

[felbinger]

Same issue here, I tried to get it working with output = lib.mkForce { logstash = ...} but it doesn't work also...

I copied the service, renamed services.filebeat to my-services.filebeat (except for systemd.services.filebeat), removed the default option for elasticsearch completly and it's working now.

If anyone has a solution how to clear the default value please tell us :)

[SuperSandro2000]

@roberth is that intended behavior? What do we miss?

[felbinger]

@SuperSandro2000
How do you normally deal with problems like this? Removing the option as I described above would be a breaking change, so that's probably not a good solution. Is it possible to customize the options in some way (e. g. output becomes submodule, which is set to this value by default, but can be overridden) to have the ability to disable this option?

[SuperSandro2000]

Then it is a breaking change. Sometimes it is unavoidable. We just need to make sure to add a release notes entry.

[felbinger]

Actually, no idea how I could oversee this in the first place. But it's possible to disable the output using services.filebeat.settings.output.elasticsearch.enabled = false; (@fifthecho (ping to force notification, in case you still need this)).

I am going to continue working on #322425 to improve the documentation of the module for the option search.