External Logging

[lollerfirst]

Description

Defines a ElasticsearchLayer which implements Layer and can subscribe to tracing events and transmit them to ElasticSearch (specifically, another service might require another apposite Layer)

ElastiSearccLayer creates a multi producer single consumer channel and spawns a task that continuously attempts to receive log messages. In the Layer trait implementation on_event is called everytime a thread logs message and it write that message to the channel.

[crodas]

It'd be great to have the JoinHandle in the struct and to kill it on drop

[lollerfirst]

Yea sorry for the delay.

[lollerfirst]

I opted to move the handle in the ElasticsearchLayer object without wrapping it in a Option because there is always a task in this case. Wdyt?

[lollerfirst]

Not quite sure how to test the API key in the "Authentication" header, since ElasticSearch won't mint one for me. It says some bullshit about the current license...

Currently only tested with http authentication embedded in the URL (e.g. http://elastic:password@localhost:9200) which is a bad auth.

[lollerfirst]

Ok scrap that. Managed to get one.

[lollerfirst]

Should we gate this under a feature?

[thesimplekid]

Thanks for this. I have a mild preference for putting this behind a feature, but since its in mintd and not cdk and only adds deps we mostly already have from cdk not a strong one.

Should we add elasticsearch search to the docker-compose file with a few lines of docs of how to run it? Or what is the recommended way to run with elasticsearch?

[thesimplekid]

Suggested change

	chrono = "0.4.39"
	serde_json = "1.0.133"
	reqwest = "0.12.9"
	chrono = "0.4.39"
	serde_json = "1"
	reqwest = "0.12"

Since for MSRV we need 24.3 of reqwest we should not set the patch version, cargo will use the lastest one we can.

This change plus rebasing onto main should fix the MSRV CI errors

[thesimplekid]

Suggested change

	use chrono::Utc;
	use reqwest::Client;
	use serde::Serialize;
	use std::fmt::Write;
	use tokio::sync::mpsc::{self, Sender};
	use tokio::task::JoinHandle;
	use tracing::field::{Field, Visit};
	use tracing::Event;
	use tracing_subscriber::layer::{Context, Layer};
	use tracing_subscriber::registry::LookupSpan;
	use std::fmt::Write;
	use chrono::Utc;
	use reqwest::Client;
	use serde::Serialize;
	use tokio::sync::mpsc::{self, Sender};
	use tokio::task::JoinHandle;
	use tracing::field::{Field, Visit};
	use tracing::Event;
	use tracing_subscriber::layer::{Context, Layer};
	use tracing_subscriber::registry::LookupSpan;

Std imports should be first, this will fix formatting ci

[thesimplekid]

Imports under the -Z minimal-versions are for the case where a dep of ours does not properly define their minimum deps I don't think that is the case here.

Do we need the dep in cdk I don't see where it is used?

[lollerfirst]

No I forgot to remove it because previously was working on cdk crate not cdk-mintd so that's left over from that..