Calyptia Enterprise Dummy plugin

A Calyptia Enterprise plugin providing dummy input.

The canonical source of this repository including all copyright and licensing information is here: https://github.com/chronosphereio/calyptia-core-fluent-bit-dummy

The following example code implements an input plugin that works with separated input collecting threads that is introduced in Fluent Bit 1.9. It describes how to share context from the specified instance configuration to the input callback.

Every output plugin go through four callbacks associated to different phases:

Plugin Phase	Callback
Registration	FLBPluginRegister()
Initialization	FLBPluginInit()
Input Callback	FLBPluginInputCallback()
Exit	FLBPluginExit()

And Input Cleanup Callback is optional.

This callback is called right after Input Callback.

Plugin Phase	Callback
Input Cleanup Callback	FLBPluginInputCleanupCallback()

Plugin Registration

When Fluent Bit loads a Golang input plugin, it looks up and loads the registration callback that aims to populate the internal structure with plugin name and description:

//export FLBPluginRegister
func FLBPluginRegister(def unsafe.Pointer) int {
	return input.FLBPluginRegister(def, "gdummy", "dummy Go!")
}

Plugin Initialization

Before the engine starts, it initialize all plugins that were requested to start. Upon initialization a configuration context already exists, so the plugin can ask for configuration parameters or do any other internal checks:

//export FLBPluginInit
func FLBPluginInit(ctx unsafe.Pointer) int {
	return input.FLB_OK
}

The function must return FLB_OK when it initialized properly or FLB_ERROR if something went wrong. If the plugin reports an error, the engine will not load the instance.

Input Callback

When Fluent Bit wants to collect logs from Golang input plugin, the input callback will be triggered.

The callback will send a raw buffer of msgpack data with it proper bytes length into Fluent Bit core.

data will collect the assigned pointer and this passing pointer should be allocated by C style allocation (C.calloc/C.malloc).

//export FLBPluginInputCallback
func FLBPluginInputCallback(data *unsafe.Pointer, size *C.size_t) int {
	now := time.Now()
	// To handle nanosecond precision on Golang input plugin, you must wrap up time instances with input.FLBTime type.
	flb_time := input.FLBTime{now}
	message := map[string]string{"message": "dummy"}

	entry := []interface{}{flb_time, message}

	// Some encoding logs to msgpack payload stuffs.
	// It needs to Wait for some period on Golang input plugin side, until the new records are emitted.

	length := len(packed)
	*data = C.CBytes(packed)
	*size = C.size_t(len(packed))
	return input.FLB_OK
}

Input Cleanup Callback

For cleaning up some sort of allocated resources, this callback will be triggered after Input Callback.

This callback is mainly used for cleaning up resources not for the first argument of input callback.

//export FLBPluginInputCleanupCallback
func FLBPluginInputCleanupCallback(data unsafe.Pointer) int {
	// Some sort of cleaning up resources

	return input.FLB_OK
}

Returning Status Values

for more details about how to process the sending msgpack data into Fluent Bit core, please refer to the in_gdummy.go file.

When done, there are three returning values available:

Return value	Description
FLB_OK	The data have been processed normally.
FLB_ERROR	An internal error have ocurred, the plugin will not handle the set of records/data again.
FLB_RETRY	A recoverable error have ocurred, the engine can try to flush the records/data later.

Plugin Exit

When Fluent Bit will stop using the instance of the plugin, it will trigger the exit callback. e.g:

//export FLBPluginExit
func FLBPluginExit() int {
	return input.FLB_OK
}