Created to facilitate simplified construction of distributed systems, the Atomizer library was built with simplicity in mind. Exposing a simple API which allows users to create "Atoms" (def) that contain the atomic elements of process logic for an application, which, when paired with an "Electron"(def) payload are executed in the Atomizer runtime.
To use Atomizer you will first need to add it to your module.
go get -u go.atomizer.io/engine@latestI highly recommend checking out the Test App for a functional example of the Atomizer framework in action.
To create an instance of Atomizer in your app you will need a Conductor (def). Currently there is an AMQP conductor that was built for Atomizer which can be found here: AMQP, or you can create your own conductor by following the Conductor creation instructions.
Once you have registered your Conductor using one of the Element Registration methods then you must create and register your Atom implementations following the Atom Creation instructions.
After registering at least one Conductor and one Atom in your Atomizer instance you can begin accepting requests. Here is an example of initializing the framework and registering an Atom and Conductor to begin processing.
// Initialize Atomizer
a := Atomize(ctx, &MyConductor{}, &MyAtom{})
// Start the Atomizer processing system
err := a.Exec()
if err != nil {
...
}Now that the framework is initialized you can push processing requests
through your Conductor for your registered Atoms and the Atomizer framework
will execute the Electron(def) payload by
bonding it with the correct registered Atom and returning the resulting
Properties over the Conductor back to the sender.
Since the concepts here are new to people seeing this for the first time a capstone team of students from the University of Illinois Springfield put together a test app to showcase a working implementation of the Atomizer framework.
This Test App implements a MonteCarlo π simulation using the Atomizer framework. The Atom implementation can be found here: MonteCarlo π. This implementation uses two Atoms. The first Atom "MonteCarlo" is a Spawner which creates payloads for the Toss Atom which is an Atomic Atom.
To run a simulation follow the steps laid out in this blog post which will describe how to pull down a copy of the Monte Carlo π docker containers running Atomizer.
This test application will allow you to run the same MonteCarlo π simulation from command line without needing to setup a NodeJS app or pull down the corresponding Docker container.
Thank you to
- Matthew N Meyer (@MatthewNormanMeyer)
- Megan Pugliese (@mugatupotamus)
- Nick Heiting (@nheiting)
- Benji Vesterby (@benjivesterby)
Creation of a conductor involves implementing the Conductor interface as described in the Atomizer library which can be seen below.
// Conductor is the interface that should be implemented for passing
// electrons to the atomizer that need processing. This should generally be
// registered with the atomizer in an initialization script
type Conductor interface {
// Receive gets the atoms from the source
// that are available to atomize
Receive(ctx context.Context) <-chan *Electron
// Complete mark the completion of an electron instance
// with applicable statistics
Complete(ctx context.Context, p *Properties) error
// Send sends electrons back out through the conductor for
// additional processing
Send(ctx context.Context, electron *Electron) (<-chan *Properties, error)
// Close cleans up the conductor
Close()
}Once you have created your Conductor you must then register it into the framework using one of the Element Registration methods for Atomizer.
The Atomizer library is the framework on which you can build your distributed system. To do this you need to create "Atoms"(def) which implement your specific business logic. To do this you must implement the Atom interface seen below.
type Atom interface {
Process(ctx context.Context, c Conductor, e *Electron,) ([]byte, error)
}Once you have created your Atom you must then register it into the framework using one of the Element Registration methods for Atomizer.
Electrons(def) are one of the most important
elements of the Atomizer framework because they supply the data necessary
for the framework to properly execute an Atom since the Atom implementation
is pure business logic.
// Electron is the base electron that MUST parse from the payload
// from the conductor
type Electron struct {
// SenderID is the unique identifier for the node that sent the
// electron
SenderID string
// ID is the unique identifier of this electron
ID string
// AtomID is the identifier of the atom for this electron instance
// this is generally `package.Type`. Use the atomizer.ID() method
// if unsure of the type for an Atom.
AtomID string
// Timeout is the maximum time duration that should be allowed
// for this instance to process. After the duration is exceeded
// the context should be canceled and the processing released
// and a failure sent back to the conductor
Timeout *time.Duration
// CopyState lets atomizer know if it should copy the state of the
// original atom registration to the new atom instance when processing
// a newly received electron
//
// NOTE: Copying the state of an Atom as registered requires that ALL
// fields that are to be copied are **EXPORTED** otherwise they are
// skipped
CopyState bool
// Payload is to be used by the registered atom to properly unmarshal
// the []byte for the actual atom instance. RawMessage is used to
// delay unmarshal of the payload information so the atom can do it
// internally
Payload []byte
}The most important part for an Atom is the Payload. This []byte holds data
which can be read in your Atom. This is how the Atom receives state information
for processing. Decoding the Payload is the responsibility of the Atom
implementation. The other fields of the Electron are used by the Atomizer
internally, but are available to an Atom as part of the Process method if
necessary.
Electrons are provided to the Atomizer framework through a registered Conductor, generally a Message Queue.
The results of Atom processing are contained in the Properties struct in
the Atomizer library. This struct contains metadata about the processing that
took place as well as the results or errors of the specific Atom which was
executed from a request.
// Properties is the struct for storing properties information after the
// processing of an atom has completed so that it can be sent to the
// original requestor
type Properties struct {
ElectronID string
AtomID string
Start time.Time
End time.Time
Error error
Result []byte
}The Result field is the []byte that is returned from the Atom that was
executed, and in general the Error property contains the error which was
returned from the Atom as well, however if there are errors in processing
an Atom that are not related to the internal processing of the Atom that
error will be returned on the Properties struct in place of the Atom error
as it is unlikely the Atom executed.
Atomizer exports a method called Events which returns an
go.devnw.com/events.EventStream and Errors which returns an
go.devnw.com/events.ErrorStream. When you call either of
these methods with a buffer of 0 they utilize an internal channel within the
go.devnw.com/event package which then MUST be monitored for events/errors
or it will block processing.
The purpose of these methods is to allow for users to monitor events
occurring inside of Atomizer. Because these use channels either pass a buffer
value to the method or handle the channel in a go routine to keep it from
blocking your application.
Along with the Events and Errors methods, Atomizer exports two important
structs. atomizer.Error and atomizer.Event. These contain information such
as the AtomID, ElectronID or ConductorID that the event applies to as well
as any message or error that may be part of the event.
Both atomizer.Error and atomizer.Event implement the fmt.Stringer
interface to make for easy logging.
atomizer.Error also implements the error interface as well as the
Unwrap method for nested errors.
atomizer.Event also implements the go.devnw.com/events.Event interface to
ensure the the event package can correctly handle the event.
// Event indicates an atomizer event has taken
// place that is not categorized as an error
// Event implements the stringer interface but
// does NOT implement the error interface
type Event struct {
// Message from atomizer about this error
Message string `json:"message"`
// ElectronID is the associated electron instance
// where the error occurred. Empty ElectronID indicates
// the error was not part of a running electron instance.
ElectronID string `json:"electronID"`
// AtomID is the atom which was processing when
// the error occurred. Empty AtomID indicates
// the error was not part of a running atom.
AtomID string `json:"atomID"`
// ConductorID is the conductor which was being
// used for receiving instructions
ConductorID string `json:"conductorID"`
}
// Error is an error type which provides specific
// atomizer information as part of an error
type Error struct {
// Event is the event that took place to create
// the error and contains metadata relevant to the error
Event *Event `json:"event"`
// Internal is the internal error
Internal error `json:"internal"`
}There are three methods in Atomizer for registering Atoms and Conductors.
Atoms and Conductors can be registered in an init method in your code as seen
below.
func init() {
err := atomizer.Register(&MonteCarlo{})
if err != nil {
...
}
}Atoms and Conductors can be passed as the second argument to the Atomize
method. This parameter is variadic and can accept any number of registrations.
a := Atomize(ctx, &MonteCarlo{})Direct registration happens when you use the Register method directly on an
Atomizer instance.
NOTE: The Register call can happen before or after the a.Exec() method call.
a := Atomize(ctx)
// Start the Atomizer processing system
err := a.Exec()
if err != nil {
...
}
// Register the Atom
err = a.Register(&MonteCarlo{})
if err != nil {
...
}