Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add additional config layer to filesystem and add content to README.md and CONTRIBUTING.md #13

Merged
merged 13 commits into from
Jan 4, 2024
14 changes: 11 additions & 3 deletions .markdownlint.yaml
Original file line number Diff line number Diff line change
@@ -1,10 +1,18 @@
# Documentation: https://github.com/DavidAnson/markdownlint-cli2#configuration

# Setting the special default rule to true or false includes/excludes all rules by default.
"default": true
default: true

# Do not restrict line length: https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#MD013
"MD013": false
MD013:
code_blocks: false
headings: false
line_length: 120
tables: false

# Multiple headings with the same content: https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#MD024
MD024:
siblings_only: true

# Do not restrict inline HTML: https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#MD033
"MD033": false
MD033: false
34 changes: 34 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Contribution guide

Everyone is more than welcome to contribute to this project! That's what open source is all about!

In the following, we try to provide some help and guidance on how to participate, contribute and develop on the project.

## Raising an issue or feature request

The simplest way to contribute to the project is to use it!

Whenever you encounter any issues or missing features while using the project, please create a
[GitHub Issue](https://github.com/cluetec/lifeboat/issues) in this project and describe what you found or what you need.

### Git workflow

The development is orientated on GitHub flow. Therefore, every source code contribution needs to be provided through a
[GitHub Pull-Request](https://github.com/cluetec/lifeboat/pulls) against the `main` branch.

#### Releases & Versioning

Releases will be made by tagging a specific commit on the `main` branch. For the versioning we are using the
[Semantic Versioning Schema](https://semver.org/):

> Given a version number MAJOR.MINOR.PATCH, increment the:
>
> 1. MAJOR version when you make incompatible API changes
> 2. MINOR version when you add functionality in a backwards compatible manner
> 3. PATCH version when you make backwards compatible bug fixes

#### Patching of older release

Sometimes it's happening that security issues appear in older releases. Regarding the fact that the community behind
this project is not very large, we are not able to provide patches for each release we have ever published. Therefore,
_**we will only maintain the latest minor version with security patches and bug fixes!**_
5 changes: 3 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ MAIN_FILE_PATH ?= ./main.go
# Path where the binary will be stored
BINARY_FILE_PATH = out/lb

GOOS ?= linux
GOARCH ?= amd64
GOOS ?= darwin
GOARCH ?= arm64

# Setting SHELL to bash allows bash commands to be executed by recipes.
# Options are set to exit when a recipe line exits non-zero or a piped command fails.
Expand Down Expand Up @@ -73,3 +73,4 @@ ci: dependencies vet test build ## Run certain recipes for CI pipeline.
.PHONY: build
build: ## Build binary.
GOOS=${GOOS} GOARCH=${GOARCH} go build --ldflags="-s -w" -o ${BINARY_FILE_PATH} ${MAIN_FILE_PATH}
chmod u+x ${BINARY_FILE_PATH}
127 changes: 127 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,136 @@
# Lifeboat

Lifeboat is a backup tool provided by [cluetec GmbH](https://cluetec.de). Lifeboat enables the user to create backups
for a range of different source systems (e.g. PostgreSQL, MongoDB, HashiCorp Vault) and storing the backup also in
different destination storage systems (e.g. S3 Buckets, Azure Blob, S/FTP, local filesystem).

## ✅ Supported Systems

Source systems:

- [x] Local filesystem
- [ ] PostgreSQL (Not implemented yet)
- [ ] MongoDB (Not implemented yet)
- [ ] HashiCorp Vault (Not implemented yet)

Destination storage systems:

- [x] Local filesystem
- [ ] S3 Bucket (Not implemented yet)
- [ ] Azure Storage Account (Not implemented yet)
- [ ] S/FTP (Not implemented yet)

## 🔥 Motivation

cluetec has been offering [software development services](https://www.cluetec.de/development/digitale-transformation/)
for several years now. We have been contracted with the implementation and initial operation of the software for a large
number of projects. Here we repeatedly encountered the backup issue for various database systems. To avoid having to
copy and adapt shell scripts back and forth every time, we thought about turning these shell scripts into an application
that could be used to back up various database systems with as little adaptation effort as possible.

## 💻 Installation

At the moment we don't provide any installation methods. As we just started the project, we will start with providing
the compiled binaries within the GitHub Releases. Later container images as also helm charts will follow.

## ⚙️ Usage

Lifeboat is a CLI tool which makes it possible to use it in various kind of environments like on a local machine, in a
Unix cronjob, in Kubernetes, in a VM, wherever the user wants. As the tool needs a quite complex configuration it's
possible to provide a config file. In addition, it's also possible to provide all configuration via environment
variables.

```shell
$ lb
Lifeboat is a general purpose backup tool which supports backups for arbitrary sources and destinations.

Usage:
lb [command]

Available Commands:
backup Execute the backup.
completion Generate the autocompletion script for the specified shell
help Help about any command

Flags:
-h, --help help for lb

Use "lb [command] --help" for more information about a command.
```

### Configuration

The configuration is divided in three different parts:

1. General configs
2. Source system configs
3. Destination storage configs

If you want to set a config via an environment variable, just concatenate the yaml structure (in uppercase characters)
with underscores (`_`). For example like this: `SOURCE_FILESYSTEM_PATH`

To get an idea how the configuration can look like, have a look at the [`config.yaml`](./config.yaml).

#### General configuration

| Yaml Config | Default | Required | Description |
|-------------|---------|----------|------------------------------------------------------------------------------------------------|
| `logLevel` | `info` | 👎 | Defines the log level of the application. Possible value are: `debug`, `info`, `warn`, `error` |

#### Source system configuration

All configurations for the source system needs to be placed under the `source` object in the configuration.
Furthermore, we need to define, which source system we want to use. This will be done by setting the `type` field, like
it's done in the following example. The possible values can be found in the respective subsections for each source
system.

```yaml
source:
type: filesystem
```

##### Filesystem

The following configs need to be place under the following yaml structure:

```yaml
source:
type: filesystem
filesystem:
...
```

| Yaml Config | Default | Required | Description |
|-------------|---------|----------|--------------------------------------------------------------------------------------------------------------|
| `path` | | 👍 | Defines the path in the local filesystem (relative or absolute) to a file or folder that should be backed up. |

#### Destination storage configuration

All configurations for the destination storage systems needs to be placed under the `destination` object in the
configuration. Furthermore, we need to define, which destination storage system we want to use. This will be done by
setting the `type` field, like it's done in the following example. The possible values can be found in the respective
subsections for each destination storage system.

```yaml
destination:
type: filesystem
```

##### Filesystem

The following configs need to be place under the following yaml structure:

```yaml
destination:
type: filesystem
filesystem:
...
```

| Yaml Config | Default | Required | Description |
|-------------|---------|----------|------------------------------------------------------------------------------------------|
| `path` | | 👍 | Defines the path in the local filesystem (relative or absolute) where to store the file. |

## 🤝 Contribution

Everyone is more than welcome to contribute to this project! That's what open source is all about!
Expand Down
6 changes: 4 additions & 2 deletions config.yaml
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
source:
type: filesystem
path: /tmp/source.txt
filesystem:
path: /tmp/source.txt

destination:
type: filesystem
path: /tmp/destination.txt
filesystem:
path: /tmp/destination.txt

logLevel: debug
8 changes: 6 additions & 2 deletions internal/destination/filesystem/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,16 @@ import (

const Type = "filesystem"

type metaConfig struct {
Filesystem config
}

type config struct {
Path string
}

func newConfig(rc *globalConfig.ResourceConfig) (*config, error) {
var c config
var c metaConfig

err := mapstructure.Decode(rc, &c)

Expand All @@ -38,5 +42,5 @@ func newConfig(rc *globalConfig.ResourceConfig) (*config, error) {
return nil, err
}

return &c, nil
return &c.Filesystem, nil
}
8 changes: 6 additions & 2 deletions internal/source/filesystem/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,16 @@ import (

const Type = "filesystem"

type metaConfig struct {
Filesystem Config
}

type config struct {
Path string
}

func newConfig(rc *globalConfig.ResourceConfig) (*config, error) {
var c config
var c metaConfig

err := mapstructure.Decode(rc, &c)

Expand All @@ -38,5 +42,5 @@ func newConfig(rc *globalConfig.ResourceConfig) (*config, error) {
return nil, err
}

return &c, nil
return &c.Filesystem, nil
}