Add environment variable configuration for storage

[joao-zanutto]

@wwoytenko I've added some of the variables proposed in #41. I already tested locally that these changes work and if you agree to add this PR to the project, I can also add the related docs to the other PR.

I'm still getting used to Go and the project, but I couldn't figure out where I could add the toggle to switch between directory and s3 storage as an environment variable (the proposed STORAGE_TYPE env var)

[joao-zanutto]

@wwoytenko I figured how to add the STORAGE_TYPE env var and added the directory env var with a validation function, but couldn't figure out where I should call the Validate() function just by checking the s3 storage code

[wwoytenko]

Consider mentioned comments

[wwoytenko]

Hi! Greenmask employs Viper and Cobra frameworks for managing the configuration. This means you should not hardcode os.Getenv.

Have a look at an example

[joao-zanutto]

@wwoytenko I see, while investigating the code further I realized that Viper would take care of the configuration. As I understand, I believe that it was supposed to load all the configuration from environment variables automatically since viper.AutomaticEnv() is being called in the initConfig() function here.

However, I can't seem to make it work using any environment variables following Viper's docummentation (like STORAGE_DIRECTORY_PATH or COMMON_TMP_DIR). I'm still investigating but I'll surely drop those modifications.

[joao-zanutto]

@wwoytenko it seems I was able to fix it, it was simply the lack of the separator replacer. I opened a new PR #43 with the change.

On a side note I would like to propose a couple of other changes (and I'm unsure where would be the ideal location to communicate those, so I'm just adding them here):

• make config.yaml not required -- this would make it easier to run greenmask as a container without needing to rebuild the image with a config file or mounting a volume with the file
• set common.tmp_dir default value to linux /tmp
• change Dockerfile entrypoint to greenmask instead of /bin/bash -- this is more of a matter of following convention of other dockerized tools
• define a user greenmask on the container to avoid running it as root -- this is a security best practice I believe

I can work on all of those if you agree, I've tested a few of those changes locally already and they are really simple, the first one being the most impactful for my use case. To give you some context of my use case: I'm planning to run greenmask once a day as a ECS container inside a VPC in AWS, export dumps to S3 and also use it in Github actions to restore the dump to a postgres database so we can run database migration tests in our CI. So, having all the configurations as environment variables would be ideal, even the transformer configurations for the dump operation (which I still haven't tested yet).

[wwoytenko]

@joao-zanutto sure you can start doing it with all the listed fixes/features. Kindly remind you to review the documentation according to the changes will make.

Thank you!

[wwoytenko]

It is better to implement the configuration in the next way:

storage:
  type: "s3"

   s3_config: 
     bucket: "test"
  
   directory_config: 
     path: "/tmp"

We should introduce the storage.type parameter and depending on the type provided you can use either S3 config or Directory.

Keep in mind that the env vars mapping should be done via viper/cobra usage here

[wwoytenko]

I think it is important now, because AutomaticEnv does not work when we provide the variable STORAGE_DIRECTORY_PATH=/tmp. The default config file assembles the S3 storage (not the Directory). It would be fine to introduce the next keys in the config:

• storage.type
• storage.s3
• storage.directory

Then the variables will be dynamically working using the next variable naming:

• For path setting in Directory STORAGE_DIRECTORY_PATH=/tmp
• For bucket setting in S3 STORAGE_S3_BUCKET=testbucket

[wwoytenko]

Closing, because implemented in #51. Feel free to create new Merge Requests.