LoudML wants create "output-bucket" influxdb database with "input-bucket" user/pass config

[toni-moreno]

Hello @regel

I've a little surprised because , I don't understand the input/output bucket logic

this is my configuration

buckets:
  - name: test-loudml #output
    type: influxdb
    addr: influxdb:8086
    database: loudml
    retention_policy: autogen
    measurement: loudml
    annotation_db: loudml
    dbuser: admin
    dbuser_password: admin1234
    create_database: false

  - name: influxdb-linux #input
    type: influxdb
    addr: influxdb:8086
    database: telegraf
    retention_policy: autogen
    measurement: loudml #<--- don't understand why needed here in input database
    dbuser: telegraf
    dbuser_password: telegraf
    annotation_db: loudml  #<--- don't understand why needed here in input database
    create_database: false
 
storage:
    path: /var/lib/loudml
server:
    listen: 0.0.0.0:8077

This is my loudml log.

loudml_1    | 192.168.48.3 - - [2020-07-31 05:15:47] "POST /models/telegraf_cpu_mean_usage_system_host_telegraf_time_5m/hooks HTTP/1.1" 201 123 0.004570
loudml_1    | INFO:root:job[1cc737b3-f35a-45b1-8516-e0f8bc0a9d6b] starting, nice=5
loudml_1    | INFO:root:connecting to influxdb on influxdb:8086, using database 'loudml' 
loudml_1    | ERROR:root:403: {"error":"error authorizing query: telegraf not authorized to execute statement 'CREATE DATABASE loudml', requires admin privilege"}

influxdb_1  | [httpd] 192.168.48.4 - telegraf [31/Jul/2020:05:15:47 +0000] "POST /query?db=loudml&q=CREATE+DATABASE+%22loudml%22 HTTP/1.1" 403 133 "-" "python-requests/2.23.0" e48673c5-d2ec-11ea-8bf8-0242c0a83002 3164

The loudml engine is trying to create the output database 'loudml' (defined in output bucket test-loudml) as user telegraf defined in input bucket influxdb-linux.

I've configured Loudml/Models to read data from input db (influxdb-linux) and write all results ( predictions/annotations ) in the output db (test-loudml) [right now both in the same influx server, but I have plans to separate both db's in different influxdb servers]

What I expect?

1. not create database because I've configured create_database = false
2. if create_database was true I expect loudml creates output database with user admin (not telegraf)

Could be a bug? or perhaps something in the configuration didn't understand?

Thank you very much

[toni-moreno]

Only as related info.

I've tested again with this data flow in mind.

[input-bucket] => [LOUDML] => [output-bucket](predictions/annotations)

And right now loudml are saving annotations in the input-bucket database instead of output-bucket as expected.
This behavior could be a problem in some environments where no admin user is unknown.

Thank you very much

[regel]

Hi Toni, ok I understand the setup. I reproduced a similar setup, with read-only access to a database and write-only access to a second database. There are multiple issues. Some of them are solved. See the setup here:

#383

Long story short: annotations were historically saved in a special 'chronograf' database for compatibility with the Javascript UI. Loud ML needs read-write access to annotations DB in order to read and write abnormal time windows and exclude them from future training operations.

I have to patch the Javascript UI to finish this bug fix. ETA e/o the month (vacations)

[toni-moreno]

Hello @regel , how about this issue? , we are planning to build a fix for this issue. Would you merge it when done?