lookup('env', 'PWD') returns an empty string when deploying TDP via TDP-server running as a Linux service

[sergkudinov]

Error

A deployment via TDP-server or TDP-ui, when TDP-server running as a Linux service, craches with the error like:

2023-07-06 19:02:45,683 p=28584 u=root n=ansible | fatal: [dcgfy6pq]: FAILED! => changed=false
  msg: |-
    Could not find or access '/files/jmx_prometheus_javaagent-0.16.1.jar' on the Ansible Controller.

Cause

The base path to binaries files is defined in the variable binaries_local_dir: "{{ lookup('env', 'PWD') }}/files". This works fine when launching a deployment via TDP-lib (tdp deploy) or via TDP-server (or TDP-ui) when tdp-server is started as a process in terminal.

However, it doesn't work when TDP-server is started as a Linux service, because {{ lookup('env', 'PWD') }} returns an empty string. This because PWD is a shell variable, in the sense its maintained by the shell.

in the systemd unit there is no shell hence the issue

To reproduce

1. Install TDP-server while installing TDP-getting-started

2. Configure TDP-server as a Linux service (configuring tdp-server.service file)

Example /etc/systemd/system/tdp-server.service file

[Unit]
Description=TDP Server

[Service]
User={{ service_user }}
WorkingDirectory={{ service_directory }}
ExecStart={{ service_directory }}/venv/bin/python {{ service_directory }}/venv/bin/uvicorn tdp_server.main:app --reload
EnvironmentFile=-/etc/sysconfig/tdp-server
Environment=PATH={{ service_directory }}/venv/bin/:/sbin:/bin:/usr/sbin:/usr/bin
Environment=GIT_PYTHON_GIT_EXECUTABLE=/usr/bin/git
Restart=always

[Install]
WantedBy=multi-user.target

Example /etc/sysconfig/tdp-server file:

# common
TDP_COLLECTION_PATH=ansible_collections/tosit/tdp:ansible_collections/tosit/tdp_extra
TDP_RUN_DIRECTORY=.
TDP_VARS=./tdp_vars
# tdp-lib
TDP_DATABASE_DSN=sqlite:///sqlite.db
# tdp-server
PROJECT_NAME=tdp-server
BACKEND_CORS_ORIGINS="[\"http://{{ tdp_ui.host }}:{{ tdp_ui.port }}\"]"
SERVER_NAME={{ tdp_server.host }}
SERVER_HOST=http://{{ tdp_server.host }}:{{ tdp_server.port }}
# OPENID_CONNECT_DISCOVERY_URL=http://localhost:8080/auth/realms/tdp_server/.well-known/openid-configuration
OPENID_CLIENT_ID=tdp_server
DATABASE_DSN=sqlite:///sqlite.db
DO_NOT_USE_IN_PRODUCTION_DISABLE_TOKEN_CHECK=True
UVICORN_HOST={{ tdp_server.host }}
UVICORN_PORT={{ tdp_server.port }}

3. Start TDP-server service with systemctl start tdp-service

4. Run deployment with the command like:

curl -X POST http://UVICORN_HOST:UVICORN_PORT/api/v1/deploy/dag

Workaround

There are multiple choices:

1. When running TDP-server as a Linux service, provision it with the environment variable PWD equals the root folder of your files folder with binaries.

Exemple /etc/sysconfig/tdp-server:

# ... previous vars

# pointing the root folder of `tdp-getting-started` (same as `{{ service_directory }}`)
PWD=.

2. Overwrite the binaries_local_dir variable with smth without lookup, for example:

binaries_local_dir: "{{ service_directory }}/files"

Solution

Define binaries_local_dir differently without using PWD.

[sergkudinov]

@rpignolet what do you think, can we avoid using lookup('env', 'PWD')?