Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fails to trigger in presence of other middlewares #13

Open
flargargoyl opened this issue Jul 10, 2022 · 27 comments
Open

Fails to trigger in presence of other middlewares #13

flargargoyl opened this issue Jul 10, 2022 · 27 comments

Comments

@flargargoyl
Copy link

using the available information, configured servcie
manager:
image: adyanth/traefik-container-manager-service
expose:
- 10000
volumes:
- '/var/run/docker.sock:/var/run/docker.sock'
networks:
t2_proxy:
labels:
- traefik.enable=true
- traefik.http.routers.manager.entrypoints=http
- traefik.http.routers.manager.rule=HostRegexp({host:.+})
- traefik.http.routers.manager.priority=1
- traefik.http.middlewares.manager.errors.status=404
- traefik.http.middlewares.manager.errors.service=manager
- traefik.http.middlewares.manager.errors.query=/
- traefik.http.routers.manager.middlewares=manager-timeout
- traefik.http.services.manager.loadbalancer.server.port=80
- traefik.http.middlewares.manager-timeout.plugin.traefik-container-manager.Name=generic-container-manager
- traefik.http.middlewares.manager-timeout.plugin.traefik-container-manager.ServiceUrl=http://manager:10000/api

lables on target service:

labels:
  - "traefik.enable=true"
  ## HTTP Routers
  - "traefik.http.routers.nyc-rtr.entrypoints=https,http"
  - "traefik.http.routers.nyc-rtr.rule=Host(`nyc.$DOMAINNAME`) || Host(`nyc.lan`)"
  - "traefik.http.routers.nyc-rtr.tls=true"
  - "traefik.http.routers.nyc-rtr.tls.options=myTLSOptions@file"
  ## Middlewares
  - "traefik.http.routers.nyc-rtr.middlewares=chain-no-auth@file" # No Authentication

- "traefik.http.routers.nyc-rtr.middlewares=chain-oauth@file" # Google OAuth 2.0

  ## HTTP Services
  - "traefik.http.routers.nyc-rtr.service=nyc-svc"
  - "traefik.http.services.nyc-svc.loadbalancer.server.port=8080"

manager

  - traefik.http.routers.nyc-rtr.middlewares=nyc-timeout
  - traefik.http.middlewares.nyc-timeout.plugin.traefik-container-manager.timeout=60
  - traefik.http.middlewares.nyc-timeout.plugin.traefik-container-manager.name=nyc
  - traefik.http.middlewares.nyc-timeout.plugin.traefik-container-manager.ServiceUrl=http://manager:10000/api
  - traefik-container-manager.name=nyc
  - traefik-container-manager.host=nyc 
  
  
  logs of the manager:
  
  today at 10:57:56 PMUsing name: nyc

today at 10:57:56 PM- Service nyc is up
today at 10:57:56 PM&{nyc 60 0xc0000560e0 true}
today at 10:57:56 PMSent delay
today at 10:58:02 PMSleeping 60
today at 10:59:02 PMStopping service nyc
today at 10:59:02 PMService: &{name:nyc timeout:60 host: path: time:0xc0000560e0 isHandled:true}
today at 10:59:02 PMContainers [{9d15618ecffef4ee6a7b15d05bd1309a5c122a8b248ffefc413ba4f66657f876 [/nyc] modem7/newyearcountdown:latest sha256:1f724b5e683ddf48732153b7bc8792a60e991ecbd9df46a3ad998b2c09dbbc1e /docker-entrypoint.sh nginx -g 'daemon off;' 1657482972 [{ 8080 0 tcp}] 0 0 map[com.docker.compose.config-hash:aaaed66675816690a37325ccfee1567536a9b314cddf1e7ce27e0059d042022a com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:1f724b5e683ddf48732153b7bc8792a60e991ecbd9df46a3ad998b2c09dbbc1e com.docker.compose.oneoff:False com.docker.compose.project:docker com.docker.compose.project.config_files:/home/flar/docker/docker-compose.yml com.docker.compose.project.working_dir:/home/flar/docker com.docker.compose.service:nyc com.docker.compose.version:2.6.0 maintainer:NGINX Docker Maintainers [email protected] traefik-container-manager.host:nyc traefik-container-manager.name:nyc traefik.enable:true traefik.http.middlewares.nyc-timeout.plugin.traefik-container-manager.ServiceUrl:http://manager:10000/api traefik.http.middlewares.nyc-timeout.plugin.traefik-container-manager.name:nyc traefik.http.middlewares.nyc-timeout.plugin.traefik-container-manager.timeout:60 traefik.http.routers.nyc-rtr.entrypoints:https,http traefik.http.routers.nyc-rtr.middlewares:nyc-timeout traefik.http.routers.nyc-rtr.rule:Host(nyc.zayir.org.ua) || Host(nyc.lan) traefik.http.routers.nyc-rtr.service:nyc-svc traefik.http.routers.nyc-rtr.tls:true traefik.http.routers.nyc-rtr.tls.options:myTLSOptions@file traefik.http.services.nyc-svc.loadbalancer.server.port:8080] running Up 2 minutes (healthy) {t2_proxy} 0xc0003ac030 []}]
today at 10:59:02 PMUsing name: nyc
today at 10:59:02 PMmodem7/newyearcountdown:latest running

After the last record, if i try to access the URL of the nyc service, nothing happens. URL returns 404.
Can it be related to the fact that after the container stopped, traefik dosen`t have its route anymore?

@adyanth
Copy link
Owner

adyanth commented Jul 11, 2022

Hey @flargargoyl have you set up the traefik-container-manager-service?

When the container is stopped, the service takes over listening to the traefik route (by listening to HostRegexp(`{host:.+}`) ), looks up the stopped container, and starts it back up.

@flargargoyl
Copy link
Author

@adyanth yes, i have. Copied what was in
https://github.com/adyanth/traefik-container-manager/blob/main/docker-compose.yml
for that purpose. Is there a way to see more deeper logs of it? havent found that in its repo. If that matters, im using subdomain method. Does host in that regexp needs to be exact name of the container, hostname of the container, or just the route used?

@flargargoyl
Copy link
Author

just got a guess. I have used another middleware handler to catch the http error codes and redirect them to be using a custom page. Feels like this may have broken this catching route. I`ll shut down that custom error page handler and re-test and come back with the results.

@adyanth
Copy link
Owner

adyanth commented Jul 11, 2022

Ahh yes, that would do it. You can try keeping the custom page middleware after this one to see if that helps.

For logs, the combination of traefik and the service logs usually gets to the issue.

@flargargoyl
Copy link
Author

flargargoyl commented Jul 11, 2022

ive checked both but it didn't gave me the reason for not starting the container up again, nor any attempt off doing so and ive noticed absence of the route for 'nyc' service, hence thought its related. I'll test around the error handler, its not required in my setup, just a gimmic anyways. Will come back here in either case to update. Thanks for your support.

@flargargoyl
Copy link
Author

flargargoyl commented Jul 11, 2022

@adyanth okay, that was not it. I think i'm lost in those configs.
My setup is using lables and some rules.
I also define middlewares in a file, so i can include them in a chain.
Where exactly i need to define this

http:
middlewares:
my-traefik-container-manager:
plugin:
traefik-container-manager:
name: whoami
serviceUrl: http://manager:10000
timeout: "60"

Do i need to change it`s names?

ive changed entropion to mine as i have http and https, so that's working.
However, no matter what i do, it STOPS the servcie after the timeout and i do not see any single attempt to start it.

heres manager and service compose parts

manager:
container_name: manager
image: adyanth/traefik-container-manager-service
expose:
- 10000
volumes:
- '/var/run/docker.sock:/var/run/docker.sock'
environment:
<<: *default-environment
networks:
- t2_proxy
labels:
- traefik.enable=true
- traefik.http.routers.manager.entrypoints=http,https
- traefik.http.routers.manager.rule=HostRegexp({host:.+})
- traefik.http.routers.manager.priority=1
- traefik.http.middlewares.manager.errors.status=404
- traefik.http.middlewares.manager.errors.service=manager
- traefik.http.middlewares.manager.errors.query=/
- traefik.http.routers.manager.middlewares=manager-timeout
- traefik.http.services.manager.loadbalancer.server.port=80
- traefik.http.middlewares.manager-timeout.plugin.traefik-container-manager.Name=generic-container-manager
- traefik.http.middlewares.manager-timeout.plugin.traefik-container-manager.ServiceUrl=http://manager:10000/api

nyc:
image: modem7/newyearcountdown:latest
container_name: nyc
hostname: nyc.${DEPLOYMENT_LOCATION}
restart: unless-stopped
environment:
<<: *default-environment
networks:
- t2_proxy

ports:

- "${NYC_PORT:-8080}:8080"

labels:
  - "traefik.enable=true"
  ## HTTP Routers
  - "traefik.http.routers.nyc-rtr.entrypoints=https,http"
  - "traefik.http.routers.nyc-rtr.rule=Host(`nyc.$DOMAINNAME`)"
  - "traefik.http.routers.nyc-rtr.tls=true"
  - "traefik.http.routers.nyc-rtr.tls.options=myTLSOptions@file"
#      ## Middlewares
  - "traefik.http.routers.nyc-rtr.middlewares=chain-no-auth@file" # No Authentication
#      - "traefik.http.routers.nyc-rtr.middlewares=chain-oauth@file" # Google OAuth 2.0
#      ## HTTP Services
  - "traefik.http.routers.nyc-rtr.service=nyc-svc"
  - "traefik.http.services.nyc-svc.loadbalancer.server.port=8080"
  - traefik.http.routers.nyc-rtr.middlewares=nyc-timeout
  - traefik.http.middlewares.nyc-timeout.plugin.traefik-container-manager.timeout=30
  - traefik.http.middlewares.nyc-timeout.plugin.traefik-container-manager.name=nyc
  - traefik.http.middlewares.nyc-timeout.plugin.traefik-container-manager.ServiceUrl=http://manager:10000/api

- traefik.http.services.nyc.loadbalancer.server.port=80

  - traefik-container-manager.name=nyc

- traefik-container-manager.path=/whoami # Prefix matched to the incoming path.

  - traefik-container-manager.host=nyc 
  
  What am i doing wrong, please?

@flargargoyl
Copy link
Author

to add to that, i`ve copied the whoami service configuration https://github.com/adyanth/traefik-container-manager/blob/main/docker-compose.yml from here, changed entripoint name (i use http and https). Manager uses both.

whoami service is running and i see its routers, servicies and middleware in Traefik, however i CANNOT reach it via my subdomain. Container is accessible via same network as Traefik, and is started, but something in routing dosen`t work.

Ive also saved this
http:
middlewares:
my-traefik-container-manager:
plugin:
traefik-container-manager:
name: whoami
serviceUrl: http://manager:10000
timeout: "60"

in a separate file in my dynamic configuration folder, but im not sure i understand how its going to be used properly due to namings?
all-and-all, im quite lost. Sorry if im being dumb. Service does stop the containers by their timeouts but does not start them back whatever i do, and i cant seems to see any log suggesting it trying or seeing it needs to, including in Traefik Debug.

@adyanth
Copy link
Owner

adyanth commented Jul 11, 2022

Can you please upload the yaml files or quote them using code blocks?

From your initial log:

today at 10:59:02 PMUsing name: nyc
today at 10:59:02 PMmodem7/newyearcountdown:latest running

That comes from https://github.com/adyanth/traefik-container-manager-service/blob/760c4570422cb52c129c789d6b2101cdd3ecafed/main.go#L287

Which means that the container was indeed started. Can you please let the container be stopped, send a request and run docker ps or docker compose ps to see if the container is actually started? It looks to me that it is, but after that, somehow traefik does not send traffic to it.

PS: You need to define both the plugin and the plugin config per container. If you share the yaml properly, I can take a look and correct it for you.

@flargargoyl
Copy link
Author

sorry, only now noticed broken formatting. Here`s my configuration for manager, nyc and whoami.
Double-checking now what you said about the container, tho ive seen it staying dead, but just to be sure im not missing anything.

manager:
container_name: manager
image: adyanth/traefik-container-manager-service
expose:
- 10000
volumes:
- '/var/run/docker.sock:/var/run/docker.sock'
environment:
<<: *default-environment
networks:
- t2_proxy
labels:
- traefik.enable=true
- traefik.http.routers.manager.entrypoints=http,https
- traefik.http.routers.manager.rule=HostRegexp({host:.+})
- traefik.http.routers.manager.priority=1
- traefik.http.middlewares.manager.errors.status=404
- traefik.http.middlewares.manager.errors.service=manager
- traefik.http.middlewares.manager.errors.query=/
- traefik.http.routers.manager.middlewares=manager-timeout
- traefik.http.services.manager.loadbalancer.server.port=80
- traefik.http.middlewares.manager-timeout.plugin.traefik-container-manager.Name=generic-container-manager
- traefik.http.middlewares.manager-timeout.plugin.traefik-container-manager.ServiceUrl=http://manager:10000/api

nyc:
image: modem7/newyearcountdown:latest
container_name: nyc
hostname: nyc.${DEPLOYMENT_LOCATION}
restart: unless-stopped
environment:
<<: *default-environment
networks:
- t2_proxy
labels:
- "traefik.enable=true"
- "traefik.http.routers.nyc-rtr.entrypoints=https,http"
- "traefik.http.routers.nyc-rtr.rule=Host(nyc.$DOMAINNAME)"
- "traefik.http.routers.nyc-rtr.tls=true"
- "traefik.http.routers.nyc-rtr.tls.options=myTLSOptions@file"
- "traefik.http.routers.nyc-rtr.middlewares=chain-no-auth@file" # No Authentication
- "traefik.http.routers.nyc-rtr.service=nyc-svc"
- "traefik.http.services.nyc-svc.loadbalancer.server.port=8080"
- traefik.http.routers.nyc-rtr.middlewares=nyc-timeout
- traefik.http.middlewares.nyc-timeout.plugin.traefik-container-manager.timeout=30
- traefik.http.middlewares.nyc-timeout.plugin.traefik-container-manager.name=nyc
- traefik.http.middlewares.nyc-timeout.plugin.traefik-container-manager.ServiceUrl=http://manager:10000/api
- traefik-container-manager.name=nyc
- traefik-container-manager.host=nyc

whoami:
image: containous/whoami
container_name: whoami
environment:
<<: *default-environment
networks:
- t2_proxy
labels:
- traefik.enable=true
- traefik.http.routers.whoami.entrypoints=http,https
- traefik.http.routers.whoami.rule=Host(whoami.$DOMAINNAME)
- traefik.http.routers.whoami.middlewares=whoami-timeout
- traefik.http.middlewares.whoami-timeout.plugin.traefik-container-manager.timeout=30
- traefik.http.middlewares.whoami-timeout.plugin.traefik-container-manager.name=whoami
- traefik.http.middlewares.whoami-timeout.plugin.traefik-container-manager.ServiceUrl=http://manager:10000/api
- traefik.http.routers.whoami.service=whoami
- traefik.http.services.whoami.loadbalancer.server.port=80
- traefik-container-manager.name=whoami
- traefik-container-manager.host=whoami

@flargargoyl
Copy link
Author

did another test. Made sure nyc is started, seen it`s "face", looking at logs of Manager:

today at 1:51:08 PMServer listening on port 10000.
today at 1:53:23 PMnyc 30
today at 1:53:23 PMService: &{name:nyc timeout:30 host: path: time:0xc0000a8070 isHandled:false}
today at 1:53:23 PMContainers [{c8b9bd74cfb097826ca9f37c183f24b6f2840d8d5e48e3e7fa444a8f3f64402f [/whoami] containous/whoami sha256:0f6fbbedd3777530ea3bedadf0a75b9aba805a55f6c5481ef0ebd762c5eeb818 /whoami 1657536654 [{ 80 0 tcp}] 0 0 map[com.docker.compose.config-hash:1201648ffe1da81d2323470e6e817f9829078107a9ce05f483c759751360c0f5 com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:0f6fbbedd3777530ea3bedadf0a75b9aba805a55f6c5481ef0ebd762c5eeb818 com.docker.compose.oneoff:False com.docker.compose.project:docker com.docker.compose.project.config_files:/home/flar/docker/docker-compose.yml com.docker.compose.project.working_dir:/home/flar/docker com.docker.compose.service:whoami com.docker.compose.version:2.6.0 traefik-container-manager.host:whoami traefik-container-manager.name:whoami traefik.enable:true traefik.http.middlewares.whoami-timeout.plugin.traefik-container-manager.ServiceUrl:http://manager:10000/api traefik.http.middlewares.whoami-timeout.plugin.traefik-container-manager.name:whoami traefik.http.middlewares.whoami-timeout.plugin.traefik-container-manager.timeout:30 traefik.http.routers.whoami.entrypoints:http,https traefik.http.routers.whoami.middlewares:whoami-timeout traefik.http.routers.whoami.rule:Host(whoami.zayir.org.ua) traefik.http.routers.whoami.service:whoami traefik.http.services.whoami.loadbalancer.server.port:80] running Up 2 minutes {t2_proxy} 0xc0004261d0 []} {0f183442f0eaf0a294de833c7369d6f66d350421f26bf17307dd028906135179 [/nyc] modem7/newyearcountdown:latest sha256:1f724b5e683ddf48732153b7bc8792a60e991ecbd9df46a3ad998b2c09dbbc1e /docker-entrypoint.sh nginx -g 'daemon off;' 1657536651 [{ 8080 0 tcp}] 0 0 map[com.docker.compose.config-hash:c06ddb72068e9362be5d99060e659119858bcec44f43a1456d5c775cf86cec0b com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:1f724b5e683ddf48732153b7bc8792a60e991ecbd9df46a3ad998b2c09dbbc1e com.docker.compose.oneoff:False com.docker.compose.project:docker com.docker.compose.project.config_files:/home/flar/docker/docker-compose.yml com.docker.compose.project.working_dir:/home/flar/docker com.docker.compose.service:nyc com.docker.compose.version:2.6.0 maintainer:NGINX Docker Maintainers [email protected] traefik-container-manager.host:nyc traefik-container-manager.name:nyc traefik.enable:true traefik.http.middlewares.nyc-timeout.plugin.traefik-container-manager.ServiceUrl:http://manager:10000/api traefik.http.middlewares.nyc-timeout.plugin.traefik-container-manager.name:nyc traefik.http.middlewares.nyc-timeout.plugin.traefik-container-manager.timeout:30 traefik.http.routers.nyc-rtr.entrypoints:https,http traefik.http.routers.nyc-rtr.middlewares:nyc-timeout traefik.http.routers.nyc-rtr.rule:Host(nyc.zayir.org.ua) traefik.http.routers.nyc-rtr.service:nyc-svc traefik.http.routers.nyc-rtr.tls:true traefik.http.routers.nyc-rtr.tls.options:myTLSOptions@file traefik.http.services.nyc-svc.loadbalancer.server.port:8080] running Up 2 minutes (healthy) {t2_proxy} 0xc0004261f0 []}]
today at 1:53:23 PMNo MATCH!!!
today at 1:53:23 PMUsing name: nyc
today at 1:53:23 PM- Service nyc is up
today at 1:53:23 PM&{nyc 30 0xc0000a8070 false}
today at 1:53:23 PMSent delay
today at 1:53:23 PMIn stopAfterTimeout
today at 1:53:23 PMSleeping 30
today at 1:53:53 PMStopping service nyc
today at 1:53:53 PMService: &{name:nyc timeout:30 host: path: time:0xc0000a8070 isHandled:true}
today at 1:53:53 PMContainers [{c8b9bd74cfb097826ca9f37c183f24b6f2840d8d5e48e3e7fa444a8f3f64402f [/whoami] containous/whoami sha256:0f6fbbedd3777530ea3bedadf0a75b9aba805a55f6c5481ef0ebd762c5eeb818 /whoami 1657536654 [{ 80 0 tcp}] 0 0 map[com.docker.compose.config-hash:1201648ffe1da81d2323470e6e817f9829078107a9ce05f483c759751360c0f5 com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:0f6fbbedd3777530ea3bedadf0a75b9aba805a55f6c5481ef0ebd762c5eeb818 com.docker.compose.oneoff:False com.docker.compose.project:docker com.docker.compose.project.config_files:/home/flar/docker/docker-compose.yml com.docker.compose.project.working_dir:/home/flar/docker com.docker.compose.service:whoami com.docker.compose.version:2.6.0 traefik-container-manager.host:whoami traefik-container-manager.name:whoami traefik.enable:true traefik.http.middlewares.whoami-timeout.plugin.traefik-container-manager.ServiceUrl:http://manager:10000/api traefik.http.middlewares.whoami-timeout.plugin.traefik-container-manager.name:whoami traefik.http.middlewares.whoami-timeout.plugin.traefik-container-manager.timeout:30 traefik.http.routers.whoami.entrypoints:http,https traefik.http.routers.whoami.middlewares:whoami-timeout traefik.http.routers.whoami.rule:Host(whoami.zayir.org.ua) traefik.http.routers.whoami.service:whoami traefik.http.services.whoami.loadbalancer.server.port:80] running Up 2 minutes {t2_proxy} 0xc000426208 []} {0f183442f0eaf0a294de833c7369d6f66d350421f26bf17307dd028906135179 [/nyc] modem7/newyearcountdown:latest sha256:1f724b5e683ddf48732153b7bc8792a60e991ecbd9df46a3ad998b2c09dbbc1e /docker-entrypoint.sh nginx -g 'daemon off;' 1657536651 [{ 8080 0 tcp}] 0 0 map[com.docker.compose.config-hash:c06ddb72068e9362be5d99060e659119858bcec44f43a1456d5c775cf86cec0b com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:1f724b5e683ddf48732153b7bc8792a60e991ecbd9df46a3ad998b2c09dbbc1e com.docker.compose.oneoff:False com.docker.compose.project:docker com.docker.compose.project.config_files:/home/flar/docker/docker-compose.yml com.docker.compose.project.working_dir:/home/flar/docker com.docker.compose.service:nyc com.docker.compose.version:2.6.0 maintainer:NGINX Docker Maintainers [email protected] traefik-container-manager.host:nyc traefik-container-manager.name:nyc traefik.enable:true traefik.http.middlewares.nyc-timeout.plugin.traefik-container-manager.ServiceUrl:http://manager:10000/api traefik.http.middlewares.nyc-timeout.plugin.traefik-container-manager.name:nyc traefik.http.middlewares.nyc-timeout.plugin.traefik-container-manager.timeout:30 traefik.http.routers.nyc-rtr.entrypoints:https,http traefik.http.routers.nyc-rtr.middlewares:nyc-timeout traefik.http.routers.nyc-rtr.rule:Host(nyc.zayir.org.ua) traefik.http.routers.nyc-rtr.service:nyc-svc traefik.http.routers.nyc-rtr.tls:true traefik.http.routers.nyc-rtr.tls.options:myTLSOptions@file traefik.http.services.nyc-svc.loadbalancer.server.port:8080] running Up 2 minutes (healthy) {t2_proxy} 0xc000426218 []}]
today at 1:53:53 PMNo MATCH!!!
today at 1:53:53 PMUsing name: nyc
today at 1:53:53 PMmodem7/newyearcountdown:latest running

tried accessing the web of nyc few more times. Checked container status with docker compose ps - its been nyc exited (0) every time.
Note the timings of Manager logs. It seems to state itll shut down, but then that its running, and does not react further?
In Traefik console, there`s obviously not nyc router as the container is down.
generic regexp router is there.
10 minutes and few more attemts to access the hyc web. No changes. Container is stopped, no new logs from the Manager.

@adyanth
Copy link
Owner

adyanth commented Jul 11, 2022

You also have whoami running right? Does it work as expected? Could you also share the logs from Traefik itself? The middleware logs are written to the traefik logs itself.

Let me try this on my system to make sure I am not missing something obvious here.

@flargargoyl
Copy link
Author

no, whoami is still not working. Container is up, traefik sees it, but i cant access it whatsoever, getting 404. It feels like theres something wrong - or something have changed - for how those lables should work\be defined.

let me see traefik logs for ncy and whoami, i need to clear few things from them but will post them soon.

@flargargoyl
Copy link
Author

nyc traefik logs
https://pastebin.com/kmC9LBzt

whoami traefik logs
https://pastebin.com/gSJ5FXBv

@flargargoyl
Copy link
Author

note, ive removed FQDN and email from those logs

@adyanth
Copy link
Owner

adyanth commented Jul 11, 2022

Can you please re pull and update the adyanth/traefik-container-manager-service image? It might not have been up to date with the repository. I just built/pushed it and tested it with the default whoami config and it works with both host and port configs and it worked fine for me.

Those logs look acceptable to me, the problem is with the manager service itself, if any. Please attach the manager logs after updating it if it fails.

@flargargoyl
Copy link
Author

updated the image, done docker compose up -d nyc manager traefik whoami --force-recreate

https://pastebin.com/frytaDeM manager logs

same deal. nyc is accesible once, killed with timeout and never spun up again.
whoami not accessible and returns 404.

was i correct taking this
http:
middlewares:
my-traefik-container-manager:
plugin:
traefik-container-manager:
name: whoami
serviceUrl: http://manager:10000
timeout: "60"

and putting into a .yml file in my dynamic configuration folder? Do i need to configure it for other services as well?

@flargargoyl
Copy link
Author

ive managed to make whoami work using my "template" of lables

whoami:
image: containous/whoami
environment:
<<: *default-environment
networks:
- t2_proxy
labels:
- "traefik.enable=true"
- "traefik.http.routers.whoami-rtr.entrypoints=https,http"
- "traefik.http.routers.whoami-rtr.rule=Host(whoami.$DOMAINNAME)"
- "traefik.http.routers.whoami-rtr.tls=true"
- "traefik.http.routers.whoami-rtr.tls.options=myTLSOptions@file"
- "traefik.http.routers.whoami-rtr.middlewares=chain-no-auth@file" # No Authentication
- "traefik.http.routers.whoami-rtr.service=whoami-svc"
- "traefik.http.services.whoami-svc.loadbalancer.server.port=80

  nyc still, "if it dies, it dies". 

@adyanth
Copy link
Owner

adyanth commented Jul 11, 2022

You do not need to add the dynamic config separately. The labels are all you need.

The logs you shared do not show any activity after the services were stopped due to timeout. This most definitely points to an issue with the configuration.

I tried to reproduce the issue. Adding the below section to the docker-compose.yml in the repository works fine for me.

I see one problem though, that is the container has a very slow starting healthcheck that does not let it start until 5-10 sec. Using the YAML below works every time for me. Even without it, you need to wait 10+ seconds once the container stops to start it back up.

  nyc:
    image: modem7/newyearcountdown
    healthcheck:
      disable: true
    labels:
      - traefik.enable=true
      - traefik.http.routers.nyc.entrypoints=web
      - traefik.http.routers.nyc.rule=Host(`localhost`)
      - traefik.http.routers.nyc.middlewares=nyc-timeout
      - traefik.http.middlewares.nyc-timeout.plugin.traefik-container-manager.timeout=5
      - traefik.http.middlewares.nyc-timeout.plugin.traefik-container-manager.name=nyc
      - traefik.http.middlewares.nyc-timeout.plugin.traefik-container-manager.ServiceUrl=http://manager:10000/api
      - traefik.http.services.nyc.loadbalancer.server.port=8080
      - traefik-container-manager.name=nyc
      - traefik-container-manager.host=localhost

@flargargoyl
Copy link
Author

flargargoyl commented Jul 11, 2022

i'll try it out. If clearly feels that middlewares i use clashes with this one somehow, but i'm not an expert enough to figure out which and why.

@adyanth
Copy link
Owner

adyanth commented Jul 11, 2022

I'd recommend trying with this middleware as the first one in the chain (or the last one, won't hurt to try both) once the default configuration works for you.

@flargargoyl
Copy link
Author

in my case i had to:
change entrypoint name
update host rule
add

  • "traefik.http.routers.nyc.service=nyc"
    then ive managed to get access to the container that started intially. However it does not start again when i access it after the stop. Ive no idea what's with my setup but its seems to be clear it is not issue with the code\feature. I will continue my investigations without bothering you further.
    Quite possibly i need to refresh my install and doublecheck for some depreciated settings, as it has been running for quite a while with barely any change, and as it is a homelab and i like adventure, it is autoupdated :D
    Thank you for your help. Closing the issue. Will continute to dig around when will have a bit more time for it, something inside my setup is bothering your code and so far ive no clue.

@adyanth
Copy link
Owner

adyanth commented Jul 11, 2022

Homelab and autoupdate

Haha, I'm in the same boat! Sure, do remember to reply here if you get it sorted, I'd like to know what it was. If you single it out back to this plugin, feel free to open it back :)

@flargargoyl
Copy link
Author

Ive managed to make it working with default configs.
However, if i add any other middlewares, it does not work - not even shutting the container down.
Managers logs just say no matches found.

Is there a way to diagnose this or build around? Im using middlewares for security and authentication.

And another question. Is there a way to make it work with TCP service \ entrypoint? I've tried, but didn't managed to, traefik was not creating a tcp router. I believe there`s no way to attach custom middleware to a tcp router?

@adyanth
Copy link
Owner

adyanth commented Jul 26, 2022

Hey @flargargoyl I remember using other middlewares along with this but it's been quite some time and I might be remembering it wrong. I will look into that when I get some time.

Regarding TCP, yes, there is no way to attach a middleware to a TCP router, and since TCP does not need SNI, there isn't much you can do anyway. Maybe you can add a http middleware to a dummy http route pointing to the same container just as a turn on switch, but then you would need to keep the timeouts large enough.

@flargargoyl
Copy link
Author

@adyanth i see, understood. Will it help if i give you my middlewares configurations? Nothing too fancy, just ensuring proper HTTPS and oauth redirections.
Honestly, this functionality of having on-demand containers is a dream for some portion of my home lab infrastructure, but i`m basically useless in terms of code itself to make it work myself, but i can bring some value from infrastructure point of view :)

@adyanth
Copy link
Owner

adyanth commented Jul 26, 2022

Sure, please do. If it is any middleware, I should be good to troubleshoot, but if not, your config would definitely help.

@adyanth adyanth reopened this Jul 26, 2022
@adyanth adyanth changed the title services is properly stopped after a timeout, but not started if accessed again Fails to trigger in presence of other middlewares Jul 26, 2022
@flargargoyl
Copy link
Author

middlewares.toml

[http.middlewares]
[http.middlewares.middlewares-basic-auth]
[http.middlewares.middlewares-basic-auth.basicAuth]
realm = "Traefik2 Basic Auth"
usersFile = "/shared/.htpasswd" #be sure to mount the volume through docker-compose.yml

[http.middlewares.middlewares-rate-limit]
[http.middlewares.middlewares-rate-limit.rateLimit]
average = 100
burst = 50

[http.middlewares.middlewares-secure-headers]
[http.middlewares.middlewares-secure-headers.headers]
accessControlAllowMethods= ["GET", "OPTIONS", "PUT"]
accessControlMaxAge = 100
hostsProxyHeaders = ["X-Forwarded-Host"]
# sslRedirect = true #replaced with middlewares-https-redirectscheme for v2.5.x
stsSeconds = 63072000
stsIncludeSubdomains = true
stsPreload = true
forceSTSHeader = true
#frameDeny = true #overwritten by customFrameOptionsValue
customFrameOptionsValue = "allow-from https:zayir.org.ua" #CSP takes care of this but may be needed for organizr.
contentTypeNosniff = true
browserXssFilter = true
##sslForceHost = true # add sslHost and all of the
##sslHost = "example.com"
referrerPolicy = "same-origin"
##Setting contentSecurityPolicy is more secure but it can break things. Proper auth will reduce the risk.
##the below line also breaks some apps due to 'none' - sonarr, radarr, etc.
##contentSecurityPolicy = "frame-ancestors '.example.com:';object-src 'none';script-src 'none';"
# Line below, featurePolicy, was deprecated in v2.5.x in favor permissionPolicy
# featurePolicy = "camera 'none'; geolocation 'none'; microphone 'none'; payment 'none'; usb 'none'; vr 'none';"
permissionsPolicy = "camera=(), microphone=(), geolocation=(), payment=(), usb=(), vr=()"
[http.middlewares.middlewares-secure-headers.headers.customResponseHeaders]
X-Robots-Tag = "none,noarchive,nosnippet,notranslate,noimageindex,"
server = ""

[http.middlewares.middlewares-oauth]
[http.middlewares.middlewares-oauth.forwardAuth]
address = "http://oauth:4181" # Make sure you have the OAuth service in docker-compose.yml
trustForwardHeader = true
authResponseHeaders = ["X-Forwarded-User"]

[http.middlewares.middlewares-authelia]
[http.middlewares.middlewares-authelia.forwardAuth]
address = "http://authelia:9091/api/verify?rd=https://authelia.example.com"
trustForwardHeader = true
authResponseHeaders = ["Remote-User", "Remote-Groups"]

middlewares-chains.toml

[http.middlewares]
[http.middlewares.chain-no-auth]
[http.middlewares.chain-no-auth.chain]
middlewares = [ "middlewares-rate-limit", "middlewares-secure-headers"]

[http.middlewares.chain-basic-auth]
[http.middlewares.chain-basic-auth.chain]
middlewares = [ "middlewares-rate-limit", "middlewares-secure-headers", "middlewares-basic-auth"]

[http.middlewares.chain-oauth]
[http.middlewares.chain-oauth.chain]
middlewares = [ "middlewares-rate-limit", "middlewares-secure-headers", "middlewares-oauth"]

[http.middlewares.chain-authelia]
[http.middlewares.chain-authelia.chain]
middlewares = [ "middlewares-rate-limit", "middlewares-secure-headers", "middlewares-authelia"]

[http.middlewares.chain-nextcloud]
[http.middlewares.chain-nextcloud.chain]
middlewares = [ "middlewares-rate-limit", "nextcloud-middlewares-secure-headers", "nextcloud-redirect"]

my "regular" list of traefik lables, example for service hastebin (selfhosted pastebin). ive tried to make it on-demand with your >plugin, but whenever i used any other middleware than your plugins, it didn`t work.
Initially, i thought it might be my oauth one, but even with "No Auth" it does nothing.

 labels:
  - "traefik.enable=true"
  ## HTTP Routers
  - "traefik.http.routers.hastebin-rtr.entrypoints=https"
  - "traefik.http.routers.hastebin-rtr.rule=Host(`hastebin.$DOMAINNAME`)"
  - "traefik.http.routers.hastebin-rtr.tls=true"
  ## Middlewares
  - "traefik.http.routers.hastebin-rtr.middlewares=chain-no-auth@file" # No Authentication
  ## HTTP Services
  - "traefik.http.routers.hastebin-rtr.service=hastebin-svc"
  - "traefik.http.services.hastebin-svc.loadbalancer.server.port=7777"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants