Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Node IPs are not automatically detected (dird-global) #37

Open
megaezz opened this issue Jun 6, 2024 · 1 comment
Open

Node IPs are not automatically detected (dird-global) #37

megaezz opened this issue Jun 6, 2024 · 1 comment

Comments

@megaezz
Copy link

megaezz commented Jun 6, 2024

Hello, friend. I'm trying to use DIRD as docker global service.
I have several nodes: 10.0.0.2, 10.0.0.10, 10.0.0.26, 10.0.0.40
I created a service:
docker service create --name=dird-global --mode=global --env="DOCKER_NODE_HOSTNAME={{.Node.Hostname}}" --mount=type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock newsnowlabs/dird:latest --global-service --install --preexisting
But only node with ip 10.0.0.2 got real client ip. Other nodes doesn't. Because service doesn't detect node ips automatically and by default apply rules for nodes 10.0.0.2 - 10.0.0.9 - I see it from logs.
So that's why 10.0.0.2 works and other doesn't.

For example, logs of node 10.0.0.26:

2024-06-06.22:04:29.345030|5c8207ded8dc|21710| Docker Ingress Routing Daemon 4.2.0 starting, with args: --install --preexisting
2024-06-06.22:04:29.448612|5c8207ded8dc|21710| Detecting ingress network and node IP:
2024-06-06.22:04:29.453620|5c8207ded8dc|21710| - Ingress subnet: 10.0.0.0/24
2024-06-06.22:04:29.458686|5c8207ded8dc|21710| - This node's IP: 10.0.0.26
2024-06-06.22:04:29.463785|5c8207ded8dc|21710| Cleaning up any stale load-balancer rules ...
2024-06-06.22:04:29.495699|5c8207ded8dc|21710| !!! -------------------------- WARNING ------------------------------------
2024-06-06.22:04:29.500513|5c8207ded8dc|21710| !!! Assuming --ingress-gateway-ips 10.0.0.2 10.0.0.3 10.0.0.4 10.0.0.5 10.0.0.6 10.0.0.7 10.0.0.8 10.0.0.9
2024-06-06.22:04:29.505879|5c8207ded8dc|21710| !!!
2024-06-06.22:04:29.510757|5c8207ded8dc|21710| !!! Please compile a list of the ingress network IPs of each of your nodes
2024-06-06.22:04:29.515838|5c8207ded8dc|21710| !!! that you will be using as a load-balancer.
2024-06-06.22:04:29.520827|5c8207ded8dc|21710| !!!
2024-06-06.22:04:29.525623|5c8207ded8dc|21710| !!! You only have to do this once, or whenever you change your set of
2024-06-06.22:04:29.530684|5c8207ded8dc|21710| !!! load-balancer nodes.
2024-06-06.22:04:29.535341|5c8207ded8dc|21710| !!!
2024-06-06.22:04:29.540332|5c8207ded8dc|21710| !!! Then relaunch using:
2024-06-06.22:04:29.545040|5c8207ded8dc|21710| !!! /opt/docker-ingress-routing-daemon --install --ingress-gateway-ips "<Node Ingress IP List>"
2024-06-06.22:04:29.552642|5c8207ded8dc|21710| !!! ----------------------------------------------------------------------
2024-06-06.22:04:29.565163|5c8207ded8dc|21710| Enumerating load balancers from --ingress-gateway-ips 10.0.0.2 10.0.0.3 10.0.0.4 10.0.0.5 10.0.0.6 10.0.0.7 10.0.0.8 10.0.0.9
2024-06-06.22:04:29.586411|5c8207ded8dc|21710| - Load balancer 10.0.0.2 will have ID 2
2024-06-06.22:04:29.600651|5c8207ded8dc|21710| - Load balancer 10.0.0.3 will have ID 3
2024-06-06.22:04:29.608568|5c8207ded8dc|21710| - Load balancer 10.0.0.4 will have ID 4
2024-06-06.22:04:29.616590|5c8207ded8dc|21710| - Load balancer 10.0.0.5 will have ID 5
2024-06-06.22:04:29.624547|5c8207ded8dc|21710| - Load balancer 10.0.0.6 will have ID 6
2024-06-06.22:04:29.645762|5c8207ded8dc|21710| - Load balancer 10.0.0.7 will have ID 7
2024-06-06.22:04:29.656456|5c8207ded8dc|21710| - Load balancer 10.0.0.8 will have ID 8
2024-06-06.22:04:29.666895|5c8207ded8dc|21710| - Load balancer 10.0.0.9 will have ID 9
2024-06-06.22:04:29.673977|5c8207ded8dc|21710| This node is not a specified load balancer; so skipping installing ingress namespace iptables rules
2024-06-06.22:04:29.679190|5c8207ded8dc|21710| Setting ingress_sbox namespace sysctl variables:
2024-06-06.22:04:29.684294|5c8207ded8dc|21710| - Setting net.ipv4.vs.conn_reuse_mode=0 net.ipv4.vs.expire_nodest_conn=1 net.ipv4.vs.expire_quiescent_template=1
2024-06-06.22:04:29.692749|5c8207ded8dc|21710| Checking for preexisting containers that need routing rules added ...
2024-06-06.22:04:29.890578|5c8207ded8dc|21710| Detected container launch for service 'dird-global', with ID '0562e3fc0ab3' and NID '21527': no ingress network interface found, so skipping.
2024-06-06.22:04:29.896040|5c8207ded8dc|21710| Launching docker event watcher to monitor for container launches (pgroup 21710) ...

How to enable auto detecting of node ips?

@struanb
Copy link
Contributor

struanb commented Jun 25, 2024

Hi @megaezz. Thanks for using DIRD.

DIRD cannot autodetect the node IPs across your clutster, as it can only detect the node IP for the node it is running on.

You seem to know your node IPs though: 10.0.0.2, 10.0.0.10, 10.0.0.26, 10.0.0.40

You could try therefore:

docker service create --name=dird-global --mode=global --env="DOCKER_NODE_HOSTNAME={{.Node.Hostname}}" --mount=type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock newsnowlabs/dird:latest --global-service --install --preexisting --ingress-gateway-ips 10.0.0.2,10.0.0.10,10.0.0.26,10.0.0.40

N.B. Running the newsnowlabs/dird:latest image is not yet officially supported!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants