Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HA-Proxy killing connection to server #9

Open
abn opened this issue Sep 9, 2013 · 4 comments
Open

HA-Proxy killing connection to server #9

abn opened this issue Sep 9, 2013 · 4 comments
Assignees

Comments

@abn
Copy link
Member

abn commented Sep 9, 2013

haproxy seems to be closing its connection before the response is completed

uwsgi_response_write_body_do() TIMEOUT !!!
IOError: write error
[pid: 28897|app: 0|req: 299/8675] IP () {52 vars in 1198 bytes} [Mon Sep  9 06:22:50 2013] GET /service/v2/update/1970-01-01T00:00:00/ => generated 25326655 bytes in 68404 msecs (HTTP/1.1 200) 5 headers in 354 bytes (343 switches on core 1)

uwsgi_response_write_body_do() TIMEOUT !!!
IOError: write error
[pid: 2925|app: 0|req: 1/47] IP () {52 vars in 1198 bytes} [Mon Sep  9 06:29:42 2013] GET /service/v2/update/1970-01-01T00:00:00/ => generated 46276210 bytes in 69777 msecs (HTTP/1.1 200) 5 headers in 354 bytes (459 switches on core 0)
@ghost ghost assigned abn Sep 9, 2013
@abn
Copy link
Member Author

abn commented Sep 9, 2013

Isolated a fix for the issue. It seems that haproxy drops the connection to client as a side-effect to http-server-close.

    option http-server-close
+    option http-pretend-keepalive
    #option forwardfor       except 127.0.0.0/8

@abn abn closed this as completed Sep 9, 2013
@abn abn reopened this Sep 9, 2013
@abn
Copy link
Member Author

abn commented Sep 10, 2013

Had to add the following option to uwsgi.ini

http-keepalive = true

Also modified haproxy.cfg for good measure:

-    timeout client          1m
+    timeout client          3m
-    timeout server          1m
+    timeout server          3m

@abn abn closed this as completed Sep 10, 2013
@abn
Copy link
Member Author

abn commented Nov 15, 2013

Adding a mitigation to handle slow response generation:
victims/victims-server-openshift@87db085

@abn abn reopened this Nov 15, 2013
@abn
Copy link
Member Author

abn commented Nov 15, 2013

The error message seems to be triggered due to a socket-timeout looking at the uwsgi code base. The default was 4s.

Seems to me that there is some delay in writing records to response stream. This will be an issue irrelevant of the update size.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant