Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GPCLI fails with large files. #145

Open
ducharmg opened this issue Jun 25, 2019 · 2 comments
Open

GPCLI fails with large files. #145

ducharmg opened this issue Jun 25, 2019 · 2 comments

Comments

@ducharmg
Copy link

We have some files with 5000 strings.
Gp will fail intermittently on upload of english/translations or download of same.
Typical error stacks are:

*** uploading large english file ***

ava.lang.RuntimeException: com.ibm.g11n.pipeline.client.ServiceException: Error while processing API request PUT 26d0f13ecafad0d0007f0581e310385f/v2/bundles/gpTest-endor-bigfile4/en
at com.ibm.g11n.pipeline.tools.cli.ImportCmd._execute(ImportCmd.java:137)
at com.ibm.g11n.pipeline.tools.cli.BaseCmd.execute(BaseCmd.java:103)
at com.ibm.g11n.pipeline.tools.cli.GPCmd.main(GPCmd.java:80)
at com.ibm.g11n.pipeline.GPClient.execute(GPClient.java:12)
at com.ibm.g11n.pipeline.GPipe.importEnglishFile(GPipe.java:324)
at com.ibm.g11n.pipeline.GPipe.main(GPipe.java:90)
Caused by: com.ibm.g11n.pipeline.client.ServiceException: Error while processing API request PUT 26d0f13ecafad0d0007f0581e310385f/v2/bundles/gpTest-endor-bigfile4/en
at com.ibm.g11n.pipeline.client.impl.ServiceClientImpl.invokeApiJson(ServiceClientImpl.java:1476)
at com.ibm.g11n.pipeline.client.impl.ServiceClientImpl.invokeApiJson(ServiceClientImpl.java:1445)
at com.ibm.g11n.pipeline.client.impl.ServiceClientImpl.uploadResourceEntries(ServiceClientImpl.java:456)
at com.ibm.g11n.pipeline.tools.cli.ImportCmd._execute(ImportCmd.java:135)
... 5 more
Caused by: com.ibm.g11n.pipeline.client.ServiceException: Received HTTP status: 504 with non-JSON response from PUT 26d0f13ecafad0d0007f0581e310385f/v2/bundles/gpTest-endor-bigfile4/en
at com.ibm.g11n.pipeline.client.impl.ServiceClientImpl.invokeApiJson(ServiceClientImpl.java:1464)
... 8 more
*** deleting large english file ***

java.lang.RuntimeException: com.ibm.g11n.pipeline.client.ServiceException: Error while processing API request DELETE 26d0f13ecafad0d0007f0581e310385f/v2/bundles/gpTest-endor-bigfile4
at com.ibm.g11n.pipeline.tools.cli.DeleteBundleCmd._execute(DeleteBundleCmd.java:33)
at com.ibm.g11n.pipeline.tools.cli.BaseCmd.execute(BaseCmd.java:103)
at com.ibm.g11n.pipeline.tools.cli.GPCmd.main(GPCmd.java:80)
at com.ibm.g11n.pipeline.GPClient.execute(GPClient.java:12)
at com.ibm.g11n.pipeline.GPipe.deleteBundles(GPipe.java:360)
at com.ibm.g11n.pipeline.GPipe.main(GPipe.java:115)
Caused by: com.ibm.g11n.pipeline.client.ServiceException: Error while processing API request DELETE 26d0f13ecafad0d0007f0581e310385f/v2/bundles/gpTest-endor-bigfile4
at com.ibm.g11n.pipeline.client.impl.ServiceClientImpl.invokeApiJson(ServiceClientImpl.java:1476)
at com.ibm.g11n.pipeline.client.impl.ServiceClientImpl.invokeApiJson(ServiceClientImpl.java:1445)
at com.ibm.g11n.pipeline.client.impl.ServiceClientImpl.deleteBundle(ServiceClientImpl.java:301)
at com.ibm.g11n.pipeline.tools.cli.DeleteBundleCmd._execute(DeleteBundleCmd.java:31)
... 5 more
Caused by: com.ibm.g11n.pipeline.client.ServiceException: Received HTTP status: 504 with non-JSON response from DELETE 26d0f13ecafad0d0007f0581e310385f/v2/bundles/gpTest-endor-bigfile4
at com.ibm.g11n.pipeline.client.impl.ServiceClientImpl.invokeApiJson(ServiceClientImpl.java:1464)
... 8 more

Repeated attempts to do the operation eventually succeed.

@ducharmg
Copy link
Author

The behaviour suggests that the http request is timing out. I.e. the request has been received and being processed but the client is timing out waiting for a completion status.

@ducharmg
Copy link
Author

ducharmg commented Jul 4, 2019

setting the following jvm properties resolves the http 504 issues...
System.setProperty("sun.net.client.defaultReadTimeout", "90000");
System.setProperty("sun.net.client.defaultConnectTimeout", "90000");

some operations where taking 30-40 seconds

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant