-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
StyleGAN example Processing 4.01A #17
Comments
Hi @j3nsykes I'd be curious to follow this, while I'm not teaching StyleGAN -> Processing right now, it comes up often for me! Last year I did a bunch of examples with p5.js which I could maybe use to test. I've also switched to some other model hosting services (like hugging face!) but it is trickier to train your own models! If you want to share the full code example I'm happy to take a look! ccing @yining1023 who is teaching a similar class in case she also wants to follow! |
Hi @shiffman thanks for replying so quickly and offering to help! Currently I am just running the examples from the Runway ML Processing library. This one I've also got some hosted model examples in P5 too but I hoped to show a range of methods that join up workflows and interfaces for teaching purposes. With the advantage of running locally in Processing and perhaps joining up other sketches students have been working on. (they are more practiced in processing than p5 right now). However, I will move back towards P5 if I can't resolve this I think. |
Just to update : my P5JS examples that return a base 64 image from Runway also no longer work. Any example that just returns text or numbers (bounding box co-ords for example) still work fine but examples which receive imagery from runway either to P5 or Processing 4.01A no longer work . I would normally think this would be to do with a change in JSON data structure coming from Runway. However, the examples still work in Processing 3.5.4 which doesn't support that theory. |
Hi @j3nsykes I just checked and my p5 example is still working. I have it paired with a node server to make the request. https://runway-ml-template.glitch.me I can try to fire up Processing later today and see if this same model will work there! |
Interesting! This made me triple check and it materialises my P5JS examples (from the web editor) run in Firefox but not Chrome or Safari. The Debug Inspector is indicating a CORS permission issue which is OK I can work around that for future reference. |
Ah, so looking at the Processing example you linked I believe it's using the OSC connection to the Runway desktop app, is that correct? I bet there's some kind of networking permissions (or firewall settings?) that Processing 4 requires? My p5 examples all use the hosted models and it would be possible to do the same with Processing using the https://github.com/runemadsen/HTTP-Requests-for-Processing Or am I confused? I using HTTP Requests is helpful, I can see if I can get an example working! Are you on windows or mac? |
Oh its using a local network HTTP request already . Although not via hosted models but via the Network tab interface in Runway Workspaces. I've printed the runwayData to console that's received into Processing from Runway and I can see it is receiving the Base64 String but throws an error when trying to execute this |
Thank you @j3nsykes for reporting/details and @shiffman for the support. Unfortunately the soonest I can look into this issue is next weekend. In the meantime, @j3nsykes could you also post the full stack trace error message ? I'm using the okhttp to handle the post requests. Apologies in advance I won't be able to fix this ASAP. |
Thanks George . My colleague had some examples set up for AttGAN and GPT2 using the HTTP-Requests-For-Processing library to communicate directly without the use of the RunwayML Processing lib. This is still using the local network query not hosted models but a good Processing 4.01A solution. I will post a repo of examples in a few hours so others teaching with these tools can use them. |
Templates for HTTP request from Processing 4.01A to RunwayML StyleGAN model. The same methods can be used for all other models by just changing the JSON input and output Strings per model. Thanks to Catherine Weir who provided a template for the AttGAN model which these templates were adapted from. |
I appreciate Runway ML Lab isn't as supported now however I'm hoping I might be able to find a solution for teaching purposes.
I am running the StyleGAN example in processing 4.01A (August 9th 2022 release). The example communicates via HTTP successfully. I can see the Runway input interface and preview window update. However, the image is not returned to processing. The error is
Error, disabling runwayDataEvent() null
I have printed the infoEvent details and they are as follows.
{ "outputs": [{ "defaultOutputFormat": "JPEG", "channels": 3, "name": "image", "description": null, "type": "image" }], "inputs": [ { "samplingMean": 0, "default": null, "name": "z", "length": 512, "description": null, "type": "vector", "samplingStd": 0.5 }, { "default": 0.8, "min": 0, "max": 1, "name": "truncation", "description": null, "step": 0.01, "type": "number" } ] }
The example works perfectly in processing 3.5.4 so i don't think it is a message string issue or change on Runway's side. I have tried giving Processing 4 permission access to all files and folders but no change. Other examples run without issue in Processing 4.01 A (eg:GPT2 example that is text based) therefore it is maybe a Base64 issue? Any pointers greatly appreciated as I hope to use this in a teaching example and would rather not request students mix versions of Processing.
The text was updated successfully, but these errors were encountered: