generate images from arbitrary strings?

[sterlingcrispin]

Hey I'm not deeply familiar with text to image GAN's or pytorch and I'm interested in trying your code out but from what I can tell it uses some preprocessed text embeddings rather than accepting arbitrary string input. Am I missing something? What would be the best way to generate new images with arbitrary strings?

it looks like https://github.com/reedscot/icml2016 that you linked to accepts arbitrary input, can I use that to output the latent space of those text queries to disk and then give them to your code base?

any guidance you have would be greatly appreciated

[tohinz]

Hi, have a look at this issue from our follow-up work which addresses the same thing: tohinz/semantic-object-accuracy-for-generative-text-to-image-synthesis#8 (comment)

Fair warning though: if you want to use this with "difficult" captions it most likely will not work. Current state-of-the-art models can only generate good images when they are trained in very limited domains (e.g. dataset of birds or flowers). More complex datasets/captions (e.g. COCO) at the moment will result in images that 99% of the time do not look very good.

[sterlingcrispin]

Hey thanks, I think I had reviewed that code and noticed that it reads "example_filenames.txt" which is a list of names of image files, rather than arbitrary strings, so I guess I need to populate my own data_dic in that code with strings?

as an outsider to this area of research its very funny and perplexing how many text-to-image GAN's there are on github which don't explicitly seem to take arbitrary string input but I understand why its easier to focus on a set of known sentences that are already processed which everyone uses as a benchmark