the dataset source

[Delicious-Bitter-Melon]

Thank you for your contribution to open-source datasets. I would like to confirm whether these video resources were collected independently by your team or if they are based on public datasets such as CelebV-Text, which have been processed or utilized further by you?

[SHYuanBest]

Thanks for your interest. The current open-source dataset is collected from youtube by our team independently, the other part of our training data that cannot be opened to the public is sourced from open-sora plan. The CelebV-Text dataset seems to have the following problems, so we did not use it:

[Delicious-Bitter-Melon]

The dataset uploaded in the preprint version appears to be incomplete. While the dataset contains 54,239 videos, masks are only available for 31,947 of them. @SHYuanBest

[SHYuanBest]

We only used 31,947 of them.

[Delicious-Bitter-Melon]

We only used 31,947 of them.

Thank you for your reply. Could you clarify whether you use a face mask or a head mask to obscure unrelated areas during training?

[SHYuanBest]

We use the face first, and use the head if the face mask is missing.

[Delicious-Bitter-Melon]

We use the face first, and use the head if the face mask is missing.

Thank you again for your reply. I have some confusion regarding the process: Do you first concatenate the VAE features of keypoint maps and the VAE features of facial images along the frame dimension, and then concatenate these with the noise video along the channel dimension to achieve a total of 32 channels?

[SHYuanBest]

yes

[Delicious-Bitter-Melon]

yes

Thanks.

[Delicious-Bitter-Melon]

Hi. Could you provide further clarification regarding the preference for "half-body or full-body images" as mentioned in the Hugging Face Space documentation? From my understanding, the system utilizes FaceXLib to detect and crop faces, which should effectively handle face images, half-body images, and full-body images alike. However, your statement seems to emphasize half-body and full-body images while not explicitly mentioning face images. Could you explain how this preference impacts the processing of face images and why there might be a specific emphasis on half-body and full-body shots?

[SHYuanBest]

If we only input 'crop face', FaceXLib is likely to fail in detecting faces. However, using half-body or full-body images can significantly reduce this likelihood.

[Delicious-Bitter-Melon]

If we only input 'crop face', FaceXLib is likely to fail in detecting faces. However, using half-body or full-body images can significantly reduce this likelihood.

Got it. Thank you very much for your prompt response.

[SHYuanBest]

The dataset uploaded in the preprint version appears to be incomplete. While the dataset contains 54,239 videos, masks are only available for 31,947 of them. @SHYuanBest

oh, i miss some important details. The 31947 are single face, and the remain are multi face. #21