Skip to content

Commit

Permalink
add video
Browse files Browse the repository at this point in the history
  • Loading branch information
@zhangfan-p
zhangfan-p committed Nov 27, 2023
1 parent f8eed63 commit 1b66abb
Showing 1 changed file with 34 additions and 17 deletions.
51 changes: 34 additions & 17 deletions index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ <h1 class="title is-1 publication-title"><span style="color:#B9770E; font-weight
</span>
<!-- Video Link. -->
<span class="link-block">
<a href="https://vchitect.github.io/VBench-project/" target="_blank"
<a href="https://www.youtube.com/watch?v=7IhCC8Qqn8Y" target="_blank"
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="fab fa-youtube"></i>
Expand Down Expand Up @@ -187,7 +187,7 @@ <h1 class="title is-1 publication-title"><span style="color:#B9770E; font-weight
<section class="hero teaser">
<div class="container is-max-desktop">
<div class="hero-body">
<img src="assets/vbench/images/dim_results/fig_teaser_new.png" style="width:1000px; margin-bottom:20px" alt="Teaser."/>
<img src="assets/vbench/images/dim_results/fig_teaser_new.png" style="width:1000px; margin-bottom:10px" alt="Teaser."/>
<p style="margin-top: 0;">
<b>Overview of Vbench.</b> We propose VBench, a comprehensive benchmark suite for video generative models. We design a comprehensive and hierarchical <b>Evaluation Dimension Suite</b> to decompose "video generation quality" into multiple well-defined dimensions to facilitate fine-grained and objective evaluation. For each dimension and each content category, we carefully design a <b>Prompt Suite</b> as test cases, and sample <b>Generated Videos</b> from a set of video generation models. For each evaluation dimension, we specifically design an <b>Evaluation Method Suite</b>, which uses carefully crafted method or designated pipeline for automatic objective evaluation. We also conduct <b>Human Preference Annotation</b> for the generated videos for each dimension, and show that VBench evaluation results are <b>well aligned with human perceptions</b>. VBench provides valuable insights and will be open-sourced.
</p>
Expand All @@ -196,6 +196,37 @@ <h1 class="title is-1 publication-title"><span style="color:#B9770E; font-weight
</section>


<!-- Paper video. -->
<section class="hero is-light">
<div class="columns is-centered has-text-centered" style="margin-top: 0px; margin-bottom: 10px;">
<div class="column is-three-fifths">
<h2 class="title is-3">Video</h2>
<div class="publication-video">
<iframe width="560" height="315" src="https://www.youtube.com/embed/7IhCC8Qqn8Y?si=_gpWAOXWOi2OcaIJ" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
</div>

</div>
</div>
</section>
<!-- / Paper video. -->


<!-- Abstract. -->
<section>
<div class="container is-max-desktop">
<div class="columns is-centered has-text-centered" style="margin-top: 10px; margin-bottom: 0px;">
<div class="column is-four-fifths">
<h2 class="title is-3 is-centered">Abstract</h2>
<div class="content has-text-justified">
<p>
Video generation has witnessed significant advancements, yet evaluating these models remains a challenge. A comprehensive evaluation benchmark for video generation is indispensable for two reasons: 1) Existing metrics do not fully align with human perceptions; 2) An ideal evaluation system should provide insights to inform future developments of video generation. To this end, we present <b>VBench</b>, a comprehensive benchmark suite that dissects "video generation quality" into specific, hierarchical, and disentangled dimensions, each with tailored prompts and evaluation methods. VBench has three appealing properties: <b>1) Comprehensive Dimensions</b>: VBench comprises 16 dimensions in video generation (e.g., subject identity inconsistency, motion smoothness, temporal flickering, and spatial relationship, etc). The evaluation metrics with fine-grained levels reveal individual models' strengths and weaknesses. <b>2) Human Alignment</b>: We also provide a dataset of human preference annotations to validate our benchmarks' alignment with human perception, for each evaluation dimension respectively. <b>3) Valuable Insights</b>: We look into current models' ability across various evaluation dimensions, and various content types. We also investigate the gaps between video and image generation models. We will open-source VBench, including all prompts, evaluation methods, generated videos, and human preference annotations, and also include more video generation models in VBench to drive forward the field of video generation.
</p>
</div>
</div>
</div>
</section>
<!--/ Abstract. -->


<!-- Radar_big -->
<section class="section" style="margin-top:-50px; margin-bottom:-50px;">
Expand All @@ -222,21 +253,7 @@ <h2 class="title is-3 is-centered">VBench Evaluation Results of Video Generative
</section>


<!-- Abstract. -->
<section>
<div class="container is-max-desktop">
<div class="columns is-centered has-text-centered" style="margin-top: 10px; margin-bottom: 0px;">
<div class="column is-four-fifths">
<h2 class="title is-3 is-centered">Abstract</h2>
<div class="content has-text-justified">
<p>
Video generation has witnessed significant advancements, yet evaluating these models remains a challenge. A comprehensive evaluation benchmark for video generation is indispensable for two reasons: 1) Existing metrics do not fully align with human perceptions; 2) An ideal evaluation system should provide insights to inform future developments of video generation. To this end, we present <b>VBench</b>, a comprehensive benchmark suite that dissects "video generation quality" into specific, hierarchical, and disentangled dimensions, each with tailored prompts and evaluation methods. VBench has three appealing properties: <b>1) Comprehensive Dimensions</b>: VBench comprises 16 dimensions in video generation (e.g., subject identity inconsistency, motion smoothness, temporal flickering, and spatial relationship, etc). The evaluation metrics with fine-grained levels reveal individual models' strengths and weaknesses. <b>2) Human Alignment</b>: We also provide a dataset of human preference annotations to validate our benchmarks' alignment with human perception, for each evaluation dimension respectively. <b>3) Valuable Insights</b>: We look into current models' ability across various evaluation dimensions, and various content types. We also investigate the gaps between video and image generation models. We will open-source VBench, including all prompts, evaluation methods, generated videos, and human preference annotations, and also include more video generation models in VBench to drive forward the field of video generation.
</p>
</div>
</div>
</div>
</section>
<!--/ Abstract. -->


<!-- LeaderBoard -->
<section class="section" style="margin-top:-50px; margin-bottom:-50px;">
Expand Down

0 comments on commit 1b66abb

Please sign in to comment.