and the distribution of digital products.
HyperHuman Ablation Study: Optimal Expert Branch Design for Improved Image Generation
:::info Authors:
(1) Xian Liu, Snap Inc., CUHK with Work done during an internship at Snap Inc.;
(2) Jian Ren, Snap Inc. with Corresponding author: [email protected];
(3) Aliaksandr Siarohin, Snap Inc.;
(4) Ivan Skorokhodov, Snap Inc.;
(5) Yanyu Li, Snap Inc.;
(6) Dahua Lin, CUHK;
(7) Xihui Liu, HKU;
(8) Ziwei Liu, NTU;
(9) Sergey Tulyakov, Snap Inc.
:::
Table of Links3 Our Approach and 3.1 Preliminaries and Problem Setting
3.2 Latent Structural Diffusion Model
A Appendix and A.1 Additional Quantitative Results
A.2 More Implementation Details and A.3 More Ablation Study Results
A.5 Impact of Random Seed and Model Robustness and A.6 Boarder Impact and Ethical Consideration
A.7 More Comparison Results and A.8 Additional Qualitative Results
5.2 ABLATION STUDY
\ Simultaneous Denoise with Expert Branch. We explore whether latent structural diffusion model helps, and how many layers to replicate in the structural expert branches: 1) Denoise RGB, that only learns to denoise an image. 2) Denoise RGB + Depth, which also predicts depth. 3) Half DownBlock & UpBlock. We replicate half of the first DownBlock and the last UpBlock, which contains one down/up-sample ResBlock and one AttnBlock. 4) Two DownBlocks & UpBlocks, where we copy the first two DownBlocks and the last two UpBlocks. The results are shown in Tab. 2 (top), which prove that the joint learning of image appearance, spatial relationship, and geometry is beneficial. We also find that while fewer replicate layers give more spatially aligned results, the per-branch parameters are insufficient to capture distributions of each modality. In contrast, excessive replicate layers lead to less feature fusion across different targets, which fails to complement to each other branches.
\
\
:::info This paper is available on arxiv under CC BY 4.0 DEED license.
:::
\