Zhenyu Zhang

I am a staff research scientist at Tencent Youtu Lab, where I work on computer vision, graphics and machine learning.

I got my Ph.D degree from Department of Computer Science and Engineering, Nanjing University of Science and Technology in 2020, supervised by Jian Yang. In 2019, I spent 10 wonderful months as a visiting student at MHUG group in Unviversity of Trento, Italy, supervised by Nicu Sebe.

Email  /  Google Scholar  /  Linkedin  /  CV

profile photo
Research

I'm interested in computer vision, photography and reconstrcution. Much of my research is about inferring the 3D model (depth, normal, mesh, point cloud, etc) and intrinsic cues (light, albedo, specular, roughness, etc) from images, and rendering high-realistic photo based on such information. My recent works are focused on 3D face modelling and neural rendering.

Recent News
  • 07/2022 – 2 papers on depth completion are accepted by ECCV'22

  • 03/2022 – 2 papers on face reconstruction and neural rendering are accepted by CVPR'22, with the acceptance rate to be 25.3%

  • 01/2022 – I obtain Outstanding Doctoral Thesis Honorable Mention of CCF (China Computer Federation) 2021. See this page (in Chinese)

  • 07/2021 – 1 paper on nighttime depth estimation accepted by ICCV'21, with the acceptance rate to be 25.9%

  • 07/2021 – Our ASFD on face detection is accepted by ACM MM'21

  • 03/2021 – 1 paper on 3D face modelling accepted by CVPR'21 (Oral!), with the acceptance rate to be 23.7%

  • 02/2020 – 2 papers accepted by CVPR'20, with the acceptance rate to be 22.1%

  • 11/2019 – 1 paper accepted by AAAI'20, with the acceptance rate to be 20.6%

  • 10/2019 – Our ECCV'18 extended paper is accepted by TPAMI!

  • 04/2019 – Our paper on online depth adaptation is released at Arxiv

  • 03/2019 – 1 paper accepted by CVPR'19

  • 12/2018 – I join MHUG as a visiting student. Wonderful people and city!

  • 06/2018 – 1 paper accepted by ECCV'18

        Publications (* equal contribution)
Physically-Guided Disentangled Implicit Rendering for 3D Face Modeling
Zhenyu Zhang, Yanhao Ge, Ying Tai, Weijian Cao, Renwang Chen, Kunlin Liu, Hao Tang, Xiaoming Huang, Chengjie Wang, Dongjin Huang, Zhifeng Xie.
CVPR 2022
Paper /

3D face modeling is limited by classical graphics rendering, so that we let it benefit from a novel neural rendering approach.

Learning to Restore 3D Face from In-the-Wild Degraded Images
Zhenyu Zhang, Yanhao Ge, Ying Tai, Xiaoming Huang, Chengjie Wang, Hao Tang, Dongjin Huang, Zhifeng Xie.
CVPR 2022
Paper /

When restoring degraded faces, you need to restore the 3D geometry-aware effect.

Regularizing Nighttime Weirdness: Efficient Self-supervised Monocular Depth Estimation in the Dark
Kun Wang*, Zhenyu Zhang*, Xiang Li, Jun Li, Baobei Xu, Jian Yang.
ICCV 2021
Paper / Code

Nighttime is challenging, but we make it. Dataset and code are now released.

Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection
Zhenyu Zhang, Yanhao Ge, Renwang Chen, Ying Tai, Yan Yan, Jian Yang, Chengjie Wang, Jinlin Li and Feiyue Huang.
CVPR 2021,   (Oral Presentation)
Paper / Arxiv / Code

Improving the non-parametric 3D face reconstruction by leveraging consistency of unconstrained photo collection.

Cross-modal attention network for temporal inconsistent audio-visual event localization
Hanyu Xuan, Zhenyu Zhang, Shuo Chen, Jian Yang, Yan Yan.
AAAI 2020

Audio-visual event localization on temporal inconsistent videos

Pattern-Structure Diffusion for Multi-Task Learning
Ling Zhou, Zhen Cui, Chunyan Xu, Zhenyu Zhang, Chaoqun Wang, Tong Zhang, Jian Yang.
CVPR 2020

A graph-based method to mine multi-task relationship.

Online Depth Learning against Forgetting in Monocular Videos
Zhenyu Zhang, Stephane Lathuiliere, Elisa Ricci, Nicu Sebe, Yan Yan, Jian Yang.
CVPR 2020

Depth estimation method fails in new scenes, but we can continuously align it against domain shift.

Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentation
Zhenyu Zhang, Zhen Cui, Chunyan Xu, Yan Yan, Nicu Sebe, Jian Yang.
CVPR 2019

Feel difficult to combine different tasks? Here we provide a pair-wise similarity based method to leverage multi-task correlation.

Joint Task-Recursive Learning for RGB-D Scene Understanding
Zhenyu Zhang, Zhen Cui, Chunyan Xu, Zequn Jie, Xiang Li, Jian Yang.
TPAMI 2019

A recursive approach for joint-task learning in RGBD scenes.

Joint Task-recursive Learning for Semantic Segmentation and Depth Estimation
Zhenyu Zhang, Zhen Cui, Chunyan Xu, Zequn Jie, Xiang Li, Jian Yang.
ECCV 2018

A new framework for joint depth estimation & semantic segmentation.

Progressive Hard-Mining Network for Monocular Depth Estimation
Zhenyu Zhang, Chunyan Xu, Jian Yang, Junbin Gao, Zhen Cui.
TIP 2018

Improving monocular depth estimation by mining difficult regions.

Deep hierarchical guidance and regularization learning for end-to-end depth estimation
Zhenyu Zhang, Chunyan Xu, Jian Yang, Ying Tai, Liang Chen.
Pattern Recognition 2018

A new framework for monocular depth estimation.

      Awards

  • 2021, Outstanding Doctoral Thesis of CCF (China Computer Federation), honorable mention

  • 2017-2018, 2018-2019 National Graduate Scholarship (top 2%)


      Others

  • I am a fan of drama. I used to be one of the organizers of modern drama troupe of NJUST.


  • Last modified in Aug. 2021. For the style of my personal website, Please refer to the wonderful page from Jon Barron.