Recent Research Highlights

GEAR-NERF

Gear-NeRF: Free-Viewpoint Rendering and Tracking with Motion-aware Spatio-Temporal Sampling, CVPR 2024

Xinhang Liu, Yu-Wing Tai, Chi-Keung Tang, Pedro Miraldo, Suhas Lohit, Moitreya Chatterjee
CVPR Highlight, 2.8% of 11532
[Paper] [Project] [Youtube]

SANeRF-HQ

SANeRF-HQ: Segment Anything for NeRF in High Quality, CVPR 2024

Yichen Liu, Benran Hu, Chi-Keung Tang, Yu-Wing Tai
Segment Anything in High Quality for NeRF
[Paper] [Project] [Youtube]

C3Net

C3Net: Compound Conditioned ControlNet for Multimodal Content Generation, CVPR 2024

Juntao Zhang, Yuehuai Liu, Yu-Wing Tai, Chi-Keung Tang
Multimodal (Text, Image, Audio) Content Generation from Multimodal Compound Conditioned Input
[Paper] [Project]

HQ-SAM

Segment Anything in High Quality, NeurIPS 2023

Lei Ke*, Mingqiao Ye*, Martin Danelljan, Yifan Liu, Yu-Wing Tai, Chi-Keung Tang, Fisher Yu (* denotes equal contribution)
HQ-SAM receives 2000+ Github stars in one month.
[Paper] [Project]

FaceDNeRF

FaceDNeRF: Semantics-Driven Face Reconstruction, Prompt Editing and Relighting with Diffusion Models, NeurIPS 2023

Hao Zhang*, Tianyuan Dai*, Yanbo Xu*, Yu-Wing Tai, Chi-Keung Tang (* denotes equal contribution)
3D faces from a single image with prompt editable and relighting ability.
[Paper] [Project]

BiMatting

BiMatting: Efficient Video Matting via Binarization, NeurIPS 2023

Haotong Qin*, Lei Ke*, Xudong Ma, Martin Danelljan, Yu-Wing Tai, Chi-Keung Tang, Xianglong Liu, Fisher Yu (* denotes equal contribution)
An accurate and efficient video matting model using binarization.
[Paper] [Project]

InstanceNeRF

Instance Neural Radiacne Field, ICCV 2023

Yichen Liu*, Benran Hu*, Junkai Huang*, Yu-Wing Tai, Chi-Keung Tang (* denotes equal contribution)
This is the first instance segmentation framework for NeRF.
[Paper] [Project]

CascadeDet

Cascade-DETR: Delving into High-Quality Universal Object Detection, ICCV 2023

Mingqiao Ye*, Lei Ke*, Siyuan Li, Yu-Wing Tai, Chi-Keung Tang, Martin Danelljan, Fisher Yu (* denotes equal contribution)
Promoting DETR's detection accuracy in universal domains via cascade attention.
[Paper] [Project]

EgoHOI

EgoPCA: A New Framework for Egocentric Hand-Object Interaction Understanding, ICCV 2023

Yue Xu, Yong-Lu Li, Zhemin Huang, Michael Xu LIU, Cewu Lu, Yu-Wing Tai, Chi Keung Tang
We contribute comprehensive pre-train sets, balanced test sets and a new baseline for Egocentric Hand-Object Interaction (Ego-HOI).
[Paper] [Project]

NeRF-RPN

NeRF-RPN: A general framework for object detection in NeRFs, CVPR 2023

Benran Hu*, Junkai Huang*, Yichen Liu*, Yu-Wing Tai, Chi-Keung Tang (* denotes equal contribution)
This is the first object detection framework for NeRF.
[Paper] [Project][Youtube]

Mask-Free Video Instance Segmentation

Mask-Free Video Instance Segmentation, CVPR 2023

Lei Ke, Martin Danelljan, Henghui Ding, Yu-Wing Tai, Chi-Keung Tang, Fisher Yu
Removing video and image mask annotation necessity for highly accurate VIS.
[Paper] [Project]