Jae Shin Yoon (윤재신)

   

I am a Research Scientist at Adobe Research.

Email: jaeyoon [at] adobe.com, jsyoon4325 [at] gmail.com
CV | Google Scholar

I received Ph.D. degree in Computer Science and Engineering at the University of Minnesota (UMN), Twin Cities, advised by Prof. Hyun Soo Park.  I received MS degree in Department of Robotics from Korea Advanced Institute of Science and Technology (KAIST) under the supervision of Prof. In So Kweon. I received my bachelor degree in Electrical Engineering from Hanyang University under the supervision of Prof. Yong Ho Song. I was an exchange student at Kobe University. My general research interest lies in computer&robot vision and machine learning.

Industry

  • Research Scientist at Adobe Research, San Jose , USA | 2022 - present
  • Internship at Adobe Research, San Jose , USA | 2021 (June - Aug.)
  • Internship at NVIDIA Research, Santa Clara, USA | 2019 (Feb. - Aug.)
  • Internship at Facebook Reality Labs, Oculus, Pittsburgh, USA | 2018 (June - Aug.)

  • Research & Productization

    JOG3R: Towards 3D-Consistent Video Generators
    Chun-Hao Paul Huang, Niloy Mitra, Hyeonho Jeong, Jae Shin Yoon, Duygu Ceylan
    The British Machine Vision Conference (BMVC) 2025
    [PDF] [Project Page]
    Can3Tok: Canonical 3D Tokenization and Latent Modeling of Scene-Level 3D Gaussians
    Quankai Gao, Iliyan Georgiev, Tuanfeng Yang Wang, Krishna Kumar Singh, { Ulrich Neumann*, Jae Shin Yoon* }
    *indicates joint last authors
    IEEE International Conference on Computer Vision (ICCV) 2025
    [PDF] [Project Page] [Code]
    IntrinsicEdit: Precise generative image manipulation in intrinsic space
    Linjie Lyu, Valentin Deschaintre, Yannick Hold-Geoffroy, Miloš Hašan, Jae Shin Yoon, Thomas Leimkühler, Christian Theobalt, Iliyan Georgiev
    SIGGRAPH 2025 (TOG)
    [PDF] [Project Page]
    3D-Fixup: Advancing Photo Editing with 3D Priors
    Yen-Chi Cheng, Krishna Kumar Singh, Jae Shin Yoon, Alex Schwing, Liangyan Gui, Matheus Gadelha, Paul Guerrero, Nanxuan Zhao
    SIGGRAPH 2025
    [PDF] [Project Page]
    Comprehensive Relighting: Generalizable and Consistent Monocular Human Relighting and Harmonization
    Junying Wang, Jingyuan Liu, Xin Sun, Krishna Kumar Singh, Zhixin Shu, He Zhang, Jimei Yang, Nanxuan Zhao, Tuanfeng Y. Wang, Simon S. Chen, Ulrich Neumann, Jae Shin Yoon
    IEEE Computer Vision and Pattern Recognition (CVPR) 2025
    [PDF] [Project Page]
    Text2Relight: Creative Portrait Relighting with Text Guidance
    Junuk Cha, Mengwei Ren, Krishna Kumar Singh, He Zhang, Yannick Hold-Geoffroy, Seunghyun Yoon, HyunJoon Jung,
    { Jae Shin Yoon*, Seungryul Baek* }
    *indicates joint last authors
    The Association for the Advancement of Artificial Intelligence (AAAI) 2025
    [PDF] [Project Page]
    Generative Portrait Shadow Removal
    Jae Shin Yoon, Zhixin Shu, Mengwei Ren, Xuaner Zhang, Yannick Hold-Geoffroy, Krishna Kumar Singh, He Zhang
    ACM Transactions on Graphics, Vol.43(6) (TOG), SIGGRAPH Asia 2024
    The upsampling module is integrated into Adobe's Photoshop to support perfect identity preservation. [Link] [2025]
    [PDF] [Project Page]
    High-Fidelity Modeling of Generalizable Wrinkle Deformation
    Jingfan Guo, Jae Shin Yoon, Shunsuke Saito, Takaaki Shiratori, Hyun Soo Park
    European Conference on Computer Vision (ECCV) 2024
    [PDF] [Project Page]
    COMPOSE: Comprehensive Portrait Shadow Editing
    Andrew Hou, Zhixin Shu, Xuaner Zhang, He Zhang, Yannick Hold-Geoffroy, Jae Shin Yoon, Xiaoming Liu
    European Conference on Computer Vision (ECCV) 2024
    [PDF] [Project Page]
    IntrinsicDiffusion: Joint Intrinsic Layers from Latent Diffusion Models
    Jundan Luo, Duygu Ceylan, Jae Shin Yoon, Nanxuan Zhao, Julien Philip, Anna Frühstück, Wenbin Li, Christian Richardt, Tuanfeng Y. Wang
    ACM SIGGRAPH 2024
    [PDF] [Project Page]
    Text2HOI: Text-guided 3D Motion Generation for Hand-Object Interaction
    Junuk Cha, Jihyeon Kim, { Jae Shin Yoon*, Seungryul Baek* }
    *indicates joint last authors
    IEEE Computer Vision and Pattern Recognition (CVPR) 2024
    [PDF] [Project page] [Code]
    Relightful Harmonization: Lighting-aware Portrait Background Replacement
    Mengwei Ren, Wei Xiong, Jae Shin Yoon, Zhixin Shu, Jianming Zhang, HyunJoon Jung, Guido Gerig, He Zhang
    IEEE Computer Vision and Pattern Recognition (CVPR) 2024
    Shipped to Adobe's Photoshop for Object-Background harmonization feature. [Link] [2025]
    [PDF] [Project Page]
    Bidirectional Temporal Diffusion Model for Temporally Consistent Human Animation
    Tserendorj Adiya, Jae Shin Yoon, Jungeun Lee, Sanghun Kim, Hwasup Lim
    International Conference on Learning Representations (ICLR) 2024
    [PDF] [Video] [Project page]
    3D Reconstruction of Interacting Multi-Person in Clothing from a Single Image
    Junuk Cha, Hansol Lee, Jaewon Kim, Bao Truong, { *Jae Shin Yoon, *Seungryul Baek }
    *indicates joint last authors
    IEEE Winter Conference on Applications of Computer Vision (WACV) 2024
    [PDF] [Project Page]
    Complete 3D Human Reconstruction from a Single Incomplete Image
    Junying Wang, Jae Shin Yoon, Tuanfeng Y. Wang, Krishna Kumar Singh, and Ulrich Neumann
    IEEE Computer Vision and Pattern Recognition (CVPR) 2023
    [PDF] [Video] [Project page]
    Metaverse in the Wild: Modeling, Adapting, and Rendering of 3D Human Avatars from a Single Camera
    Jae Shin Yoon
    PhD Thesis, Computer Science and Engineering, University of Minnesota 2022
    [PDF]
    Learning Motion-Dependent Appearance for High-Fidelity Rendering of Dynamic Humans from a Single Camera
    Jae Shin Yoon, Duygu Ceylan, Tuanfeng Y. Wang, Jingwan (Cynthia) Lu, Jimei Yang, Zhixin Shu, and Hyun Soo Park
    IEEE Computer Vision and Pattern Recognition (CVPR) 2022
    Shipped to Adobe's biggest demo, Max Sneak as Project MotionMix. [Link] [2023]
    [PDF] [Video] [Project page] [Article]
    HUMBI: A Large Multiview Dataset of Human Body Expressions and Benchmark Challenge
    Jae Shin Yoon, Zhixuan Yu, Jaesik Park, and Hyun Soo Park
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) Journal
    [PDF] (Extended version of CVPR 20) [Dataset Page]
    Pose-Guided Human Animation from a Single Image in the Wild
    Jae Shin Yoon, Lingjie Liu, Vladislav Golyanik, Kripasindhu Sarkar, Hyun Soo Park, and Christian Theobalt
    IEEE Computer Vision and Pattern Recognition (CVPR) 2021
    [PDF] [Video] [Project page] [Code] [Poster] [5-min video]
    Novel View Synthesis of Dynamic Scenes with Globally Coherent Depths from a Monocular Camera
    Jae Shin Yoon, Kihwan Kim, Orazio Gallo, Hyun Soo Park, and Jan Kautz
    IEEE Computer Vision and Pattern Recognition (CVPR) 2020
    [PDF] [PDF_supple] [Video] [Video_supple] [Project Page]
    HUMBI: A Large Multiview Dataset of Human Body Expressions
    {Zhixuan Yu*, Jae Shin Yoon*}, In Kyu Lee, Prashanth Venkatesh, Jaesik Park, Jihun Yu, and Hyun Soo Park
    *indicates joint first authors
    IEEE Computer Vision and Pattern Recognition (CVPR) 2020
    [PDF] [Video] [Project Page]
    Self-Supervised Adaptation of High-Fidelity Face Models for Monocular Performance Tracking
    Jae Shin Yoon, Takaaki Shiratori, Shoou-I Yu, and Hyun Soo Park
    IEEE Computer Vision and Pattern Recognition (CVPR) 2019 Oral presentation
    [PDF] [Supple] [Video] [Project Page] [Poster]
    3D Semantic Trajectory Reconstruction from 3D Pixel Continuum
    Jae Shin Yoon, Ziwei Li, Hyun Soo Park
    IEEE Computer Vision and Pattern Recognition (CVPR) 2018
    [PDF] [Video] [Project Page]
    KAIST Multi-spectral Day/Night Dataset for Autonomous and Assisted Driving
    Yukyung Choi, Namil Kim, Soonmin Hwang, Kibaek Park, Jae Shin Yoon, Kyounghwan An, and In So Kweon
    Transactions on Intelligent Transportation Systems (T-ITS)
    [PDF] [Project Page]
    Pixel-level Matching for Video Object Segmentation using Convolutional Neural Networks
    Jae Shin Yoon, Francois Rameau, Junsik Kim, Seokju Lee, Seunghak Shin, and In So Kweon
    IEEE International Conference on Computer Vision (ICCV) 2017
    [PDF] [Project Page] [Slide] [Poster]
    VPGNet: Vanishing Point Guided Network for Lane and Road Marking Detection and Recognition
    Seokju Lee, Junsik Km, Jae Shin Yoon, Seunghak Shin, Oleksandr Bailo, Namil Kim, Tae-hee Lee, Hyun Seok Hong, Seung-Hoon Han, In So Kweon
    IEEE International Conference on Computer Vision (ICCV) 2017
    [PDF] [Project Page]
    Robust Road Marking Detection and Recognition Using Density-Based Grouping and Machine Learning Techniques
    Oleksandr Bailo, Seokju Lee, Francois Rameau, Jae Shin Yoon, In So Kweon
    IEEE Winter Conference on Application and Computer Vision (WACV) 2017
    [PDF]
    Thermal-Infrared based Drivable Region Detection
    Jae Shin Yoon, Kibaek Park, Soonmin Hwang, Namil Kim, Yukyung Choi, Francois Rameau and In So Kweon
    IEEE Intelligent Vehicle Symposium (IV) 2016
    [PDF] [Project Page] [Dataset]
    All-Day Visual Place Recognition: Benchmark Dataset and Baseline
    Yukyung Choi, Namil Kim, Kibak Park, Soonmin Hwang, Jae Shin Yoon,In So Kweon
    IEEE Computer Vision and Pattern Recognition Workshop 2015 on Visual Place Recognition in Changing Environments (CVPR-VPRICE) 2015
    [PDF] [Project Page]
    Low-Cost Synchronization for Multispectral Cameras
    Yukyung Choi, Namil Kim, Kibak Park, Soonmin Hwang, Jae Shin Yoon, In So Kweon
    IEEE International Conference on Ubiquitous Robots and Ambient Intelligence (URAI) 2015
    [PDF]
    Geometrical calibration of multispectral calibration
    Namil Kim, Yukyung Choi, Soonmin Hwang, Kibaek Park, Jae Shin Yoon, In So Kweon
    IEEE International Conference on Ubiquitous Robots and Ambient Intelligence
    [PDF]
    Technical Report
    Neural 3D Clothes Retargeting from a Single Image
    Jae Shin Yoon, Kihwan Kim, Jan Kautz, and Hyun Soo Park
    Technical Report 2021
    [PDF]