CV

Employment

Senior Research Manager, META, Reality Labs Research, Redmond, WA, USA 07/2022 – present

·  Haptic AR/VR glove tracking.

Principal Research Manager, Wormpex AI Research, Bellevue, WA, USA 11/2018 – 05/2022

·  3D SfM/SLAM and localization of 2500+ store captures per day (constantly achieving 99.5% success rate) with cameras and robots, and automatically extract high-level semantics.

·  Supervising 3D reconstruction platform to link in-store video captures with digital REVIT/CAD models.

·  Lead teams for viewing space, lighting and acoustic simulation to optimize placement for cameras, lighting and microphone/speaker arrays. Helping company cutting 40%+ in-store lighting energy cost.

·  Technical consulting for CTO strategical planning.

·  Managing intern programs for university collaborations.

Senior Research/Dev. Manager, Zillow Group, Seattle, WA, USA, 11/2017 – 11/2018

·  Managing backend 3D algorithms for Zillow 3D Home for remote house viewing.

·  Leading an R&D team to create advanced indoor 3D modeling services and algorithms, including frameworks to combine Deep Learning with classical 3D vision algorithms such as Structure from Motion and SLAM.

·  Organizing Zillow Computer Vision, Machine Learning and SLAM reading groups.

·  Coordinate Zillow-University long-term collaborations, including mentoring research interns, review university funding proposals.

Principal Researcher, Zillow Group, Seattle, WA, USA, 07/2015 – 10/2017

·  Worked on next generation house viewing experience with 3D capturing, modeling, VR/AR.

Research Scientist, Amazon Go, Amazon, Seattle, WA, USA, 05/2014 – 06/2015

·  Created the full hardware and software pipeline on Amazon Go product 3D modeling.

·  Worked on customer action detection using depth segmentation.

·  Participated in camera rig calibration and real-time re-calibration.

Lead Computer Scientist, GE Research Center, Schenectady, NY, USA, 08/2010 – 04/2014

·  Lead project (325K USD, two-years) of Low-cost SoC Vision System for kitchen situational awareness, such as hazard detection and cooking experience improvement. The project was funded by GE Appliance.

· Mil-level accuracy (1/1000 inch) dimension measurement system with single consumer level camera for industrial part defect measurement/inspection. The project was funded by GE Aviation.

·  2D borescope image/video to 3D CAD model automatic alignment algorithm and platform for Industrial crack geometric inspection. The project was funded by GE Oil & Gas Life Extension Services.

·  Multiple sensor robust fusion: produced a robust readout based on the consensus of multiple sensors; so as to reduce the impact of individual sensor noise. The system has been applied to many GE projects, such as in GE Healthcare IntelliRoom 360 project and US Department of Homeland Security (DHS) funded TLRI project, where people or vehicles are tracked by multiple cameras in patient rooms for hand-hygiene compliance analysis.

·  Camera network calibration for wide-area cloud tracking to determine the cloud-sun occlusion events for GE Energy solar plant short-term ramp prediction. Cloud height estimation with multi-view stereo and homography parallax. The project was funded by GE Power&Water.

·  Kalman filter based tracker for moving vehicle tracking from cameras mounted on Unmanned Aerial Vehicle (UAV) for GE Aviation. Developed moving camera alignment algorithm to compensate large camera motion due to flight turbulence for accurate tracking. The project was funded by GE Aviation.

·  Calibrated special designed optical sensors such as Microsoft Kinect RGB-D camera to align the color image and depth image for GE Healthcare IntelliRoom 360 project for patient healthcare room 3D reconstruction, and ultrasound probe 3D tracking.

Post-Doctoral Researcher, UNC-Chapel Hill, Chapel Hill, NC, USA, 09/2009 – 07/2010

·  Multi-view 3D motion field computation, dynamic object skeleton initialization and 3D mesh tracking. Applied to BASE-IT project funded by US Office of Naval Research (ONR) for marine virtual environment training.

Summer Intern, Honda Research Institute, Mountain View, CA, USA, 06/2005 – 08/2005

·  Construct camera network, and vision-based reconstruction for humanoid robots. Design and implement the cross-platform generic sensor network protocol the "SensorTalk", as listed in Patent section. This system is primarily used to provide Honda ASIMO humanoid system (one of the most advanced humanoid system in the world) a robust, generic and flexible platform to sense the peripheral environment, so that the robot can learn smartly from its surroundings.

Part time System Designer, Zhejiang University, Hangzhou, China, 03/2004 – 06/2004

·  3D reconstruction for personalized shoe design. Foot 3D shape automatic acquisition using multiple inward-looking digital cameras setup and a multi-view stereo algorithm framework. This project is sponsored and industrialized by Aokang Corp., one of the top shoe manufacturing corporations in China.

Education

University of North Carolina at Chapel Hill, NC, USA                        08/2007 – 05/2010

·  Ph.D. Degree, Computer Science, Advisor: Prof. Marc Pollefeys

University of North Carolina at Chapel Hill, NC, USA                        08/2004 – 07/2007

·  M.S. Degree, Computer Science, Advisor: Prof. Marc Pollefeys

Zhejiang University, Hangzhou, Zhejiang, China                               09/2000 – 06/2004

·  Bachelor’s degree in Computer Science and Technology

Research 

Post Doc, Department of Computer Science,

University of North Carolina at Chapel Hill, NC, USA                        09/2009 – 07/2010

·  Proposed multi-view 3D motion field computation, dynamic object skeleton initialization and 3D mesh tracking.

Academic Guest, Computer Vision & Geometry Group,

ETH-Zürich, Switzerland                                                                       09/2007 – 07/2008

·   Multi-view multi-object dynamic 3D reconstruction & tracking analysis.

·   Multi-modal sensor network, focus on robust probabilistic sensor fusion of camcorders and Time-of-Flight cameras for 3D reconstruction, and heterogeneous sensor network calibration.

Research Assistant, Department of Computer Science,

UNC at Chapel Hill, USA                                                                         08/2004 – 08/2009

·   Propose the idea of “effective boundary” for automatic 2D occlusion mask generation from a single video.

·   Propose a Bayesian framework for 3D static occluder inference from silhouette cues, organize multi-camera datasets acquisition and implement the algorithm in C++.

·   nVIDIA CUDA implementation of Real-time multi-view probabilistic visual hull, and occlusion inference.

Teaching 

Graduate-level course of 3D Photography, ETH at Zürich, Switzerland            Fall 2007

·   Graduate-level computer vision course, taught jointly with Prof. Dr. Marc Pollefeys.

Publication 

Journal papers

[1]       A. Sadagic, M. Kolsch, G. Welch, C. Basu, C. Darken, J. Bachs, H. Fuchs, H. Towels, N. Rowe, J.-M. Frahm, L. Guan, R. Kumar, and H. Cheng, “Smart Instrumented Training Ranges: Bringing Automated System Solutions to Support Critical Domain Needs”, The Journal of Defense Modeling and Simulation: Applications, Methodology, Technology, 2013

[2]   L. Guan, J.-S. Franco and M. Pollefeys, “Multi-view Occlusion Reasoning for Probabilistic Silhouette-Based Dynamic Scene Reconstruction”, International Journal of Computer Vision (IJCV), 2010

Conference papers, media reports and demos

[3]     S. Yan, P. Yang, H. Li, C. Song, L. Guan, H. Kang, H. Gang, Q. Huang, Implicit Autoencoder for Point-Cloud Self-Supervised Representation Learning, IEEE International Conference on Computer Vision (ICCV), Sep. 2023 

[3]     Y. Sun, Q. Huang, D. Hsiao, L. Guan, G. Hua, Learning View Selection for 3D Scenes, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2021

[4]       M. Eder, P. Moulon, L. Guan, Pano Popups: Indoor 3D Reconstruction with a Plane-Aware Network, International Conference on 3D Vision (3DV), Sep. 2019

[5]       L. Guan, Behand Zillow 3D Home – Backend Algorithms, Zillow Group Engineering Blog, https://www.zillow.com/engineering/behind-zillow-3d-home-backend-algorithms/

[6]       L. Guan, A. Colburn,  I. Boyadzhiev, A. Mullen, P. Moulon, and Q. Shan, “Zillow 3D Home”, CVPR Demo, Jun. 2018

[7]       S.-N. Lim, L. Guan, S. Liu, and X. Yang, "Automatic Registration of Smooth Object Image to 3D CAD Model for Industrial Inspection Applications", IEEE 3rd 3D Vision Conference (3DV), the joint conference of 3D Digital Imaging and Modeling (3DIM) and 3D Data Processing Visualization and Transmission (3DPVT), Seattle, Jun. 2013

[8]       M. Dou, L. Guan, J.-M. Frahm and H. Fuchs, “Exploring High-Level Plane Primitives for Indoor 3D Reconstruction with a Hand-held RGB-D Camera”, ACCV Workshop on Color Depth Fusion, in conj. with 11th Asian Conference on Computer Vision (ACCV), 2012

[9]       D. Gao, Y. Yao, F. Pan, T. Yu, L. Guan, B. Yu, T.-P. Tian, D. III Walter, B. Yanoff and N. Krahnstoever, "Computer Vision Aided Target Linked Radiation Imaging", IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, Jun. 2012

[10]     L. Guan, T. Yu and P. Tu, "Simultaneous Image Segmentation and 3D Plane Fitting for RGB-D Sensors - An Iterative Framework", International Workshop on Point Cloud Processing, in conjunction with CVPR 2012, Providence, RI, Jun. 2012

[11]     J. Tu, A. D. Amo, Y. Xu, L. Guan, M.-C. Chang, and T. Sebastian, "A fuzzy bounding box merging technique for moving object detection", Fuzzy Information Processing Society (NAFIPS) Annual North American Meeting, 2012

[12]     L. Guan, J.-S. Franco, E. Boyer and M. Pollefeys, Probabilistic 3D Occupancy Flow with Latent Silhouette Cues, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, 2010

[13]     J.-S. Franco, L. Guan, E. Boyer and M. Pollefeys, 3D Occupancy Flow from Latent Silhouettes (in French), Reconnaissance des Formes et Intelligence Artificielle (RFIA), Jan. 2010

[14]     L. Guan and M. Pollefeys, A Unified Approach to Calibrate a Network of Camcorders & ToF cameras, IEEE workshop on Multi-camera and Multi-model Sensor Fusion Algorithms and Applications (M2SFA2), in conjunction with 10th European Conference on Computer Vision (ECCV), 2008 (Oral presentation)

[15]     L. Guan, J.-S. Franco and M. Pollefeys, Multi-Object Shape Estimation and Tracking from Silhouette Cues, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Anchorage, AK, Jun. 2008

[16]     L. Guan, J.-S. Franco and M. Pollefeys, 3D Object Reconstruction with Heterogeneous Sensor Data, International Symposium on 3D Data Processing, Visualization & Transmission (3DPVT), 2008 (Oral presentation)

[17]     J.-S. Franco and M. Pollefeys, 3D Occlusion Inference from Silhouette Cues, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2007 (Oral presentation, 4.8% acceptance rate)

[18]     L. Guan, S. Sinha, J.-S. Franco and M. Pollefeys, Visual Hull Construction in the Presence of Partial Occlusion, International Symposium on 3D Data Processing, Visualization & Transmission (3DPVT), 2006 (Oral presentation)

[19]     L. Guan, C. Guo and R. Xiong, Robocup Field Distortion Calibration with Variational Image Registration, Inter-national Symposium on Intelligent Multimedia, Video & Speech Processing (ISIMP), 2004

Theses, Book chapters and other papers

[20]     L. Guan, Multi-view Dynamic Scene Modeling, Ph.D. Thesis, 2010

[21]     M. Pollefeys, S.Sinha, L. Guan, J.-S. Franco, Multiview Calibration Synchronization and Dynamic Scene Recon-struction, invited chapter, Multi-Camera Networks: Concepts and Applications, ELSEVIE corp., 2008

[22]     L. Guan, Sensor-Based Cooperative Multi-Robot 3D Environment Reconstruction, Integrative Paper, 2006, UNC Chapel Hill (Full my Master's Degree requirement)

[23]     L. Guan, Algorithms of Object Extraction in Digital Images based on Alpha value, Bachelor Thesis, CAD & CG National Key Lab, Zhejiang University, Hangzhou, 2004 (Excellent Thesis Award)

Patents

[1]      H. Gonzalez-Banos, J.Wormer, and L. Guan, "Interface for Sensor Query and Control (SensorTalk)", US8355804B2, 2005.

[2]      L. Guan, P. Tu, Y. Yao, and M.-C. Chang, "Apparatus and method for spatially relating views of sky images acquired at spaced apart locations", US8750566B2, 2011.

[3]      Y. Yao, P. Tu, M.-C. Chang, L. Guan and Y. Tong, "Apparatus and method for predicting solar irradiance variation", US8923567B2, 2011.

[4]      D. Gao, T. Yu, Y. Yao, K. Patwardhan, L. Guan, and P. Tu, "Real-Time Video Tracking System Using Multi-cue Fusion and Scene Context", US8842881B2, 2012.

[5]      S. N. Lim, L. Guan, S. Liu, X. Yang, J. Garza, D. Diwinsky, and J. Rittscher, "Methods and Systems for Enhanced Automated Visual Inspection of a Physical Asset", US9251582B2, 2012.

[6]      P. Tu, J. Ritscher, and L. Guan, "System and method for generating semantic annotations", US9251421B2, 2013.

[7]      S. N. Lim, L. Guan, S. Liu, and X. Yang, "System and method for image based inspection of an object ", US9305345B2, 2014.

[8]      L. Guan, "System and Methods for Boil Detection", US9330469B2, 2014.

[9]      L. Guan, "System and Methods for Cookware Detection", US9449220B2, 2014.

[10]    S. N. Lim, J. Tu, L. Guan, and C. Hoffman, "Method and system for detecting a damaged component of a machine", US9483820B2, 2014.

[11]    S. N. Lim, L. Guan, S. Liu, S. Dhanvantri, Y. Agarwal, B. Pandey, "System and method for engine inspection", US20160178532A1, 2014.

[12]    J. Armstrong, J. Wiseman, and L. Guan, "Sensing system for a cooktop appliance with airflow protected sensor", US20150302569A1, 2014.

[13]    L. Guan, "Personalized Household Object Classification via Visual Sensors", Filed by GE Appliance, 2014.

[14]    G. Hua, L. Guan, "System to search three-dimensional object data", US10902280B1, Amazon Go, Amazon, 2015.

[15]    Q. Shan, L. Guan, A. Colburn, I. Boyadzhiev, "Connecting and using building data acquired from mobile devices", US10834317B2, Zillow 3D Home, Zillow, 2017.

[16]    A. Colburn, Q. Shan, R. Mehran, L. Guan, Automated Mapping Information Generation From Inter-Connected Images, US20200408532A1, Zillow 3D Home, Zillow, 2017

[17]    I. Boyadzhiev, A. Colburn, L. Guan, Q. Shan, Capture, Analysis And Use Of Building Data From Mobile Devices, US20190306424A1, Zillow 3D Home, Zillow, 2017

[18]    Y. Li, L. Guan, Presenting image transition sequences between viewing locations, US10643386B2, Zillow 3D Home, Zillow, 2018

[19]    M. Dawson, L. Guan, A Otwell, D. Hsiao, Automated control of image acquisition via use of acquisition device sensors, US10708507B1, Zillow 3D Home, Zillow, 2018

Invited Talks & Presentation

·  CVPR demo, Salt Lake City, Jun. 2018

·  Amazon, Seattle, WA, Mar. 2014

·  CVPR, Providence, RI, Jun. 2012

·  Zhejiang University, Hangzhou, China, Dec. 2011

·  CVPR, San Francisco, CA, USA, Jun. 2010

·  GE Global Research Center, Niskayuna, NY, USA, Feb. 2010

·  Intel Research Lab Seattle, Seattle, WA, USA, Jan. 2010

·  University of Southern California, Los Angeles, CA, USA, Aug. 2009

·  Utopia Compression Co., Los Angeles, CA, USA, Aug. 2009

·  Object Video, Reston, VA, USA, Apr. 2009

·  CVPR, Anchorage, AK, USA, Jun. 2008

·  3DPVT, Atlanta, GA, USA, Jun. 2008

·  CVPR, Minneapolis, MN, USA, Jun. 2007

·  3DPVT, Chapel Hill, NC, USA, Jun. 2006

Membership & Service

·  Omnidirectional Computer Vision Workshops organizer (in conjunction with CVPR) 2020 ~ 2023

·  Ph.D, defense committee member for Dr. Marc Eder @ UNC-Chapel Hill, defended 2020.

·  Member of Sigma Xi, 2010-present

·  Member of Program Committee, International Conference on Computer Vision (ICCV), 2011; 3DIMPVT, 2012; 3DV, 2013; 3DV 2014.

·  Reviewer of 3DV 2013 - present, ECCV 2010 - present, ICCV 2007 – present, IROS 2012, CVPR 2009 - present, 3DIMPVT 2012, AVSS 2008, VECIMS 2008, ACCV 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Machine Vision and Applications Journal, Journal on Advances in Signal Processing (JASP), Journal of 3D Research, Journal of Computer Vision and Image Understanding, Journal of Computer Graphics Forum, Journal of Visual Communication and Image Representation, Journal of Neurocomputing, ELSEVIE Book chapter of “Multi-Camera Networks: Concepts and Applications”

Certificates

GE Award to Inventors                                                                                                2012

GE certified Green Belt in Lean Six-Sigma and DFSS                                               2011

Skills

Proficient in C/C++. Python, OpenCV, OpenMVG, Ceres, VXL, VTK, ITK & Cmake, and OpenGL. Visual Studio, Eclipse, Xcode and Windows/Linux operating systems. Experienced with Unreal Engine, Unity, GLSL, cg, CUDA. iOS development with both Swift and React Native. Proficient with Matlab, Latex, Adobe Photoshop, Illustrator, Poser, Blender, and Coral Painter.

Hands on experience with ARKit, Kinect, ARM processor cross-compilation. Fluent in spoken and written Chinese and English.