Si Lu
Computer Vision Engineer - Autonomous Vehicles at NVIDIA, CA
Ph.D.in Department of Computer Science Maseeh College of Engineering and Computer Science Portland State University, OR

Email: lusi "at" pdx.edu
Resume (updated on 09-19-2019)
 
Working on Autonomous Driving at NVIDIA. PhD student in Computer Science at Portland State University. My advisor is Dr. Feng Liu. I have More than 9 years' experience on Computer Vision & Graphics topics such as 3D construction/enhancement in light field VR, novel view synthesis for camera arrays, video stabilization, image stitching as well as computational photography such as image/RGBD denoising and patch matching. Passionate about developing/implementing state-of-the-art computer vision and computer graphics algorithms. Skilled programmer with C/C++/Python/Matlab/OpenCV/OpenGL/Pytorch. Familiar with machine learning and deep learning techniques such as CNN/RNN/LSTM/ResNet/DenseNet/Faster R-CNN/Mask R-CNN.
Enducation
Portland State University Ph.D. candidate in Computer Science GPA 3.92 2012-present
Portland State University Master of Computer Science GPA 3.92 2012-2019
Tsinghua University Master of Engineering in Fluid Mechanics GPA 3.63 2009-2012
Tsinghua University Bachelor of Engineering in Mechanics GPA 3.61 2005-2009
Publications (Click here to see my peer-review expericen)
 
Experience
Computer Vision Engineer - Autonomous Vehicles NVIDIA, Santa Clara, CA, 06/2018 - present
My work appears at GTC 2019!
I am proud that my work appeared in the GTC keynote demo at GTC 2019. Click to watch.

NVIDIA teamed up with leading HD mapping partners to deliver a global mass market end-to-end autopilot solution. Click to watch.

Autonomous Vehicles: HD Map Generation
• HD map fusion: design and implement lane boundary fusion.
• HD map fusion: design and implement pole/sign/traffic lights fusion.
• HD map emitting: implement the emitting of all fused map contents.
• HD map rendering: construct locallayout for rendering and visualization.
• HD map rendering: implement graphics pipeline to render HD map contents.

Autonomous Vehicles: Localization
• Localization: KPI tracking/visualizationand in Athena websites.
• Localization: fast KPI computing in the cloud computing platform.
• Localization: design/implement a high-accuracy camera-based localization
• Localization: so far the first one to purly use low cost sensors (IMU, GPS ...).
• Localization: design/implemente core algorithm for lidar/radar localization
 
Computer Vision Intern - Lytro Immerge (VR) Lytro, Moutain View, CA, 06/2017 - 09/2017
  Multi-view 3D reconstruction ( C/C++/OpenCV )
• Processing HD (2048×2048) RGB-D images
• Signigicant fine detal improvements for VR depth maps (left)
• Depth enhancement for light filed (multi-view) camera arrays (middle)
• Depth enchancement for stiched 360-degree videos in Virtual Reality (right)
• Integrated into Lytro Immerge (VR) production pipeline for depth
 
Research Projects at Universities
Highspeed Videos from Camera Arrays (2016-2017)
• Develope an algorithm to generate highspeed videos from camera arrays
• Use novel views synthesis to do frame interpolation among cameras 
• Use image-based rendering techniques, e.g., feature matching/warping
• Plausible and parallax-free novel views can be robustly interpolated
• Works for challenging scenes with large camera and object motions
• Accepted to WACV 2019: Paper, Video demo, Poster, Dataset.
   
Patch Matching for Image Denoising (2017-present)
 Develope a clustering-based approach with unreliable pixel estimation to consistently improve patch-based denoising techniques' (like BM3D) performance via better similar patch searching for image denoising. (Accepted to WACV 2019: Paper, Video demo, Poster, Code (New!))
   
No-reference Image Denoising Quality Assessment (2015-2017)
Present a noreference image denoising quality assessment method that can be used to select for an input noisy image the right denoising algorithm with the optimal parameter setting. Project website (New!). Code (New!). Github (New!)
   
Depth Enhancement (2012-2014)
Develop a depth map enhancement algorithm that performs depth map completion and denoising simultaneously for RGBD cameras like Microsoft Kinect and Xtions Pro. Paper. Project website. Poster.
   
Flash Light Detection for Unmanned Aerial Vehicles (2015)
Developed an algorithm that allows real-time flash light detection for Unmanned Aerial Vehicles. By designing specific color-based features, we achieved a 89.2% detection rate with a small false positive rate (0.6%).
 
Robocup Team Leader (2007-2012) Project website.
• Developed a robot vision system with accurate and real-time (20 FPS) objects (football, field lines and goals) detection and ball-locating/self-locating (C++/Windows Mobile SDK/Linux).
• Developed a wireless vision parameter tuning interface to allow fast camera calibration before real games. By utilizing this interface, I reduced the parameter tuning time by more than 70% before each game (Matlab, C).
 
Teaching Experience
CS 447/547-Computer Graphics (instructor): teach Computer Graphics topics on movies, games, animations and 3d rendering. Helped students with two course projects - a mini-Photoshop (FLTK/OpenCV) with basic image processing and an Amusement Park 3D animation rendering project (OpenGL).
CS 410/510-Computational Photography (instructor): teach research topics ranging from concepts of digital camera and photography to computer vision/graphics techniques, including high dynamic range imaging, panorama stitching, image segmentation & matting, video stabilization, virtual reality basics, deep learning in computer vision etc..
Academic Peer-Reviewing experience
• IEEE Transactions on Image Processing, Lightening Network for Low-Light Image Enhancement(in progress)
• IntelliSys2020, Recommender - A Personalized Subreddit Recommendation Engine
• IntelliSys2020, Perspectives of Applying the e-Learning Model in Educational Institutions in Bosnia and Herzegovina
• IntelliSys2020, Neural Network Modeling of Productive Intellectual Activity in Older Adolescents
• IntelliSys2020, AI Scrum Master - An automated approach for agile project planning based on artificial intelligence
• IntelliSys2020, Enhanced Cognition aided by Reinforced Learning and Real-Time Decision over Shared-Spectrum
• IEEE Access, A Novel Bayesian Patch-Based Approach for Image Denoising
• IEEE Transactions on Image Processing, Spatial-temporal Networks for Temporally Coherent Video Super-Resolution
• IEEE Transactions on Image Processing, FusionNet: An Unsupervised Convolutional Variational Network for Hyperspectral and Multispectral Image Fusion
• IEEE Transactions on Image Processing, A Multi-path Deep Neural Network for Visual Smoke Surveillance
• IEEE Transactions on Image Processing, Automatic Content-preserving and Temporal-consistent 360° Video Projection
• IEEE Transactions on Multimedia, A Multi-Attribute Blind Quality Evaluator for Tone-mapped images
• IEEE Transactions on Multimedia, No-reference Image Quality Assessment via Local Structural Information Representation
• IEEE Transactions on Multimedia, Visual Importance and Distortion Guided Deep Image Quality Assessment Framework
• IEEE Signal Processing Letters, Color Image Denoising via Cross-channel Texture Transferring
• IEEE Signal Processing Letters, A geometric-separating convolutional neural network for blind image inpainting
• IEEE Transactions on Circuits and Systems for Video Technology, Depth Estimation Using an Infrared Dot Projector and an Infrared Color Stereo Camera
• IEEE Transactions on Circuits and Systems for Video Technology, Deep Convolutional Shearlet Transform for Densely-Sampled Light Field Reconstruction
• IEEE Signal Processing Letters, Strong Edge-Aware Depth Image Completion with Multidirectional Filtering
• ACCV Workshop on RGB-D-Sensing and understanding via combined colour and depth, Unsupervised RGBD Video Object Segmentation Using GANs
• ACCV Workshop on RGB-D-Sensing and understanding via combined colour and depth, Environment-aware non-rigid registration in surgery using physics-based simulation