Professor: Marek A. Perkowski, Electrical Engineering.
Goal of the class.
Students should learn the main ideas of humanoid robotics: motion control, sensing environment and intelligent
control. Emphasis is on robot vision methods that allow human-robot interaction.
Students should get practical understanding how robot vision
algorithms are used in robot systems.
Where is this knowledge useful?
Previous students who have done projects found this class useful to find industrial positions in the
following areas: robot vision, flexible automation systems with vision, medical image processing,
hardware design of image processors, bank image processing, general software design, industrial board-level design.
The ideas can be used also in startup companies that build innovative mobile
and humanoid head robotic products (one company so far).
Software used in class depends on every year projects. This quarter the projects are:
1) KHR-1 (motion control, programming, human gesture mimicking using OpenCV).
Programming in Visual Basic and C++ (OpenCV).
2) Sonbi, theatrical stationary robot. (motion control, motion programming, speech, vision).
Programming in Visual Basic and C++ (OpenCV).
In general, students can use C, C++, Basic, Visual Basic.NET, Lisp and Matlab.
- Towards Robot Theatre. Introduction to the class.
Slides in PPT (PowerPoint). This lecture in addition reviews the basic concepts from previous robotic class.
We discuss decision trees, search, automata, combinational mappings (functions), evolutionary algorithms and neural nets in robot control.
Robotic System Servo Control. Presentation of Jeff Allen as a lecturer in class. Slides in PPT.
Sensors and their role in new approaches to perception. Slides in PPT.
- Digital Resistive sensors. Slides in PPT.
- Analog Resistive sensors. Slides in PPT.
- Overview of sensors. Slides in PPT.
Robot soccer competitions. Introduction to Robot Vision. Slides in PPT.
Cameras. Visual Servoing. Slides in PPT.
Edge detection and feature extraction algorithms. Slides in PPT.
Labeling and sequential algorithms. Slides in PPT.
Histogramming. Soccer robot vision. Slides in PPT.
Walsh Transforms and butterflies. Slides in PPT.
- Walsh Matrix. Slides in PPT.
Spectral Transforms and Image Processing software. Slides in PPT.
Walsh and Fourier Transforms. Butterflies. Fast algorithms and their properties. Use of spectral methods
in robot vision. Slides in PPT.
- Hough Transforms. Slides in PPT.
Hough Transform application in a mobile robot for corridor navigation. Slides in PPT.
- Quad trees and Oct-trees. Slides in PPT.
- Thinning algorithms. Slides in PPT.
Review. Questions in Intelligent Robotics. In Word format.
How many you can answer?
Auxiliary Materials for class and projects. Mostly our reports.
FROM JEFF ALLEN FOR OPEN CV VISION PROJECT - HUMAN GESTURE RECOGNITION.
- cv096.dll DLL from Jeff for project.
- cxcore096.dll DLL from Jeff for project.
FROM STEFAN GEBAUER FOR SPEECH-VISION SONBI THEATRE PROJECT.
Stefan Gebauer. Humanoid Robot based on Vision and Speech Recognition. In PDF format.
Report1 from Stefan. In PDF format.
Report2 from Stefan. In PDF format.
Report3 from Stefan. In PDF format.
Report4 from Stefan and Normen Giesecke's group. In PDF format.
- ZIP files for vision project.
robotics2nn.zip-DEFANGED ZIP files with software and reports.
MORE ON OPEN CV.
Thesis by Mikhail Pivtoraiko about using OpenCV on Stanton board. In Word format.
- Instruction about using OpenCV. In PPT format.
- Contact to Sam Siziliano who is OpenCV expert.
- Email from Anthony Kautz
who build speech for robot and worked on OpenCV. Helpful. But may be obsolete now. In txt format.
MORE ON VISION SYSTEMS.
Vision Guided Motion. Slides in PDF.
Robot Soccer. Birgit Graf, student of Prof. Braunl. Here you can learn about Robot Soccer
and their vision system in full detail. In PDF format.
Auxiliary Materials for class and projects. Various Sources.
Lectures on Matlab. Lecture 1. Introduction to Matlab.
Lectures on Matlab. Lecture 2. More Matlab Programming.
Lectures on Matlab. Lecture 3. Finishing with Matlab.
Lectures on Matlab. Lecture 4. Finishing with Matlab.
- Lecture on Introduction and Control Basic to Matlab. In PDF.
- The same lecture in PPT.
- Matlab Primer in PDF.
Introduction to Matlab in PPT.
Matlab two-dimensional plots in PPT.
Matlab Script and Function files in PPT.
- Simple Programming in Matlab in PPT.
Solution of non-linear algebraic equations in Matlab. PPT format.
- Vision for robot localization by Ulrich. Paper in PDF.
Visual Servoing for a mobile robot. Paper in PDF.
- Paper by Zhang on camera calibration in PDF.
Region Segmentation. Slides in PDF format.
Fast Parallel Algorithms for Matrix problems. Slides in PPT Format.
- Beyond wavelets and JPEG 2000. Slides in PPT format.
- Camera Calibration for 3D vision. In PDF.
- Slides in PDF about wavelets.
Robot Competitions and societies.
Paper about autonomous driving mobile robot competitions.
In PDF format.
Technical Activities of RAS. Slides in PPT.
Research related to Book of Prof. Braunl.
1999 paper by Prof. Braunl about the EyeBot robot Family. In PDF format.
The Eysim Mobile Robot Simulator by Prof. Braunl. In PDF format.
- Client server for Eyebot. Paper in PDF.
Behavior Based Robotics and Biological Models.
Slides in PPT about behavior based robot design. We discussed several similar ideas in class.
- Robots and biological intelligence. PDF.
- From Sensory
Substitution to Situated Robots. Slides in PPT.
Slides in PPT by Mark Brosnan about Affective computing.
Class about autonomous mobile robots. In PDF.
Homework in mobile robot kinematics. In PDF format.
Measurement and correction of systematic odometry errors in mobile robots. Paper in PDF format.
- Paper in PDF.
Structural Properties and Classification of Kinematic and Dynamic Models
of Wheeled Mobile Robots.
Robot Navigation and related topics.
- Mobile robot positioning using sensor. In PDF.
Paper in PDF about Monte Carlo localization for mobile robots.
Paper on Experimental Comparison of localization methods in PDF.
Bayesian estimation and Kalman filtering: A Unified framework for Mobile
Robot Localization. Paper in PDF.
Navigation and Metric Path Planning.
Thesis by Philip Kedrowski about Self-building global maps for autonomous navigation.
In PDF format.
Lecture from CMU about Path planning for mobile robots. Slides in PDF format.
Lecture from CMU about Mapping. Slides in PPT format.
Paper about fast obstable avoidence based on vector field histogram. In PDF.
Dynamic Markov localization approach, by Burgard, Derr, Fox and Cremers. Paper
in PDF format.
- Paper about path planning for a mobile robot.
- Fingerprint for mobile robot localization in PDF.
- Posters about evolutionary robotics from
Introduction to robot control. Slides in PDF.
- Example of a world for a robot.
Walking robots, especially KHR-01.
- Slides on Kinematics and Animation of Humanoid Robots in PPT.
- High Level Motion Control Slides in PPT.
- Modeling humanoid robots in computer graphics. Slides in PPT.
- Animation for computer graphics. Slides in PPT format.
David Vernon. Inexpensive humanoid robot architectures. In PDF.
- Gait evolution for biped robots using visual feedback.
Paper in PDF.
Baltes et al. Design Walking gaits for small humanoid robot. In PDF format.
Paper about Active Balancing using Gyroscopes for a Small Humanoid Robot.
Paper about humanoid Robots. They also use KHR-01. In PDF format.
Research in Humanoid robots from Brown University. In PPT format.
Random Morphology robot - Locomotion learning. One page, interesting and new.
In PDF format.
Human-Robot and Human-Computer Interaction.
Prof. Fumio Harashima about State of the Art in Human-Computer Interaction. Much about robotics.
Slides in PDF format.
- Lectures about Human-Robot Interaction. Slides in PDF.
Lecture about Tracking Devices for humans, for instance built into glasses. In PDF format.
Lecture from CMU about developing autonomy for robots in teams. Slides in PDF format
This has applications to robot soccer and robot theatre.
Paper about robot colony for entertainment. In PDF.
A Probabilistic Approach to Collaborative Multi-Robot Localization. Paper in PDF.
- Heterogeneous Team of Modular
Robots for Mapping and Exploration. Paper in PDF by Robert Grabowski.
Lecture from CMU about coordination using search. In PDF format.
- New Paradigm for robot learning. In PDF.
Motion Planning for stationary robots.
Motion planning methods good for stationary robot with hands. In PDF.
- Motion Planning that may be applied to
any kind of robots. PDF.
Other robotics classes.
- Overview of AI-based robotics class. Slides in PDF.
For Robot Theatre.
- Slides on Theory of Science in PDF.