(Partially supported by the Center for Telecommunication and Network Security at Oklahoma State University)
Authors: Ronny S. Lim, Hung M. La and Weihua Sheng
Today there are bridges everywhere around the world. One of the important tasks to maintain the bridge deck is to detect the cracks. There many kind of cracks inspection, the easiest one is using human eyes, however it is limited of accuracy. Other approach is using robot to do the inspection.
In this project, our problem is to build a system that detect cracks using a mobile robot. Basically, the robot takes the images of the surface of the bridge. Then using some image processing techniques we detect the cracks and get their location. The locations of the cracks in the images are local, hence we need to merge them together based on camera calibration technique to build the global map of cracks. There are several challenges in this project such as localization, path planning, crack detection and camera calibration.
Laplace of Gausian Algorithm: The main purpose of the Laplace of Gaussian algorithm is to find out the edge points of an image. These edge points can be detected by finding the zero crossings of the second derivative of the image intensity. However, calculating second derivative is very sensitive to noise. This noise should be filtered out before edge detection. To achieve this, Laplacian of Gaussian is used. This method combines Gaussian filtering with the Laplacian for edge detection.
Camera Calibration: In order to find the cracks location in the world coordinate, we need to find the relationship between the image frame and world frame. Using camera calibration we find out that relationship such as translation and rotation matrix. Using camera calibration Matlab toolbox, we acquire the translation and rotation matrix. In the application, we define the calibration before we start the experiment, and we assume that the orientation of the camera will not be changed. Hence, we can apply parameters that we have from the calibration to find out the world coordinate of the cracks.
Localization and path planning: We use some software from Mobile Robot Inc to solve the localization problem. In other hand, we use tele-operating to implement the path planning. The software that we use is ARNL (Advanced Range Navigation Laser) and Mobile eyes. The algorithm for localization is based on Monte Carlo Localization (MCL). Before we run the ARNL, we need to build 2D map for the robot to localize using MCL. We do tele-operating on the robot to scan the environment using laser scan. Then we use Mapper3 to build 2D map. Finally, the map will be given to the robot in order to localize.
Publications of this project:
- R. S. Lim, H. M. La and W. Sheng, A robotic crack inspection and mapping system for bridge deck maintenance, IEEE Transactions on Automation Science and Engineering,pp. 367-378, Vol. 11, No. 2, April 2014.
- R. S. Lim, H. M. La, Z. Shan and W. Sheng, Developing a crack inspection robot for bridge maintenance, in the proceedings of the IEEE International Conference on Robotics and Automation (ICRA), May 9 – 13, 2011, Shanghai, China.
Graphical user interface for crack detection and crack mapping on the global coordinate system.
Golden Bridge in San Francisco.
Bridge cracks inspection based on human’s eyes.
Bridge cracks inspection using mobile robot.
This picture shows the Pioneer 3DX robot with a high resolution camera used in the experiment.
Video of Crack Detection Using Pioneer 3DX robot.
Video of Path Planning Using Virtual Obstacle.