The Project

For our project we implemented a non-holonomic differential drive robot capable of retrieving payloads from different distinct locations, marked by colour, by means of on-board visual servoing.

Our motivation for this stems from the Amazon Picking Challenge, where warehouse robots must retrieve items from shelves and bring them back to a packaging location. While our solution ignores the complexity involved in selecting individual items from a collection, as well as localizing shelves in a third dimension, we feel the visual servoing aspect still presents a significant challenge. It also would allow for infinitely reconfigurable warehouses with little reconfiguration of the robot itself; instead of following tracks to a pre-programmed location, our robot offers the ability for real-time localization, discovery, and movement.

Visual Servoing

How can you identify objects of interest in a video stream?

• Took an image frame from source IP Camera
• Converted to HSV
• Find image in range of selection
• Open to eliminate noise
• Close to fill holes

Exploration

If no objects are in the robot's field of view, what is the best way to explore the environment to find the objects of interest?

• Grid occupancy mapping
• Tachometric Localization
• Follow walls
• Turn inside and outside corners
• Difficulty detecting motor stalls
• Scan for the ball when turning

The Architecture

• Decided to build a robot that was fully teleoperated
• Better performance on the PC side for computationally expensive tasks (image processing, pathing finding, etc).
• Communication occurred between the PC and EV3 in JSON.
• Wanderer,Braitenberg,and Home Behavior

JSON Example:

{ "motor": "A", "command": { "type": "rotate to", "argument": "50" } }