Frame difference approach for Movement detection using Kinect and OpenCV and ROS.

Frame Difference

Movement detection is very salient feature in modern image processing based application. There are several approaches can be taken for detecting movement. Background subtraction is a well-known approach for computer vision based object detection. Therefore, there are already a lot of works in the literature focused on it. Background subtraction is very useful when the background is fixed and also the camera is stationary position.

To detect any kind of change from a camera vision or video, the difference of two consecutive frames is one of the most effective approaches. The frame difference can be defined as equation below: 

Frame Difference

f(x, y, t) is a frame at time t. The immediate next frame at (t+1) is f(x, y, t+1) and th represents the threshold value parameter for decision. Three Frame difference is an improved method from two frame difference and it uses three consecutive frames to find the difference. From the difference of three frame images, the changes or movement detection is done using the contour object.

Next two equations are for three frame difference method. The additional option is that there is a scope for adding multiple threshold value. For n frame difference method there is an option to use (n-1) threshold values which can give more flexibility to find the moving contours.

Three Frame difference

fk(x,y)is the k-th frame. T1, T2 is thresholds with different values.  and  are the binarization of two adjacent frames difference. 

ROS

Robot operating system is chosen for robot based implementation of movement detection.Robot Operating System (ROS) is an open-source, heterogeneous and scalable P2P network-based robotics framework. ROS provides wide range of libraries and tools to help software developers creating innovative robot applications in an organized pattern. The benefit of using ROS is to get some useful essential features which will reduce the development time and complexity by providing device drivers, libraries, visualizers, message-passing, package management, access to third party tools, hardware abstraction, and more. ROS is licensed under an open source, BSD license. The basic advantage of ROS is code reuse for the robotic developer and researcher in a common and generalized platform. ROS is a language-independent architecture (C++, python, lisp, java, and more), scalable platform (ARM CPUs to Xeon Clusters) and intent to enable researchers to rapidly develop new robotic systems by increasing the reusability through use of standard tools and interfaces.

There are not many Robot frameworks out there which support the development of general purpose robot software across many platforms. This makes it difficult to accomplish even simple and trivial tasks which might seem simple to humans but the Robot developer has to navigate and sort through many variations between instances of tasks and environments. Having a general purpose framework like ROS helps us to create complex and robust robot behavior across many platforms which have got the support of ROS.

How to use Kinect in ROS?

Quick Intro about Kinect:

Kinect is a RGB-D based motion sensing input device product line by Microsoft. It was announced and demonstrated for Xbox 360 video game platform named as Project Natal in June 2009. The first generation Kinect was first released on November 2010. Since then it attracts the attention of researchers and robotics developer community due to its immense possibility compare to traditional vision based system.

 Kinect was developed based on the software technology by Rare and on range camera technology by Prime Sense which developed a system that can interpret specific gestures, making completely hands-free control of electronic devices. Kinect features an RGB camera, depth sensor and multi-array microphone. The depth sensor consists of IR laser projector combined with CMOS sensor which captures 3D video data. The RGB camera is located between the IR projector and IR camera, which has no role in depth measurement.

Kinect was originally designed and released to be used in Microsoft Xbox 360 game console. No external driver was provided by the manufacturer. But as Kinect grabs attraction to the researchers and robotic development community, in November 2010, Adafruit Industries offered a bounty for an open-source driver for Kinect and in the same month Adafruit announced Hector Martin as the winner who produced a Linux driver for Kinect that allows using both of the RGB camera and depth sensitivity functions. It was named Open Kinect libfreenect which is the core library for accessing the Microsoft Kinect. Another open source framework, OpenNI framework was introduced by OpenNI organization. It provides for Kinect an application programming interface (API). There are 12 different nodes available in OpenNI framework for different functionalities.

Quick Intro on ROS:

Robot Operating System (ROS) is an open-source, heterogeneous and scalable P2P network-based robotics framework. ROS provides wide range of libraries and tools to help software developers creating innovative robot applications in an organized pattern. The benefit of using ROS is to get some useful essential features which will reduce the development time and complexity by providing device drivers, libraries, visualizers, message-passing, package management, access to third party tools, hardware abstraction, and more. ROS is licensed under an open source, BSD license. The basic advantage of ROS is code reuse for the robotic developer and researcher in a common and generalized platform. ROS is a language-independent architecture (C++, python, lisp, java, and more), scalable platform (ARM CPUs to Xeon Clusters) and intent to enable researchers to rapidly develop new robotic systems by increasing the re-usability through use of standard tools and interfaces.

There are not many Robot frameworks out there which support the development of general purpose robot software across many platforms. This makes it difficult to accomplish even simple and trivial tasks which might seem simple to humans but the Robot developer has to navigate and sort through many variations between instances of tasks and environments. Having a general purpose framework like ROS helps us to create complex and robust robot behavior across many platforms which have got the support of ROS.

• Peer to Peer
• Multi Language Support
• Free and open source

Now to work with Kinect and ROS in Linux environment, a ROS based kinect driver is needed.  Once the Linux environment is set with ROS installation the Freenect stack setup is enough to communicate with kinect sensor through ROS. A sudo apt get command is enough for that.

sudo apt-get install libfreenect-dev
sudo apt-get install ros-indigo-freenect-launch

For executing the command a internet connection is needed. This will install the kinect driver into ROS. After that connect the kinect device with USB and also make sure the power cable is connected. Then run the freenect launch command to start driver.

roslaunch freenect_launch freenect.launch

The package link is available here.