A software platform for “smart” video tracking 

– Ranjini Raghunath

 Researchers at the Indian Institute of Science (IISc) have developed a novel software platform from which apps and algorithms can intelligently track and analyse video feeds from cameras spread across cities. Such analysis is not only useful for tracking missing persons or objects, but also for “smart city” initiatives such as automated traffic control.  

 Many cities worldwide have set up thousands of video cameras. Machine learning models can scour through the feeds from these cameras for a specific purpose ‒ tracking a stolen car, for example. These models cannot work by themselves; they have to run on a software platform or “environment” (somewhat similar to a computer’s operating system). But existing platforms are usually set in stone, and do not offer much flexibility to modify the model as the situation changes, or test new models over the same camera network.  

 “There has been a lot of research on increasing the accuracy of these models, but sufficient attention hasn’t been paid to how you make [the model] work as part of a larger operation,” says Yogesh Simmhan, Associate Professor at the Department of Computational and Data Sciences (CDS).  

The spotlight algorithm narrows the search space for analysing video feeds if the missing person is found within a camera’s field of view. It gradually expands the set of video feeds analysed when the person falls in a blindspot between cameras. This intelligence helps reduce the computation required for analysing videos from thousands of cameras while not sacrificing accuracy. Credit: Yogesh Simmhan

To address this gap, Simmhan’s lab has developed a software platform called Anveshak. It can not only run these tracking models efficiently, but also plug in advanced computer vision tools and intelligently adjust different parameters ‒ such as a camera network’s search radius ‒ in real time.  

 In a recently published paper, the researchers show how Anveshak can be used to track an object (like a stolen car) across a 1,000-camera network. A key feature of the platform is that it allows a tracking model or algorithm to focus only on feeds from certain cameras along an expected route, and tune out other feeds. It can also automatically increase or decrease the search radius or “spotlight” based on the object’s last known position.  

 The platform also enables the tracking to continue uninterrupted even if the resources ‒ the type and number of computers that analyse the feeds ‒ are limited. “In the field, the amount of computing power you have is not really negotiable on the fly. The devices are static. You have to do the best you can with what is available,” explains Simmhan. For example, if the search radius needs to be increased and the computer becomes overwhelmed, the platform will automatically start dropping the video quality to save on bandwidth, while continuing to track the object.  

 In 2019, as part of a winning entry for the IEEE TCSC SCALE Challenge award, Simmhan’s lab showed how Anveshak could potentially be used to control traffic signals and automatically open up “green routes” for ambulances to move faster. The platform used a machine learning model to track an ambulance on a simulated Bengaluru road network with about 4,000 cameras. It also employed a “spotlight tracking algorithm” to automatically restrict which feeds needed to be analysed based on where the ambulance was expected to go.  

 Simmhan’s lab is also working on incorporating privacy restrictions within the platform. “We can decide what are the kinds of analyses that we are comfortable running. We could say, for example, that we would allow analytics that track vehicles, but not analytics that track people, or analytics that track adults but not children,” he says. They are also working on ways to use Anveshak to track multiple objects at the same time.  

A. D. Khochare, A. Krishnan and Y. Simmhan, “A Scalable Platform for Distributed Object Tracking across a Many-camera Network,” in IEEE Transactions on Parallel and Distributed Systems, doi: 10.1109/TPDS.2021.3049450 

CONTACT:
Yogesh Simmhan
Associate Professor
Department of Computational and Data Sciences (CDS)
Indian Institute of Science (IISc)
simmhan@iisc.ac.in
+91-80-2293 3421

NOTE TO JOURNALISTS: 

a) If any of the text in this release is reproduced verbatim, please credit the IISc press release.
b) For any queries about IISc press releases, please write to news@iisc.ac.in or pro@iisc.ac.in