George is a zebrafish. Along with Tom and 98 other mates, George swims freely in a laboratory tank at the Champalimaud Centre for the Unknown (CCU), in Lisbon, Portugal. A camera records from above a video of all the animals’ comings and goings.
Is it possible to distinguish individual fish from the video images at every moment? Yes, says Gonzalo de Polavieja, principal investigator of the Collective Behavior Lab, who, with his team, has developed a program called idtracker.ai that can do the job quickly and with extreme accuracy. Their results were published in the journal Nature Methods.
“The ultimate goal of our team is understanding group behavior,” says de Polavieja, whose preferred names for the fish are George and Tom. We want to understand how animals in a group decide together and learn together.” The researchers sought to extract very high quality data from the videos such as the position and shape of each of the animal, as well as their individual paths, without mistakes.
Recognizing each individual among dozens of similar peers would be difficult for humans—or, for that matter, a conventional computer program. “We would just go crazy trying,” says de Polavieja. For large crowds, without artificial intelligence in the mix, even a powerful computer might have to run the program for years to get results. And these would probably not be very accurate.
That’s where idtracker.ai comes in. The new software, says de Polavieja, supplies the quality data required for a second phase that analyzes the rules that drive the animals’ collective behaviour.
Four years ago, before joining the CCU, de Polavieja published the first version of the software in Nature Methods, which did not rely on artificial intelligence. The results were much more modest. “We could track 10 animals back then,” he says.
De Polavieja and his co-authors, Francisco Romero-Ferrero, Mattia Bergomi, Robert Hinz and Francisco Heras, have now tested the new AI version with groups of up to 100 zebrafish. “We didn’t test more than 100 because our tank is not big enough for that.” Nonetheless, using another approach to record the images, they showed that the software can identify up to 150 individual fish with very little loss of accuracy. “I didn’t believe we could reach those numbers; it was a surprise,” notes de Polavieja. “I thought there wouldn’t be enough information in the images.”
Deep learning
Idtracker.ai is composed of two so-called deep-learning neural networks and a few more conventional algorithms. A deep-learning neural network is a computer simulation of real networks of neurons in the brain that are capable of learning from experience.
Using the video images of the zebrafish in the tank, the first network in the chain is trained to tell whether every visible blob in the images corresponds to a single animal or to several.
With this output, the second neural network is then trained to assign a name (or number) to each blob that contains just one fish—in other words, to identify each individual fish. The recognition is based on the unique features of each zebrafish. “People think zebrafish are all alike, but this proves they are, in fact, all different from each other,” notes de Polavieja.
Lastly, the program applies two conventional algorithms. “One is to gain some certainty about the few individuals whose identities are still somewhat uncertain,” says de Polavieja. “And the other determines which animal is which when their paths cross—that is, when their trajectories appear superimposed on the video.
The results speak for themselves: It takes about an hour for idtracker.ai to identify each and every one of the 100 zebrafish in the video at all times with almost 100 percent accuracy. “If you show the network a random part of the video it has never seen before and ask it: ‘who is this?”, the network will correctly assign the right name—or number—to that fish 99.997 percent of the time,” says de Polavieja. And if you ask it where George, or Tom, or any other zebrafish is at a given moment, it will find it in the crowd almost beyond the shadow of a doubt.
The team also tested the software with fruit flies, medaka fish (Japanese rice fish), ants and mice. It also works, though with smaller numbers of individuals. Zebrafish are best for these studies, says de Polavieja. “Mice are more difficult because they tend to cluster and deform.”
“This is the first time such high-quality data has been obtained for 100 fish,” says de Polavieja. The team has now used idtracker.ai, which is freely available, to extract a set of rules that explain zebrafish behavior in groups. They describe their results in another paper, which they have posted on bioarxiv.org and submitted for publication in a scientific journal.
As for possible applications, this software could allow for tracking persons, or to identify a given person in a crowd based on information about his or her physical appearance. “There is now a whole industry for this type of software,” says de Polavieja. “People are applying these [AI] techniques to develop other similar tracking tools. But before we proved we could do it in animals, it was hard to believe it was even remotely possible.”
[“source=techxplore”]
