Computer vision model gets entomologists, bumblebee buffs all abuzz

Breaking a bottleneck in bee research

By Pat Melgares

A Kansas State University project that harnesses the power of artificial intelligence and the curiosity of nature lovers may just be the bee’s knees for saving one of the world’s most efficient pollinators.

K-State entomologist Brian Spiesman is leading the project, called BeeMachine, which uses computer vision to identify North American bumblebee species with images submitted from public databases and ordinary citizens.

As of summer 2021, the database has more than 125,000 images and feeds a computer algorithm that Spiesman hopes will break a bottleneck in getting important research data to conservationists and others who put that knowledge to use.

“The way research currently works is you go out into the field, capture a lot of bees and bring them back to the lab to identify them,” said Spiesman, research assistant professor of entomology in the College of Agriculture.

Lab workers then identify the bees under a microscope one at a time. Spiesman said researchers often collect bees all summer, identify them during the winter, then analyze and write data for publication — a process that can sometimes take years.

“BeeMachine, and what will likely be future iterations of the use of artificial intelligence for insect identification, will transform the field of bee and insect conservation more broadly,” said Claudio Gratton, professor of entomology at the University of Wisconsin-Madison, who has used the program with his students to confirm the identities of species. “The possibility of having species-level identifications from numerous places will finally give us the ability to accurately and rapidly monitor bee occurrence, population sizes and how they fluctuate over time.”

• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 

Photos courtesy of BeeMachine.

A citizen science approach

There are more than 20,000 native bee species — including bumblebees — around the world. It is commonly believed among scientists that the population of bumblebees in North America is declining. One species — Bombus pensylvanicus — once was widely spread across the eastern U.S., but its range has greatly contracted, according to Spiesman. While the species still exists in Kansas, it has not been seen in many places around the U.S., including Pennsylvania.

“Bumblebees are great pollinators, especially for things like tomatoes and blueberries and other crops that require buzz pollination,” Spiesman said. “They vibrate very rapidly and that helps them more effectively deposit pollen on the flowers that require that.”

Rich Hatfield, a senior conservation biologist with the Xerces Society for Invertebrate Conservation, has contributed thousands of images to BeeMachine from his own site, Bumble Bee Watch, which gathers images submitted by citizens and has them reviewed by an expert taxonomist.

That process — best known as citizen science — is valuable to researchers because it connects them to an endless number of interested citizens who are contributing to scientific knowledge.

But current efforts to engage the public in bee research fall short because of the amount of time it takes to accept samples and then wait for an expert taxonomist to identify the species and provide feedback. Often, people submitting the samples simply want quick answers on what they have seen.

Spiesman said BeeMachine shows promise for bridging that gap. One idea is that BeeMachine’s computer vision system could be integrated with Bumble Bee Watch and similar sites to provide quicker feedback to those submitting pictures.

The power of artificial intelligence

K-State computer scientist William Hsu with the Center for Artificial Intelligence and Data Science is developing the computer vision that is key to making quicker identification a reality.

“Computer vision is the scientific field of using machines to understand images and videos, meaning that we have a picture or sequence of pictures captured on a camera, phone or other device, and we might want to name all the objects or people, mark out the borders of objects, label and track living organisms, or even explain events or actions that are happening,” said Hsu, professor of computer science in the Carl R. Ice College of Engineering.

As computer programs analyze images, Hsu said, those programs store the information for use in identifying similar images sometime in the future. Initially, images may be identified by a human being, and that information is stored alongside the image.

“The learning system we use is a deep neural network, a mathematical model resembling some parts of the visual processing system of the human brain,” Hsu said. “This model is modified to make it better at predicting the labels that people already put on images. This is called training and the result is a trained model that we can use on bees we haven’t seen before.”

Future buzz

While 120,000 images currently in the BeeMachine database seems like a lot, it’s still not enough. The more that the program is fed, the better it gets at identifying the correct species.

“An expert taxonomist usually has to look at two or three images of the same individual bee that are taken at different angles,” Spiesman said. “In the artificial intelligence model, we are realizing that it needs that, too. If it has a really good look at it, it will give us a good result. If you want to make sure you have the right answer, you have to load in two or three or four images of the same bee.”

In the year since BeeMachine debuted, Spiesman said the K-State researchers have increased the number of species identified in the database from 36 to 41. There are an estimated 45 species of bumblebees in North America. Eventually, BeeMachine could be used to identify bumblebees around the world, and then — much more long term — any of the 20,000 native bee species found across the globe.

Getting there means advancing the technology even further. Already, Hsu said researchers are looking at other ways to gather images, such as using thermal sensors, drone-mounted cameras, wearable or implantable sensors and other devices. Spiesman said something as simple as mounting motion-sensitive cameras in a field to track bees as they pollinate flowers also could be valuable — similar to remote wildlife cameras.

A computer vision model for farmers

Kansas State University entomologist Brian McCornack half-jokingly says he’s counted close to a billion aphids — a minute, sap-sucking bug that feeds on plants — in his career of 20-plus years, and he’s just plain tired of doing it.

So, he’s pretty happy that computers may soon be able to do it for him. Computer vision modeling also could be used to aid farmers in their crop fields.

“Counting insects, especially across large geographic regions, is time-consuming and sometimes impractical,” said McCornack, professor and head of the entomology department. “Insects are quite mobile, and populations can change drastically from one week to the next. Farmers simply can’t walk every single acre of a field as frequently as needed to make management decisions. Machines can reduce the workload while improving the reliability in pest or natural enemy density estimates, which are the basis of all treatment thresholds. The exciting part is that we are not that far away from seeing these tools in the hands of farmers and consultants.”

K-State entomologists, agronomists and biological engineers are developing technologies, such as drones, to identify defoliated soybean leaves or to identify lady beetles on sorghum.

But automation does not mean removing people from production agriculture, McCornack said.

“It does mean that we need to be sure that our students and stakeholders know that these tools have a place in decision-making, and that we teach them how to use this technology to solve some of the grand challenges facing agriculture,” McCornack said.