End User
Government
Sector
Research
Aviation
Source Data
Images
Technology
Machine learning
Machine vision
A few months ago, Kim Griffith was gainfully employed, bouncing between two jobs she loved: directing dance shows on Royal Caribbean cruises and teaching hot yoga in Baltimore, where she lives when on land.
“That’s what I do in real life,” she says, laughing in a way one might call “rueful.”
She means, of course, the real life of the Before Times—not the surreal, bad-dream-like life of the pandemic. In this life, her jobs—which require tight-quarters ships and stifling rooms full of people breathing deeply—have mostly dried up (although she has been videostream-teaching some yoga classes next to a space heater, while wearing a sauna suit).
Griffith, thus underemployed, perked up right away when her friend Jonathan Obermaier, furloughed from a local brewery, pointed her to a gig on their Hampden neighborhood Facebook page: Astronomers from the Space Telescope Science Institute, just down the road, wanted to pay out-of-work service professionals like her to analyze images from Hubble, Earth’s most iconic telescope. The analyzers would pore over pretty pictures and determine how similar they were to each other for about $1,500, at a rate of around $20 per hour. It’s called the Hubble Image Similarity Project.
Game on, thought Griffith, who started the gig in mid-May.
Before this, she didn’t really know what went on at the Space Telescope Science Institute, and definitely didn’t feel connected to it, even though it was so nearby. But things have changed. “I ran right past it on my run yesterday,” she says. “I was like, oh, hey, I’m doing something for them!”
And they are doing something for her in return—paying her. That’s in contrast with most citizen science projects, which rely on volunteers. It was a very conscious choice for the institute’s astronomers, as cutting paychecks has to be. Joshua Peek and Rick White, the scientists who founded this project, saw so many of their neighbors suddenly without incomes that they wanted to help out. Their own careers and funding are more insulated from the immediate economics of Covid-19, and the astronomers saw an opportunity to create short-term work for others while helping themselves.
Peek—whose family helped organize their block’s pandemic mutual aid group—felt like the aphorism “Think globally; act locally” applied to this situation too. So, for cosmic mutual aid, he says, instead of turning to the whole internet—where most citizen science work is sourced—he turned to his neighbors, “rather than people who are anonymous around the world.”
Peek and White have been trying to make an algorithm—a convolutional neural network—that can determine whether this picture of Galaxy X looks like that picture of Galaxy Y, or whether this gassy wave resembles that nebulous wisp. Then, other astronomers could search Hubble’s 200,000-plus-image archive for shots that have similar visual qualities. “What people want to do is be able to look at something—point at it on the screen—and say, ‘Show me some other objects that look like this,’” says White.
Right now, scientists can search for an image in Hubble’s archival database using parameters like the object’s location in space, or which instrument took the data, or when it was photographed. But “dark dust cloud that has a tail on it?” says Peek. “That’s not a search you can do.”
Peek and White thought such a capability would be useful—kind of like a reverse image search on Google. But perfect matches aren’t necessary, because no two spots in space are going to look exactly alike. Instead, they want to find images that are similar to one another, sort of like putting a picture of your dad into Google and getting back a bunch of photos of other guys with bald spots and brown eyes.
The neural nets that Peek and White have been using were already “trained”—that is, they’d learned to recognize certain objects. For example, you can train a neural net to recognize cats by having it look at a bunch of cat pictures and figure out which visual elements give a cat its cat-ness. Then it can search for feline features in other images. Humans generally create the data sets that train the networks, curating a set of images that show lots of the objects that the net is learning to seek.
But a network schooled in images taken on this planet has a harder time parsing space photos. The network compares images of celestial stuff to terrestrial objects, trying to see how similar, say, a distant galaxy is to something you’d find in your house. “How much does it look like a rug? How much does it look like a hat? How much does it look like a cat?” says Peek.
Still, the researchers were able to work with this, because the software would spit out match percentages for each picture, and these were still a useful basis for comparison, even if they didn’t technically make much sense. “We would basically find that astronomical images that were, for example, 10 percent rug and 6 percent cat and 2 percent shark, and so forth, looked similar to other astronomical images that had those same percentages,” says White. “So that’s how we used the pretrained networks to ‘cluster’ astronomical images.”
But the astronomers still had no real way to do quality control—to verify that the program makes accurate estimates of which images look similar or different. “We don’t know how badly it’s doing or how much better it does when we change something,” says White.
“What we’ve needed for a while now is a test data set,” adds Peek. “This is a check you can run to see if it makes sense.” What you need, for such tasks, are humans and their not-silicon brains. And when astronomers need a lot of Homo sapiens-style seeing done, they often turn to the crowd. They make a citizen science project.
Chris Lintott, who cofounded the website Zooniverse, a clearinghouse of citizen science projects spanning a spectrum of topics, says that modern citizen science started in the 19th century. That’s when, for example, scientists started snapping up meteorology reports from volunteers, including statistics on things like rainfall, so they could understand how weather appeared from place to place and, eventually, predict how it would appear in the future. In astronomy, these public projects began earlier this century, when cosmic data sets became both huge and open. One of the first programs was 2006’s Stardust@Home, in which people scoured what Lintott calls “blurry images of dust grains” from NASA’s Stardust mission, looking for the few that might have come from outside the solar system. A year later, when Lintott and his astronomy team, based at the University of Oxford, suddenly had a million images of galaxies to classify, he thought, “if people were willing to look at dust grains for fun, surely they’d be willing to look at galaxies.”
His first project was called GalaxyZoo; volunteers did tasks like deciding whether a collection of stars grouped into an elliptical or spiral galaxy. Lintott later helped spin that site into Zooniverse, which tackles not only astronomy but also fields like climate studies, medicine, and even literature. Right now, you can look for gravitational waves, track Kenyan giraffes, find asteroids, or spot penguins, among dozens of other options. (“The projects that have pictures of penguins do much better,” Lintott notes.)
The best part of citizen science, in Lintott’s mind, is that because people have neurons, not just coded instructions, they’re more flexible thinkers. “People find things they weren’t told to notice,” he says. “People have found the unexpected or the things we thought were rare,” like the famous “Tabby’s Star”—the object that, for a while, people thought might host an alien-built megastructure that blocked some of its light. (That would be chill, but astronomers generally agree its weird fluctuations are due to dust and debris).
But the Zooniverse projects, and most citizen science programs, are unpaid. That can be complicated, as Max Liboiron pointed out in a presentation last March. Liboiron runs the Civic Laboratory for Environmental Action Research, which does plastic-pollution research with the help of community members. “One of the main reasons I don’t often identify as a practitioner of citizen science is because a lot—though not all—citizen science projects are based on a sacrifice economy,” read Liboiron’s presentation notes. “In a sacrifice economy, value continually accrues to people with more privilege (usually accredited scientists) and it’s usually drawn from folks with less privilege. Perhaps your citizen science projects gain value from retired white guys with castles and good pensions, but mine do not.” That’s part of why Liboiron pays participants.
Many for-profit crowd gigs, which people are turning to during the pandemic, also add much more value to the creators’ enterprise than the participants’ lives, as they tend to pay very little. A paper presented at 2018’s Association for Computing Machinery’s Human Factors in Computing Systems conference found that workers toiling through tasks on Amazon Mechanical Turk—where the Hubble Image Similarity Project is hosted—earn a median wage of around $2 per hour. By contrast, the Hubble Image Similarity Project will pay an average hourly wage of $20.
Astronomers have some financial fortune during this crisis: Their enterprise is insulated from the immediate economic downturn in a way that shuttered bars, curbside-only restaurants, and cruise ships are not. The federal science budget will likely suffer in the future—along with most other budgets on this planet. But grants that have already paid out are generally still on the table, salaries are still mostly being paid, astronomers can work from home, and for the most part, it was never their mandate to turn a profit. When Peek and White asked the higher-ups at the Space Telescope Science Institute for some seed money to pay their citizen scientists, they got a yes: This initial run will pay about 30 people $1,500 each for about two weeks’ worth of work. Peek is working on securing additional funding for the future.
“I think it’s an amazing project,” says Lintott. And, he adds, it mirrors citizen science history: Amateur meteorologists, for instance, didn’t give away free rainfall measurements forever. “[Scientists] started paying people for these records,” he says.
When Covid-19 started closing in, the Hubble astronomers’ universe simultaneously shrank and expanded, as if the principles of relativity had gotten confused. Collaborators across the planet were essentially as close by as the postdoc across the street. The postdoc across the street was also as far away as a Martian. Everything felt both global and hyperlocal. It made Peek and White want to connect to their own community, to join Hampden’s self-isolated islands into an archipelago. That’s why they only opened the Hubble Image Similarity Project up to workers in their area, rather than to the entire internet.
To find participants, Peek reached out to a local restaurant owner, who directed him to a guy who goes by the pseudonym Lou Catelli—Hampden’s unofficial mayor, according to The Baltimore Sun, which also chronicled the Hubble project’s debut. Catelli helped spread the word on Facebook. That’s where Obermaier, the furloughed brewery worker, found the gig. “I was super excited when I saw it,” he says, “to be able to look at pictures of space and to work from home too. I’ve been a little nervous to go back to work in a building.”
He calls this his “space job.” As part of the gig, Obermaier and his new colleagues will be going through 2,000 images, chosen by White, comparing each one to all the others. That’s about 2 million pairs. Their task is to evaluate which pictures resemble each other, comparing them side by side. “Which ones are most similar, which ones are sort of similar but not very close, and which ones are not very similar?” asks White.
That’s a little subjective, but that’s OK. “There’s no right or wrong answers,” Obermaier says—just your own assessment. The project involves autonomy and trust, providing a small sense of control in a world that seems to spin out in new ways every day.
Griffith, the underemployed director and yoga instructor, gets that. “It’s nice to have—even if it’s just a little project—to have a purpose,” she says. At first, she valued the work mostly as work, and a way to get some cash. But as she began to click through, its appeal grew deeper. The pictures, for instance, were beautiful. “That’s probably the artsy part of me,” she says.
Initially, she didn’t think she was bringing any special skills to the job, but she's come to see herself as qualified in an interesting way. Looking at whether this noctilucent arm resembles that one, or whether two star sets look like fraternal twins or thrice-removed cousins—it reminds her of her Before Times job, directing shows on cruise ships. “It’s similar to what my eye does onboard, looking for something different, trying to get the dancers to move in unison,” she says. But, for now, she’s trying to shape the universe into a sensible, searchable place, knowing there are no right or wrong answers, just the ones she finds herself.