Astronomy reduced to pixel archive science

A University of Oregon professor of physics frets that astronomy is drowning in data that threatens to reduce it to a “pixel archive science.” His solution is something right out of Star Trek.

Dr. Gregory Bothun made a presentation titled, “Big Data, Discovery, and a New Kind of Astronomy: Are We Prepared?” at the February meeting of the Rose City Astronomers at the Oregon Museum of Science and Industry in Portland. Bothun noted that efforts such as the Sloan Digital Sky Survey provide the stuff of discovery.

Prof. Gregory Bothun of the University of
Oregon spoke about astronomy’s challenges
with big data to a meeting Feb. 15
of the Rose City Astronomers in Portland.
Photo: Greg Scheiderer.

“The great thing about surveys is that they produce a catalog of calibrated sources which serve the community on a worldwide basis and involve more people in astronomy,” he said, adding that, for this reason, surveys should come before more targeted observations of individual objects.

“We’ve done it the other way around, mostly because of some kind of fetish with large-aperture glass,” Bothun said. “We have spent far more money building large telescopes than we have on building real, useful surveys that serve the community.”

Bothun pointed out that sometimes a big telescope will do a survey, such as Hubble’s ultra-deep-field work, and this leads to tremendous advances.

“Every time an instrument does a calibrated survey, science moves forward much more rapidly than some individual working with some piece of aperture doing a follow-up observation,” Bothun said.

A pipeline problem

While Sloan was useful, Bothun said, it also illuminated a problem. It took eight years to get the survey’s 20 terabytes of data into the hands of scientists.

“We’re not good at pipeline processing of survey data in a timely manner to feed a community,” Bothun said. “We shouldn’t have to wait eight years to go from acquired pixels to reduced data to analysis. It should just happen instantly. To the extent that it doesn’t is the extent that we’re going to shoot ourselves in the foot and turn astronomy into a science that archives pixels.”

The problem is about to get more challenging. A coalition of institutions is building the Large Synoptic Survey Telescope (LSST) in Chile. The LSST camera will have 3.2 billion pixels, and at 16 bits per pixel, each image it captures will be a whopping 6.4 gigabytes.

“Try to take a selfie of that and send it to your mom over wifi,” Bothun quipped. The challenge, though, is no laughing matter. It’s difficult to move that much data around, and it’s hard to look at it, too.

What you see is not all you get

“Every pixel in astronomy has a source in it. We need to see every pixel. We’re nowhere close to that,” Bothun said. A short-term answer may be visualization walls, commonly called viz walls. These are banks of high-definition monitors that scientists could use to display and manipulate vast amounts of data in one place. This would be perfect for looking at such large, high-resolution images. If you’re seeing a scaled-down version of a photo, Bothun said, the really interesting stuff may simply get averaged out. In addition, it’s better to look at a entire image at native resolution. This will take some training of our brains, but they’re capable.

“Your brain is a great visualizing machine. It’s a great parallel processing machine,” Bothun said. He said if it wasn’t we couldn’t drive on I-5. Think about how it would be if you tried to consciously track the speed and location of every other vehicle around you on the freeway. It’s not possible.
“Your brain does this automatically,” Bothun said. “It’s about time we we started to do data analysis in a forum that matches your brain’s algorithm.”

This would allow us “to take on extremely challenging problems, which is what leads to discovery in science,” he added.

Star Trek to the rescue

Viz walls may not be enough when it comes to the data from LSST. Its ten-year survey of the universe will generate a mind-boggling 60 petabytes of information. To meet the challenge, Bothun’s office is working on advanced visualization tools, a sort of three-dimensional viz cloud.

“It could be the holodeck,” Bothun said in reference to the virtual reality facility in Star Trek. “That’s how you should think of this.”

In this viz cloud trained humans could look at data in real time, and quickly sort out and discard what isn’t useful. After all, Bothun noted, the scientifically interesting data is usually just a tiny fraction of what is collected, and there’s no good reason to be pack rats with the rest.

“If all we’re going to do is take the raw data set and write it to disk, this is not a useful instrument,” he said of the LSST. “We have to do business differently if we want to optimize discovery.”

Big data is here, and visualization of this sort will help astronomers, but it will go beyond that; It can help in fields from finance and business to medicine, climate change, and counter-terrorism. To make effective use of the information available will require solutions to the pipeline and database challenges.

“All of this is absolutely vital for observational astronomy to continue to progress and continue to engage in discovery,” Bothun concluded.