NCAR partners with UC San Diego on visualization capabilities

New tools for understanding big data in the biological and geosciences

December 17, 2014

BOULDER - The National Center for Atmospheric Research (NCAR) is partnering with the University of California, San Diego (UC San Diego) to expand and enhance visualization capabilities in the bio- and geosciences through a grant from the National Science Foundation.

The collaboration builds on existing software capabilities developed at NCAR and UC San Diego, and it will combine them to produce new open-source tools for scientists to explore large data sets.

The project is known as WASP (Wavelet-enabled Progressive Data Access and Storage Protocol).

Current advances in digital imaging and numerical modeling technologies have enabled the creation of vast amounts of data. A challenge for many researchers is making sense out of these digital outputs. One way of dealing with extremely large data sets is known as progressive data access (PDA), which is the enabling technology behind consumer applications like Google Maps. In mapping applications, PDA reduces data volumes by only loading areas of interest, not the entirety of the map database, and allows the user to view these images in greater or lesser detail.

The problem is that similar tools are scarce in the biosciences, despite a need for analyzing data gathered from advanced imaging technologies such as MRI and CT scans. Also, as the size and complexity of the data increase, the computing resources commonly available for data analysis are over-subscribed. The geosciences encounter similar issues, with models for weather, climate, oceans and other Earth systems generating very large and complex data.

Given the similar nature of the challenge across various disciplines, researchers at NCAR and UC San Diego put their heads together to work on a solution, capitalizing on complementary work that was already ongoing at both institutions. Though the bio- and geosciences are very different scientific disciplines, with different data, the underlying forms and structure of the data are very similar, allowing for shared methods of dealing with the information.

Two software approaches

3D visualization of Hurricane Katrina
This high-resolution simulation of 2005's Hurricane Katrina as is it makes landfall on the Gulf Coast was visualized with VAPOR. (Image courtesy John Clyne, NCAR.)

An NCAR team has developed a software solution known as VAPOR (Visualization and Analysis Platform for Ocean, Atmosphere and Solar Researchers). VAPOR provides an interactive 3D-visualization environment that runs on most UNIX and Windows systems. At the heart of VAPOR is a progressive data access scheme based on mathematical linear transforms using wavelets. An NSF grant launched the development of the technology in 2003, and VAPOR is currently on its third major release, with over 6,000 users worldwide.

“VAPOR is an application specifically designed to facilitate researchers’ interaction with very large data sets, but while using only relatively modest computing resources,” said John Clyne, a software engineer and computer scientist who is the principal investigator for VAPOR in NCAR’s Computational and Information Systems Laboratory (CISL). “VAPOR is already widely used in the geosciences community, and with this award we will not only be able to expand and improve it for its current users, but also make it usable for the biosciences and bioimaging communities.”

At UC San Diego, the Center for Scientific Computation in Imaging has been developing a general analysis and visualization software toolkit for the bioimaging community, known as the Shape Analysis for Phenomics from 3D Imaging Data (SAPID) ToolKit (STK).

The goal of the SAPID project is to develop advanced computational methods for researchers in evolutionary biology to characterize subtle morphological variations from high-resolution 3D voxel-based digital imaging modalities. (A voxel is analogous to a pixel, but in three-dimensional space.)

A critical issue that arose during this project was the necessity of being able to handle very large datasets.  This new NSF award will address this important issue and thus provide innovative capabilities to the bioimaging community.

“This award is exciting because it allows us to take our existing software, combine and reuse it in new ways, and expand its capabilities to serve more broadly across scientific disciplines,” said Lawrence Frank, the principal investigator for the WASP award and the SAPID Project and a researcher at UC San Diego’s Institute of Engineering in Medicine at the Jacobs School.

Progressive data access applied to high-resolution CT scan of seadragon
This image demonstrates progressive data access applied to a high-resolution CT scan of a seadragon. The leftmost view of the seadragon head was constructed using a fraction of the data (1/100th), yet is morphologically identical to the rightmost image constructed using all of the available CT data. (Image courtesy UC San Diego.)

Both Frank and Clyne point out that the most important aspect of this collaboration is it will reuse existing NSF-funded software to provide a common framework benefitting both biological digital imaging and geosciences numerical modeling communities. It will have a profound impact for scientists working with large data sets.

“We’ll be able to provide better tools to the climate and weather science communities, while providing the first such tools for the biosciences community,” said Clyne. “It’s especially gratifying that NSF’s initial investments in both VAPOR and STK can be augmented with this award to produce an impactful and inter-disciplinary tool.”

 


*Media & nonprofit use of images: Except where otherwise indicated, media and nonprofit use permitted with credit as indicated above and compliance with UCAR's terms of use. Find more images in the NCAR|UCAR Multimedia & Image Gallery.

The University Corporation for Atmospheric Research manages the National Center for Atmospheric Research under sponsorship by the National Science Foundation. Any opinions, findings and conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.