Unidata

EarthCube Webinar: Survey Analysis Via Visual Exploration (SuAVE)

The November EarthCube webinar features Ilya Zaslavsky from the San Diego Supercomputer Center on SuAVE (Survey Analysis via Visual Exploration), a new online tool for sharing and visually exploring surveys and image collections.

Free EarthCube Webinar: Doing Geoscience with EarthCube Tools

The EarthCube Tools Webinar series provides practical demonstrations of how NSF-funded EarthCube projects can help you to collect, access, share, and visualize geoscience data.  Each one-hour webinar showcases an EarthCube project followed by ample time for questions and conversation. For instructions on how to call in to this free event, click here.

Our next webinar is Friday, Oct. 7, at noon (2 pm EDT)

Forecast for big data: Mostly cloudy

May 31, 2016 | The rise of big data has big implications for the advancement of science. It also has big implications for the clogging of bandwidth. The growing deluge of geoscience data is in danger of maxing out the existing capacity to deliver that information to researchers. In response, scientific institutions are experimenting with storing data in the cloud, where researchers can readily get the relatively small portion of the data they actually need. Helping blaze the way is Unidata, which partnered with Amazon Web Services last year to make Next Generation Weather Radar (NEXRAD) data from the National Oceanic and Atmospheric Administration (NOAA) available in the cloud in near real time. The project is one of the ways Unidata, a community program of the University Corporation for Atmospheric Research (UCAR), is exploring what the future of data access may look like. "One of the roles we play at Unidata is to see where the information technology world is going and monitor the new technologies that can advance science," said Unidata Director Mohan Ramamurthy. "In the last 10 years, we've watched the cloud computing environment mature. It's become robust and reliable enough that it now makes sense for the scientific community to begin to adopt it." Inside an Amazon Web Services data center. (Photo courtesy Amazon.) The data deluge Since 1984, Unidata has been delivering geoscience data in near real time to researchers who want it. Today, Unidata also offers those scientists tools they can use to analyze and visualize the data. In 2008, Unidata's servers delivered 2.7 terabytes of data a day to 170 institutions. Just five years later, the program was providing 13 terabytes—or the equivalent of about 4.5 million digital photos—a day to 263 institutions. Today, Unidata is delivering about 33 terabytes of data a day. And the volume is only expected to grow.  For example, NOAA's new weather satellite, GOES-R (Geostationary Operational Environmental Satellite R-Series), is scheduled to launch in October. When GOES-R is up and running, it alone will produce a whopping 3.5 terabytes of data a day. "We've been pushing out data for 30-plus years here at Unidata," said Jeff Weber, who is heading up Unidata's collaboration with Amazon. "What we're finding now is that the volume of available data is just getting to be too large," We can't keep putting more and more data into the pipe and pushing it out—there are physical constraints." The physical constraints are not just on Unidata's side. Many universities and other institutions that rely on Unidata do not have the local bandwidth to handle a huge increase in the incoming stream of data. To address the problem, Unidata decided a few years ago to begin transitioning its services to the cloud—a network of servers hosted on the Internet that allow you to access and process data from anywhere. The vision is to create a future where scientists could go to the cloud, access the data they need, and then use cloud-based tools to process and analyze that data. At the end of their projects, scientists would download only their finished products: a map or graph, perhaps, or the results from a statistical analysis. "With cloud computing, you can bring all your science and the analytic tools you use to the data, rather than the old paradigm of bringing the data to your tools," Ramamurthy said.  'Navigating the waters' These advantages were part of the motivation behind the U.S. Department of Commerce's announcement last spring that NOAA would collaborate with Amazon, Google, IBM, Microsoft, and the Open Commons Consortium with the goal of "unleashing its vast resources of environmental data" using cloud computing. A NEXRAD data product available to researchers through Unidata. (Image courtesy Unidata.) Amazon Web Services was one of the first out of the gate on the NOAA Big Data Project, uploading the full archive of NEXRAD data to the cloud last summer. But to figure out how to continue to feed the archive with near real time observations and to help make sense of the data — how people might want to use it and what kinds of tools they would need — Amazon turned to Unidata. "It made a lot of sense for Unidata to partner with Amazon and vice versa," Ramamurthy said. "They wanted expertise in atmospheric science data. We wanted an opportunity to introduce cloud-based data services to our community and raise awareness about what it can do." The scientific community is perhaps more hesitant to rely on the cloud than other user groups. Datasets are the lifeblood of many research projects, and knowing that the data are stored locally offers a sense of security for many scientists, Ramamurthy said. Losing access to some data could nullify years of work. But the truth is that the data are likely more secure in the cloud than on a local hard drive, Ramamurthy said. "Mirroring" by multiple cloud servers means that data are always backed up. If the Amazon project, and the NOAA Big Data Project in general, are successful in winning scientists over, it could go a long way toward helping Unidata make its own transition to the cloud. Unidata will be studying and learning from the project – including how to make a business model that will work -- with an eye toward its own future. "We're navigating the waters to find out what works and what doesn't so we can report back to the National Science Foundation," Weber said. "We want to see how this paradigm shift might play out — if it makes sense, if it doesn't, or if it makes sense in a few ways but not others." Writer/contactLaura Snider, Senior Science Writer and Public Information Officer

UCAR to support EarthCube: Cyberinfrastructure will advance science

BOULDER – EarthCube, a landmark initiative to develop new technological and computational capabilities for geosciences research, will be supported by the University Corporation for Atmospheric Research (UCAR) under a new agreement with the National Science Foundation (NSF). Created by NSF in 2011, EarthCube aims to help researchers across the geosciences from meteorology to seismology better understand our planet in ways that can strengthen societal resilience to natural events. More than 2,500 EarthCube contributors – including scientists, educators, and information professionals – work together on the creation of a common cyberinfrastructure for researchers to collect, access, analyze, share, and visualize all forms of data and related resources. "EarthCube offers the promise to advance geoscience research by creating and delivering critical new capabilities,” said UCAR scientist Mohan Ramamurthy, principal investigator and project director of the new EarthCube office at UCAR. "This is a great opportunity for UCAR to leverage its successful track record in managing large scientific projects that advance our understanding of the planet," said Michael Thompson, interim UCAR president. "The EarthCube project offers the potential to significantly benefit society by helping scientists use the power of diverse big datasets to better understand and predict the natural events, from severe storms to solar disturbances, that affect all of us." EarthCube is designed to foster collaborations across the geosciences. The technology helps scientists in different disciplines better understand the far-reaching influences of natural events, such as how major storms like Sandy (above) affect coastal and inland flooding. This unique view of Sandy was generated with NCAR's VAPOR visualization software, based on detailed computer modeling. (©UCAR. Visualization by Alan Norton, NCAR, based on research by NCAR scientists Mel Shapiro and Thomas Galarneau. This image is freely available for media & nonprofit use. Click here for higher resolution.) UCAR will administer the day-to-day operations of EarthCube under the three-year, $2.8 million agreement with NSF. The EarthCube science support office, currently funded through an NSF grant to the Arizona Geological Survey in Tucson, Arizona, will move to UCAR's Boulder offices starting this month. EarthCube is designed to help researchers across the geosciences address the challenges of understanding and predicting the complexity of the Earth system, from the geology and topography to the water cycle, atmosphere, and space environment of the planet. This approach is critical for improved understanding of the environment and better safeguarding society. In order to better predict the potential effects of a landfalling hurricane on inland mudslides, for example, scientists from multiple disciplines, including meteorology, hydrology, geography, and geology, need a common platform to work together to collect observations, ingest them into advanced computer models of the Earth system, and analyze and interpret the resulting data. "The EarthCube Science Support Office will help us find and share the data geoscientists collect and use to answer critical science questions about the Earth," said Eva Zanzerkia, program director in NSF’s Division of Earth Sciences. Ramamurthy said UCAR is well positioned to help EarthCube meet its goals, since UCAR provides technological support to the geosciences community, including its 109 member universities. UCAR has been involved with EarthCube since NSF launched the initiative. "Currently researchers are spending an enormous amount of time on routine tasks because there is no data system, database, or data infrastructure where they can get all the information they need in some kind of a uniform way from a single interface," Ramamurthy said. "If EarthCube can facilitate the integration of data from multiple domains in a way that is easier and faster, and if there is interoperability in terms of standards for data to be input into a common environment, then integration becomes more easily possible." UCAR is a nonprofit consortium of more than 100 member colleges and universities focused on research and training in the atmospheric and related Earth system sciences. UCAR’s primary activity is managing the National Center for Atmospheric Research (NCAR) on behalf of NSF, NCAR’s sponsor. UCAR also oversees a variety of education and scientific support activities under the umbrella of the UCAR Community Programs, which will administer EarthCube.

Patricia: A global view

October 23, 2015 | As the most intense hurricane on record in the Western Hemisphere, Hurricane Patricia is taking direct aim at Mexico’s Pacific coast and expected to spawn impacts as far away as Texas and the central United States. This image, created with a specialized visualization system built at the University Corporation for Atmospheric Research, shows Patricia's strength as it approaches Mexico as well as the connections the storm is making with other atmospheric disturbances further away. It uses global satellite infrared measurements that produce thermal images of clouds. The reds and blues show colder, higher cloud tops that are associated with towering cumulus clouds and more powerful storms. With Patricia and its 200 mile-per-hour winds bearing down on them, residents across a broad region of southwestern Mexico are being warned to prepare for potentially catastrophic impacts. The storm also is beginning to interact with low pressure over the central United States. The image shows how Patricia's outer reaches flow into a band of clouds that stretches across the country's mid-section. This connection is expected to feed the low-pressure system, potentially bringing more rain to parts of the country. After Patricia moves inland tonight, the storm’s central core will be torn apart by Mexico’s mountains but its incredible moisture will continue on to Texas, where forecasters fear a major flooding event in the next two to three days. Patricia also appears to be interacting with Hurricane Olaf, a less-intense system clearly visible in this image in the central Pacific, far to the west of Patricia. "When we look at a hemispheric view like this, we can see teleconnections across thousands of miles," said Jeff Weber, a meteorologist with UCAR’s Unidata program who created this image. "Olaf and Patricia are in phase with each other, and their movements appear to be related. Patricia may be drawing in some of the moisture that Olaf is bringing up into the atmosphere." Although not directly linked to the hurricanes, the V-shaped cloud structure in the northern Pacific also signals stormy weather ahead. These are a pair of extratropical cyclones creating an amplified wave train. This will likely affect the U.S. west coast over the next seven to ten days, creating stormy and windy conditions, with cold air to the south and warm air to the north. "We’ve got a lot of activity in this part of the world right now," Weber said. "It's all related to El Niño and the unusually warm waters in the Pacific." Weber made the image using the Integrated Data Viewer. This specialized software, created by Unidata, helps scientists visualize and analyze atmospheric conditions. Writer/contactDavid Hosansky

Unidata Users Workshop

The Unidata Users Committee invites you to join Unidata staff, community members, and distinguished speakers this June for a workshop is to raise awareness of important new trends in geoscience technology, including cloud computing, data management, and the place of the Python language in geoscience computing infrastructure. The workshop is a chance for the academic community to share hands-on activities, course materials, and ideas for improving research and education.

Seminar on Ocean, Atmosphere, and Climate Modelling

The Unidata Seminar Series presents a talk by Dr. Richard P. Signell of the U.S. Geological Survey. The talk, titled Ocean, Atmosphere & Climate Model Assessment for Everyone, highlights the role of Unidata technologies in the U.S. Integrated Ocean Observing System (US-IOOS). Dr. Signell will be giving the talk after the presentation of Unidata's Russell L. DeSouza award, of which he is the 2014 recipient.

Seminar on GEOSS Common Infrastructure

The Unidata Seminar Series presents a talk by Dr. Stefano Nativi of the National Research Council of Italy (IIA-CNR, Florence Division). The talk, titled GEOSS Common Infrastructure and the Brokering Framework, will describe progress on using service brokering techniques in a federated multidisciplinary data environment as part of the Global Earth Observation System of Systems.

Geoscience data services to expand with NSF sponsorship

BOULDER—A program that provides unique data support to geoscientists worldwide will expand its services over the next five years, under a renewal of its grant with the National Science Foundation (NSF). Unidata, managed by the University Corporation for Atmospheric Research (UCAR), provides atmospheric science data to university departments in near real time. Its services encompass a wide range of cyberinfrastructure technologies that make geoscience data more useful and accessible for scientists and educators at more than 3,000 educational, government, and research institutions worldwide, including 700 U.S. universities. This 3-D depiction of the flow in and around 2008's Hurricane Gustav was created using Unidata's Integrated Data Viewer. Click on image to animate. (Visualization courtesy Unidata.) Under the new award with NSF of up to $25 million, Unidata will tap emerging technologies to better serve the geoscience community. This includes using cloud computing in ways that will enable researchers worldwide to access data and collaborate more effectively with colleagues at distant organizations and across scientific disciplines in order to tackle major scientific challenges. “We’re working to leverage the advantages of and advances in cloud-based computing paradigms that have emerged and become robust in recent years,” said Unidata director Mohan Ramamurthy. “The goal is to help advance scientific understanding of the physical world by better enabling scientists to extract knowledge from a deluge of observations and other data.” By gathering information into a cloud environment, the Unidata approach will also reduce the amount of data that must be transferred over computer networks and ease the computing requirements at universities and other research organizations. Unidata focuses on enabling scientists to better access, analyze, and integrate large amounts of data. It has also developed sophisticated tools to visualize information. Although Unidata’s core activities focus on serving scientists and educators in the atmospheric and related sciences, virtually every project that Unidata undertakes has a broader impact on the geosciences community and society at large. Unidata-developed cyberinfrastructure is in wide use among U.S. federal agencies, private industry, and non-governmental and international organizations, including the National Oceanic and Atmospheric Administration, the Department of Energy, Department of Defense, and NASA. More than 100,000 university students across the country are expected to use Unidata’s products and services, and hundreds of scholarly articles reference Unidata annually. Professors and other Unidata users said its services are critical for geoscience education and research. “Unidata provides the superhighway needed to connect my students to critical weather observations used for education and teaching in the atmospheric and related sciences,” said Jim Steenburgh, professor of atmospheric sciences at the University of Utah. At Millersville University, scientists and education experts in the Earth Sciences and Computer Science departments used a Unidata analysis and visualization tool to create a 3-D virtual immersion experience known as GEOpod. This allows the user to navigate a virtual probe within a computer simulation of the atmosphere, capturing temperature, humidity, and other parameters while using navigational aids and tracking capabilities. "With the help of Unidata, we can essentially bring students into a numerical weather model, helping them better understand the actual atmosphere as well as the modeling process,” said Richard Clark, chairman of the Earth Sciences Department at Millersville University. Unidata is a community data and software facility for the atmospheric and related sciences, established in 1984 by U.S. universities with sponsorship from NSF.

Students get a forecaster's view

May 18, 2012  •  University students and faculty soon will have the chance to peer at day-to-day weather through the same lens used by National Weather Service meteorologists. A new version of the NWS’s workhorse graphics software could begin making its way to campuses in fiscal year 2013 through UCAR’s Unidata program. Up to now, the Advanced Weather Interactive Processing System (AWIPS) has been restricted to NWS employees. With the creation of AWIPS II, NOAA and longtime contractor Raytheon are moving to an open-source, Java-based approach. The upgraded system is now being tested at a handful of NWS offices, including those in Omaha, Norman, and Boulder. Forecasters Mark Klein and Keith Brill (NOAA Hydrometeorological Prediction Center) evaluate AWIPS II features at a testbed facility in Camp Springs, Maryland. (Photo courtesy NOAA.) Unidata is a natural choice for bringing AWIPS II to universities. From its beginnings in the 1980s, Unidata has supported GEMPAK, a graphics program used heavily at NOAA’s National Centers for Environmental Prediction (NCEP) as part of its own version of AWIPS. Through Unidata, more than 300 universities have gained free access to GEMPAK, along with user support. The software has been a vital tool for both real-time weather analysis and research, says Unidata director Mohan Ramamurthy. “It’s been a terrific partnership,” he says. The need for GEMPAK will lessen as the NWS, NCEP, and Unidata all shift to AWIPS II. However, Unidata will continue to fully support GEMPAK for 18 months after the advent of AWIPS II. Existing tutorials and documentation will be kept online even after that point. Gerry Creager (Texas A&M University) and Kevin Tyle (University at Albany, State University of New York) are the first two beta testers to receive the AWIPS II software from Unidata. Testing at other campuses will ramp up later this year, with a goal of making evaluation copies of the software available to all interested Unidata members by the time AWIPS II is adopted nationally by the NWS in 2013. Several Unidata staff paid a visit to the Boulder NWS office in March for a first-hand look at the benefits of AWIPS II, led by meteorologist in charge Nezette Rydell and forecaster David Barjenbruch. The system’s new features include improved zooming and panning capabilities. "Speed, efficiency, and visualization quality are essential to us as working forecasters," Barjenbruch told the group. Forecasts and other products from NOAA’s Space Weather Prediction Center for a variety of solar phenomena, including auroras, will be accessible to university students and faculty through Unidata’s AWIPS II feed. (Wikimedia photo by Senior Airman Joshua Strang, U.S. Air Force.) Along with imagery of supercell thunderstorms and hurricanes, AWIPS II will provide Unidata members with a stream of solar data. NOAA’s nine national centers include the Space Weather Prediction Center (SWPC), whose products will become part of AWIPS II. NOAA’s Michelle Mainelli—who serves as a liaison to Unidata in the AWIPS II transition—pointed out some of the resulting benefits at an April 24 talk at the UCAR-facilitated Space Weather Workshop in Boulder. She noted that AWIPS II will allow new short-term forecasts of aurora potential to be harmonized with cloudiness outlooks, so users will know if those shimmering displays might actually be visible at a given spot should they develop. “Many meteorologists haven’t thought much about space weather products,” says Ramamurthy, who notes the potential for expanding Unidata’s user base to solar specialists. Thinking along the same lines is SWPC space weather services branch chief Brent Gordon. “I would like to see the space weather research and university communities be able to take advantage of the same programs the terrestrial weather community has been offered through Unidata,” he says. “Bringing space weather to AWIPS II is the first step in this process, and I’ll be excited to see how the community responds.”

Pages

Subscribe to Unidata