big data package

Key moments in NCAR supercomputing

 October 5, 2012 State-of-the-art computing and big-data processing have been part of NCAR from the center’s earliest years. Here's a snapshot of selected hardware advances. Micro-processorpeakteraflops  Memory(terabytes) Processors  Years inservice 1963 First supercomputer   CDC 3600 (Control Data)NCAR staff wrote the operating system  0.000001   0.000000032    1    2.5   1977 First vector-based system   C1 (CRAY-1A, Cray Computer)NCAR becomes Cray Computer's first customer for this vector-processing system  0.000160    0.000008    1    12   1988 First parallel processing system Capitol (Connection Machine 2,Thinking Machines)With over 8,000 processors, thismachine enabled the NCAR-University of Colorado Center for AppliedParallel Processing  0.00717    0.001    8,192    4.5   1999 Transition to clusteredshared-memory processors Blackforest (SP RS/6000, IBM)NCAR makes a major investment inconversion from vector-based toparallel processing systems   1.96     0.7    1,308   5.5   2008 Supercharged speed and efficiency Bluefire (Power 575 Hydro-Cluster, IBM)The first in a highly energy-efficientclass of water-cooled machines to beshipped anywherein the world arrives,with speed that more than triplesNCAR's computing capacity  77    12.2    4,096    4.5   2012 The next great leap   Yellowstone (iDataPlex, IBM)NCAR enters petascale computing withthis massively parallel machine  1,600    149.2    74,592    new     More supercomputing history at NCAR NCAR's Computational and Information Systems Lab provides a detailed timeline of our supercomputers.

First up: Accelerated scientific discovery

October 8, 2012 | First in the queue for Yellowstone is a set of 11 computing-intensive projects approved as part of the two-month-long Accelerated Scientific Discovery initiative. Chosen from applicants at NCAR and in the university community, the ASD projects will serve to inaugurate Yellowstone, carrying out large amounts of computing over a short period, while tackling major problems in Earth and atmospheric science. For more detail, see the formal descriptions on the ASD site for university-led projects and NCAR-led projects. TopicsOcean fronts and turbulence3-D mapping of Earth's upper layersInside cumulus cloudsBetter long-range forecastsHigh-resolution climate: local to globalTurbulent clouds in detail               Air pollution projections through 2055When turbulence and rotation meetExtending predictabilitySolar magnetism and electricity in spaceSea spray and the atmosphere Ocean fronts and turbulence        Baylor Fox-Kemper, University of Colorado Boulder, project leadArrest of frontogenesis in oceanic submesoscale turbulence Goal: Resolve the nature of the process that impedes sharpening of oceanic fronts (roughly 1–10 kilometers or 0.6–6 miles wide) in regions of turbulence. Such fronts are common over the globe’s ocean surface; they separate areas of differing temperature and salinity and influence circulation patterns and other key aspects of oceans and air-sea exchanges. 3-D mapping of Earth's upper layers        Thomas Jordan, University of Southern California, project leadCommunity computational platforms for developing three-dimensional models of Earth structure Goal: Image Earth’s upper layers by full 3-D tomography using two different approaches. The resulting improvements in regional and global models will better characterize the flow within Earth’s mantle, the evolution of plate-tectonic forces, and the potential seismic effects of earthquakes and nuclear explosions. Inside cumulus clouds      Lance Collins, Cornell University, project leadDirect numerical simulation of cumulus cloud core processes over larger volumes and for longer times Goal: Simulate particle-turbulence interactions in conditions that mimic cumulus cloud cores, over distances ranging from millimeters up to a few meters, for periods of about 20 minutes. These simulations will help improve models of cloud dynamics and could also benefit climate modeling. Better long-range weather forecasts      William Skamarock, NCAR, project leadGlobal cloud-permitting atmospheric simulations using MPASGoal: Test the performance of the Model for Prediction across Scales (MPAS) by producing 10-day forecasts for two periods. By varying the spacing between the centers of each hexagonal grid cell from 60 kilometers down to 3 km in five steps, the experiment will help determine where and when higher resolutions are most critical to forecast quality. High-resolution climate: local to global      R. Justin Small, NCAR, project leadMeso- to planetary-scale processes in a global ultra-high-resolution climate model Goal: Conduct and analyze simulations using the Community Earth System Model with roughly 100 times more grid points than commonly used. Among other outcomes, the study will shed light on how climate responds to the coupling of ocean and atmospheric fronts within a model and how features such as polynyas (holes within Arctic sea ice) affect climate. Turbulent clouds in detail      Andrzej Wyszogrodzki, NCAR
, project leadPetascale simulation of physics and dynamics of turbulent clouds Goal: This project will close the spatial gap between two numerical approaches to model cloud dynamics and cloud microphysics at scales ranging from the size of small cumulus clouds (about 1.0 mile) down to cloud droplets (about 0.0005 inches). The results will benefit weather and climate models at regional and global scales. Air pollution projections through 2055      Gabriele Pfister, NCAR
, project leadPrediction of North American air quality Goal: Perform high-resolution simulations with a nested regional climate model that includes interactions between chemistry and meteorology, in order to study possible changes in weather and air quality over North America between present and future time periods (2020–2030 and 2045–2055). The analysis will focus on summertime U.S. air pollution events. When turbulence and rotation meet      Annick Pouquet, NCAR, project leadRotation and stratification at unit Burger number in turbulent flows Goal: Examine the role of helicity (corkscrew-like motion) in turbulent fluids that are both stratified and rotating. Because this type of fluid behavior resembles key aspects of Earth’s atmosphere, the findings could help illuminate some of the complexities of flow in and around tornadoes, hurricanes, and other cyclones. Extending predictability      James Kinter (pictured) and Ben Cash, Center for Ocean-Land-Atmosphere Studies, project leadsTowards seamless high-resolution prediction at intraseasonal and longer timescales Goal: Explore the impact of increased atmospheric resolution on model fidelity and prediction skill in the operational ECMWF (European Centre for Medium-Range Weather Forecasts) coupled climate model. The results will help understand and quantify predictability in weather and climate for periods ranging from days to years. Solar magnetism and electricity in space      Project lead: Michael Shay, University of DelawareTurbulence in the heliosphere: The role of current sheets and magnetic reconnection Goal: Perform the first systematic study of how thin sheets of electric current generated by the Sun reconnect in the presence of turbulence. Understanding the process through which intense concentrations of energy dissipate during these reconnections is among the major challenges of solar physics. Sea spray and the atmosphere      David Richter, NCAR, project leadTurbulence modification in the spray-laden atmospheric marine boundary layer Goal: Examine the effect of sea spray suspended by turbulence above the ocean on the transfer of heat and momentum to the ocean surface. The results will help improve understanding of how effects related to sea spray could influence the development of hurricanes.  

Power meets efficiency

October 5, 2012 | Though it’s become a trendy concept, “big data” is a realm that NCAR scientists and their collaborators pioneered to help shape the nation’s infrastructure for weather prediction and related research. Largely because the atmosphere is so vast, and because tiny changes can have a major impact over time, atmospheric research involves enormous amounts of data. Over the last 40 years, hundreds of scientists at dozens of institutions have relied on NCAR’s computational muscle and massive data storage systems in order to make progress in forecasting weather, projecting future climate, and other key tasks. (See Key moments in NCAR supercomputing.) Located on the western fringe of Cheyenne, Wyoming, the NCAR-Wyoming Supercomputing Center officially opens in October 2012. (©UCAR. Photo by Carlye Calvin. This image is freely available for media & nonprofit use.) The latest chapter in the story of NCAR computing begins in the fall of 2012, as the NCAR-Wyoming Supercomputing Center opens for business. The center’s flagship hardware—an IBM system dubbed Yellowstone, in honor of Wyoming’s iconic national park—will rank among the speediest supercomputers in the world when it debuts. Yellowstone will feature 74,592 parallel processors, each carrying out calculations simultaneously when the supercomputer is running at its peak. “We are delighted that Yellowstone is here,” says UCAR president Tom Bogdan. “This supercomputer and the NWSC as a whole will provide a long-sought and much-needed boost to the capabilities of researchers in the atmospheric and Earth sciences.” "We've tried to design a balanced system to support data-intensive supercomputing." —Anke Kamrath, director of operations and services for NCAR's Computational and Information Systems Laboratory Years ago, it became clear that the computing center at NCAR’s Mesa Laboratory was reaching the limits of its ability to provide space, power, and cooling for ever-larger supercomputers. The center’s search for an alternative led to Cheyenne, where the NWSC now stands as the latest example of an expanding research corridor of tech-oriented development along the Front Range of the Rocky Mountains. The NWSC will advance a broad range of geoscience research, with potential benefits to society in everything from storm prediction to air quality monitoring and the assessment of energy and water resources. More results, more detail, more efficiency Yellowstone will provide almost 30 times the computing power of its NCAR predecessor, Bluefire. All else being equal, that might allow an experimental short-term weather forecast to be carried out in nine minutes versus three hours, or a climate projection to be produced in a week as opposed to half a year. The speed boost can also support more and better science in other ways, producing a greater number of simulations or increasing their complexity, depending on the goals of a given study. The NWSC offers more than sheer power, though. Both the building and the supercomputing inside have been designed with maximum efficiency in mind. Cheyenne’s windy High Plains climate provides a great deal of natural cooling; the NWSC takes advantage of it through a louvered air lock system. With the help of this and other energy-saving features, more than 90% of the power entering the NWSC goes directly to its computational mission—a notable improvement on the rates of 70% or lower found at many data centers. And even before its doors opened, the NWSC garnered a LEED Gold rating for Leadership in Energy and Environmental Design from the U.S. Green Building Council. More than 90% of the power entering the NWSC goes directly to its computational mission (IT LOAD), compared to about 66% for most data centers operating today. (Image courtesy NCAR/CISL.) The Yellowstone supercomputer is highly efficient in its own right. Compared to Bluefire, the amount of power required by Yellowstone per unit of computing will be reduced by more than 85%, thanks to a variety of improvements in computing hardware. The machine promises to serve as a powerful proving ground for computational scientists—at NCAR and at research universities and labs around the country and the world—working to maximize the usefulness and nimbleness of weather and climate models. Built with the future in mind While the NWSC’s grand opening is the climax to more than five years of planning and construction, it’s also just the beginning: the center is well equipped to serve scientists far into the future. Yellowstone occupies only part of the NWSC’s available computing spaces, or modules. Moreover, the facility is designed so that next-generation capacity can be added without disrupting current operations. Housed in a set of 100 interconnected cabinets, the Yellowstone system includes more than 70,000 processors, as well as high-performance bandwidth, memory, and visualization functions to transmit, store, and view the results. (©UCAR. Photos by Carlye Calvin.) Expanded scientific horizons will be part of the picture from the outset. The roster of initial projects encompasses a wide range of our environment’s fluid behavior, from the effects of sea spray on hurricanes to the motion of tectonic plates that leads to earthquakes. The NWSC is setting the stage for major progress in such critical areas as geophysical turbulence, which not only rattles aircraft but influences the behavior of countless processes all around us. With an ever-broadening range of disciplines represented, and new scientific partnerships already taking shape, the NWSC could go a long way toward helping us steward our planet’s resources, preserve its health, and avoid the natural hazards that so profoundly affect people and the economy. “The vision for Yellowstone parallels the principles that have guided the design of the NWSC,” says NCAR director Roger Wakimoto. “In both instances, we have taken an approach that maximizes the science we can do and the benefit of that science to society.” The NCAR-Wyoming Supercomputing Center in Cheyenne, Wyoming, takes advantage of naturally cool air and many other energy-saving features to maximize efficiency of both the building and its supercomputers. The building has also achieved LEED Gold certificaton for Leadership in Energy and Environmental Design from the U.S. Green Building Council.   Related Links NWSC Fact Sheet

A public-private success story

September 17, 2012 | A great deal of work had already taken place before workers turned over the first mound of soil on the prairie that now houses the NWSC. The public-private partnership that led to the NWSC’s creation emerged through a fortuitous mix of geographical, technological, institutional, and human variables, plus the ability to recognize opportunity and act on it. We asked two of the principals involved to share the story as they saw it first hand: Randy Bruns, head of the nonprofit economic development group Cheyenne LEADS, and Jeff Reaves, associate UCAR vice president for business services. A roster of contributing partners is here. Jeff Reaves Randy Bruns How did Cheyenne emerge as the location for the NWSC? Reaves: We looked at a number of options for meeting our need for a new data center: expanding on the NCAR mesa, leasing or remodeling an existing data center, retrofitting some other building. Then we decided to look at co-hosting it with a university along the Front Range, somewhere between Golden and Fort Collins, where there would be adequate transmission-level power, high-speed bandwidth fiber, and room for future growth. We were pretty far down the road in selecting a Front Range site when we got the call from Randy asking if we would entertain a proposal from Cheyenne. Bruns: We’d been trying to recruit companies involved in computer-centric technology. Our university and our community colleges are turning out tech-capable people, but for the most part, they leave the state for jobs. We don’t have a huge workforce in Wyoming. But we do have abundant power. And here in Cheyenne, we’re sitting on a lot of fiber that runs coast to coast but is largely untapped. Cheyenne exists because of the transcontinental railroad, and we also became an interstate corridor. And both rail and highway corridors tend to serve as corridors for data. The NWSC (turquoise pin) is located several miles west-northwest of downtown Cheyenne. (Map © 2012 Google.) Reaves: There were several proposed sites in Wyoming. The ones in Cheyenne had access to transmission-level power, which was important for us. The site we ended up choosing was about double the size of our minimum requirement for land, which offered a number of advantages in terms of mechanical and electrical infrastructure and potential expansion. Also, they were willing to deed the land [24 acres] to us, whereas some of the universities wanted to retain ownership, even with a no-cost lease. Then there were financial incentives: the land at no cost, plus infrastructure to the site. The state legislature offered $20 million in cash toward construction of the facility, and the university offered $1 million per year for the next 20 years for the ongoing operation of the facility. Cheyenne Light, Fuel & Power agreed to provide both primary and secondary power at no extra cost, so that if one substation goes down, they can switch us to another. They were also willing to provide 10% of our power needs with renewable energy from the nearby Happy Jack wind farm. Also, NSF encouraged us to consider Wyoming because it was an EPSCoR state. So there was this added benefit, which opened up opportunities to serve a broader community. [EPSCoR, the Experimental Program to Stimulate Competitive Research, helps direct funding to 31 U.S. states and territories that receive a disproportionately small fraction of NSF research funding.] Bruns: We involved the Wyoming Business Council [the state’s economic development arm], the governor’s office, and the University of Wyoming, all at the highest levels. The fortunate thing is that, in a state this size, we all know one another—the academic world, the private sector, and the government side, both state and local—so it’s fairly easy to reach across organizational boundaries. Essentially the entire legislature understood the significance of this. In a matter of hours and days, they were all on board. It wasn’t the largest construction project by any means that we’ve attracted here. It certainly wasn’t the largest employer. And yet in a very subtle but profound way, everybody recognized the potential and the power of this relationship. UW’s stake in NWSC computing Seven major University of Wyoming projects on the NWSC's Yellowstone supercomputer will address topics ranging from hydrology of the Colorado River Basin to planet formation from star debris to fluid dynamics of wind turbines. In addition, through EPSCoR (see above), the university recently received its largest-ever research grant, a $20 million, five-year award for a variety of water-related studies involving four UW colleges and 11 departments.  “UW’s computational researchers are working on projects that are of great importance to Wyoming, the U.S. and, in fact, the entire world,” says Bryan Shader, UW’s special assistant to the vice president of research and economic development. “All of this research is big science.” Have you already seen benefits in Wyoming from the NWSC? Bruns: With the prospect of this center being here, the university immediately began to make changes, with undergraduate and graduate offerings, and began expanding the relationship with NCAR in terms of personnel and shared staff. It also helped certify the case that Cheyenne in particular, and Wyoming in general, had substantial capabilities in the tech sector. It highlighted those not only for outsiders but in some respects even for people right here in Wyoming. Echostar Communications has expanded—they’re now up to more than 400 employees, all tech workers. Then in April 2012, Microsoft announced that they’re putting a huge data center complex here in Cheyenne. So the momentum is building. What about having a data center located nearly 100 miles from the rest of NCAR? Reaves: We recognized that technology had evolved to the point where you really didn’t have to have the machines collocated with the scientists. We’d already been serving the university community all over the world. Then it was a matter of how far away we could go from Boulder, which depended on broadband accessibility. One of the things that made Cheyenne doable was that BiSON already existed. [BiSON, the Bi-State Optical Network, connects the University of Wyoming to the Denver-based Front Range GigaPop, which in turn links research and higher education institutions across northern Colorado, including NCAR.] Looking to the future of big data and supercomputing, what is next for Wyoming, and what role might the NWSC play? Bruns: One of the things we’ve learned is that there’s a lot of commercial fiber here. We believe we’re in an excellent position for Microsoft and companies like them. We’re also very confident of our long-range ability to provide power to these centers. And NCAR has been willing to share much of what they learned with us, so we can talk with some authority about the unique advantages that our high, dry, cool environment provides for efficient operation of data centers. I think the real power of this center being here is an expansion of the Colorado technology corridor into Wyoming in a very tangible way. We’ve also opened the doors for expanded science at the University of Wyoming and elsewhere. At the end of the day, it’s about creating the capability to do better science, more science, faster science, in an area that’s critically important to the United States and arguably the world: understanding the environment that we live in. What are the keys to success for a public-private partnership like this one? Reaves: It’s crucial to know what your needs and requirements are and whether or not you can really satisfy them. You might be willing to make some tradeoffs, but at least you know the impacts of those tradeoffs. We knew what our then-current computing capacity was and where we needed to go in the next 10 to 20 years. We had a pretty good idea of what was required in terms of power, cooling, and computing to meet that demand. So our specs and requirements were very clearly spelled out. Another extremely important aspect is the partners themselves. I can’t emphasize enough what a great partnership we’ve developed with people in Wyoming—the legislature, the business community, the academic community. We listened and did what we could to meet their needs while satisfying our own. We developed a true partnership, up and down the line. Bruns: First of all, you have to get the right people and the right organizations to the table. In our case, there were key people at very significant points who grasped a vision for this project. [Former NCAR director] Tim Killeen was extraordinarily open to shifting directions at a time when NCAR’s selection process was starting to narrow. Governor [Dave] Freudenthal grasped the potential for this to expand the university, expand Wyoming’s presence in the tech sector, and tie our state even more solidly to science and research. The university president, Tom Buchanan, very quickly saw the big-picture, long-term implications of this center as the basis for an expanding relationship and for making significant positive changes at his institution. So there were a handful of people who intuitively understood the potential of this and committed their organizations to exploring how to make it happen. I think having people of vision like that was an amazing alignment of stars. This photo of the NWSC, looking northeast, was taken during final stages of building construction. It shows meeting areas (left) and the entrance foyer (right). (©UCAR. Photos by Carlye Calvin.)  

10 ways Yellowstone will make a difference

September 11, 2012 “The Yellowstone supercomputer will dramatically advance our understanding of Earth," says Al Kellie, director of NCAR’s Computational and Information Systems Laboratory (CISL). “Its computing capacity and speed will allow us to investigate a wide range of phenomena that affect our lives, with more detail than ever before.” Here are just 10 examples of how the new system coming online at the NCAR-Wyoming Supercomputing Center—and systems to come—are poised to tackle major challenges confronting our society.  Thunderstorms and tornadoes Scientists will be able to simulate these small but dangerous systems in remarkable detail, zooming in on the movement of winds, raindrops, and other features at different points and times within an individual storm. By learning more about the structure and evolution of severe weather, researchers will be able to help forecasters deliver more accurate and specific predictions, such as which locations within a county are most likely to experience a tornado within the next hour. Water availability Yellowstone will support global and regional climate modeling in much greater detail, helping to answer such questions as which regions will receive more or less precipitation in a warming world and which may face especially withering drought. Yellowstone’s high-resolution capabilities will also enable scientists to represent the heights of mountains more precisely within climate models, which will help project future snowpack in regions that supply water to vast agricultural tracts and millions of residents, such as California’s Sierra Nevada and the Colorado Rockies. Subsurface energy and carbon storage As the nation’s portfolio of energy options continues to grow, oil and gas remain key elements of the mix. Yellowstone can be used to model large oil and gas deposits in difficult-to-access regions several miles below Earth’s surface in greater detail. Subsurface models can also help identify areas that could be used as reservoirs to store carbon, keeping it out of the atmosphere and helping to reduce the effects of burning fossil fuels. Long-term forecasting Farmers, shipping companies, utilities, and other planners would benefit enormously from forecasts that accurately predict weather conditions a month in advance. Because large-scale oceanic and atmospheric patterns play such a major role at this time scale, scientists will rely on supercomputers such as Yellowstone to provide needed detail on the effects of these big patterns on future local weather events. Yellowstone’s size also allows for more ensembles—multiple runs of the same simulation, each with a small change in the initial conditions—that can shed important light on the skill of longer-term forecasts. Wildfires Wildfires create their own weather, interacting with nearby atmospheric conditions to drive the blazes through terrain in ways that are highly complex and difficult to simulate. By enabling more realistic modeling, Yellowstone can lead to improved predictions of fire patterns that may help both residents in threatened areas and firefighters trying to control the blaze. Arctic sea ice As the extent and thickness of sea ice diminishes, industries in the United States and elsewhere are eying the Arctic for potential shipping lanes and increased extraction of natural resources. Using supercomputers such as Yellowstone, scientists can work toward the development of seasonal forecasts of sea ice to help decision makers anticipate sea ice patterns months in advance. Climate modeling on Yellowstone also will help determine how soon the Arctic might experience ice-free conditions in summer, which could have major effects on local ecosystems and on regional weather and climate. Hurricanes While forecasters have made great strides in predicting the track of a hurricane several days in advance, it remains difficult to predict major changes in intensity or to identify which clusters of tropical thunderstorms will develop into hurricanes. The supercomputers at the NWSC will help answer those questions by enabling researchers to decipher the impacts of the complex processes involved, including sea surface temperatures, upper-level winds, regions of dry and moist air in the larger environment, and small-scale changes of temperature and humidity within a tropical cyclone. Air quality Pollutants such as particulate matter and ozone-filled smog threaten both human health and the environment, including crops. Atmospheric chemists plan to run simulations on Yellowstone that will help us better understand the regional and global evolution and movement of pollutants, potentially leading to forecasts of local air quality several days in advance. Renewable energy sources To harness more wind and solar energy, utilities must better anticipate shifts in local wind and cloud conditions. Yellowstone will help that effort by enabling scientists to simulate very detailed wind flows over various types of terrain, as well as cloud development and change for several types of clouds that have differing effects on solar radiation. Earthquakes As scientists gain new insights into the geometry of tectonic plates and their dynamic interactions, they need more powerful supercomputers to simulate faults worldwide and better understand various types of earthquakes. Such knowledge can ultimately help lead to earlier warnings of potentially deadline climate events.   Accelerating science Eleven initial experiments scheduled for fall 2012 will help put Yellowstone through its paces. See Accelerated scientific discovery for the details. All photos ©UCAR, except as indicated.
Subscribe to big data package