CISL

CISL Seminar Series: A Novel Recurrent Convolutional Neural Network for Ocean and Weather Forecasting

A Novel Recurrent Convolutional Neural Network for Ocean and Weather Forecasting
Robert Firth

Numerical weather prediction is a computationally expensive task that requires not only the numerical solution to a complex set of non-linear partial differential equations, but also the creation of a parameterization scheme to estimate sub-grid scale phenomenon.

UCAR collaboration with The Weather Company to improve weather forecasts worldwide

BOULDER, Colo. — The University Corporation for Atmospheric Research (UCAR) today announced a new collaboration with The Weather Company, an IBM business, to improve global weather forecasting. The partnership brings together cutting-edge computer modeling developed at the National Center for Atmospheric Research (NCAR) with The Weather Company's meteorological science and IBM's advanced compute equipment."This is a major public-private partnership that will advance weather prediction and generate significant benefits for businesses making critical decisions based on weather forecasts," said UCAR President Antonio J. Busalacchi. "We are gratified that taxpayer investments in the development of weather models are now helping U.S. industries compete in the global marketplace."UCAR, a nonprofit consortium of 110 universities focused on research and training in the atmospheric and related Earth system sciences, manages NCAR on behalf of the National Science Foundation.With the new agreement, The Weather Company will develop a global forecast model based on the Model for Prediction Across Scales (MPAS), an innovative software platform developed by NCAR and the Los Alamos National Laboratory.The Model for Prediction Across Scales (MPAS) enables forecasters to combine a global view of the atmosphere with a higher-resolution view of a particular region, such as North America. (@UCAR. This image is freely available for media & nonprofit use.)MPAS offers a unique way of simulating the global atmosphere while providing users with more flexibility when focusing on specific regions of interest. Unlike traditional three-dimensional models that calculate atmospheric conditions at multiple points within a block-shaped grid, it uses a hexagonal mesh resembling a honeycomb that can be stretched wide in some regions and compressed for higher resolution in others. This enables forecasters to simultaneously capture far-flung atmospheric conditions that can influence local weather, as well as small-scale features such as vertical wind shear that can affect thunderstorms and other severe weather.Drawing on the computational power of GPUs — graphics processing units — such as those being used in a powerful new generation of IBM supercomputers, and on the expertise of NCAR and The Weather Company, the new collaboration is designed to push the capabilities of MPAS to yield more accurate forecasts with longer lead times. The results of NCAR's work will be freely available to the meteorological community. Businesses, from airlines to retailers, as well as the general public, stand to benefit.Mary Glackin, head of weather science and operations for The Weather Company, said, "As strong advocates for science, we embrace strong public-private collaborations that understand the value science brings to society, such as our continued efforts with UCAR to advance atmospheric and computational sciences.""Thanks to research funded by the National Science Foundation and other federal agencies, society is on the cusp of a new era in weather prediction, with more precise short-range forecasts as well as longer-term forecasts of seasonal weather patterns," Busalacchi said. "These forecasts are important for public health and safety, as well as enabling companies to leverage economic opportunities in ways that were never possible before."About The Weather CompanyThe Weather Company, an IBM Business, helps people make informed decisions and take action in the face of weather. The company offers weather data and insights to millions of consumers, as well as thousands of marketers and businesses via Weather’s API, its business solutions division, and its own digital products from The Weather Channel (weather.com) and Weather Underground (wunderground.com).This webpage was last updated on July 5, 2017.

CISL Seminar Series - "Opportunities and Challenges: Diversifying Your Workforce" - Toni Collis

CISL Seminar Series

 Opportunities and Challenges: Diversifying Your Workforce
Toni Collis
Co-Founder of Women in HPC & Applications Consultant in HPC Research & Industry, EPCC,

Promoting diversity in high-performance computing

May 2, 2017 | Justin Moore was supporting his family of four with a job at an auto parts store while juggling classes at Salish Kootenai College, a Native American college in Montana, when he heard about a computing internship in 2014 at the National Center for Atmospheric Research (NCAR) in Boulder, Colo.The internship, which used a small, low-cost computer called Raspberry Pi to teach key concepts of high-performance computing, quickly paid off. Today, Moore works full-time as an IT network specialist at Energy Keepers Inc., which manages the hydroelectric plant on the Flathead Indian Reservation in Montana, while he continues to chip away at his degree."I believe the skills I obtained in the internship can be directly attributed to my success in my field," Moore said. "It also gave me the chance to network with some of the brightest minds in the country."Justin Moore turned a summer internship at NCAR into a full-time computer networking job at a hydroelectric plant on the Flathead Indian Reservation in Montana. (©UCAR. Photo by Carlye Calvin. This image is freely available for media & nonprofit use.)Since 2014, NCAR has been using Raspberry Pi as part of the SIParCS (Summer Internships in Parallel Computational Science) program to teach "hot" computing skills to small groups of university students, including one or two who are underrepresented in the sciences. In March, in efforts to reach more students, NCAR pivoted to an "externship" model, bringing the Raspberry Pi training to Miami Dade College faculty who can teach the skills to dozens of students at a time. “Raspberry Pi is a perfect platform for high-performance computing education because the credit-card sized mother boards can be linked together to mimic the parallel processing capabilities of a supercomputer and perform simplified geoscience applications,” said Rich Loft, director of technology development in NCAR's Computational and Information Systems Laboratory.A Raspberry Pi, which costs $35 or less, can run a full Linux operating system — the same system used by nearly all supercomputers, in more than 90 percent of smartphones, and in many other electronic devices.A Raspberry Pi kit used during the NCAR training at Miami Dade College. The Raspberry Pi circuit board is in the upper right-hand corner, connected to a blue cable. Components plug into a breadboard in the center of the picture (Photo courtesy Rich Loft, NCAR.)"It's inexpensive. It levels the playing field," said Loft, who led the training at Miami Dade College. "In my view it busts the digital divide."Loft noted that the previous internship approach wasn't reaching as many students as NCAR had hoped, partly because many students found it too difficult to relocate to Boulder during the summer. Miami Dade proved an ideal testbed for an externship model, since it's one of the country's largest universities, with eight campuses and more than 90,000 students, 70 percent of whom are Hispanic and 17 percent of whom are African American."This approach has scalability," Loft said, shortly after returning from the intensive two-day faculty workshop. "You can't scale up a program training one student at a time, even though it's very rewarding."The NCAR directorate, which supported the Miami Dade training through a diversity grant, hopes that an expanded program will reap even greater outcomes.A legacy of successThe Raspberry Pi internship approach already has yielded additional success stories, with students going on to graduate school and receiving prestigious scholarships.Lauren Patterson, for example, was a student at Hampton University in Virginia when she learned Raspberry Pi as a SIParCs intern at NCAR, also in 2014. "I loved that I was able to work hands-on and assemble the Raspberry Pi cluster myself," Patterson said.Lauren Patterson has received an Apple scholarship and will start a job at Google after completing her summer internship on Raspberry Pi at NCAR. (@UCAR. Photo by Carlye Calvin. This image is freely available for media & nonprofit use.) She said her experience led to an Apple internship under its scholars program, a $25,000 scholarship, and a software engineering job at Google starting next fall in New York City. Apple scholars participate in a 12-week internship at Apple headquarters in California, receive ongoing coaching and guidance, and serve as Apple ambassadors on their campuses.Gaston Seneza, a senior at Philander Smith College in Arkansas, said that before NCAR's SIParCS 2015 internship he had no practical knowledge of computers.He learned about Linux, sensors, programming, cloud storage, and scientific research, and now has a passion for computer science. "Raspberry Pi was a game-changer for me," he said.Gaston Seneza, who is from Rwanda, also won an Apple scholarship after his summer internship at NCAR. (@UCAR. Photo by Carlye Calvin. This image is freely available for media & nonprofit use.) The Rwandan native also was named an Apple scholar, and aspires to go into the field of artificial intelligence. "My dream is to see a world where intelligent machines work for us."Said Loft: "We're trying to get these kids on the hi-tech career onramp. You put machine learning or experience with parallel computing on your resume and you can get hired by Apple, Google, or Amazon – or get into graduate school. These are hot skills." Machine learning is a type of artificial intelligence in which a computer program can change or "learn" as it encounters new data.Moore, Patterson, and Seneza all praised the mentoring by Loft, an NCAR senior scientist, and Raghu Raj Prassana Kumar, an NCAR project scientist who has worked with the Raspberry Pi training project since its beginning."It's a lot of fun, and it's very rewarding to help these young people learn," Kumar said.Kumar is also known at NCAR for creative uses of Raspberry Pi, including connecting 12 of them to calculate Pi to a million digits on Pi Day in 2015. (It took longer than a day and one Raspberry Pi burned out from exertion, but it was successful.)Connecting learning to everyday lifeAt the recent Miami Dade workshop, Kumar and Loft, along with University of Wyoming Professor Suresh Muknahallipatna and three of his students, taught 20 Miami Dade faculty members how to set up and program simple projects with a Raspberry Pi. One group used sensors to measure things like temperature, pressure, and humidity, while another created a word frequency histogram from the complete works of William Shakespeare using a Raspberry Pi Hadoop cluster.Ana Guzman (far right), a Miami Dade College associate professor of electricial engineering, gets Raspberry Pi tips from Cena Miller, a University of Wyoming student. A group of Miami Dade faculty members were trained recently on using the low-cost computers for hands-on teaching by a team that included NCAR computer scientists and University of Wyoming students. (Photo courtesy Rich Loft, NCAR.) David Freer, a Miami Dade computer science professor, said he and his colleagues thought the workshop was terrific. "We worked with flame sensors that sent messages to users on their cell phones, along with other cool projects," he said.Djuradj Babich, director of Miami Dade's School of Engineering and Technology, said he hopes to "ride the excitement wave" from the training and develop an ongoing relationship with NCAR. Loft said NCAR also hopes to reach out to additional universities.Qiong Cheng, an assistant professor at Miami Dade, has since set up a Raspberry Pi in her office, complete with a motion detector. She said she will use the Raspberry Pi platform in her classes this fall, which are part of a new bachelor's program in data analytics.She likes the fact that Raspberry Pi, combined with sensors, is an inexpensive way to measure data in the real world, and thus connect learning to everyday life.  "Students are more interested in that," she said, adding that Raspberry Pi supports "our mission to reach underrepresented students — to motivate them, to inspire them, and to provide them with a hands-on learning experience."That's the kind of talk that excites Loft."We want to continue to collaborate to drive this home. Which means that Miami Dade is using this in their curriculum as the workhorse in their computer lab for students," he said. "That's what's going to make me very happy."Writer/Contact:Jeff Smith, Science Writer and Public Information Officer  

CISL Seminar Series- Sophie Hou

Data Stewardship and Engineering Team (DSET) - Building Community Informed and Driven Data Services

Sophie Hou
NCAR/CISL

Building roads to match tomorrow's weather

April 20, 2017 | When engineers design roads, bridges, and other types of transportation infrastructure, they need to account for local weather patterns. Extreme heat or freeze-thaw cycles can lead to ruts and cracks in roads, and heavy rains can overwhelm inadequate drainage systems, washing out bridges and flooding key transportation corridors.But how should engineers design new transportation projects, which may last for a half-century, if climate change will greatly alter weather patterns? The extent to which temperatures and precipitation may change in the future has become a major concern for the transportation industry.To address this issue, climate scientists at the National Center for Atmospheric Research (NCAR) are launching an innovative collaboration with civil and environmental engineers at Carnegie Mellon University and the RAND Corporation. They're using global and regional computer models, along with statistical techniques, to generate projections of future climate in ways that will be most helpful to infrastructure designers and planners, especially when it comes to drainage.A girl looks at a washed-out road in Louisville, Colorado, after damaging floods in 2013. Engineers are teaming up with climate scientists to design transportation infrastructure that can withstand shifting weather patterns. (Photo by David Hosansky.)The three-year project, funded by the National Science Foundation, will focus on Pittsburgh and several other cities across the country that will likely be affected in different ways by future climate."Our overriding goal is to enable transportation agencies to maximize the lifetime performance of new infrastructure while minimizing the costs to ensure its resilience to extreme weather events," said NCAR senior scientist Linda Mearns, the principal investigator on the project.Several recent studies led by NCAR scientists have underscored the extent to which climate change may affect future temperature and precipitation extremes in the United States. One concluded that, if emissions of greenhouse gases continue along a business-as-usual course, record daily high temperatures will outpace record daily lows by about 15 to 1 later in the century. A second study, also looking at emissions continuing on a business-as-usual path, concluded that incidents of extreme rainfall may increase by as much as five times in parts of the country.More detail means more uncertaintyTo conduct the new project, Mearns and her colleagues are working closely with local transportation officials and other stakeholders. Rather than analyzing the overall ways that climate is likely to change in the target cities, they're focusing on information that will be most useful to the design and construction of drainage infrastructure and other transportation systems."This requires very active engagement with stakeholders," Mearns said. "It's working together to determine what they want versus what we can actually provide and coming up with measures of uncertainty that are meaningful for them. This is in the realm of true coproduction of knowledge."For example, an engineer designing a drainage system along a highway might want an estimate of how much precipitation will fall in 15-minute increments. Although climate models do not provide such detailed information, Mearns and her colleagues can provide a partial answer by using a combination of techniques to produce projections of future precipitation every hour to several hours, as well as characterizing the uncertainty around those projections.A major challenge is that more detailed projections have greater uncertainty. While climate models consistently show that emissions of greenhouse gases lead to higher average global temperatures, the outlook is less clear for temperature and precipitation patterns by region. The type of information most needed by infrastructure planners and designers—projections of extreme temperatures and precipitation for specific locations and time periods—is even more uncertain. As a result, the study team will have to make compromises between the need for high-resolution data and the need for reliable data.Mearns said it's critical to give engineers a clear understanding of the uncertainty of a particular projection in order to prevent transportation projects from being based on a false sense of precision in climate projections. "The challenge," she said, "is developing sound engineering strategies for extremes under uncertainty."In addition to Mearns, the NCAR scientists working on the project include Seth McGinnis, Melissa Bukovsky, Rachel McCrary, and Doug Nychka. The Carnegie Mellon team is being led by Costa Samaras, who directs the school's Center for Engineering and Resilience for Climate Adaptation.“This project is a unique interdisciplinary collaboration that will advance the ways engineers and climate scientists will work together in the future,” said Samaras. “Infrastructure can last for many decades, and engineers need to design infrastructure to be resilient at the end of the infrastructure life span as well as in the beginning. Working with NCAR is critical to advancing the research needed to transform the way we design infrastructure in the United States."The benefit of different techniquesTo generate climate projections, Mearns and her colleagues will use two types of techniques to translate the coarse resolution of a global computer model, which typically simulates climate processes that are larger than about 100 miles, into the localized weather events that are of interest to transportation experts.One of these techniques, known as dynamical downscaling, will use a combination of three coarser-resolution global climate models and two higher-resolution regional models (including the NCAR-based Weather Research and Forecasting model, or WRF). This will enable the researchers to simulate the entire globe in coarse resolution while zooming in on selected regions with much higher resolution. This approach doesn't need as much supercomputing power as trying to simulate the entire globe in high resolution, although it still can be computationally intensive.The other technique, known as statistical downscaling, involves developing statistical relationships between large-scale atmospheric patterns and local temperatures and precipitation. This technique, which requires even less computing, can help scientists link conditions in a global model (such as a large area of low pressure) to a localized weather event (such as intermittent downpours).The combined approaches will enable the scientists to generate projections for at least every six hours, and possibly—with the use of additional specialized techniques—as frequently as every hour. Using both the dynamical and statistical approaches also will enable the team to better understand the uncertainties around future climate as well as evaluate the relative strengths of the techniques."Transportation systems are critical to the U.S. economy, and they represent some of the largest investments of our tax dollars," Mearns said. "We want to make sure that they'll hold up to a future climate."FunderNational Science FoundationPartnersCarnegie Mellon UniversityRAND CoroporationWriter/contactDavid Hosansky, Manager of Media Relations

Our People - Rory Kelly

April 19, 2017 | On weekdays in winter, while most people are still sleeping, Rory Kelly is driving to a ski area in the dark, slipping into his ski mountaineering gear, and training for two hours before heading back to Boulder for his job as a software engineer in NCAR's Computational and Information Systems Laboratory (CISL).

A tribute to Steve Worley

April 14, 2017 | At the 30-year mark of his career at NCAR, Steve Worley is announcing his plans to begin a phased retirement. From now into 2018, Steve will be transferring his responsibilities in CISL’s Data Support Section (DSS) to DSS staff and assisting with Doug Schuster’s transition to DSS Manager. Steve will reduce his working hours to 60 percent in July 2017, and in September 2017 he will conclude his third year as Chair of NCAR’s Data Stewardship Engineering Team (DSET).

CISL Seminar Series - Building and Using CESM2

Building and Using CESM2
Jean-François Lamarque
Senior Scientist, NCAR/ACOM & CGD

Pages

Subscribe to CISL