S4
Supercomputers, and the computational power they unleash, are a necessary part of generating accurate weather forecasts – forecasts that rely on numerical weather prediction (NWP) models.
The Space Science and Engineering Center (SSEC) is home to one of the largest supercomputers on the University of Wisconsin-Madison campus. That computer just received a $1.5 million upgrade through a grant from NOAA’s National Environmental Satellite, Data, and Information Service and the Center for Satellite Applications and Research to meet the demands of these increasingly complex models.
Weather models such as the Global Forecast System (GFS) and the European Center for Medium Range Forecast’s global model (ECMWF) require powerful supercomputers like SSEC’s S4 to run simulations and prediction models. Credit: SSEC
The upgrades came as part of a regular five-year plan to keep up with advancements in high performance computing and the ever-increasing size of atmospheric datasets coming from sources like weather satellites GOES-16 and NOAA-20.
Known as the Satellite Simulations and Data Assimilation Studies computer, or S4, the supercomputer looks like a typical server sitting inside a climate-controlled room. But beneath the blinking lights is a system designed and built by SSEC computer scientists to run a series of complex and massive computer programs, including NWP models, global climate models, and Observing System Simulation Experiments.
NWP models such as the National Centers for Environmental Prediction’s (NCEP) Global Forecast System (GFS) and the European Center for Medium Range Forecasts’ (ECMWF) Integrated Forecasting System (IFS) are used around the world to predict weather days in advance, from local rainfall events to hurricane tracks. Consisting of thousands of lines of code, these models rely on large datasets of atmospheric measurements – pressure, wind, temperature and moisture – to form a representation, or simulation, of the physical processes of the atmosphere over time. The resulting simulations are used to generate weather forecasts.
SSEC computer scientist Scott Nolin stands next to the S4 system which is housed within a climate controlled room in the UW-Madison Atmospheric, Oceanic and Space Sciences Building. The S4 recently received a $1.5 million upgrade to help improve weather modeling and predictions. Credit: SSEC
“The models that run on this system have a lot of interconnected pieces, with all the different variables affecting their neighbors,” says Scott Nolin, SSEC computer scientist. “To handle the massive amounts of data, the system must have a high-throughput while also having low latency.”
In total, the new S4 consists of 80 computers, 2,500 processors and four petabytes of storage, helping to reduce the time required to run simulation models. With everything running together, Nolin says the S4 has a theoretical performance limit of 120 teraflops, or 120 trillion floating point operations per second. For comparison, an average consumer laptop is capable of producing anywhere from 200 to 400 gigaflops. To run one of these weather models, a consumer laptop would take more than a year to complete one simulation, if it doesn’t crash. Nolin says that although the system is fast comparatively, it is a relative term when describing machines like the S4.
“Researchers will be quick to fully utilize the new system’s hardware capabilities by running higher resolution models,” he says. “However we will also see noticeable improvements in the wait-times for researchers looking to run their experiments.”
Among them are SSEC researcher Agnes Lim who uses the S4 system to test new methods of optimizing and assimilating hyperspectral infrared data from the Cross-track Infrared Sounder (CrIS) on NOAA-20. She says running a GFS simulation on the previous SSEC system can take as long as a day to complete depending on the resolution used. In most instances, the higher the resolution, the more accurate a forecast can be, but the tradeoff is time.
“These upgrades give us a way to test new ideas more quickly and efficiently, but also to run simulations at higher resolutions and ultimately improve data assimilation techniques,” says Lim.
In the last five years, the resolution of imagery and data from GOES-16 and other weather satellites has increased significantly. Facilities like SSEC’s Satellite Data Services receive and archive more than eight terabytes of data per day from 13 polar orbiting and geostationary satellites. The increased resolution is a boon for improving weather models, but remains an ever-growing challenge to build computer systems capable of crunching that data in a timely manner.
Nolin and his group work closely with researchers to help them run their simulations on the specialized machine and the team plans to have the new system online and ready for users by early Spring 2019. The S4 will continue to serve as a testbed for new ideas and experiments for improving weather models like the latest FV3-GFS model.
This work is supported by NOAA National Environmental Satellite, Data, and Information Service and the Center for Satellite Application and Research (NESDIS STAR).