High Performance Computing (HPC) may not be a household term, but the computing power it unlocks is the reason I love my job. I get to apply my skills and passion in a way that directly affects people’s lives. As the Environmental Quality Modeling Technical Lead for the DoD’s User Productivity Enhancement, Technology Transfer, and Training (PETTT) program, I help engineers, primarily at the Army Corps of Engineers, improve the wellbeing of humanity and the environment.
HPC in ActionIn 2010, Pakistan experienced historic levels of flooding. The Army Corps of Engineers used its premiere shallow-water simulation software, Adaptive Hydraulics Suite (AdH), to assist the U.S. military in aiding those displaced by the floods. The software calculated what areas of the country would be dry and approximately how long it would take for flood waters to recede1. The suite is also used to study environmental impacts of storm surges, dam building, levee breaches, and ship-channel dredging. The Pakistan information was time sensitive, so it was critical that AdH could run efficiently in a massively parallel environment (using lots of processers simultaneously).
My RoleI help engineers run their models, like AdH, on DoD HPC systems. I am also part of the development team for AdH. Together with Army engineers, I plan what features need to be added and the best strategy for improving AdH. I then ensure that any added features run efficiently in parallel on DoD HPC systems.
Commanding Complexity with CouplingAs powerful as AdH is, it does not simulate the full complexity of a hurricane. What happens when the information required can’t be generated by a single piece of software? AdH is good, but sometimes you need more physics to fully simulate the complexity of a hurricane. The best way to do this is by coupling analysis codes. As the environmental software and HPC, itself, become more advanced, more information is required from simulations. At the same time, HPC systems are becoming more expensive to run, making efficiency ever more critical.
One way that simulations are becoming more effective is through model coupling2. Two phrases come to mind when I think of model coupling:
- Jack of all trades, master of none: No single piece of software can simulate everything. Each piece of software running on DoD HPC systems is written by a team of experts in that field. It makes sense that different software is appropriate for modeling different physical processes.
- The whole is greater than the sum of its parts: Models must communicate with each other. When models effectively couple to each other, they produce information that could never be obtained by running the models separately. In the beginning of model coupling, the models would write files to disk which would then be read by another model, creating an efficiency bottleneck. This might work for some systems, but what about modelling the coastal effects of a hurricane?
One Size Does Not Fit AllSince no single model can predict wind models, surface waves, tidal circulation and storm surge, the Army created the Coastal Storm Modeling and Database System (CSTORM). CSTORM couples several different environmental models in order to make a complete picture of a hurricane’s coastal impact. A wind model communicates with a long-range wave model that then feeds forces into a tightly coupled system consisting of a tidal-circulation model, a short-term wave model, and a surge/shallow-water model (e.g., AdH). This is all happening at the same time on an HPC system. CSTORM tightly controls performance load across the processors and data management between the models. It is used every time a system threatens coastal areas. Many CSTORM simulations are run as the system approaches land to advise coastal authorities. It has even correctly predicted that the storm surge would permanently cut a barrier island in half (Ship Island, Gulf of Mexico)3.
Ship Island before (top) and after (below) Hurricane Katrina. Storm surge submerged the island and severely widened the breach between East and West Ship Islands. Source: USGS
A Front Row SeatI am very happy that my position with Engility on the PETTT program allows me to work on software systems like AdH and CSTORM. I get to work on state-of-the-art HPC systems and implement new applied math and software techniques. This keeps me constantly learning and challenged as a scientist. All the while, my efforts are helping to keep people safe.
- Tate, J., Savant, G., and McVan, D., 2012. Rapid Response Numerical Modeling of the 2010 Pakistan Flooding. Leadership and Management Engineering, Vol. 12(4), 315–323.
- Lucas Pettey, Corey Trahan, Gajanan Choudhary, Clint Dawson and Gaurav Savant. 201. Separating Software and Science in the Adaptive Hydraulics Suite. Proceedings from the Computational Methods for Water Resources Saint-Malo France June 3-7, 2018.
- Ty V. Wamsley, Raymond S. Chapman, Mark B. Gravens, Alison S. Grzegorzewski, Bradley D. Johnson, David B. King, Rusty L. Permenter, Michael W. Tubman, Barry Bunch, Dorothy Tillman, Elizabeth S. Godsey 2013. Mississippi Coastal Improvements Program; Evaluation of Barrier Island Restoration Efforts USACE ERDC Final Technical Report September 2013.