If you’ve never seen a Where’s Waldo picture, imagine a large drawing with hundreds of people and objects and the task to find just one character (Waldo) in that mess. The Germans, with their wonderful talent for naming things, call this type of book a Wimmelbilderbuch (teeming picture book). The illustrations offer hours of entertainment…which is great when you aren’t in a hurry. When the readings from your sensor network arrive as a Wimmelbilderbuch, it’s not nearly as fun. Enter extreme-scale analysis.
That’s a Bit ExtremeExtreme-scale computing (think one thousand million million operations per second or 1015) works with problem sizes that require the largest systems available in the world to solve. In extreme-scale analysis, we apply this power to the problem of understanding data, whether that data comes from computational simulations or large-scale measurements, such as enormous sensor arrays. Analysis and visualization (viz), however, have often happened on a smaller scale than the computations or measurements that generate the data to analyze, which means extreme-scale analysis/viz has a somewhat lower bar to be deemed “extreme.”
That disparity is currently plaguing decision makers. In the first Waldo book, he was surrounded by only 250 characters. By the time he got to his fourth book, 850 characters filled the page! Data visualization is trending in the same direction. Imagine trying to find Waldo without a nice big picture of him on the front page. Imagine trying to find trends or insights in a dataset without a clear understanding of what you want to find. Today, rather than treating analysis/viz as an afterthought, the wisest scientists and policy makers are investing in it in proportion to the scale of science they are doing.
Making Sense of the MessWhen I attend Supercomputing Conference 2018 (SC18), I will be participating on the program committee of the In Situ Infrastructures for Enabling Extreme-scale Analysis and Visualization (ISAV 2018) workshop. The ISAV workshop is for (1) analysis/viz experts to collaborate on how to best facilitate extreme scale, especially for actively running simulations (mostly because of I/O bottlenecks); and (2) scientists to convey to analysis/viz experts what they really need to analyze and understand from their simulations. If we’re successful in our collaboration and understanding objectives, future in situ analysis and visualization will offer cost savings (time and money), increased accuracy, and better use of resources. Ultimately, it will help the scientific HPC community translate the most challenging computational results into understandable information, which is what every decision maker needs to make decisions.
As my colleagues and I begin to apply machine learning and I/O enhancements to in situ visualization, we are going to empower decision makers with smarter and faster tools to analyze their data. So don’t fret the next time you’re tasked to find a red-and-white striped beanie in a sea of data, we’ve got you covered.
Want to try your hand at a supercomputing Wimmelbilderbuch? See if you can find Maxwell’s equations hidden on this chalkboard. Please no ISAV cheating.