Data visualization gets artificial intelligence boost with $5 million NSF grant

Researchers at University of Hawai’i at Mānoa, University of Illinois Chicago, and Virginia Tech were awarded a $5 million National Science Foundation grant to synergize two complementary technologies — large-scale data visualization and artificial intelligence — to create the Smart Amplified Group Environment (SAGE3) open-source software.

SAGE, soon to be on its third iteration as SAGE3, is the most widely used big-data visualization and collaboration software in the world.

SAGE and SAGE2 are software to enable data-rich collaboration on high-resolution display walls. SAGE2 moved SAGE into cloud computing and SAGE3 ushers in the inclusion of artificial intelligence.

Principal investigator Jason Leigh is a computer and information science professor at University of Hawai’i at Mānoa  and the inventor of SAGE. SAGE is software to enable teams of collaborators to work together with data in the form of data visualizations.

SAGE3’s novel interactive artificial intelligence methods will learn from human analytic activity and use the knowledge gained to assist people in synthesizing relevant data connections that lead to new hypotheses and findings, explained Chris North, a professor of computer science in the College of Engineering at Virginia Tech and a co-principal investigator.

Scientists analyzing their data in SAGE3 will collaborate with each other and with artificial intelligence through large interactive visualization spaces, such as multi-monitor workstations, tiled display walls, and virtual reality headsets.

Leigh said the applications for the SAGE3 technology are endless. “It will be tremendously useful to enable evidence-based response during natural disasters and events like the current COVID-19 pandemic,” Leigh said.

He created SAGE in collaboration with researchers from the University of Illinois Chicago 16 years ago. They’ve since further developed SAGE and SAGE2 with 15 years of research funding by the National Science Foundation.

“This new funding provides us with an opportunity to take SAGE to the next level — providing smart end-user services to amplify users’ productivity making commercial and open-source artificial intelligence solutions available that can autonomously and transparently analyze data while continually learning and improving through user interactions,” said Maxine Brown, director of the Electronic Visualization Laboratory at the University of Illinois Chicago. Brown has been involved with SAGE since its inception.

“SAGE3 also supports resource orchestration services that integrates these smart services with applications, workflows, and visualizations and collaboration services, and makes it easy for users to connect to local and remote display walls and computational systems and to securely run reproducible work models,” explained Luc Renambot, Electronic Visualization Laboratory associate research professor and co-architect with Leigh of the SAGE software since its inception.

“As a ‘co-pilot’ to individuals or teams who are trying to examine large datasets, SAGE3 will provide a discovery pathway from data sources to human insights and knowledge, which is currently a gap in the e-Science pipeline that the National Science Foundation has called the ‘missing middle,'” said Mahdi Belcaid, assistant professor at University of Hawai’i at Mānoa and the artificial intelligence co-principal investigator on the project.

According to the researchers, SAGE3 will make artificial intelligence technologies broadly accessible, not just a privilege for the technically savvy. North said the project will conduct user-centered design research studies to ensure usability of this novel combination of data visualization and artificial intelligence in SAGE3.

Through seed funds provided by Virginia Tech’s Institute for Creativity, Arts, and Technology and the Hawai‘i Data Science Institute, North had the opportunity to work side-by-side with Leigh in the Laboratory for Advanced Visualization & Applications at the University of Hawai’i during the spring 2020 semester. This in-person exchange, and numerous video-teleconferences with the Electronic Visualization Laboratory, was essential to helping plan and initiate the five-year SAGE3 collaboration, noted North. SAGE3 will serve the Institute’s missions of bringing together computer scientists, domain scientists, engineers, artists, and designers to tackle some of the world’s most complex challenges.

The user base goes well beyond scientists. Leigh said he sees SAGE3 helping teachers use and share artificial intelligence concepts to future generations of students. And for those who work in creative media, such as filmmakers and video game designers, Leigh envisions it being a highly visual tool to brainstorm new ideas and analyze all forms of digital media to see how they relate to each other, which ones were most popular and why.

— Written by Jenise L. Jacques, Virginia Tech