Lu Cheng receives NSF CAREER grant to improve AI accuracy, reliability
Lu Cheng receives NSF CAREER grant to improve AI accuracy, reliability
Assistant Professor Lu Cheng received a National Science Foundation CAREER award for her research into developing reliable and fair large language models, which are essential to responsible AI use. She aims to bridge the gap between conceptual AI principles and responsible AI practices.
“Language models, especially those large language models and foundation models, are very popular nowadays, and basically almost everywhere. People use them daily,” Cheng said. “But one question that we often ask is, can we trust these large language models?”
The large language models behind AI often make errors, such as returning incorrect information. Sometimes the models have hallucinations, making up nonsensical results. This inability of models to make fully confident predictions is known as uncertainty.
Cheng’s grant, “CAREER: Conformal Methods for Responsible Language Models,” seeks to develop responsible models with rigorous guarantees. This project will explore uncertainty’s decisive yet largely unrecognized role in enhancing language model reliability, using conformal prediction, which uses past experience to help determine confidence in new predictions.
Cheng will quantify the uncertainty of a large language model’s output, developing a statistically rigorous way to quantify how uncertain the model is and establish how much trust can be embedded in the output. While quantification is not a new concept, traditional methods to quantify uncertainty aren’t directly applicable to larger foundation models. Instead, Cheng will develop a series of new uncertainty methods built upon a very powerful statistical tool, conformal prediction.
As companies have raced to launch their AI models, they have been closed source, meaning they are essentially black boxes; you don’t know what the models do or how the models do it. It’s all proprietary. This makes uncertainty even more difficult to assess.
“If we are able to estimate the uncertainty, we can find out why large models behave the way they do,” Cheng said. “The next question is, so what can we do to mitigate this uncertainty?”
Once methods to mitigate uncertainties are in place, Cheng hopes the outputs of these large models can be more reliable. Uncertainty and fairness are related, and uncertainty can be a very important signal to identify biases and improve the fairness of machine learning models.
In a previous study, Cheng found that for the same set of fairness metrics and for the same input data, the fairness metrics can be very different each time you run the algorithm. Improving these metrics will lead to fairer algorithms.
Today, more open-source models such as DeepSeek are being developed, where users can access and modify source code freely. This approach helps build trust and encourages collaboration. These models have proved to be more reliable, and Cheng hopes that companies will use the same techniques to improve their closed models, to make them more reliable and fair.
“We know there are biases in the algorithm. And we also know there are biases in the data, which can be further like propagated into the algorithm side,” Cheng said. “As an AI researcher, I think it’s important to recognize or acknowledge that there’s an issue, and mitigate these for when algorithms are applied in the real world.”