UNC Faculty Land $1 Million Grant to Develop Assessment Tool
May 27, 2020
The National Science Foundation awarded two University of Northern Colorado faculty a $1.077 million grant to improve teaching in college-level environmental science courses.
Assistant Professor Chelsie Romulo and Professor Steven Anderson will spend the next three years developing a machine-learning program that will assess students’ understanding of the connections among food-energy-water concepts in their classes. Their goal is to help instructors make evidence-based decisions on how to best test their students’ understanding of complex concepts.
Top: Chelsie Romulo giving a lecture to students in one of her previous Introduction to Environmental Studies (ENST 100) class.
“In all of education, our main objective is for students to learn, and there are lots of different ways to teach so students can learn, as well as tests to see if learning is happening — that’s the piece that’s difficult,” Romulo said. “In order to assess, we need tools for assessment; we need to test instructors and students together to see if what the instructor is doing is effective.”
Traditional testing methods, which form what are known as concept inventories, differ from multiple-choice questions, to short answer to essay, so Romulo and Anderson are hoping the computer program they’re creating will identify patterns of understanding based on testing outcomes.
Anderson co-developed a similar assessment to evaluate teaching and learning methods in geological sciences.
"Our work 15 years ago on the Geoscience Concept Inventory showed us that it was very
difficult and time-consuming to make a multiple-choice test that was both statistically
valid 
and reliable, but we were able to accomplish that by administering it to more than
4,000 students," Anderson said. "For our current project, there are additional challenges
in scoring more complex test questions, such as short answer questions, and that’s
where the Michigan State group comes in: they will apply artificial intelligence to
the grading of our new test and questions so that we can quickly grade these and move
toward making a concept inventory that will give us results that are research grade."
The UNC researchers will collaborate with 10 other higher-educational institutions across the country to gather an assortment of test questions given to students, as well as collecting the students’ answers.
“We’re trying to get a broad array of colleges from all over the nation, such as minority-serving and research institutions,” she said. “It’s a good idea to have a lot of representation.”
Above: The universities that are participating in Romulo and Anderson's
research for the concept inventory devleopment.
Over the course of the grant’s three years, Romulo hopes to create a larger concept inventory for the food-energy-water connections and involve more universities.
“There’s a huge need for this, which is why we received the funding,” Romulo said. “I’m excited to work with these other universities, especially with COVID-19 and teaching online. There are lots of opportunities for us to work with other universities and have that synergy of sharing activities and assignments across the board.”
Romulo discusses the Introduction to Environmental Studies class and the importance of Liberal Arts Core curriculum in the following video:
Amanda Manzanares, a UNC doctoral student, will assist Romulo and Anderson. Manzanares' master’s thesis analyzed students’ understanding on basic geology topics, similar to Romulo and Anderson’s upcoming research with the NSF grant. She’s starting the Educational Psychology Ph.D. program this fall with UNC Associate Professor Molly Jameson, Ph.D., and Professor Kevin Pugh, Ph.D., as her co-advisors.
“I am excited to work with Drs. Romulo and Anderson on this project," Manzanares said. "While completing my master’s research at UNC, I learned that it is important that educators have a better understanding of the initial knowledge state of their students. Instructors can then use this information to structure their classroom activities to help students build a strong foundational knowledge that they can use to learn more complicated concepts.”
About the Award
- Project Title: Improving Undergraduate Science, Technology, Engineering and Mathematics
- Grant award: $1 million over three years
- Funding Agency: National Science Foundation
- Researchers: Principal investigator Chelsie Romulo, Ph.D., an assistant professor in UNC’s Geography, GIS, and Sustainability Program; co-principal investigator Steve Anderson, Ph.D., a professor of Earth Sciences at UNC; co-principal investigator Shirley Vincent, Ph.D., a consultant and former director of Education Research and Evaluation for the National Council for Science and the Environment. Other research assistance is being provided by UNC doctoral students Amanda Manzanares Brian Tucker, Education Studies, and Lee Arakawa, Sport and Exercise Science.
- Of note: In collaboration with the Automated Assessment of Constructed Response Lab at Michigan State University, the researchers are also collaborating with the Science Education Resource Center at Carleton College in Minnesota to help evaluate the assessment tool and conduct outreach.
—Written by Katie Corder
