Northwestern’s Technological Innovations for Inclusive Learning and Teaching Lab uses Minecraft to reimagine outcomes for students with learning differences.
Computer science and learning sciences Prof. Marcelo Worsley started TIILT in 2016. He said he envisioned a space for students to design and conduct research on different learning technologies.
“We work on tilting the balance in favor of people who have been and continue to be marginalized,” Worsley said.
Worsley said he wanted it to be a place where community members could also test out the lab’s new technologies.
The lab is made up of undergraduate, graduate and doctorate students, along with other researchers and scholars. Worsley said many projects are student-led.
“It’s the people that make the lab what it is,” Worsley said. “And I think that feeds into the research and gives purpose to it.”
Weinberg sophomore Jannah Issa began working on a project for TIILT called Gesturecraft in summer 2023. She described Gesturecraft as an interface where children use their hands to control Minecraft instead of a keyboard and mouse.
According to Issa, Gesturecraft is used to gauge memory retention by comparing what kids remember in the game when they use traditional controls versus hand recognition.
“The whole point is seeing how we can apply this in the classroom,” Issa said. “Something as small as a teacher using more hands-on activities could help kids understand and learn better.”
Issa introduced Gesturecraft to several computer science-based camps and programs in 2023. She said she worked with kids who were first-generation immigrants who did not speak English. She added it was cool to see how the technology was useful despite the language barrier.
Gesturecraft is a continuation of a previous project called Multicraft.The project’s interface uses input methods like wooden blocks, eye tracking and verbalization in the physical world to translate actions into the Minecraft world.
Worsley said the inspiration for Multicraft and Gesturecraft came from seeing local middle schoolers with disabilities play Minecraft. Since they faced difficulties engaging with the game, he said he knew the game could become more accessible.
“The vision for these interfaces is to set it up so that a user can use whatever modalities they feel comfortable with to control the game,” Worsley said.
Worsley said he also wants to think about how data from interface users can be used to study learning.
Postdoctoral researcher Vishesh Kumar has been a part of TIILT for two and a half years. He said he hopes the lab’s technologies become useful for all types of learning.
“There are flexible ways to think about these materials, such that teachers can use them, out-of-school facilitators can use them, kids and families can use them and really just anyone.”
Kumar also said people often conceptualize ideas using their bodies, but a majority of schooling experiences do not embrace that idea. He said body movements deepen learning and added that Gesturecraft uses modalities to broaden this type of learning.
In the future, Kumar said he hopes TIILT will distribute their resources to the public so people can use their technologies to easily adapt to various learning environments.
“We are trying to create these embodied, gesture-based experiences to engage different kinds of kids, different sets of abilities and different sets of understandings,” Kumar said.
Email: [email protected]
Related Stories:
— SESP introduces undergraduate learning sciences major
— Shin: NU should encourage students of all disciplines to study computer science
