Microsoft Research's Project Zanzibar showcases the potential of bringing the physical and digital worlds together

Microsoft Research today revealed its latest project named Project Zanzibar which showcases the potential of bringing the physical and digital worlds together. Project Zanzibar is a sensing platform in the form of a flexible, portable mat that can also locate, sense and communicate with objects as well as sense a user’s touch. The mat uses capacitive sensing and NFC to enable multitouch and hover gesture input along with physical object manipulation and control. MSR team explored many different scenarios that this new platform will enable in the play and learning spaces.

Here’s how it can be used in games for kids:

Kids can tell stories by manipulating toys and props on the mat to control corresponding graphical avatars, previewing and capturing the result as a movie on the screen of a connected device. Camera and light source objects frame and light the mise-en-scène. Additional props can be used to change the setting of a virtual scene. For example, a palm tree can change the virtual environment to a tropical setting. Movie maker is thus a story authoring and storytelling prototype, envisioned for children aged 5 to 12. In this age group, socio-dramatic or pretend play are prevalent play patterns, with children experimenting with role playing and ascribing unique personalities to their toys.

Here’s how it can be used in learning activities:

The team took inspiration from traditional Montessori exercises for young children and extended them with digital content and feedback using Project Zanzibar. For example, the Letter Plates Montessori exercise provides feedback as words are formed and tactile letter-shapes are traced. Paper overlays on the Project Zanzibar mat do not interfere with functionality and provide additional context to the exercise.

The team envisions a rich family of educational apps including spelling and simple coding exercises using blocks, again with or without screens. By allowing kids to incorporate their tactile, natural world actions and senses when interfacing with computers, learning can conceivably be enhanced.

Microsoft Research will reveal more information on this project in a paper to be presented later this month at CHI 2018 in Montreal.

Read more about this project here.