Microsoft and Liebherr working together on next gen smart refrigerators

Last night, we reported about LG Smart Instaview Signature fridge with full Windows 10. Today, Microsoft is announcing their collaboration with Liebherr household appliances division in the development of Liebherr’s new generation of the SmartDeviceBox.

With the SmartDevice capability, future Liebherr refrigerators will allow you to monitor stored groceries using internal cameras and object recognition technology. It not only captures images for viewing but also recognizes individual food items inside the refrigerator. The captured info flows automatically into an inventory list, which lets the customer see quickly and clearly what is in the refrigerator from anywhere. Additional groceries can be added to a shopping list that customers can access when on the move using an app for iOS, Android or Windows devices using the SmartDeviceBox voice module.

Microsoft is helping them with computer vision capability for developing the above described technology. Microsoft Cognitive Services Computer Vision API can annotate new images with a wide variety of generic category labels and object tags. Liebherr will be using them to develop their solution.

Using the deep learning algorithms contained within CNTK, Microsoft data scientists worked with Liebherr to build a new image processing system to detect specific food products present inside a Liebherr refrigerator. Starting with a general purpose computer vision model learned from millions of generic images, the Microsoft system can learn to recognize new types of objects – milk cartons, ketchup bottles, pickle jars and much more – from example images. When a new image from inside a refrigerator is provided to the newly learned model, it can detect the presence of the objects it has seen before during training.

Even though the current system they have is just a prototype, Microsoft and Liebherr are actively working together to bring these kind of experiences to next generation smart appliances.