At the 2016 Neural Information Processing Systems (NIPS) Conference in Barcelona, Cray announced its deep learning collaboration with Microsoft, and the Swiss National Supercomputing Centre (CSCS). They worked with Microsoft and CSCS to scale the Microsoft Cognitive Toolkit (formerly CNTK) on a Cray XC50 supercomputer at CSCS nicknamed “Piz Daint”. By accelerating the training process, instead of waiting weeks or months for results, data scientists can obtain results within hours or even minutes.
“Applying a supercomputing approach to optimize deep learning workloads represents a powerful breakthrough for training and evaluating deep learning algorithms at scale,” said Dr. Xuedong Huang, distinguished engineer, Microsoft AI and Research. “Our collaboration with Cray and CSCS has demonstrated how the Microsoft Cognitive Toolkit can be used to push the boundaries of deep learning.”
A team of experts from Cray, Microsoft, and CSCS have scaled the Microsoft Cognitive Toolkit to more than 1,000 NVIDIA® Tesla® P100 GPU accelerators on the Cray XC50 supercomputer at CSCS. The result of this deep learning collaboration opens the door for researchers to run larger, more complex, and multi-layered deep learning workloads at scale, harnessing the performance of a Cray supercomputer.
“Only Cray can bring the combination of supercomputing technologies, supercomputing best practices, and expertise in performance optimization to scale deep learning problems,” said Dr. Mark S. Staveley, Cray’s director of deep learning and machine learning. “We are working to unlock possibilities around new approaches and model sizes, turning the dreams and theories of scientists into something real that they can explore. Our collaboration with Microsoft and CSCS is a game changer for what can be accomplished using deep learning.”