Microsoft open sources its data compression algorithm

At the Open Compute Project (OCP) Global Summit 2019, Microsoft yesterday announced that it is open sourcing Project Zipline, a data compression algorithm that offers up to 2X high compression ratios versus the commonly used Zlib-L4 64KB model. Along with the data compression algorithm, Microsoft is also open sourcing hardware design specifications, and Verilog source code for register transfer language (RTL). Microsoft expects that Project Zipline will be used in various scenarios such as network data processing, smart SSDs, archival systems, cloud appliances, general purpose microprocessor, IoT, and edge devices.

Project Zipline is a cutting-edge compression technology optimized for a large variety of datasets, and our release of RTL allows hardware vendors to use the reference design to produce hardware chips to allow the highest compression, lowest cost, and lowest power out of the algorithm. It’s available to the OCP ecosystem, so they can contribute to it, and create further benefit for the entire ecosystem, including Azure and our customers.

Project Zipline will be available on GitHub soon.

Source: Microsoft