At Spark Summit, Microsoft is announcing an extensive commitment for Apache Spark to power Microsoft’s big data and analytics offerings. Microsoft today announced the general availability of Apache Spark v1.6.1 for Azure HDInsight. Read more about it here. In addition, Microsoft is announcing the following,
- R Server for HDInsight in the cloud powered by Spark: R Server for HDInsight will be generally available in the summer making the Spark integration available both on-premises and in the cloud. This makes it easy to move code and projects to the cloud with a few clicks and within a few minutes without buying hardware or hiring specialized operations teams typically associated with big data infrastructure.
- R Server for Hadoop on-premises now powered by Spark: R Server for Hadoop will support both Microsoft R and native Spark execution frameworks available in June. Combining R Server with Spark gives users the ability to run R functions over thousands of Spark nodes letting you train your models on data 1000x larger and 100x faster than was possible with open source R and nearly 2x faster than Spark’s own MLLib.
- Power BI support for Spark Streaming: Spark support in Power BI is now expanded with new support for Spark Streaming scenarios. This allows you to publish real-time events from Spark Streaming directly into one of the fastest growing visualization tools in the market today.