At DataWorks Summit, Microsoft announced the general availability of Azure HDInsight 3.6. HDInsight 3.6 comes with various updates to various open source components in Apache Hadoop & Spark eco-system, allowing Azure users to easily deploy them and run them reliably. HDInsight 3.6 has the latest Hortonworks Data Platform (HDP) 2.6 platform. Microsoft also announced the general availability of Apache Spark 2.1 with support for real-time streaming solutions with Spark integration to Azure Event Hubs and leveraging the structured streaming connector in Kafka for HDInsight.
It also supports Apache Hive 2.1 which enables ~2X faster ETL with robust SQL standard ACID merge support and many more improvements. The new Interactive Hive using LLAP (Long Lived and Process) enables 25x faster queries. The updated Hive views (Hive view 2.0) provides an easy to use GUI for developers to get started with Hadoop. Anyone can easily upload data to HDInsight, define tables, write queries and get insights from data faster using Hive views 2.0.
Learn more about this release here.