According to Andrej Kaparthy, there are four main factors holding back AI:Compute, Data, Algorithms, and Infrastructure. In this talk, we will show howwe attack the Data and Infrastructure challenges for Deep Learning.Specifically, we will show how we integrated Tensorflow with the world's mostscalable and human-friendly distribution of Hadoop, Hops. Hopsis a new European distribution of Hadoop with a distributed metadataarchitecture and 16X the performance of HDFS. Hops also includes a human-friendly UI, called Hopsworks, with support for the Apache Zeppelin Notebook.We will show how users can run tensorflow programs in Apache Zeppelin on hugedatasets in Hadoop. Moreover, we will show how Hopsworks makes discovering anddownloading huge datasets a piece of cake with peer-to-peer sharing ofdatasets between Hopsworks clusters. Within minutes, you can installHopsworks, discover curated important datasets and download them to train DeepNeural networks using Tensorflow. Hops is the first Hadoop distribution tosupport Tensorflow. Hops and Hopsworks are both Apache v2 licensed projectsand have been developed primarily at KTH Royal Institute of Technology andSICS Swedish ICT in Stockholm.
Hops and Hopsworks are both Apache v2 licensed projects and have beendeveloped primarily at KTH Royal Institute of Technology and SICS Swedish ICTin Stockholm. |