We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Industrial Machine Learning

00:00

Formal Metadata

Title
Industrial Machine Learning
Subtitle
Horizontally scalable Data Pipelines with Airflow
Alternative Title
Industrial Machine Learning Pipelines with Python & Airflow
Title of Series
Number of Parts
132
Author
License
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
Industrial Machine Learning This talk will provide key insights on the learnings I have obtained throughout my career building & deploying machine learning systems in critical environments across several sectors. I will provide a deep dive on how to build scalable and distributed machine learning data pipelines using Airflow with a Celery backend. I will also compare Airflow with other technologies available out there and how it differentiates, such as Luigi, Chronos, Pinball, etc. If you attend the talk, you will obtain an understanding on the solid fundamentals of Airflow, together with its caveats and walk-arounds for more complex use-cases. As we proceed with the examples, I will cover the challenges that you will run into when scaling Machine Learning systems, and how Airflow can be used to address these using a manager-worker-queue architecture for distributed processing with Celery. By the end of this talk you will have the knowledge required to build your own industry-ready machine learning pipelines to process data at scale, and I will provide further reading resources so people are able to implement the knowledge obtained almost right away.
Expert systemScalabilityMachine learningEndliche ModelltheorieComponent-based software engineeringBuildingCryptographyMathematical analysisPredictionScale (map)CAN busSlide ruleMachine codePredictabilityCryptographyWave packetMachine codeMathematical analysisProduct (business)Presentation of a groupGroup actionMachine learningEndliche ModelltheorieSlide ruleVirtual machineBasis <Mathematik>Computing platformConnectivity (graph theory)NeuroinformatikCASE <Informatik>DataflowAlgorithmPotenz <Mathematik>BitMereologyInterface (computing)ScalabilityStructural loadDifferent (Kate Ryan album)Spectrum (functional analysis)Transformation (genetics)Set (mathematics)CryptanalysisComputer animation
Function (mathematics)outputTriangleSquare numberShape (magazine)Plot (narrative)AreaPerimeterGroup actionState of matterPredictionCAN busVirtual machineWeightMenu (computing)WordSequenceEndliche ModelltheorieSeries (mathematics)Curve fittingPoint (geometry)Linear regressionComputer networkCryptographyYouTubeRecurrence relationMachine learningBuffer overflowMachine codeMaxima and minimaFeature spaceFocus (optics)Transformation (genetics)Sample (statistics)Data modelUser profilePublic key certificateLinear mapAsynchronous Transfer ModeModul <Datentyp>Line (geometry)Software testingWave packetKolmogorov complexityInterface (computing)AlgorithmServer (computing)Read-only memoryDistribution (mathematics)Task (computing)Queue (abstract data type)Computer architectureGastropod shellExponentialabbildungPersonal digital assistantSource codeProcess (computing)DataflowTerm (mathematics)Perfect groupSoftware frameworkLambda calculusRevision controlSoftware developerData managementGraph (mathematics)Operations researchDigital-to-analog converterBroadcast programmingOperator (mathematics)View (database)Electronic mailing listDefault (computer science)Price indexUsabilityProcess (computing)Data acquisitionRule of inferenceDatabaseMiniDiscCore dumpSimilarity (geometry)Different (Kate Ryan album)Slide ruleFeature spaceCASE <Informatik>Different (Kate Ryan album)Set (mathematics)Endliche ModelltheorieData storage deviceProcess (computing)Machine codeRevision controlBranch (computer science)Functional (mathematics)Software frameworkTelecommunicationDataflowCore dumpLambda calculusFront and back endsSimilarity (geometry)outputOperator (mathematics)CryptographyRule of inferenceLinear regressionDatabaseMachine learningVirtual machinePredictabilityView (database)Information1 (number)Software repositoryElectronic mailing listScheduling (computing)Order (biology)ImplementationGraph (mathematics)Projective planeData managementFile formatExterior algebraUniform boundedness principleType theoryProduct (business)Parameter (computer programming)MereologyWindowWave packetSoftwareControl flowStack (abstract data type)Perspective (visual)Multiplication signStructural loadTime seriesPublic key certificateVideoconferencingProcess (computing)AreaSheaf (mathematics)Civil engineeringSequencePerimeterCycle (graph theory)PlotterService (economics)Basis <Mathematik>Complex (psychology)Maxima and minimaTriangleExpert systemTwitterGraph coloringSquare numberLatent heatFrame problemTask (computing)Abstract machineArtificial neural networkCost curveResultantCartesian coordinate systemIterationBitComputer architectureLibrary (computing)Term (mathematics)NumberPlastikkarteShape (magazine)International Date LineSource codeBuffer overflowRepository (publishing)Letterpress printingIntegrated development environmentQueue (abstract data type)YouTubeFunction (mathematics)Server (computing)Scaling (geometry)WeightCellular automatonGene clusterComputer programmingLine (geometry)Object (grammar)ScalabilityBoiling pointLevel (video gaming)WordSoftware testingPoint (geometry)Interface (computing)Division (mathematics)SynchronizationVector spaceInformation securitySingle-precision floating-point formatRepresentation (politics)Uniqueness quantificationAbstractionInstance (computer science)Formal languageSubsetStrategy gameBackupTrailSpectrum (functional analysis)Intrusion detection systemInsertion lossEmailPhysical systemDefault (computer science)Semiconductor memoryProfil (magazine)AlgorithmGraph (mathematics)LoginDistribution (mathematics)Visualization (computer graphics)Mathematical optimizationQuicksortExplosionLogic programmingMessage passingSerial portMobile appComputer animation
Transcript: English(auto-generated)