We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

DeepBase: Deep Inspection of Neural Networks

00:00

Formal Metadata

Title
DeepBase: Deep Inspection of Neural Networks
Title of Series
Number of Parts
155
Author
License
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
Although deep learning models perform remarkably well across a range of tasks such as language translation and object recognition, it remains unclear what high-level logic, if any, they follow. Understanding this logic may lead to more transparency, better model design, and faster experimentation. Recent machine learning research has leveraged statistical methods to identify hidden units that behave (e.g., activate) similarly to human understandable logic, but those analyses require considerable manual effort. Our insight is that many of those studies follow a common analysis pattern, and therefore there is opportunity to provide a declarative abstraction to easily express, execute and optimize them. This paper describes DeepBase, a system to inspect neural network behaviors through a unified interface. We model logic with user-provided hypothesis functions that annotate the data with high-level labels (e.g., part-of-speech tags, image captions). DeepBase lets users quickly identify individual or groups of units that have strong statistical dependencies with desired hypotheses. We discuss how DeepBase can express existing analyses, propose a set of simple and effective optimizations to speed up a standard Python implementation by up to 72x, and reproduce recent studies from the NLP literature.
Magneto-optical driveData managementComputer networkArtificial neural networkSoftwareBasis <Mathematik>WeightExecution unitLecture/Conference
WeightDecision theoryVideo gameTask (computing)ConsistencyPhysical systemProcess (computing)VideoconferencingLecture/Conference
HierarchyVisual systemComputer-generated imageryPhysical systemMathematical modelWeightVisualization (computer graphics)Flow separationBlack boxCASE <Informatik>HTTP cookieBitConfidence intervalWhiteboardComputer animation
HierarchyComputer-generated imageryVisual systemPattern recognitionVirtual machineTranslation (relic)AutomationRecurrence relationPairwise comparisonData structureMathematical modelFormal grammarProgramming languageCategory of beingObservational studyExecution unitSystem identificationComputer networkMachine visionPhysical systemIdentity managementCNNLinear mapString (computer science)Classical physicsScale (map)Machine codeLine (geometry)Function (mathematics)Cross-correlationSurjective functionMathematical modelMathematical morphologyHypothesisMathematical modelBlogSocial classSelectivity (electronic)Software testingPoint (geometry)QuicksortProgramming languageProduct (business)SpacetimeNeuroinformatikBasis <Mathematik>outputInterpreter (computing)StatisticsCASE <Informatik>Independence (probability theory)Forcing (mathematics)Mathematical modelComplex (psychology)Type theoryStudent's t-testMereologyInheritance (object-oriented programming)Different (Kate Ryan album)Speech synthesisExecution unitVarianceMultiplication signMachine codeLatent heatDiscrete groupCoalitionCross-correlationMathematical analysisTrailConfidence intervalLevel (video gaming)Goodness of fitPredictabilityNetwork topologyChainMachine visionExtension (kinesiology)Group actionFormal grammarHypothesisArithmetic meanBitLogische ProgrammierspracheFunctional programmingFormal languagePhysical systemData structureSet (mathematics)Variable (mathematics)SubsetInstance (computer science)Function (mathematics)Pattern languageMedical imagingObject-oriented programmingGreatest elementPosition operatorDisk read-and-write headDistanceForestOpen setMUDTranslation (relic)MappingForm (programming)Logic gateTensorMeasurementSeries (mathematics)CollisionTime seriesConcurrency (computer science)Field (computer science)Visualization (computer graphics)WeightSupport vector machineLogistic distribution2 (number)Linear regressionVirtual machineBinary fileMilitary baseCartesian coordinate systemLine (geometry)Maschinelle ÜbersetzungRegular graphExpressionSoftwareSlide ruleWordElectric generatorCodierung <Programmierung>ResultantTetraederFlow separationArtificial neural networkComputer animation
Binary fileMathematical modelMaß <Mathematik>Linear regressionLogistic distributionMathematical morphologyHypothesisMathematical modelCoding theoryHypercubeLine (geometry)Scale (map)Machine codeComputer networkVisual systemComputer-generated imageryTexture mappingObject-oriented programmingPixelApproximationString (computer science)Source codeComplete metric spaceRule of inferenceFormal grammarDefault (computer science)Physical systemDeclarative programmingParameter (computer programming)Programming languageMathematical modelMedical imagingFunctional programmingPhysical systemTime seriesFormal grammarSpeech synthesisSpacetimeSensitivity analysisRule of inferenceBilderkennungRun time (program lifecycle phase)HypothesisMathematical modelExtension (kinesiology)Object-oriented programmingWordBenchmarkMathematical optimizationLine (geometry)Materialization (paranormal)TensorPoint (geometry)Cross-correlationMachine codeMereologyFormal languageExecution unitConfidence intervalVirtual machineSet (mathematics)PixelMathematical analysisDimensional analysisTable (information)NumberPredicate (grammar)Game theoryPresentation of a groupSequelFunction (mathematics)Instance (computer science)Scaling (geometry)Type theoryDifferent (Kate Ryan album)Natural languageConstructor (object-oriented programming)WindowCASE <Informatik>Row (database)outputCoalitionLatent heatInheritance (object-oriented programming)Scripting languageData managementDependent and independent variablesCartesian coordinate systemGodApproximationMeasurementAuthorizationSurfaceProcess (computing)BitInformation securityTexture mappingProteinNetwork topologySocial classVotingDistanceAuditory maskingResultantMetric systemTwitterLogistic distributionDatabaseLinear regressionCivil engineeringSemiconductor memoryPressureEllipseWeightCurveBinary fileTime domainMatrix (mathematics)Parallel portFlow separationConvolutionEnvelope (mathematics)CalculationLetterpress printingMultiplication signComputer animation
Programming languageParameter (computer programming)Declarative programmingPhysical systemGoogolMathematical modelComputer multitaskingMathematicsPhysical systemMathematical modelComputer animationXML
Transcript: English(auto-generated)