Using Advene to accompany research in AudioVisual Digital Humanities
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 490 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/46934 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 2020122 / 490
4
7
9
10
14
15
16
25
26
29
31
33
34
35
37
40
41
42
43
45
46
47
50
51
52
53
54
58
60
64
65
66
67
70
71
72
74
75
76
77
78
82
83
84
86
89
90
93
94
95
96
98
100
101
105
106
109
110
116
118
123
124
130
135
137
141
142
144
146
151
154
157
159
164
166
167
169
172
174
178
182
184
185
186
187
189
190
191
192
193
194
195
200
202
203
204
205
206
207
208
211
212
214
218
222
225
228
230
232
233
235
236
240
242
244
249
250
251
253
254
258
261
262
266
267
268
271
273
274
275
278
280
281
282
283
284
285
286
288
289
290
291
293
295
296
297
298
301
302
303
305
306
307
310
311
315
317
318
319
328
333
350
353
354
356
359
360
361
370
372
373
374
375
379
380
381
383
385
386
387
388
391
393
394
395
397
398
399
401
409
410
411
414
420
421
422
423
424
425
427
429
430
434
438
439
444
449
450
454
457
458
459
460
461
464
465
466
468
469
470
471
472
480
484
486
487
489
490
00:00
Digital signalPersonal digital assistantOpen sourceInterface (computing)VideoconferencingWorkstation <Musikinstrument>View (database)Execution unitFunction (mathematics)Projective planeContext awarenessMultiplication signSoftwareCodeDifferent (Kate Ryan album)Data structureInstance (computer science)RectangleFigurate numberProcess (computing)Knowledge engineeringMixed realityMetadataFreewareMathematicsSystem identificationShared memoryData storage deviceObservational studyCartesian coordinate systemMathematical analysisHypermediaReflection (mathematics)Right angleInterface (computing)Reading (process)Content (media)ArmTracing (software)BitQuery languageSystem administratorSinc function2 (number)Cross-platformThumbnailDomain nameComputer animation
04:20
Knowledge engineeringMathematical analysisPersonal digital assistantDigital signalVideoconferencingObservational studyView (database)Sign (mathematics)System identificationGroup actionFinitary relationTemplate (C++)Process (computing)Exploratory data analysisHypermediaCollaborationismSoftware frameworkContent (media)Bridging (networking)FeedbackInformationData modelNumbering schemeData conversionOntologyProcess modelingLocal ringTexture mappingFolksonomyLevel (video gaming)Type theoryMotif (narrative)Computer-generated imageryFormal languageVisualization (computer graphics)System programmingCodeConstraint (mathematics)Endliche ModelltheorieFile formatInformationFreewareSoftwareVisualization (computer graphics)View (database)Bridging (networking)Theory of relativityProjective planeInformation technology consultingTemplate (C++)Mathematical analysisPoint (geometry)DigitizingSoftware developerUsabilityCartesian coordinate systemPresentation of a groupElectric generatorGoodness of fitPattern languageCASE <Informatik>OntologyCategory of beingPhysical systemObservational studyLatent heatHypermediaExploratory data analysisData storage deviceData structureExecution unitGroup actionCodeWebsite10 (number)IterationStudent's t-testOrder (biology)Motion captureBootstrap aggregatingMereologyMetadataMultiplication signTask (computing)VideoconferencingAdaptive behaviorCollaborationismBroadcasting (networking)Tracing (software)Level (video gaming)Revision controlDifferent (Kate Ryan album)Menu (computing)Position operatorFood energyWorkstation <Musikinstrument>Interactive televisionOcean currentAdditionProcess (computing)1 (number)Local ring8 (number)BuildingForceTerm (mathematics)Grass (card game)Representation (politics)WordUniverse (mathematics)Lattice (order)Statement (computer science)Inheritance (object-oriented programming)Library (computing)PressureComputer animation
11:48
FreewareSoftwareComputer clusterOpen setObservational studyComputer animation
12:18
Visualization (computer graphics)SpacetimeDenial-of-service attackComputer animation
12:30
SpacetimeFacebookPoint cloudOpen sourceComputer animation
Transcript: English(auto-generated)
00:08
Yes, let's go. So, hello everyone. My name is Olivier Aubert. I'm happy to be there. Just a second. Can I? Yeah, okay. Okay, everything's fine. Good luck.
00:21
It's fine. So I'm from France, from Nantes, and I'm a research engineer and I'm currently freelance consulting for research projects. And I'm going to talk to you today about the video software I developed, which is called Advine, which is a video annotation software, and how it's been used recently, because it's a software which has some history already,
00:45
but it still has some uses and actualities, so I will present two use cases, two usage of the application, so recent. So, since it's a lightning talk, I will be quick on the project itself and then talk about the two examples, which are the Remand Method application, which is a museology investigation
01:07
method, and the Ada project, which is a media studies project with Berlin. So, Advine, it's a project which started in 2002, so that's rather old, with Yannick Pierre and Pierre-Antoine Champagne, University of Lyon, and we wanted to provide tooling
01:26
for accompanying active reading of audiovisual documents. And so, active reading is the possibility for a user to immerse himself in a document, to take notes, and to structure,
01:41
to have a scholarship workflow based on this annotation on the document, so to accompany reflections on the document. So the goal is to create, share analysis of audiovisual documents as things that we call hypervideos, basically a mix of annotations and video. It's a free software, so the project itself gave birth to a concrete artifact,
02:06
which is an application, the Advine application, which is a free software, GPL, cross-platform desktop application using Python, GTK, and Gstreamer. It's been used in different contexts, but I will talk about just two today.
02:22
Quickly, the interface is rather common. It's centered around a video. Can I show that? Yes. There's a video player here, which is always there, and then, around the video player, you have multiple places, which we call, I don't know the name we used,
02:45
yeah, okay, places for different views for interacting with your metadata. So you put metadata. For instance, here, I've got below the video a view which is a timeline view, which is rather common in audiovisual annotation domain or audiovisual manipulation.
03:01
On the right, you've got the same data, which is presented in a different way. It's as a transcription, with time codes presented as these kind of thumbnails on the video. And then, on the right, you've got an output of the process. So this is the first two views are the kind of tools, there are multiple other views, but these two views are the
03:22
kind of tools that you're using in your process, in your scholar process of exploring the content and constructing your analysis. And this, on the right, can be seen as the output of the process. So ADMIN tries to be a tool that you can use throughout the whole process, from structuring and analyzing to producing outputs here.
03:45
And we'll see how it goes. So basically, the important notion to take from this figure is the black rectangle that surrounds the annotation structure, the annotations themselves,
04:02
the annotation structure, which is user-defined, and the different views, templates, and queries that are all put in the single package, which is the unit, the documentary unit that you can exchange independently from the video. So the video is on the side, it's metadata by the video. We tackled different scientific changes in this,
04:24
so knowledge engineering, document engineering, HCI, and also data visualization and the analysis of activity traces. So that was for the scientific part. We were interested as researchers. And then I'll go through two use cases in digital humanities.
04:43
So the recent, the first one dates from four years ago, and the other one is from 2017. So Remind Method is a project done by Daniel Schmidt, which is now a professor at the University of Valenciennes. And the goal was to study the museum visitors'
05:02
experience during a visit, and through the methodology used video-based auto-confrontation. So the visitors were equipped during an exhibit with a camera on glasses to capture a subjective view of their visit, of their experience. And after the visit, they were interviewed by a researcher based on the video of the visit.
05:25
And then the video of this interview was captured. And this is the primary material, so the capture of the interview, that was used in ADVIN to be analyzed by the researchers. They transcribed the interview using ADVIN. They identified, so they had different categories
05:45
in their methodology, so they could identify the different categories. And they used relations to express courses of experience. So that's basically a group of categories that form a unit, a meaningful unit for the researchers, for the methodology.
06:04
So the underlying structure in ADVIN provided the support for expressing this kind of information. And then they could generate visualization through templates. So these visualizations, they used them for their analysis, during their exploratory analysis.
06:23
And then they could also put them on a website afterwards as a kind of publishing. So this is what it gives basically. So you've got on your left, you've got the ADVIN application with the timeline, the transcription, and so on. You see different lines here
06:40
that correspond to the different categories of analysis, so the identified categories in the discourse in the interview. And then here you've got on the right what you can find, one of the views that is produced by the tool directly through a template system that is available on the museography.fr website.
07:01
So this was the first example. So I've got to be quick, but if you have questions, I'm here today and tomorrow. The other example is the Ada project, which is carried out in collaboration with the Cinepoetics team in Freie University, Berlin, and the HPI in Potsdam.
07:24
So Cinepoetics is doing media studies, so they're the final users. HPI is about, they have an expertise, a good expertise in video analysis, so feature extraction and so on. And we brought our expertise in video annotation, manipulation, interaction, and so on. And so the goal of the project
07:44
was to study the staging patterns in the audiovisual presentations of the financial crisis. So they wanted to know if there are patterns that always come again and again when this is presented in documentaries, in feature movies, or in TV broadcasts.
08:02
And for this, they wanted to apply quantitative methods, so systematically annotate movies, and then dig into this data, this metadata they produced, to see, first, to see for themselves if there are things interesting to dig in. And then also to provide a ground truth, I think I saved my next slide, yes,
08:24
to build a ground truth for future automation of the system. So we wanted to build some feature extraction specific for the task, and we needed data for this. So I've been provided the bridge between these issues.
08:42
So the idea was to optimize the manual annotation process, so in order to be able to put students to work, so that the teams of students annotate movies, and to provide a bridge between the user's manipulation of the data,
09:00
and its semantic representation, because in the end with the HPI team, they wanted to have semantics, so we produced an ontology, it was stored in a triple store and so on, so they wanted to have semantic data, but the users at the other end didn't want to deal with semantic data, they wanted to deal with keywords or whatever. So the tool here is what bridges the gap between both sides.
09:24
And so the application is the same, basically, we did some adaptations and optimizations, and I don't have time for the process, so yeah. I'll just say that ADVIN was used to produce an ontology, so you can find the ontology that was produced through ADVIN,
09:42
through this kind of annotation structure definition that was carried out in ADVIN, and then used to build the first ontology, bootstrap ontology, that was after used for other multiple iterations to refine the ontology. And since we've now got multiple data,
10:02
so the current news for the project is that we're still working together, the Ada project itself is not yet completely over, and we're working on data visualization. So we now have tens of thousands of annotations on movies, and so this raises questions of how to visualize them for the schoolers,
10:23
for the media study schoolers. So we're working on this. Oh yeah, just one point. This is free software which has developed for a long time, and the Ada project was the opportunity for me, just the first point of the developments for the Ada project, to update code for adapting to new systems.
10:42
So it was, it's an old application which was Python 2, GTK 2, and so on. I had to do an update to the new versions of these libraries, and I didn't have the time or opportunities. And this project was the opportunity. So do not hesitate to fund projects that may not be fit for you just right now,
11:02
but may fit the task but need some developments, and yes, contribute to the free software ecosystem by funding such projects that we can advance the free software developments. And so that's it. So through these two examples, I tried to show you that this is a flexible, extensible, usable tool for digital humanities,
11:27
and I'm available also for development or consulting for this. Thank you.
11:43
Yes? Is it the same? Yeah, the question is, yes, whether it's comparable to Envivo.
12:00
So Envivo is one of the proprietary tools that are used in ethnography or ethnographic studies. This is comparable. It doesn't have the same features, obviously, because it's free software, but it's open and basically fits the bill for many of the needs.
12:21
So yes, definitely this is comparable to Envivo. Thank you.
Recommendations
Series of 4 media