The TIB AV-Portal
The TIB AV-Portal is an open-access platform for scientific videos and audio recordings, focusing on technology as well as architecture, chemistry, computer science, mathematics and physics. Over time, the scope of the portal has expanded to include a wider range of scientific disciplines, including the humanities and social sciences, economics, law and medicine (see subjects). Users of the portal have access to a diverse range of video and audio content, including lecture and conference recordings, simulations, animations, experiments, interviews and video abstracts as well as open educational resources.
The portal was developed from July 2011 to April 2014 by the Lab Non-Textual Materials of the TIB – Leibniz Information Centre for Science and Technology and University Library in cooperation with the Hasso Plattner Institute for Software Systems Engineering. The portal went online in April 2014 and was operated by yovisto GmbH until 2019 and further developed on behalf of the TIB. In September 2018, the "Scrum Team AV-Portal" was founded at the TIB, which was entrusted with the task of migrating all software projects of the portal to the TIB infrastructure. Since 2020, responsibility for the operation and continuous development of the portal has been entirely in the hands of the TIB Scrum Team.
The TIB AV-Portal simplifies the search, citation and publication of scientific videos and audio recordings while also providing options for downloading, licensing, or ordering videos and audios as DVDs. Thanks to the use of Creative Commons licenses, most of the content is freely reusable.
The services of the TIB AV-Portal include:
- Hosting and long-term archiving of videos, audio recordings and accompanying materials
- DOI registration for permanent citation
- Rights clearance and license advice
- Speech, text and image recognition
- Continuous further development of the portal
- Editorial support
- Open, ad-free and GDPR-compliant
Get to know the TIB AV-Portal in just 120 seconds!
Scene detection – shot boundary detection segments the video based on cuts. The resulting visual table of contents provides a quick overview of the entire video content and facilitates targeted access to specific sections. Each scene can be cited to the second via Media Fragment Identifier. The technology used is PySceneDetect.
Text recognition – video optical character recognition captures, indexes and makes written language, such as text on presentation slides, searchable. Tesseract is used for this purpose.
Speech recognition – automatic speech recognition transcribes spoken language in videos or audio recordings. The result is a transcript with time stamps that enables a precise search. The AI-based speech recognition "Whisper" transcribes 100 languages, including English, German, French, Spanish and Ukrainian, and translates these languages into English. Accordingly, subtitles and transcripts are offered in both the original language and in English translation.
Image recognition – visual concept detection indexes the moving image with visual concepts such as "computer animation" or "experiment". Currently, the AV-Portal Scrum Team is collaborating with the Visual Analytics Research Group to further develop image recognition using Open-Clip.
Keywording – named-entity linking associates individual segments with subject headings from the Integrated Authority File (GND). The terms are disambiguated and have relationships to other terms, which enables a more effective search.
In principle, videos and audio recordings are only analysed automatically if legally permitted. The spoken language is converted into a machine-readable transcript, which is displayed as subtitles and can be searched. Non-English media are translated into English; these translations are available both as subtitles and as searchable transcripts.
Annotations of speech, text, and images are generated exclusively for videos and audio recordings that belong to the six core subjects of the TIB – Leibniz Information Centre for Science and Technology and University Library: Engineering, Architecture, Chemistry, Computer Science, Mathematics and Physics. The reason for this is that a vocabulary for automatic annotation is currently available only for these subjects.
The TIB AV-Portal offers both a German and an English user interface, which can be selected via the language settings in the user menu (top right).
Many metadata are available in German and English, allowing users to enter search terms in either language. When a German search term is entered, its English translations are automatically included in the search, and vice versa for English search terms.
The automatic speech recognition Whisper, which we use to create subtitles and searchable transcripts, can transcribe approximately 100 languages. These include German, English, Spanish, French, Italian, Greek, Russian, Ukrainian, Polish, Japanese, Turkish, and Chinese.