The 'complex camera' user-space library is a fairly new and hot topic
in the Linux kernel video community. The issue is debated and targets
a real technical debt of Linux systems. Device producers have been
tackling this with in-house solutions that proved not to scale once
brought in the open space, so a generic solution is now required to
address the issue. The Linux kernel video input subsystem and its APIs changed greatly in
the last years to keep up with the increasing complexity and
processing power embedded in modern SoCs. Namely, the biggest
game-changing feature introduced already 5 years ago is the media controller
subsystem and its pad oriented APIs, that allows composing pipelines
of processing blocks to model the acquisition and re-processing of
video and images.
While most of the media subsystem drivers in mainline Linux have been
ported to fully support media-controller, the real missing component
is now user space support to automate setting up and controlling the
image processing pipelines.
The Video4Linux community is now tackling the issue by implementing
support for "complex camera" systems by providing a user space library
that aims to support the most recent use cases represented by mobile
consumer devices and high end laptops.
This talks provides an overview of modern media-controller capable video
device drivers, their userspace APIs and the challenges the currently
in-development "libcamera" library has to face.
It also aims to provide to both driver and application developers an
overview of the most modern implementation of the Linux video input
stack architecture, that will likely be found in most system in next
years. |