The ZIO Framework
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Untertitel |
| |
| Alternativer Titel |
| |
| Serientitel | ||
| Anzahl der Teile | 84 | |
| Autor | ||
| Lizenz | CC-Namensnennung 2.0 Belgien: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
| Identifikatoren | 10.5446/40068 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache | ||
| Produktionsjahr | 2012 |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
|
FOSDEM 20121 / 84
1
2
3
5
7
9
11
13
14
15
16
17
18
20
23
24
26
27
28
30
32
37
39
41
44
46
47
50
51
52
53
55
57
58
63
68
70
73
75
79
80
81
84
00:00
Modul <Datentyp>ProgrammierumgebungFramework <Informatik>Framework <Informatik>Arithmetisches MittelApp <Programm>Computeranimation
00:14
Offene MengeCodeProjektive EbeneEDV-BeratungHardwareAuswahlaxiomVorlesung/Konferenz
00:51
StichprobeMagnetkartep-BlockDigital Rights ManagementProgrammierumgebungZeitstempelHardwareBitrateFunktion <Mathematik>ZeitstempelMeterEin-AusgabeNebenbedingungComputeranimation
01:22
p-BlockDatenverarbeitungssystemStichprobenumfangEin-AusgabeSchnittmengeMetadatenKartesische KoordinatenEinfache GenauigkeitVorlesung/Konferenz
02:08
KonfigurationsraumFramework <Informatik>MitreissenTreiber <Programm>AnalogieschlussPuls <Technik>ZeitstempelEin-AusgabeUmwandlungsenthalpieEreignishorizontWellenformFunktion <Mathematik>TypentheorieFunktion <Mathematik>ZeitstempelProjektive EbeneSoftwareentwicklerTypentheorieSoftwareEin-AusgabeMultiplikationsoperatorRechter WinkelGewicht <Ausgleichsrechnung>GamecontrollerCASE <Informatik>Computeranimation
02:47
Objekt <Kategorie>ZeitstempelCASE <Informatik>SchnittmengeVorlesung/Konferenz
03:08
AeroelastizitätDatenverarbeitungssystemInstantiierungTypentheorieSchnittmengePuffer <Netzplantechnik>Flussdiagramm
03:38
Puffer <Netzplantechnik>MAPSchnittmengeDiagrammFlussdiagramm
03:52
p-Blockp-BlockInformationDatenstrukturMetadatenStichprobenumfangComputeranimation
04:12
Überlagerung <Mathematik>p-BlockStatistische HypotheseInformationGamecontrollerKontrollstrukturMetadatenRaum-ZeitKartesische KoordinatenDatenstrukturVorlesung/Konferenz
04:46
MehrwertnetzBitCASE <Informatik>DatenfeldEin-AusgabeWeb SiteZeitstempelStichprobenumfangComputeranimation
05:15
Kernel <Informatik>RechenwerkBimodulGruppenkeimSchnittmengeKontrollstrukturDatenstrukturMetadatenVirtuelle MaschineAttributierte GrammatikStichprobenumfangAdressraumKontrollstrukturZahlenbereichMathematische LogikWeb SiteZeitstempelGüte der AnpassungXMLComputeranimation
05:59
Kernel <Informatik>BimodulRechenwerkSchnittmengeGruppenkeimFunktion <Mathematik>Ein-AusgabeWhiteboardProzessfähigkeit <Qualitätsmanagement>RechenwerkObjekt <Kategorie>ProgrammierparadigmaRechter WinkelModulare ProgrammierungTreiber <Programm>JSONXMLVorlesung/Konferenz
06:35
SoftwareBimodulHardwareKernel <Informatik>Interrupt <Informatik>EreignishorizontTreiber <Programm>SchwellwertverfahrenGibbs-VerteilungSupremum <Mathematik>Open SourceRhombus <Mathematik>EreignishorizontMultiplikationsoperatorEreignisgesteuerte ProgrammierungHardwareKernel <Informatik>Ein-AusgabeComputeranimation
07:19
Puffer <Netzplantechnik>HardwareObjekt <Kategorie>Vorlesung/Konferenz
07:34
Puffer <Netzplantechnik>WhiteboardROM <Informatik>PufferspeicherSoftwareMetadatenUmwandlungsenthalpieSpeicherverwaltungPuffer <Netzplantechnik>WhiteboardSchreiben <Datenverarbeitung>Raum-ZeitHalbleiterspeicherMapping <Computergraphik>Modulare ProgrammierungVirtuelle AdresseComputeranimation
08:03
DatenstrukturHalbleiterspeicherp-BlockEndliche ModelltheoriePuffer <Netzplantechnik>WhiteboardVerzweigendes ProgrammMetadatenModulare ProgrammierungSoftwareEin-AusgabeDatenflussVorlesung/Konferenz
08:50
Ein-AusgabeDatenflussMereologieDivisionEin-Ausgabep-BlockMultiplikationsoperatorTreiber <Programm>Figurierte ZahlNichtlinearer OperatorCASE <Informatik>InformationsspeicherungKartesische KoordinatenSystemaufrufElektronische PublikationLesen <Datenverarbeitung>Computeranimation
09:24
p-BlockFunktion <Mathematik>Ein-AusgabePuffer <Netzplantechnik>TeilmengeTreiber <Programm>Ordnung <Mathematik>WärmeübergangSystemaufrufCASE <Informatik>Figurierte ZahlSchwingungVorlesung/Konferenz
10:20
DatenflussEin-AusgabeFunktion <Mathematik>Interface <Schaltung>InformationsspeicherungEreignishorizontSoftwareKernel <Informatik>p-BlockComputeranimation
10:46
StichprobeKontrollstrukturDatenstrukturEin-AusgabeFunktion <Mathematik>p-BlockRaum-ZeitGamecontrollerSchlussregelMetropolitan area networkEin-AusgabeFunktion <Mathematik>Elektronischer ProgrammführerDatenflussKartesische KoordinatenParallele SchnittstelleComputeranimationVorlesung/Konferenz
11:25
Raum-ZeitKontrollstrukturDatenstrukturRaum-ZeitKartesische KoordinatenDatenfeldMathematikPuffer <Netzplantechnik>GamecontrollerVirtuelle Adressep-BlockCASE <Informatik>BildschirmmaskeSnake <Bildverarbeitung>Computeranimation
11:56
PunktVorlesung/Konferenz
12:09
RechnernetzProgrammierumgebungFamilie <Mathematik>GamecontrollerPuffer <Netzplantechnik>SoftwareComputeranimation
12:30
KontrollstrukturHardwareFunktion <Mathematik>Kartesische KoordinatenInterface <Schaltung>p-BlockE-MailCAN-BusSoftwareEin-AusgabeÄquivalenzklasseStreaming <Kommunikationstechnik>Familie <Mathematik>RohdatenProtokoll <Datenverarbeitungssystem>CASE <Informatik>StatistikDivisionSchreib-Lese-KopfMarketinginformationssystemVorlesung/Konferenz
13:14
Socket-SchnittstelleKartesische KoordinatenAdressraumHardwareProtokoll <Datenverarbeitungssystem>Familie <Mathematik>HalbleiterspeicherSocketSoftwareFunktion <Mathematik>Ein-AusgabeFramework <Informatik>MAPComputeranimation
13:59
CASE <Informatik>Bus <Informatik>Einfache GenauigkeitEin-AusgabeFramework <Informatik>Treiber <Programm>Vorlesung/Konferenz
14:50
Ein-AusgabeBimodulHardwareStreaming <Kommunikationstechnik>p-BlockFunktion <Mathematik>Kernel <Informatik>Inverser LimesMIDI <Musikelektronik>BimodulHardwareElektronische PublikationEin-AusgabeSoftwaretestDruckspannungReelle ZahlBenchmarkGeradeBenutzerbeteiligungFamilie <Mathematik>XML
15:27
SoftwareOpen SourceHardwarePuffer <Netzplantechnik>DokumentenserverProjektive EbeneOffene Mengep-BlockVorlesung/Konferenz
15:53
Dokumentenserver
Transkript: Englisch(automatisch erzeugt)
00:00
I am presenting the Zio framework, which is an input-output framework. The letter Z just means nothing. It is just that Zio in Italian means uncle. And it is an open hardware project and it has been developed by me and Federico Varga,
00:24
who is this guy in the room. We both made the technical choices and wrote most of the code. And we are independent consultants and we are working for CERN.
00:43
I am sorry if I stammer, I hope you forgive me. Like this, okay. So the requirements at CERN, we are doing input-output work on accelerators.
01:03
Accelerators are several kilometers long. They have thousands of input and output devices. So the constraints is of timestamps,
01:21
which is better than one nanosecond. The data blocks are big, which means that we are not interested in acquiring a single sample. We must acquire a huge set of samples. And we need to manage the blocks offline,
01:42
which means that we are not willing to build the metadata inside each computer, but we must collect all that in a single place and elaborate it. Then we need some monitoring,
02:01
which means that the monitoring application should not know the detail of each input and output thing. Then the software requirements are please no IO control. We just want to do it in the right way. And documentation is important.
02:24
So the device types that we are going to handle are both input and output. Unfortunately, there are other projects that are concerned with input-output and do not manage output. There is no time to compare with the others, so I will not do that.
02:40
And the third, so input-output and time-to-digital, which means we return the timestamp. We are only interested in the timestamp in this case. I'm sorry, I have a problem. This counts back and my sheet counts forward, so I must use my own timer.
03:02
Okay. The most important object in ZIO is the channel set. A channel set is a container of channels. The channel is the individual wire that goes in or out of the computer.
03:24
And a channel set has a trigger type and a buffer type. There is one instance of a trigger which is common to all the channels and one instance of the buffer associated to each channel. So the layering is in three levels.
03:42
There is the device which is a collection of channel sets and the channel set which defines buffer and trigger is a collection of channels. Data is transferred in blocks. So we are not interested in the individual sample. We are interested in a data block.
04:02
So there is a block structure that handles both the meta information and the information. We may have a block with a single item, but this will be quite a waste because the control block, the control structure is half a kilobyte.
04:22
So the idea is that data within ZIO never travels without the metadata. The user space then may choose to not read the metadata but only read the data. But within ZIO we always have both. So the control is a fixed size structure with information inside.
04:45
This means that a diagnostic application without knowing whether the input is a temperature or the voltage or whatever it is can know the sample size, the timestamp.
05:03
The timestamp in our case is three fields of 64 bits. This allows to go over the nanosecond and do whatever we actually need. I'm sorry.
05:20
Okay. So there is the timestamp. There is a machine ID which may be a MAC address or whatever else and this allows all the metadata and data to be collected in a single place. This tells the number of samples, the sample size, which means how many bytes are composing each individual sample.
05:43
Then there are the attributes of the channel, which means gain and offset and whatever else, and the attributes of the trigger. So the control structure can be used independently of the individual thing.
06:01
So the device is physically a PCB or a chip. It is a probe unit, which means if your input output board is a PCI board, this is a PCB. There is a PCI driver that registers a Zio driver. If the thing is an SPI chip, there is an SPI driver that registers a Zio driver.
06:22
So the device is not actually an interesting thing. It is just a collection of Z-Sats. Then we have the trigger. The trigger is an important object inside Zio. It is a software module and it can be time driven. We already have the kernel timer
06:41
and we may have in the future some hardware internal stuff. It can be event driven. We already have an external interrupt trigger. It may be data driven, which means either it is hardware internal, some hardware device that only interrupts when there is a threshold on data,
07:04
or in-driver monitoring of input data that only fires the trigger when the data has some features. Or we can have, and we already have, a transparent trigger, which means when somebody reads the device, we get the data back.
07:25
And when somebody writes, this is immediately put to the hardware. The other very important object is the buffer. A buffer is a software module which sits between the trigger and the user.
07:40
It can be k-malloc based, and we already have a k-malloc buffer, which supports both read and write. We already have a v-malloc based, which also supports the memory map call, which means that the user space can avoid copying the data, but it just maps the buffer.
08:01
We may have onboard memory, which means if the PCI board has onboard memory and it acquires or outputs what is in the internal memory, it can be just made by a software module. And we already have, it is not in the master branch,
08:22
but I wrote it and it should work, a software buffer, which means if you are not interested in metadata, you may just read the data and all blocks are collapsed. If you come late to reading data, you will find one metadata structure
08:44
that describes a bigger block of data. So the input flow is like this. If an application reads, the file operation part of Zio will retrieve a block from the buffer,
09:02
where the trigger did store a block. Time in this case is going down, so the trigger is concerned with allocating a block. Request the driver to make the input operation. When input is done, there is a callback to the trigger. The trigger will store the block in the buffer, and the read call can retrieve the block.
09:24
In this case, the block is freed by read. So it is stored in the buffer and when it is read, it is freed. The output stuff is symmetrical. The buffer ignores if it is used for input or for output.
09:40
It is simply stored, in this case, by the write call and it is the trigger that retrieves the block when it is ready to make output or in advance in order to prepare the driver. It calls the output subset method of the driver in order to make the transfer and when it is done, the block is freed.
10:03
This is completely symmetrical. If we go back, we can see that supporting internal users, like for example the input subsystem, if we have an accelerometer instead of an accelerator, it is pretty simple.
10:21
The buffer, instead of allowing read, it will just generate an event to the input subsystem of the kernel. Or it may register as a network interface and when a store block is called, it can send a network packet to the host.
10:41
This is completely modular. So the output is symmetrical. Access from the user space happens with two devices. There is a control device which only returns the control blocks.
11:05
These are fixed size and when the application reads the control block, it knows how long the data block is. But you can also just read the data and ignore the control. Those two things are managed in parallel.
11:20
This is the data flow which works in the same way for input or output. The memory map capable buffers simply allow the application to read the control and when you know the control block, one field of the control block is the offset in the memory map buffer.
11:41
So in this case, you don't need to copy stuff to the user space or from the user space. You just need to copy the control changes. In this case, you can poll or select or read this and when it awakes, it will have a point to the data.
12:03
And this is already there and it works perfectly. The future, this is the ideas that came up for this kind of stuff. A control plus data looks like a network packet. So we may build both a buffer that sends Ethernet-like packets
12:20
because Zio has the host ID inside the control structure. Or we will write a PF-Zio which is similar to the PF-CAN. The protocol family CAN only works with CAN hardware. In this case, we may have a SOX stream for application that only know to read data
12:41
without knowing the division between blocks. We will have a SOX datagram interface that just returns the data blocks when the trigger fires for input and equivalent for output. And we will have a SOX raw interface that will return for input
13:02
and allow to send for output the control structure as an header of the network packet with the data which is following. Please note that protocol family Zio is not Zio overwritten. It is a different thing.
13:21
The idea is that application will perform input-output. Instead of accessing charged devices, which is the legacy way, they will be able to do that by sending and receiving frames. The address space is the address of the device and channel within the hardware.
13:44
So instead of using hundreds of charged devices, we may use a socket. And the problem of memory map, which we have with the charged devices, is solved by using the techniques that are already in place for zero-copy networking.
14:06
If I input a four-megabyte block, I will just receive a four-megabyte frame using the zero-copy stuff that already is in place. He also may drive hundreds of channels. Why?
14:20
Because, for example, we have an etherbond bus, which is a wishbond bus, which is remotized through Ethernet. So we may have a lot of FPGA chips that are connected to a single host driver. And in this case we actually have hundreds.
14:43
So what is the status as of tomorrow? This is not today, it is tomorrow. Both because it will not change much. And because I'm going to push a documentation file this night. And so we have software-only modules for stress test and benchmarking.
15:07
We have simple hardware modules because hardware, for the real use case, will become ready in February. We have a zero-zero device. We have a line discipline, which is for input only,
15:21
but it allows to stress test on a PTY channel. We have a GPIO and stuff. For the buffers we already have K-MALOC and V-MALOC, and we also have this Soxtreme alike thing, which collapses block in a single bigger block. Okay. That's it.
15:44
Oh, sorry. Okay, the thing is the project is zero in the open hardware repository. We have Git and documents and so on. I'm sorry. Thank you very much.
16:01
I'm sure there are people that want to...