Collabora Office as an app on iOS
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
Titel |
| |
Serientitel | ||
Anzahl der Teile | 561 | |
Autor | ||
Lizenz | CC-Namensnennung 2.0 Belgien: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
Identifikatoren | 10.5446/44193 (DOI) | |
Herausgeber | ||
Erscheinungsjahr | ||
Sprache |
Inhaltliche Metadaten
Fachgebiet | ||
Genre | ||
Abstract |
|
FOSDEM 2019136 / 561
1
9
10
15
18
19
23
24
27
29
31
33
34
35
38
39
40
43
47
49
52
53
54
55
58
59
60
63
65
67
69
70
78
80
82
87
93
95
97
102
103
104
107
110
111
114
116
118
120
122
123
126
127
131
133
136
137
139
141
142
148
153
155
157
159
163
164
168
169
170
171
172
173
174
181
183
185
187
188
193
196
197
198
199
200
201
205
207
208
209
211
213
214
218
221
223
224
226
230
232
234
235
236
244
248
250
251
252
253
255
256
257
262
263
264
268
269
271
274
275
276
278
280
281
283
284
288
289
290
293
294
296
297
300
301
304
309
311
312
313
314
315
317
318
321
322
327
332
333
334
335
336
337
338
339
340
343
345
346
352
353
355
356
357
359
360
362
369
370
373
374
375
376
377
378
383
384
387
388
389
390
391
393
394
395
396
406
408
409
412
413
414
415
419
420
425
426
431
432
433
434
435
436
438
439
440
441
445
446
447
448
453
455
457
459
466
467
471
473
474
475
476
479
480
484
485
486
489
491
492
496
499
500
502
505
507
508
512
515
517
518
529
531
533
534
535
536
539
540
546
550
551
552
553
554
555
557
558
559
560
561
00:00
Office-PaketSoftwareTotal <Mathematik>Ein-AusgabeApp <Programm>Office-PaketKollaboration <Informatik>Computeranimation
00:23
CodeServerBenutzeroberflächeHumanoider RoboterSampler <Musikinstrument>Produkt <Mathematik>SichtenkonzeptSkriptspracheSystemplattformGebäude <Mathematik>HydrostatikGasströmungSoftwaretestRechenwerkRepository <Informatik>Funktion <Mathematik>Notepad-ComputerNormalvektorRechteckClientProzess <Informatik>Serviceorientierte ArchitekturTelekommunikationSpeicherabzugMereologieMensch-Maschine-SchnittstelleMereologieSoftware Development KitSkriptspracheProzess <Informatik>ClientKomponententestKartesische KoordinatenWurzel <Mathematik>Ein-AusgabeKonfigurationsraumInformationQuellcodeHumanoider RoboterIdeal <Mathematik>BrowserBenutzeroberflächeGenerator <Informatik>SystemplattformCodePixelProgrammbibliothekApp <Programm>Automatische HandlungsplanungSichtenkonzeptSoftwaretestMessage-PassingFilesharing-SystemDifferenteDateiformatBildschirmfensterServiceorientierte ArchitekturServerUmsetzung <Informatik>Gebäude <Mathematik>RechenwerkSpeicherabzugElektronische PublikationNetzbetriebssystemMultiplikationsoperatorMobiles InternetTesselationDokumentenserverMomentenproblemVersionsverwaltungNormalvektorLaufzeitfehlerBinder <Informatik>InstantiierungOpen SourceReelle ZahlMetropolitan area networkVerkehrsinformationUltraviolett-PhotoelektronenspektroskopieAppletsinc-FunktionEinsOffice-PaketMonster-GruppeLastObjekt <Kategorie>PackprogrammTelekommunikationDatumsgrenzeInterprozesskommunikationKlasse <Mathematik>Mapping <Computergraphik>RechteckBenutzerbeteiligungDämon <Informatik>
09:25
TelekommunikationCodeSystemplattformMessage-PassingMereologieProzess <Informatik>ThreadMechanismus-Design-TheorieInterprozesskommunikationPhysikalisches SystemPuffer <Netzplantechnik>Socket-SchnittstelleÄquivalenzklasseMobiles InternetHydrostatikSpeicherabzugVektorrechnungZeiger <Informatik>Konstruktor <Informatik>Funktion <Mathematik>SkriptspracheKonfigurationsraumFormale SpracheTemplateNotepad-ComputerStabNominalskaliertes MerkmalInformationNabel <Mathematik>FehlermeldungKonfiguration <Informatik>OvalExogene VariableSampler <Musikinstrument>SimulationQuellcodeBetriebsmittelverwaltungURLKorrelationZeitstempelVersionsverwaltungThumbnailHash-AlgorithmusGeradeDickeImplementierungInhalt <Mathematik>LogarithmusProgrammierumgebungSoftwaretestTaskZeichenketteStatechartBenutzerfreundlichkeitBildschirmfensterSichtenkonzeptRechenschieberMereologieServerSichtenkonzeptHumanoider RoboterCodeEin-AusgabeSystemplattformApp <Programm>GeradeCASE <Informatik>MultiplikationsoperatorProgrammschleifeZusammenhängender GraphKonstruktor <Informatik>Mapping <Computergraphik>HoaxLineares FunktionalMetropolitan area networkThreadÄhnlichkeitsgeometrieAppletFormale SprachePhysikalische TheorieProjektive EbeneGesetz <Physik>Gebäude <Mathematik>Objekt <Kategorie>VerzeichnisdienstWald <Graphentheorie>Kartesische KoordinatenRechenwerkPuffer <Netzplantechnik>MathematikElektronische PublikationStellenringModallogikDatenstrukturBitKonfiguration <Informatik>DokumentenserverProzess <Informatik>UmwandlungsenthalpiePhysikalisches SystemMAPTemplateMechanismus-Design-TheorieZeiger <Informatik>BenutzerbeteiligungBinder <Informatik>Mobiles InternetProgrammbibliothekMessage-PassingTranslation <Mathematik>SkriptspracheKonfigurationsraumMaschinencodeFunktionalFormation <Mathematik>Zentrische StreckungSoftware Development KitSystem FVerschlingung
18:26
PunktwolkeInformationsspeicherungMaßerweiterungTopologieCodeWeb-SeiteURLSichtenkonzeptParametersystemMessage-PassingBinärcodeSpezialrechnerVektorpotenzialServerMessage-PassingObjekt <Kategorie>Physikalisches SystemGamecontrollerSichtenkonzeptInformationsspeicherungMereologieBenutzerbeteiligungWeb-SeiteBrowserElektronische PublikationMultiplikationsoperatorMaßerweiterungURLArithmetischer AusdruckFormale SpracheCodeParkettierungApp <Programm>RichtungComputeranimation
20:47
MultiplikationsoperatorCodeComputeranimation
21:44
TouchscreenComputeranimation
22:09
Office-PaketLesezeichen <Internet>Produkt <Mathematik>VersionsverwaltungUnternehmensarchitekturModul <Datentyp>FunktionalBrowserNormalvektorTypentheorieSchnelltasteElektronische PublikationBestimmtheitsmaßStichprobenumfangTouchscreenMenütechnikKartesische Koordinaten
23:45
Computeranimation
Transkript: Englisch(automatisch erzeugt)
00:05
Yes, so I'm Tor Rilquist. I'm going to speak about LibreOffice or CollaborOffice as an app on iOS. I work for, or I'm a contractor for CollaborOffice activity.
00:24
First some stuff about the company, but you probably know that already. And some history of this effort to build LibreOffice for some mobile operating system.
00:45
You first need to cross-compile it and the work to enable that was started like back in 2011 already. And the platforms that it was targeted for were iOS, Android and also cross-compiling to Windows was tried.
01:08
Initially this was just mostly something done in my spare time and there weren't really any concrete plans on to get some actual product out of it.
01:23
And cross-compilation effort for Windows was supposed to make it easier to build LibreOffice for Windows, but there were all kinds of problems with that and eventually we didn't continue that effort anymore.
01:44
Then back in 2014, there was this company called CloudOn, if you remember them. They had an app for iOS that included some file sharing and whatever and they needed to be able to also edit documents locally on an iOS device.
02:08
And they funded this work for some time, but then they got acquired and their plans changed and that didn't work out very well nicely.
02:22
Around the same time, we at Collabora started working on LibreOffice Online and this introduced this tiled rendering idea where a client or, well, some code can ask LibreOffice to render the view of a document like as rectangular tiles.
02:47
Pixel maps, that is what is used in the online. We send these tiles to the client as PNGs basically.
03:02
And for iOS, we had only a very bare bones test application or app that was kept working, but really not much work was done on improving it. Then Jeanne Ivarsson started working on a new app in 2017, but that did not get finished either.
03:27
Then last year, we made a fresh start when Michael had this idea to reuse this LibreOffice Online JavaScript user interface also in a mobile app.
03:44
And that actually turned out to work quite nicely and wasn't even very hard to get working initially. And this has also been funded by Adventist. Thank you to Nicolas there.
04:07
Something about this cross-compilation of LibreOffice code. One basic thing is that we run the configure script twice automatically, one for the build platform and one for the target or host platform.
04:29
And for the build platform, that's then macOS for iOS and Linux, for instance, for Android. We only build the build time tools that are necessary to process these ideal files, for instance, and generate code.
04:51
And when building cross-compiling for iOS or Android, we don't build any dynamic libraries as we do for the desktop platforms, but only static archives.
05:05
The main reason for this originally was that on iOS, you were not allowed to even load your own dynamic libraries. I think that has changed in the meantime, but we still do it this way anyway.
05:23
And also on Android, you could load your own shared objects, but the runtime linker had some very silly restrictions and no way could it manage with the hundreds of shared libraries that LibreOffice wants to load.
05:44
And when cross-compiling, we don't do any unit testing, which of course is a bit sad, but it isn't really possible to do it. Instead, what could be done is to make an app that includes source code from unit tests and when you run that app, it performs the unit testing.
06:10
And we actually have one very small such app already in the online repo.
06:21
The core repository doesn't contain any app source code at the moment. Well, there is Yani version's LibreOffice Lite still, but it's not built normally. And LibreOffice kit, as you might know, is this mostly C-like API that is supposed to be very simple to use, simpler than UNO.
06:52
And it was initially used only for loading files and saving files in possibly
07:01
different formats and used, for instance, by some conversion tools and things like that. Or Miklos, am I remembering wrong, but it was used for other things also initially. But then when this child rendering idea arrived, this was added to LibreOffice kit also.
07:30
And LibreOffice Online, as you know, it's a server client thing where the server part then has several processes.
07:41
For this, there is one master process, WSD. WebSocket daemon, it stands for. And there is one broker process and then there are for each document that is being edited, there is a separate kit process. And the kit name comes from LibreOffice kit, of course.
08:02
And it's only these kit processes that use LibreOffice kit, actually. And these kit processes are then strictly isolated. They're running their own CH roots so that you can't even accidentally see any information from somebody else's documents and so on.
08:27
And then the client in the browser uses lots of JavaScript and the communication between these, the client and the server, uses WebSockets.
08:41
WebSockets is an API that all browsers implement that allows you to send messages from the client to the server and get replies back.
09:01
And also because it was easy to do it, like to use that for the inter-process communication, it is used also for these, between these processes. Now then, if you want to combine this into a mobile app that runs standalone on
09:22
the mobile device, you need all the relevant parts of the C++ code of the server. And you need some platform-specific code that uses the mobile platform's own APIs to do whatever is needed.
09:44
And then you need JavaScript bits. The platform-specific code is actually not that much, maybe some thousand lines or so, or even less.
10:03
And the server code in C++, we actually don't need all of it in an app, only parts here and there, and there is a bit of ifdef spaghetti in there to filter out only those bits that are actually needed in the app.
10:23
But it's not too bad. And these JavaScript parts, they run in a WebKit web view. I think Android also has WebKit, yes, so it should be mostly compatible with both iOS and Android.
10:49
On iOS, these WebView things actually run in separate processes, but that is not really that visible in the API.
11:06
And then when this C++ code needs to communicate with the JavaScript code, if you compare to the online case there, there the server sends these WebSocket messages and the JavaScript receives them and replies with also WebSocket messages.
11:28
In that case, they go through this native code layer that when the native code needs to do something or send something to the JavaScript parts, it asks the WebView to execute one line of JavaScript code, typically.
11:52
That then invokes a function. And from there on, it continues as if it would have received it in a WebSocket message.
12:07
And what in the online server where you have these separate processes that send messages to each other in the app, they run mostly as different threads and instead of sending any messages through some system,
12:26
IPC mechanisms, we just have this simple buffering system where they add a message to a data structure and the thread notices that and takes it from there.
12:44
And we didn't want to do any large scale code changes to the online C++ code, so these, we call them fake WebSockets, make the code look more or less the same.
13:02
And if you want to create a similar app for other platforms, like mainly Android, the same basic idea should work.
13:20
You just need to write this platform specific code in Java instead of Objective C. And also as an example and experiment, I wrote one for GTK Plus and that was actually quite easy.
13:42
The API that GTK Plus and WebKit offers is in theory quite close to the one that you have on iOS. You can execute one line of JavaScript and you get callbacks when the JavaScript calls some specific function.
14:08
Then how is this iOS app actually built? Well, it's built in an Xcode project and all these static libraries that we build in LibreOffice Core, they are listed in a file.
14:25
And then in the Xcode project, we just pass this file to the linker and it looks through all these libraries and picks what it needs. And as we don't use any dynamic linking on these platforms, we have a map that's
14:46
built at build time that maps these UNO component names or constructor names to function pointers. And there is a Python script that generates these maps and that then decides what gets linked in basically.
15:12
And typically when you run the app and if you invoke some functionality that hasn't been invoked earlier, you will notice that now it needs some component that isn't actually linked in.
15:25
Then you need to update this Python script and link it again and then it should work. And all these configuration files, translations, all the XML files, RC files, et
15:48
cetera, they are mostly as in a normal LibreOffice in a very similar structure. And I thought I would go through the code a bit. Let's see if I can switch to the...
16:08
Oh, why didn't it switch? Oh yeah, I have to first exit this one. Well, I'll start by showing the initialization stuff where we initialize the
16:21
logging and language and locale stuff and we download some templates if necessary. And we start the kit and so on. So here am I now in this... Can you see this? Is it large enough?
16:41
Yeah? In the online repository and in the iOS slash mobile directory. And here this app delegate file is the one that does the initialization.
17:01
There is something related to templates and then the actual app starts in this function. The application did finish launching with options and it does some logging initialization and some language stuff.
17:30
And this here is related to the template feature. And it calls the LibreOffice kit initialization functions and then it starts a thread.
17:49
This is the objective way to start a thread asynchronously. And here it creates this low WSD object which corresponds to the WSD process in the online case.
18:08
And it runs it and once it has run it, it just loops back and creates one more.
18:31
When the app starts, you first see like a document browser that shows your documents in iCloud or on your device or in some other storage extension system.
18:47
And this actually is provided by the system. We didn't need to code anything for that. And then when you select the document thread, it creates a document object and a controller for that and stuff like that.
19:06
Mostly uninteresting details. And when it creates this WebKit web view, it shows in that view an HTML page that corresponds to the one that is shown for an online session also.
19:29
And it gets passed the document URL and the UI language chat. What time is it by the way? Five minutes left. Oh, wow.
19:41
And in this file, document.mm, you can also find this send to JS code. That's the one that executes this JavaScript expression. And here, this is one thing that should be improved as we can't send any binary data.
20:06
We have to base64 encode everything and then actually when we are sending a PNG file, we first base64 encoded it and then the receiving end decodes it.
20:21
But then it encodes it again to be able to use it as a data URL for the tile. And that file contains the code for the callback in the other direction, the things that receive the messages that the JavaScript part sends.
20:49
And yeah, that's actually it. So, any questions? Yeah? You mean like the code size? What should I say?
21:13
It's like some hundred megabytes. Sorry? Yeah.
21:21
And actually, how much time now? Okay. I will show you how it works. If you, if you have, does it show the whole screen? No.
21:57
For some reason not. Let's try again.
22:09
For some reason it doesn't show the whole screen or it does. So it is there. Okay. So here is the application. And this here is the file browser. Here you see what I have on my iCloud drive.
22:30
And here is a sample document. And this all looks very much like LibreOffice Online.
22:43
You get the same menu bar up there. And you can select stuff and use all this normal functionality or type into it and so on.
23:11
I probably should not point out the obvious improvement that needs to be made. But as you see, one thing is that the on-screen keyboard pops up very eagerly and that should be fixed.
23:26
And there is my dog. And I think that's a good thing to stop with. Okay. So thank you.