We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Collabora Office as an app on iOS

00:00

Formale Metadaten

Titel
Collabora Office as an app on iOS
Serientitel
Anzahl der Teile
561
Autor
Lizenz
CC-Namensnennung 2.0 Belgien:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen.
Identifikatoren
Herausgeber
Erscheinungsjahr
Sprache

Inhaltliche Metadaten

Fachgebiet
Genre
Abstract
After several years of only occasional progress but mostly stagnation, resources have now been available for work on a LibreOffice-based app for iOS. This talk will describe the history and technical aspects of the effort. Collabora Office for iOS is based on the C++ server-side code for Collabora Online (based on LibreOffice Online), and JavaScript code for the same (browser-side). Plus the core of Collabora Office (i.e. LibreOffice). Plus, finally, a rather small amount of glue code in Objective-C++. In the app all this code runs on the device, there is no server involved. Work on enabling compiling the LibreOffice core for iOS (cross-compiling, on a Mac, as all iOS development is done) was started originally back in 2011, by me. Initially this was mostly done in spare time, for fun. There was initially no clear product plan and no technical exact plans how, once the core code works on iOS, it can and would be used in an actual app. No work on any user interface. But just making the functionality needed to even load and save documents on iOS, for instance, was challenging enough, given the somewhat obscure and complex internal plumbing of LibreOffice with its long history. Some year later, for Collabora Online, the concept of “tiled rendering” was introduced by Michael Meeks. In this approach a client requests the core to render a document as “tiles”, rectangular pixmaps. The user interface in Online, in JavaScript, requests and displays these tiles and provides the user interface that send user input and edit request as necessary to the core. Also in the same timeframe, the CloudOn company needed an iOS app with the ability to load and edit office documents natively on an iOS device. We started working on making the LibreOffice core useful for that, based on my earlier work, and using tiled rendering for this, too. One major challenge here turned out to be the massive size of the LibreOffice core code, which was problematic as Apple at that time restricted the size of the executable code in an iOS app quite severely. Anyway, various workarounds were found, and this was progressing nicely, but then CloudOn was acquired and lost interest. No public iOS app code was released from that. As a test so see that the code works at least to some extent, I had written a minimal test app (without any actual editing UI), “TIledLibreOffice”, in the LibreOffice core code. Then no iOS work happened until a few years later when Jan Iversen got interested. He was at the time working for TDF. He worked on an iOS app, LibreOfficeLight, which is still present in the LibreOffice core code. He also worked on keeping the iOS functionality working in LibreOffice. Jon Nermut also helped. The LibreOfficeLight app as such worked but did not contain any actual editing functionality. Then half a year or so later, Michael started toying with the idea of re-using the by now quite substantial JavaScript UI we have in Online in an iOS app. And then a customer turned up that took part in funding it, Adfinis SyGroup. I started working on that, and it turned out to be surprisingly straightforward to get it working. Sure, there were lots of technical issues involved. Some of the iOS code in LibreOffice had bit-rotted over the years. The Online server code normally runs as multiple separate processes on a Linux machine. That had to be combined into code running in just one process (the iOS app) which was challenging. The JavaScript code in the iOS app is the same as that in Online. There are very few places where the JavaScript needs to know that it is running in an iOS app and not in a browser. All the code is Open Source and publicly available, in the LibreOffice “core” and “online” git repositories.
10
58
80
111
137
Vorschaubild
15:21
159
Vorschaubild
18:51
168
Vorschaubild
26:18
213
221
Vorschaubild
15:22
234
Vorschaubild
49:51
248
Vorschaubild
23:06
256
268
283
Vorschaubild
28:38
313
Vorschaubild
1:00:10
318
Vorschaubild
21:35
343
345
Vorschaubild
36:13
353
Vorschaubild
18:44
369
370
373
Vorschaubild
44:37
396
Vorschaubild
28:21
413
Vorschaubild
16:24
439
455
Vorschaubild
25:10
529
Vorschaubild
15:36
535
Vorschaubild
28:04
552
Office-PaketSoftwareTotal <Mathematik>Ein-AusgabeApp <Programm>Office-PaketKollaboration <Informatik>Computeranimation
CodeServerBenutzeroberflächeHumanoider RoboterSampler <Musikinstrument>Produkt <Mathematik>SichtenkonzeptSkriptspracheSystemplattformGebäude <Mathematik>HydrostatikGasströmungSoftwaretestRechenwerkRepository <Informatik>Funktion <Mathematik>Notepad-ComputerNormalvektorRechteckClientProzess <Informatik>Serviceorientierte ArchitekturTelekommunikationSpeicherabzugMereologieMensch-Maschine-SchnittstelleMereologieSoftware Development KitSkriptspracheProzess <Informatik>ClientKomponententestKartesische KoordinatenWurzel <Mathematik>Ein-AusgabeKonfigurationsraumInformationQuellcodeHumanoider RoboterIdeal <Mathematik>BrowserBenutzeroberflächeGenerator <Informatik>SystemplattformCodePixelProgrammbibliothekApp <Programm>Automatische HandlungsplanungSichtenkonzeptSoftwaretestMessage-PassingFilesharing-SystemDifferenteDateiformatBildschirmfensterServiceorientierte ArchitekturServerUmsetzung <Informatik>Gebäude <Mathematik>RechenwerkSpeicherabzugElektronische PublikationNetzbetriebssystemMultiplikationsoperatorMobiles InternetTesselationDokumentenserverMomentenproblemVersionsverwaltungNormalvektorLaufzeitfehlerBinder <Informatik>InstantiierungOpen SourceReelle ZahlMetropolitan area networkVerkehrsinformationUltraviolett-PhotoelektronenspektroskopieAppletsinc-FunktionEinsOffice-PaketMonster-GruppeLastObjekt <Kategorie>PackprogrammTelekommunikationDatumsgrenzeInterprozesskommunikationKlasse <Mathematik>Mapping <Computergraphik>RechteckBenutzerbeteiligungDämon <Informatik>
TelekommunikationCodeSystemplattformMessage-PassingMereologieProzess <Informatik>ThreadMechanismus-Design-TheorieInterprozesskommunikationPhysikalisches SystemPuffer <Netzplantechnik>Socket-SchnittstelleÄquivalenzklasseMobiles InternetHydrostatikSpeicherabzugVektorrechnungZeiger <Informatik>Konstruktor <Informatik>Funktion <Mathematik>SkriptspracheKonfigurationsraumFormale SpracheTemplateNotepad-ComputerStabNominalskaliertes MerkmalInformationNabel <Mathematik>FehlermeldungKonfiguration <Informatik>OvalExogene VariableSampler <Musikinstrument>SimulationQuellcodeBetriebsmittelverwaltungURLKorrelationZeitstempelVersionsverwaltungThumbnailHash-AlgorithmusGeradeDickeImplementierungInhalt <Mathematik>LogarithmusProgrammierumgebungSoftwaretestTaskZeichenketteStatechartBenutzerfreundlichkeitBildschirmfensterSichtenkonzeptRechenschieberMereologieServerSichtenkonzeptHumanoider RoboterCodeEin-AusgabeSystemplattformApp <Programm>GeradeCASE <Informatik>MultiplikationsoperatorProgrammschleifeZusammenhängender GraphKonstruktor <Informatik>Mapping <Computergraphik>HoaxLineares FunktionalMetropolitan area networkThreadÄhnlichkeitsgeometrieAppletFormale SprachePhysikalische TheorieProjektive EbeneGesetz <Physik>Gebäude <Mathematik>Objekt <Kategorie>VerzeichnisdienstWald <Graphentheorie>Kartesische KoordinatenRechenwerkPuffer <Netzplantechnik>MathematikElektronische PublikationStellenringModallogikDatenstrukturBitKonfiguration <Informatik>DokumentenserverProzess <Informatik>UmwandlungsenthalpiePhysikalisches SystemMAPTemplateMechanismus-Design-TheorieZeiger <Informatik>BenutzerbeteiligungBinder <Informatik>Mobiles InternetProgrammbibliothekMessage-PassingTranslation <Mathematik>SkriptspracheKonfigurationsraumMaschinencodeFunktionalFormation <Mathematik>Zentrische StreckungSoftware Development KitSystem FVerschlingung
PunktwolkeInformationsspeicherungMaßerweiterungTopologieCodeWeb-SeiteURLSichtenkonzeptParametersystemMessage-PassingBinärcodeSpezialrechnerVektorpotenzialServerMessage-PassingObjekt <Kategorie>Physikalisches SystemGamecontrollerSichtenkonzeptInformationsspeicherungMereologieBenutzerbeteiligungWeb-SeiteBrowserElektronische PublikationMultiplikationsoperatorMaßerweiterungURLArithmetischer AusdruckFormale SpracheCodeParkettierungApp <Programm>RichtungComputeranimation
MultiplikationsoperatorCodeComputeranimation
TouchscreenComputeranimation
Office-PaketLesezeichen <Internet>Produkt <Mathematik>VersionsverwaltungUnternehmensarchitekturModul <Datentyp>FunktionalBrowserNormalvektorTypentheorieSchnelltasteElektronische PublikationBestimmtheitsmaßStichprobenumfangTouchscreenMenütechnikKartesische Koordinaten
Computeranimation
Transkript: Englisch(automatisch erzeugt)
Yes, so I'm Tor Rilquist. I'm going to speak about LibreOffice or CollaborOffice as an app on iOS. I work for, or I'm a contractor for CollaborOffice activity.
First some stuff about the company, but you probably know that already. And some history of this effort to build LibreOffice for some mobile operating system.
You first need to cross-compile it and the work to enable that was started like back in 2011 already. And the platforms that it was targeted for were iOS, Android and also cross-compiling to Windows was tried.
Initially this was just mostly something done in my spare time and there weren't really any concrete plans on to get some actual product out of it.
And cross-compilation effort for Windows was supposed to make it easier to build LibreOffice for Windows, but there were all kinds of problems with that and eventually we didn't continue that effort anymore.
Then back in 2014, there was this company called CloudOn, if you remember them. They had an app for iOS that included some file sharing and whatever and they needed to be able to also edit documents locally on an iOS device.
And they funded this work for some time, but then they got acquired and their plans changed and that didn't work out very well nicely.
Around the same time, we at Collabora started working on LibreOffice Online and this introduced this tiled rendering idea where a client or, well, some code can ask LibreOffice to render the view of a document like as rectangular tiles.
Pixel maps, that is what is used in the online. We send these tiles to the client as PNGs basically.
And for iOS, we had only a very bare bones test application or app that was kept working, but really not much work was done on improving it. Then Jeanne Ivarsson started working on a new app in 2017, but that did not get finished either.
Then last year, we made a fresh start when Michael had this idea to reuse this LibreOffice Online JavaScript user interface also in a mobile app.
And that actually turned out to work quite nicely and wasn't even very hard to get working initially. And this has also been funded by Adventist. Thank you to Nicolas there.
Something about this cross-compilation of LibreOffice code. One basic thing is that we run the configure script twice automatically, one for the build platform and one for the target or host platform.
And for the build platform, that's then macOS for iOS and Linux, for instance, for Android. We only build the build time tools that are necessary to process these ideal files, for instance, and generate code.
And when building cross-compiling for iOS or Android, we don't build any dynamic libraries as we do for the desktop platforms, but only static archives.
The main reason for this originally was that on iOS, you were not allowed to even load your own dynamic libraries. I think that has changed in the meantime, but we still do it this way anyway.
And also on Android, you could load your own shared objects, but the runtime linker had some very silly restrictions and no way could it manage with the hundreds of shared libraries that LibreOffice wants to load.
And when cross-compiling, we don't do any unit testing, which of course is a bit sad, but it isn't really possible to do it. Instead, what could be done is to make an app that includes source code from unit tests and when you run that app, it performs the unit testing.
And we actually have one very small such app already in the online repo.
The core repository doesn't contain any app source code at the moment. Well, there is Yani version's LibreOffice Lite still, but it's not built normally. And LibreOffice kit, as you might know, is this mostly C-like API that is supposed to be very simple to use, simpler than UNO.
And it was initially used only for loading files and saving files in possibly
different formats and used, for instance, by some conversion tools and things like that. Or Miklos, am I remembering wrong, but it was used for other things also initially. But then when this child rendering idea arrived, this was added to LibreOffice kit also.
And LibreOffice Online, as you know, it's a server client thing where the server part then has several processes.
For this, there is one master process, WSD. WebSocket daemon, it stands for. And there is one broker process and then there are for each document that is being edited, there is a separate kit process. And the kit name comes from LibreOffice kit, of course.
And it's only these kit processes that use LibreOffice kit, actually. And these kit processes are then strictly isolated. They're running their own CH roots so that you can't even accidentally see any information from somebody else's documents and so on.
And then the client in the browser uses lots of JavaScript and the communication between these, the client and the server, uses WebSockets.
WebSockets is an API that all browsers implement that allows you to send messages from the client to the server and get replies back.
And also because it was easy to do it, like to use that for the inter-process communication, it is used also for these, between these processes. Now then, if you want to combine this into a mobile app that runs standalone on
the mobile device, you need all the relevant parts of the C++ code of the server. And you need some platform-specific code that uses the mobile platform's own APIs to do whatever is needed.
And then you need JavaScript bits. The platform-specific code is actually not that much, maybe some thousand lines or so, or even less.
And the server code in C++, we actually don't need all of it in an app, only parts here and there, and there is a bit of ifdef spaghetti in there to filter out only those bits that are actually needed in the app.
But it's not too bad. And these JavaScript parts, they run in a WebKit web view. I think Android also has WebKit, yes, so it should be mostly compatible with both iOS and Android.
On iOS, these WebView things actually run in separate processes, but that is not really that visible in the API.
And then when this C++ code needs to communicate with the JavaScript code, if you compare to the online case there, there the server sends these WebSocket messages and the JavaScript receives them and replies with also WebSocket messages.
In that case, they go through this native code layer that when the native code needs to do something or send something to the JavaScript parts, it asks the WebView to execute one line of JavaScript code, typically.
That then invokes a function. And from there on, it continues as if it would have received it in a WebSocket message.
And what in the online server where you have these separate processes that send messages to each other in the app, they run mostly as different threads and instead of sending any messages through some system,
IPC mechanisms, we just have this simple buffering system where they add a message to a data structure and the thread notices that and takes it from there.
And we didn't want to do any large scale code changes to the online C++ code, so these, we call them fake WebSockets, make the code look more or less the same.
And if you want to create a similar app for other platforms, like mainly Android, the same basic idea should work.
You just need to write this platform specific code in Java instead of Objective C. And also as an example and experiment, I wrote one for GTK Plus and that was actually quite easy.
The API that GTK Plus and WebKit offers is in theory quite close to the one that you have on iOS. You can execute one line of JavaScript and you get callbacks when the JavaScript calls some specific function.
Then how is this iOS app actually built? Well, it's built in an Xcode project and all these static libraries that we build in LibreOffice Core, they are listed in a file.
And then in the Xcode project, we just pass this file to the linker and it looks through all these libraries and picks what it needs. And as we don't use any dynamic linking on these platforms, we have a map that's
built at build time that maps these UNO component names or constructor names to function pointers. And there is a Python script that generates these maps and that then decides what gets linked in basically.
And typically when you run the app and if you invoke some functionality that hasn't been invoked earlier, you will notice that now it needs some component that isn't actually linked in.
Then you need to update this Python script and link it again and then it should work. And all these configuration files, translations, all the XML files, RC files, et
cetera, they are mostly as in a normal LibreOffice in a very similar structure. And I thought I would go through the code a bit. Let's see if I can switch to the...
Oh, why didn't it switch? Oh yeah, I have to first exit this one. Well, I'll start by showing the initialization stuff where we initialize the
logging and language and locale stuff and we download some templates if necessary. And we start the kit and so on. So here am I now in this... Can you see this? Is it large enough?
Yeah? In the online repository and in the iOS slash mobile directory. And here this app delegate file is the one that does the initialization.
There is something related to templates and then the actual app starts in this function. The application did finish launching with options and it does some logging initialization and some language stuff.
And this here is related to the template feature. And it calls the LibreOffice kit initialization functions and then it starts a thread.
This is the objective way to start a thread asynchronously. And here it creates this low WSD object which corresponds to the WSD process in the online case.
And it runs it and once it has run it, it just loops back and creates one more.
When the app starts, you first see like a document browser that shows your documents in iCloud or on your device or in some other storage extension system.
And this actually is provided by the system. We didn't need to code anything for that. And then when you select the document thread, it creates a document object and a controller for that and stuff like that.
Mostly uninteresting details. And when it creates this WebKit web view, it shows in that view an HTML page that corresponds to the one that is shown for an online session also.
And it gets passed the document URL and the UI language chat. What time is it by the way? Five minutes left. Oh, wow.
And in this file, document.mm, you can also find this send to JS code. That's the one that executes this JavaScript expression. And here, this is one thing that should be improved as we can't send any binary data.
We have to base64 encode everything and then actually when we are sending a PNG file, we first base64 encoded it and then the receiving end decodes it.
But then it encodes it again to be able to use it as a data URL for the tile. And that file contains the code for the callback in the other direction, the things that receive the messages that the JavaScript part sends.
And yeah, that's actually it. So, any questions? Yeah? You mean like the code size? What should I say?
It's like some hundred megabytes. Sorry? Yeah.
And actually, how much time now? Okay. I will show you how it works. If you, if you have, does it show the whole screen? No.
For some reason not. Let's try again.
For some reason it doesn't show the whole screen or it does. So it is there. Okay. So here is the application. And this here is the file browser. Here you see what I have on my iCloud drive.
And here is a sample document. And this all looks very much like LibreOffice Online.
You get the same menu bar up there. And you can select stuff and use all this normal functionality or type into it and so on.
I probably should not point out the obvious improvement that needs to be made. But as you see, one thing is that the on-screen keyboard pops up very eagerly and that should be fixed.
And there is my dog. And I think that's a good thing to stop with. Okay. So thank you.