Collabora Office Android app gory details
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 490 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/47302 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Android (robot)Office suiteMobile appAndroid (robot)Computer animation
00:21
Android (robot)TouchscreenRevision controlLinker (computing)Constructor (object-oriented programming)Component-based software engineeringBootingProcedural programmingMachine codeWeightTouchscreenLinker (computing)State of matterAndroid (robot)Connectivity (graph theory)Library (computing)Computer animation
01:47
Entire functionIterationUsabilityFile viewerWeb pageMobile WebWeb pageTouchscreenMobile appMultiplication signSlide ruleElectronic mailing listCartesian coordinate systemComputer animation
02:38
Machine codeMereologySoftware development kitTheoryFunctional (mathematics)Coordinate systemWeb applicationTesselationPixelComputer animation
04:22
View (database)Machine codeWeb 2.0PrototypeMachine codeTesselationMereologyComputer animation
05:05
Android (robot)Artistic renderingMachine codeWeb 2.0Software development kitWeb browseroutputMachine codeRevision controlView (database)TouchscreenMobile appMilitary baseType theoryComputer animation
06:43
Android (robot)Machine codeBuildingComputer configurationConfiguration spaceModulo (jargon)Core dumpLimit (category theory)Entire functionTelecommunicationMessage passingFunction (mathematics)Integral domainComputer fileAsynchronous Transfer ModeView (database)Computer-generated imageryInterior (topology)Data storage deviceHypermediaRational numberKeyboard shortcutFile formatCrash (computing)Computer fileMobile appAndroid (robot)Machine codeConfiguration spaceSet (mathematics)Letterpress printingMereologyStudent's t-testMiniDiscProjective planeGastropod shellFunctional (mathematics)TelecommunicationMessage passingMobile WebComputer fontComputer configurationWeb 2.0Multiplication signRevision controlTouchscreenInterface (computing)Shape (magazine)DemosceneoutputState of matterCrash (computing)Software bugMachine codeMedical imagingInsertion lossDifferent (Kate Ryan album)Computer animation
13:54
GoogolPatch (Unix)Android (robot)Core dumpException handlingFile formatFiber bundleHacker (term)Android (robot)MereologyFiber bundlePatch (Unix)Mobile appMultiplication signIterationLevel (video gaming)Exception handlingCore dumpRevision controlBridging (networking)Different (Kate Ryan album)Office suiteSinc functionData storage deviceFlow separationNetwork topologyComputer animation
16:03
IterationGoogolMobile WebData storage devicePlastikkarteControl flowMachine codeVirtual realityWindowFunction (mathematics)GUI widgetContent (media)PlastikkarteMereologyGUI widgetMobile appPoint cloudData storage deviceInteractive televisionOcean currentSmoothingInformationComputer animation
17:47
VideoconferencingDemo (music)Demo (music)Group actionMathematicsSelectivity (electronic)Mobile WebTable (information)Fitness functionCategory of beingType theoryRow (database)Computer animation
19:23
Game theoryVideoconferencingDemo (music)Android (robot)outputMachine codeConfiguration spaceCategory of beingSource codeTemplate (C++)Moment (mathematics)Structural loadLengthFilm editingLipschitz-StetigkeitTemplate (C++)Chaos (cosmogony)State of matterAsynchronous Transfer ModeBitCuboidMultiplication signComputer fileComputer animation
20:59
CollaborationismAndroid (robot)Mobile appSoftware developerNormal (geometry)Android (robot)Revision controlWeb 2.0MereologyVideoconferencingMachine codeoutputGodCodecMultilaterationMultiplication signBitMoment (mathematics)Computer animation
25:55
Point cloudFacebookOpen source
Transcript: English(auto-generated)
00:05
Thank you so much for being here for this presentation, which will be about the Collabar office Android app and what is inside. So, but before we can see it and I can show it the details of that.
00:22
Let me first get into the history, because as you can see, like it's a quite a lot of history in this, like the work on this started more or less in 2011. And like the first thing with LibreOffice, because you can imagine like it's a huge code base and getting
00:41
it to Android meant like first to get something on the screen, which was pioneered by Michael and Thor. So thank you for starting this at all. And it was like incredible amount of effort that was needed for this, because like, for example, the linker just allowed like 96 libraries to be loaded at the same time.
01:06
And of course, like LibreOffice itself had like several hundred and it used some component stuff so that like it was all loaded on demand. And like, you know, all this had to be overcome to do the state that actually like something got on the screen.
01:26
The debugging of this was a nightmare. Like the stuff was not ready for the native debugging at that time at all. Like you had to upload something to the device, put some weights into the startup procedure and, you know, all these nightmares.
01:46
But they succeeded. So they've got something on the screen, which like you can see tablet inside. There was the screen of LibreOffice. But of course, like this is not what you expect from the from the mobile app. So like it was unusable in this state, but it was possible.
02:04
So the next step was like trying to get some whole screens, whole pages on the screen, which seemed like a good approach at the time. So it got it to the application that like had some list of document pages.
02:27
And you were able to switch in this, but scrolling the document was like impossible because like, you know, you were able only to switch it like slides. So the next step here was to use actually the LibreOffice kit.
02:44
So LibreOffice kit that came from the need of the mobile and web applications is actually some kind of API for LibreOffice that allows you to simply access the functionality of LibreOffice from C or C++.
03:07
And it allows you to draw parts of the document into tiles. So like you split the entire documents into like 256 to 256 pixels tiles, and then like you can request them.
03:22
You can request them like by absolute coordinates. So like the theory is that like when you just want to fetch something from the document, you say, OK, well, I need it from over here, over there. And then you have some like composition that like composes that into something that looks like the document, which was great.
03:45
It functioned nicely. And like later it was extended to actually like that the editing worked too. It meant that like all the stuff had to be updated so that like when the user actually typed something into the document, like
04:08
that the part where the user actually typed is invalidated and new tiles are actually fetched for this area, which was done by Miklos. So this is what we had in 2015.
04:24
The problem of this was that, oh, didn't I skip something? I did, yeah. The important part of this was that actually like the compositing of these tiles into to form something like the document was done in Java.
04:48
We have used code or Tomas has used code for this from Mozilla, which had this Fennec prototype at that time. And so it kind of worked.
05:02
But the problem was that like it diverged from what the code was doing for the web. So, of course, the LibreOffice kit API was extended over time so that like more and more features were added there. So that like you were able to type concurrently in the web browser
05:25
and also like have there more functionality exposed like through the dialogues and everything. But like maintaining these two code bases was like very inconvenient. Like you would have to be on par with the features like that were developed for the web.
05:42
You would have to like do it just once again in Java for the mobile part as well. And so the idea that Tor was implementing for the iOS was actually like do everything like in a web browser, even in the app.
06:06
So basically in the background have the LibreOffice and LibreOffice kit. And then like have a full screen web view where you are running the JavaScript that is like normally used in the web version.
06:24
But use that on the actual device as well with all the cross compiled code in the background and all these things. And it worked for the iOS. So at some stage, like I tried to do it for Android as well.
06:43
And yeah, it had some problems as well, of course. So first of all, the old WSD has to be ported to Android. Like luckily it was not that hard because like it was pioneer for iOS as well.
07:01
It was using the same native code that like is compiled for the old like Fennec based app that is written like entirely in Java.
07:20
And but there were things that had to be done a bit differently. So first of all, like I wanted this to be built in Android Studio so that like the debugging is easier. And like it is more standard for like anybody who comes from the Android world and sees this for the first time.
07:46
So in the online Git where the like web version lives, I've created the Android sub there and like introduce the appropriate configuration options and everything.
08:02
Then edit the stuff that like had to be built natively. But for that I've created in CMake so that like Android Studio actually understands this and it is possible to build it directly from there.
08:21
Luckily, the debugging part of this worked pretty well. So now like you can debug this completely like directly from the Android Studio. You can set the breakpoints even for the native part and all these things work nicely. If you want to know the details like how to set it up for this, just read the Android slash read me.
08:42
It is described there. And yes, so the minimal app was created. So like the WebView over the entire screen and some kind of communication between the WebView that is there and between the native part that is like behind the scenes.
09:04
So we had to introduce like two ways of communication. So one is from JavaScript to native. And that is like reasonably easy on Android. Like when you have the WebView, you just add JavaScript interface and then you can run it easily.
09:27
Like this low message handler dot post mobile message and like you get it from the JavaScript to native.
09:41
But for the native to JavaScript, like it's a bit hackish because like you have to use this JavaScript prefix that actually runs like one JavaScript function. So basically, like you encode your entire message, which can be quite large.
10:03
Like it can be the entire the entire tile and like encapsulate it that way. And and you will it will be called in JavaScript and then like processed by the by the online part actually itself. I was concerned about like how this is going to be performing, but it is not that bad.
10:24
Like it is not showing that much in the profile, so it's not terrible. Then, like when this minimal app was was created, like lots of functionality has to be had to be ported from the from the old app that we had because like it had the it had the shell that showed like what are the what are the files on the on the device.
10:51
It had to be done so that like there are some assets that that come with the with the native part that like have to be present on the device so that like things work.
11:04
Some of these assets have to be some kind of unpacked so that like they behave like real normal files on the device because like we depend on many many like third party projects like font config and
11:22
these things and lots of these have some configurations that just have to be like read directly from the disk. So so like it had to it. They cannot be as assets because assets are very special on Android too. So like they have to be copied first at the first start and then can be used later.
11:42
Then some like important things like displaying the license and the notice file and of course settings so that like you can you can tweak the configuration of the of the thing. Then we had the Summer of Code student during the summer or the last summer who added lots of things that are expected on the on the Android device.
12:11
So some print support slideshow support which exports the the slideshow into SVG and shows it as a full screen device.
12:22
Inserting images again like you have to like start a separate activity that that allows you to choose the file and and provide it back to the document so that it is inserted lots of useful stuff. And of course lots of bug fixing. So one of the thing that that was a huge challenge was actually lifecycle of the thing because because like the
12:49
Android lifecycle goes through like many steps and it is a bit unclear like in what state like how the how the native part is supposed to behave.
13:02
Like when you are actually supposed to to tear it down when you are supposed to to leave it like in the background because like you know that that you will return to some state very very soon and stuff like that. So so like we had lots of crashes there luckily it seems to be in some reasonable reasonable shape just now.
13:24
Then it was necessary to debug some pieces like the startup time. It turned out that like we are starting about a minute just because the font config was parsing the huge note of fonts that are on the device.
13:44
And and so like we had to overtake it some way and fix other crashes at some other stuff. And then we tried to publish in Google Play. But just at the time like when we were ready or was nicely done Google Play like came up with a new policy
14:09
and that was that when you have a native app in the app store it has to have 64-bit version as well. And so far like everything was 32-bit so we had to make the low WSD compilable on 64-bit
14:25
which needed a poco that that we that we actually use as like the main dependency for the online itself. We had to port it to 64-bit luckily like it needed just few patches but still it was a bit annoying.
14:47
Then deliver office core itself again like it is ported to ARM 64 for a long time but on Android it behaved a bit differently. So some things were like necessary to be fixed in the bridges which which is the like very low level part of this.
15:07
So that first it compiled it all on Android and then and that it ran because like the exceptions didn't work there. So we had to do some hacks that were like already done for iOS but like not used in general.
15:21
So it had to be changed there. And of course the online build now had to count on actually having two trees of the of the compiled native stuff. So one was the 32-bit the other 64-bit and create some Android app bundle that is then uploaded to
15:42
the Google Play and Google Play like when it serves to the to the devices like repacks that into the actual apks. And then like the users actually served either with the 64-bit version or with the 32-bit version according to their device.
16:03
So finally we've got it there and we've got through several iterations since then. So again like more pieces from the old app were ported. So for example support for SD cards or for cloud storage.
16:21
Currently next cloud is supported hope to have more in the future. Then many improvements interaction and the biggest thing that was there was the was the mobile desert. We call it this way because we don't have a better name but it's basically the the stuff that you see see down there.
16:42
So some details in there. So thanks to Kylon for some design advice here. We were able to actually actually add some dumping of the of the content of the widgets that are that are in the dialogues or in the sidebars.
17:06
So we are able to dump it as JSON and transfer it to the to the JavaScript part. And based on that like we are able to to display the information that is either in the dialogues or in the sidebar some more convenient way for the users.
17:25
So like what what you see in the picture is the sidebar that you would see normally in LibreOffice. But like presented in in JavaScript rendered in JavaScript. So it works nicely you have all the things like a smooth scrolling and and interactions and all these things.
17:48
So if it works I can hopefully show you some demo. Yeah. So this is like what you get when you start it on the on the on the device.
18:08
Editing. Yes. Was started. OK. Select something. If you are more lucky than me with my fingers then you will do it on the first attempt.
18:24
Change some properties. You can insert some chart here. Sorry chart not chart but table. So then you can see the the mobile wizard in action here. Like change the columns and rows.
18:42
You can type something into the into the table. And if it doesn't fit you can resize the table. So that is more convenient like there are the special handles done for this for all this editing.
19:07
Yeah. More scrolling. And that's it I think we are at the end.
19:26
OK. So some future steps will be more to do. Of course more profile and optimizing. But at the moment like the most most terrible problems I think are are sorted out.
19:40
But still of course like more to do as usual. At the moment from time to time you could see that some document just doesn't doesn't load. But I fixed that yesterday. So this is all. Basically what was going on is that some people with some like a bit strange low chaos setting for their lip length tag was not finding the data.
20:11
So and it's it just like then didn't load and presented some like file not found found dialog box update the documentation mode.
20:24
So currently we just like copy the document from some temporary state into the new document and create from that. Like we would like to to use some real templates here for the for the documentation which is possible but not finished.
20:41
Of course lots of paper cuts here and there. For example the most visible at the moment is smooth scrolling. So like the inertia. So that like when you move with the finger quickly that like it goes on and before it reaches somewhere something. And yeah that's it. I think so if you want to get involved you are most welcome.
21:05
It is not that hard to set up. Unfortunately like you first have to cross compile the LibreOffice 3. But then the rest is like reasonably easy reasonably familiar to people who have who has done have
21:21
done some some Android development and many thanks to people who who contributed directly to the Android port. But of course like lots of other people have contributed to the to the online. Collabora did most of the work in the online there too.
21:40
So thank you so much. So that's it. Any questions please. What's your approach to synchronize releases between online.
22:10
Yeah. Yes. So at the moment like the the Google Play releases are a bit more frequent than the releases on the normal online because like we have some additional fixes that are like directly in the Android part that are not not general.
22:29
But of course like because all this code or code is shared. So if we improve something for the Android and it is in the JavaScript part it will improve the experience
22:42
even for people who are using the web version the web version normally like from the but on the mobile device. So like they will see lots of these things there improved as well. But like currently I think it is more often than than the normal normal online releases.
23:04
Or the code. Maybe it is like similar as the code and from time to time there's like one release in between. So something like that. Like in the recent past it was like every two weeks. Sorry.
23:25
And with iOS. So iOS is a bit different in the releases because like in the test flight we updated frequently. But the normal app is now based on the 4.0 still.
23:43
And we still have to update some stuff so that like we can we can like base it on 4.2. So like the official release of the iOS app will update a bit later. But currently like if you are the test flight user you will see like similar pace of the updates.
24:07
Thank you for the question. Any other question. Yes please. I imagine there's no video support on any of the mobile devices yet. No video support. No video support. I don't think there is.
24:25
But you need to negotiate codecs and God knows what. OK. Any other question.
24:42
Yes.
25:38
OK.
25:44
Any other question. Then thank you so much.