LibreOffice Online adoption into 1&1 Mail&Media ecosystem
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
Title |
| |
Subtitle |
| |
Title of Series | ||
Number of Parts | 490 | |
Author | ||
License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
Identifiers | 10.5446/47260 (DOI) | |
Publisher | ||
Release Date | ||
Language |
Content Metadata
Subject Area | |
Genre |
00:00
GMX MailEmailSource codeArchitecturePhysical systemWordElectronic mailing listSheaf (mathematics)2 (number)Presentation of a groupOpen sourceHypermediaProjective planeComputer animation
00:30
EmailHypermediaPoint cloudIdentity managementMereologyService (economics)HypermediaIdentity managementPoint cloudEmailComputer animation
00:56
Revision controlOcean currentXMLComputer animation
01:20
Point cloudEmailComputer fileHypermediaWorkstation <Musikinstrument>Cartesian coordinate systemComputer animation
01:54
Text editorPoint cloudOrder (biology)Computer fileLanding pageComputer animation
02:22
Text editorComputer fileLocal ringEmailText editorXMLProgram flowchart
02:43
Text editorWorld Wide Web ConsortiumDirectory serviceComputer fileNumberOffice suiteMaxima and minimaOcean currentCartesian coordinate systemView (database)Structural loadComputer animation
03:22
GMX MailText editorEmailTemplate (C++)Computer filePresentation of a groupSpreadsheetSheaf (mathematics)Template (C++)Computer animationProgram flowchart
04:01
Template (C++)GMX MailRevision controlWorld Wide Web ConsortiumTemplate (C++)Revision controlComputer fileLocal ringComputer animation
04:34
ArchitecturePhysical systemLanding pageData storage deviceText editorExecution unitPoint cloudInformationConnectivity (graph theory)Variable (mathematics)Integrated development environmentText editorPhysical systemModule (mathematics)DiagramTelecommunicationSystem administratorData storage devicePoint cloudOffice suiteClient (computing)Set (mathematics)Theory of relativityDigital rights managementStructural loadInstance (computer science)Level (video gaming)Function (mathematics)Cartesian coordinate systemComputer fileService (economics)Computing platformPresentation of a groupFront and back endsMereologyProxy serverOrder (biology)Domain nameRevision controlFile formatComputer animation
09:13
AbstractionSet (mathematics)Multiplication signAlgorithmInstance (computer science)Utility softwareOrder (biology)NumberCross-correlationService (economics)Stability theoryElectronic mailing listDefault (computer science)Semiconductor memoryBefehlsprozessorScaling (geometry)Computer animation
11:26
BefehlsprozessorRead-only memoryScale (map)Repeating decimalBefehlsprozessorUtility softwareServer (computing)NumberThresholding (image processing)Semiconductor memorySet (mathematics)Latent heatMetric systemComputer animation
12:39
Thresholding (image processing)Scale (map)Read-only memoryNumberOrder (biology)Event horizonInstance (computer science)Semiconductor memoryThresholding (image processing)Utility softwareElectronic mailing listTotal S.A.Metric systemCASE <Informatik>Basis <Mathematik>BefehlsprozessorOcean currentComputer animation
15:17
Metric systemFile formatRead-only memoryScale (map)Thresholding (image processing)Instance (computer science)NumberProduct (business)Computer animation
15:37
Metric systemFile formatDatabaseData storage deviceMetric systemFile formatLatent heatDescriptive statisticsBuildingOpen sourceInstance (computer science)Process (computing)Visualization (computer graphics)View (database)NumberComputer fileTime seriesGraph (mathematics)Computer animation
17:30
Electric currentSoftware development kitMetric systemMultiplication signProcess (computing)Instance (computer science)Order (biology)NumberSoftware development kit
18:32
ArchitectureExecution unitConvex hullSystem administratorBitTelecommunicationTransport Layer SecurityFluid staticsMessage passingSoftware repositoryDigital rights managementMathematicsContent (media)Crash (computing)MultimediaInstance (computer science)Text editorProjective planeFile formatMetric systemComputer fileNumberConnectivity (graph theory)Open sourceOrder (biology)Library (computing)INTEGRALStructural loadWeb applicationCross-site scriptingComputer animation
21:36
GMX MailPatch (Unix)NumberTerm (mathematics)Computer configurationWeb portalOpen setMultiplication signBasis <Mathematik>Computer animation
23:31
Slide ruleGMX MailOpen sourcePoint cloudFacebookDiscrete element methodComputer animation
Transcript: English(auto-generated)
00:06
So let's say again, the presentation is structured in three areas. First, we'll see where and why we are using LibreOffice Online. The second one is a more technical section describing the solution, the deployment and monitoring of LibreOffice Online.
00:22
The second section will go through a list of contributions that one-on-one made open source for LibreOffice project. First, a couple of words about one-on-one media. One-on-one media offers email cloud and identity services for more than 30 million active users in Europe and America.
00:46
One-on-one media is part of United Internet and it is composed from four brands, JMX, WebDE, One-on-One, and Mailcom. You can find LibreOffice Online in three out of four brands. You can find in WebDE, JMX,
01:01
both international.com and .NET, and Mailcom. Current version of LibreOffice Online that you can find in this portal is 6.2, but we are in the middle of upgrading the version to 6.3. Now that we know where we can find LibreOffice Online, let's see why.
01:22
As I said earlier, one-on-one mail and media offers cloud and email solutions. So most probably we'll have files in the cloud and attachments to the emails. Due to this, we've needed a way to preview the files quickly and to edit them without leaving the portal.
01:40
So without using an application like Microsoft or having to install LibreOffice on our workstation. The solution created using LibreOffice Online to preview and edit files is called Online Office Editor.
02:00
This is the landing page of the application, which offers some complimentary features like creating files from predefined templates, previewing your most recently edited files, browsing through your cloud documents in order to open them for edit.
02:20
And one of the most, the first features from Online Office Editor upload from local. This feature allows you to select a document from your local workstation, upload to Mail and Media Cloud and start editing it through LibreOffice Online. So after we select the file, click open,
02:42
the file is automatically added to Mail and Media Cloud in a specific directory. And the view of the application of Online Office application will change and will look like this. As you can see, the file has been added in the left sidebar in the list of open files.
03:01
Currently we're for a maximum number of file documents for an user to edit at the same time. And in the center, Nightframe has been open, which loaded LibreOffice Online load affiliate component, which allows us to start editing the file. You can find the same look and feel
03:20
both in JMX portal, here we are editing a spreadsheet file, or in Mailcom, here we are editing a presentation file. Besides upload, I want to briefly mention one feature that allows the customers to quickly create files
03:42
from a predefined templates and start editing them through LibreOffice Online. What we're mentioning is that all these predefined templates are in ODF format, and currently they are available only for German brands, like JMX and WebD.
04:00
So this is what I want to present for the first section. If we summarize, there are two important ideas to remember. First is that you can find LibreOffice Online in JMX, WebD, and Mailcom brands, or you can find the 6.2 version of it. And the second idea is that you can leverage
04:21
upload from local or templates feature to quickly create files or to upload files to Melee Media Cloud, and start editing them through LibreOffice Online. Going on to the more technical part of the presentation, we'll present the system solution
04:40
of online office editor application, while focusing on how we've deployed LibreOffice Online in Kubernetes, and also how we can monitor LibreOffice Online instances in Kubernetes using Prometheus and Grafana. Online office editor is composed from high level modules,
05:02
as you can see here in this diagram. We have the front-end components, which is the landing page, custom component from online office editor. As I said earlier, this component allows the users to create documents, to quickly browse
05:24
to the most recently edited files, to browse through cloud documents in order to edit them, or to upload new documents in cloud. So offers some features complimentary for those of editing.
05:42
When we select a file for edit, Lola Fleet component is open in a separate timeframe, which allows us to start editing the file. These are the front-end components. Moving on the back-end side, we have a middleware, which acts as a proxy
06:02
for HTTP and WebSocket requests between the front-end clients and LibreOffice Online components. It is also responsible for authentication, load balancing, and session management. We have, of course, the LibreOffice Online component
06:22
that allows us to edit files. And at the last, we have the storage adapter component, which implements the WOOP arrest specifications, and facilitates the communication between LibreOffice Online and cloud. This module allows us to upload the files edited
06:42
from LibreOffice Online through Mele Media Cloud, and to download them from Mele Media Cloud in order to be edited. What's important is that all online office editor modules are deployed in Kubernetes, which allows us to scale the modules of the application
07:02
based on our needs or based on the traffic that our application receives. Let's move forward and see how we can do this, how we can deploy LibreOffice Online in Kubernetes. First of all, Kubernetes is an open-source platform
07:23
which manages containerized services. One easy way to deploy a Docker container in Kubernetes is through Helm, which is the package manager for Kubernetes. And Helm uses a packaging format called charts, the Helm charts.
07:42
As you would see in online repo, the Helm charts are a collection of files that logically describes a related set of Kubernetes resources. LibreOffice Online Helm charts can be found in LibreOffice Online GitHub, online GitHub, under Kubernetes Helm.
08:03
If you want to test this Helm chart, we have all the needed information in the README in online GitHub, and I will iterate quickly on them. Firstly, of course, you need a Kubernetes cluster. If you want to test this locally,
08:21
the best, the easy way is to install Minikube, have all the needed information in this link. You will need the Helm, you need to install the Helm. You will need a version which is newer than 3.0. Optionally, you can customize the deployment
08:42
with some environment variables that will be passed to LibreOffice Online. You can customize these variables in values.yml. And for example, you can customize the credentials used by the LibreOffice Online admin console, or you can add the wup domain host in this file
09:02
that will be passed to LibreOffice Online. After that, you can install the Helm chart using this command, and the output of the installation will look like this. So firstly, the LibreOffice Online Helm chart will create a deployment called LibreOffice Online.
09:20
This deployment will create a replica set. Replica set purpose is to maintain a stable number of Kubernetes pods at any given time. So as you can see here, the replica set from LibreOffice Online Helm chart is supposed to create, by default,
09:42
three LibreOffice Online pods, and keep these three pods available at any given time. The list of pods can be seen here. As you can see, the correlation between the replica set and the pods is made through the replica set ID.
10:02
You can see here that the pod name contains the replica set ID. So all three pods were created by this replica set. Besides deployment replica set and pods, one brief mention to say,
10:20
pod is the smallest abstraction in Kubernetes. For us, LibreOffice Online pod contains the LibreOffice Online container, which is created using the official Dockerfile from online GitHub. Besides all these resources, we have a service
10:42
which defines a policy in order to access the pods. I think by default, it uses a round-robin algorithm in order to balance the request to LibreOffice Online pods. And one important resource that we at 1&1 Melamedia,
11:02
we are using it, is the horizontal pod autoscaler, which allows us to scale the number of LibreOffice Online instances based on some needs. The default LibreOffice Helm chart scales the number of pods based on CPU and memory utilization.
11:23
Let's talk about this feature in more details. And how the horizontal pod autoscaler works in the official Helm chart that we released in online GitHub.
11:41
Firstly, what's important to say is that Kubernetes cluster comes with a metric server, which is a cluster-wide aggregator for data, for resource utilization data like CPU and memory. And horizontal pod autoscaler will check, will query this metric server to a metrics API.
12:05
For the CPU and memory utilization for a specific set of pods. When it detects that a specific threshold is met, a specific threshold that we are setting in the horizontal pod autoscaler,
12:20
like you can see here, the threshold was 70% on memory and CPU. When it detects that this threshold is met, it will instruct the deployment, which it will instruct the replica set to increase the number of pods. Here we can see better the example, the use case.
12:44
So in our use case, we set the memory threshold to be 70%. When the horizontal pod 70% from the total amount of memory that we limit the pods to access from our Kubernetes cluster.
13:02
When it will detect that the threshold has been met, it will advise the replica set to increase the number of pods. In our case, from three to four. Here you can see the event from the horizontal pod autoscaler.
13:22
The event with the new value of the number of pods, four, and with the reason, memory resource utilization above target. You can see the target and the current memory utilization here in the horizontal pod autoscaler. So the target was 70%
13:41
and the current memory utilization was 80%. The horizontal pod autoscaler instructed the replica set to increase the number of pods. Here you can see that the new number of pods, the new desired number of pods is four. And actually in the list of pods, you can see that the LibreOffice Online,
14:01
a new LibreOffice Online instance is created. That will help us to adjust the traffic, the new traffic that we are receiving. What's worth mentioning is that the horizontal pod autoscaler, the resource from Kubernetes could be updated in order to use some specific metrics
14:23
that we are interested in. So not only CPU and memory. For example, we can tweak the horizontal pod autoscaler to check a custom metric that we exposed from LibreOffice Online, custom metric like number of active documents.
14:42
So when the total pods reaches a number of current edited documents like 200, it will instruct the deployment to create another instance in order to accommodate the new edit request. This horizontal pod autoscaler, as I've said,
15:03
is a very important feature from Kubernetes. And we at One and One, MeliMedia, we are using it in order to accommodate the traffic that we receive on a daily basis. Okay, we have a lot of instances
15:22
of LibreOffice Online in production. How we can monitor all of them? Because especially when the number of instances could increase dynamically based on the traffic that we receive. The solution for that,
15:45
the solution for monitoring LibreOffice Online instances was created using two tools, open source tools, Prometheus and Grafana. Prometheus is used to store metrics
16:02
and Grafana is used to visualize these metrics. Like a short introduction to them, Prometheus is a time series database that works by scraping monitoring endpoints,
16:21
processing them and storing them. And Grafana is a data visualization tool that help us to build dashboards and graphs for our metadata. What's important is that LibreOffice Online exposes monitoring metrics, which are compliant with Prometheus format on REST endpoint lol-get-metrics.
16:41
Description of these metrics can be find in online GitHub under vsd-metrics-takes-file. There are a lot of metrics, some of them are related to the global system, to the global memory available. Some of them are related to the process count,
17:02
for example, for counting the processes of lol-vsd or forkit or forkits. And there are some document specific metrics, like document upload duration, document download duration, the number of views for a document and so on.
17:21
You can find all the metrics with their description in this file in online GitHub. Based on these metrics, we can define in Grafana dashboards that will look like this. So in these three examples, we are counting the number of processes that LibreOffice Online started.
17:42
As you can see, we have the lol-vsd and forkit, which should have one process at any given time. Then we have the kits. From this dashboard, we can see, we can deduce the number of edited documents for example.
18:01
And yeah, we can construct a lot of dashboards. Grafana offers us a lot of features in order to construct the dashboards. It only depends on what metrics we are returning from the get metrics rest endpoint. Also, we can define alerts in Grafana that will alert us.
18:22
I don't know when there are two lol-vsd process instances on the container or something like that. If we sum up this section, there are three important details to remember is that you can deploy LibreOffice Online
18:41
in Kubernetes using the official LibreOffice Online Helm Chart that you'll find in online repo. You can leverage the horizontal pod autoscaler to scale the number of LibreOffice instances based on your needs. As I said, CPU, memory, or a number of active documents.
19:02
And you can monitor all this LibreOffice Online instances deployed in Kubernetes using Prometheus, Grafana, and the rest endpoint that returns the metrics in Prometheus format. Reaching the end of the presentation,
19:21
I want to iterate a little bit through the contribution that one-on-one made in the past year. What's next, our future contributions that we want to open source. So, one-on-one multimedia started to contribute to LibreOffice Online project in February
19:42
last year. Our first important contribution was related to WebSocket defragmentation or how we can manage defragmented WebSocket messages. We've open sourced some cross-site scripting fixes
20:01
on LolaFlit component. Also, we've done some library updates on LolaFlit. We've discovered and reported a CVE related to LibreLogo. After the LibreOffice Online conference from this year,
20:21
we've open sourced an improvement to support TLS for communication with storage. We've open sourced the rest endpoint for admin metrics, which allows us to monitor the LibreOffice Online instances with Prometheus,
20:41
and we've open sourced the Helm chart for deploying LibreOffice Online application in Kubernetes. What's next on short-term goals? We want to open source segmentation fault, a monitoring enhancement with a new metric for segmentation fault crashes.
21:03
Count, that will allow us to monitor the crashes inside the LibreOffice Online. Also, this will impose a change in the communication between load VSD and fork it through UniSockets. And also, we want to open source CDN integration
21:20
in order to allow the plug-in of a CDN, a content delivery network, for serving LolaFlit static files. All in all, OnlineOffice editor will not be made possible without the full contribution from open source community. So saying that, I want to really say big thanks
21:42
to LibreOffice community and to all the patches that you contribute. Saying that, I want to thank you for listening, and I'm open for questions if you have.
22:06
If not, thank you very much. Sorry. Do you think that you are the biggest deployment of LibreOffice Online? Because I never heard somebody doing it with Kubernetes before, so you must have lots of options. No, I don't know if we are the biggest, but in terms of deploying LibreOffice Online to Kubernetes,
22:22
yeah, I think we are the first. I don't know who is deploying. So how many documents are being added in your system at once? It depends on the portal. It depends on the number on the day, but we think that based on our monitoring,
22:44
we are having 10, 15,000 users, active users on a daily basis, and we can reach to 400 documents edited at the same time using LibreOffice Online. 400. 400, yeah.
23:03
But as I say, it depends on the day, on the portal, WebDA and JMix are portals with a large number of active users, so on these portals, yeah, we are having an increased number of documents being edited on a daily basis.
23:25
Cool. Any more questions? No, thank you very much.