Migrating from proprietary to Open-Source knowledge management tools
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 542 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/61709 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Open sourceDigital rights managementInformation securityDifferent (Kate Ryan album)Information privacyBitStrategy gameHuman migrationSoftwareGoodness of fitIntegrated development environmentConfluence (abstract rewriting)Process (computing)Lecture/Conference
01:09
Information privacyInformation securityConfluence (abstract rewriting)Revision controlStandard deviationFile formatPoint cloudPoint (geometry)Mathematics1 (number)Open sourceServer (computing)CodeOpen setCommunications protocolINTEGRALSoftwareProduct (business)Human migrationComputer animation
03:10
Data structureINTEGRALPlanningMetadataWeb pageSoftwareExtension (kinesiology)Self-organizationAuthenticationInstallation artProduct (business)CurvatureHuman migrationWave packetType theoryIntegrated development environmentConfluence (abstract rewriting)Multiplication signBitMoment (mathematics)Physical systemDigital rights managementFile formatProgram flowchart
06:03
View (database)Computer fileComputer fontExtension (kinesiology)Gamma functionGroup actionComputer animation
06:23
Human migrationInstance (computer science)TrailUser profileConfluence (abstract rewriting)Content (media)Online helpInformationWeb pageSpacetimeAuthenticationMaxima and minimaCartesian coordinate systemDiagramDigital filterTemplate (C++)Data structureSelf-organizationComputer iconAnalog-to-digital converterServer (computing)Product (business)Confluence (abstract rewriting)SharewareInstance (computer science)Macro (computer science)Data conversionMultiplication signLatent heatProfil (magazine)Human migrationPhysical systemBitProcess (computing)Web pageProper mapSystem administratorPasswordBridging (networking)Digital rights managementMereologyStreaming mediaFile formatSpacetimeWikiMathematical analysisCartesian coordinate systemSource code
09:22
Cartesian coordinate systemUniform resource locatorInstance (computer science)Human migrationDigital filterOnline helpWeb pageServer (computing)PasswordToken ringPoint cloudConfluence (abstract rewriting)DiagramRootStreaming mediaSpacetimeTemplate (C++)Content (media)Self-organizationData structureComputer iconAsynchronous Transfer ModeOrder of magnitudeCone penetration testKey (cryptography)Human migrationSpacetimeElectronic visual displayUniform resource locatorSharewareOrder (biology)Connected spaceComputer animation
10:36
Asynchronous Transfer ModeRead-only memoryHuman migrationVirtual machineWeb pagePrice indexEvent horizonSubject indexingParameter (computer programming)SpacetimeUniform resource locatorConfluence (abstract rewriting)Mathematical analysisHierarchyScripting languageCheat <Computerspiel>Software testingLoop (music)Maxima and minimaNumberDiagramTemplate (C++)Data structureSelf-organizationContent (media)Computer iconRevision controlEnterprise architectureComputer configurationLink (knot theory)Hydraulic jumpBlogServer (computing)Configuration spaceElectronic program guideComputer fileMiniDiscRevision controlWeb pagePoint cloudHuman migrationCategory of beingLoginServer (computing)Confidence interval
12:19
Confluence (abstract rewriting)WaveTrigonometrySpacetimeStandard deviationComputer fileUniform resource locatorEvent horizonContent (media)ParsingData conversionoutputHome pageWeb pageMacro (computer science)Artistic renderingFunction (mathematics)Default (computer science)Gamma functionTexture mappingGame theoryRevision controlLocal GroupConfiguration spacePhase transitionView (database)Error messageMomentumNumberFormal verificationReduction of orderScripting languageWeb pageHierarchyPower (physics)Configuration spaceConfluence (abstract rewriting)Physical systemAuthorizationSuite (music)Field (computer science)Inheritance (object-oriented programming)Point (geometry)CASE <Informatik>MappingError messageComputer fileMathematicsMetadataRandom number generationOpen sourceLoginServer (computing)Direction (geometry)Streaming mediaComputer animation
15:06
Cartesian coordinate systemConfluence (abstract rewriting)DiagramDigital filterOnline helpInstance (computer science)Human migrationInformationParameter (computer programming)SpacetimeUniform resource locatorWeb pageContent (media)Group actionFirst-order logicComputer-generated imageryCodeHierarchyMacro (computer science)Dew pointAcoustic shadowConfluence (abstract rewriting)Human migrationWeb pageHierarchyLevel (video gaming)Computer animation
15:28
Web pageGroup actionContent (media)BlogMacro (computer science)Installation artBuildingSpacetimeConfluence (abstract rewriting)Cartesian coordinate systemHydraulic jumpDiagramKeyboard shortcutHuman migrationInformationMaize1 (number)Musical ensembleElectronic visual displayComputer iconFree variables and bound variablesComputer configurationMenu (computing)TendonGame theoryOpen sourceHuman migrationContent (media)Macro (computer science)Confluence (abstract rewriting)Web pageInheritance (object-oriented programming)HierarchyBridging (networking)Computer animation
16:08
Content (media)Game theoryMacro (computer science)Gamma functionCartesian coordinate systemInstallation artConfluence (abstract rewriting)Hydraulic jumpKeyboard shortcutBoss CorporationLogicDatabaseRelational databaseWeb pageHuman migrationSpacetimeProxy serverComputer hardwareSoftwareDiagramDigital filterMathematical analysisOnline helpSoftware developerProduct (business)Computer animationSource code
16:31
BefehlsprozessorComputer hardwareSoftwareSoftware testingJava appletRevision controlDatabaseWeb browserInstallation artRead-only memoryCategory of beingPhysical systemHydraulic jumpWeb pageMetreConfluence (abstract rewriting)SpacetimeAnalog-to-digital converterOnline helpHuman migrationBoss CorporationSphereRelational databaseContent (media)Proxy serverInstallable File SystemDiagramDigital filterKeyboard shortcutCartesian coordinate systemMathematical analysisIRIS-TLemma (mathematics)Computer fileView (database)Slide ruleFile formatExtension (kinesiology)Software bugGamma functionGroup actionConfluence (abstract rewriting)Human migrationSource codeComputer animation
16:56
Gamma functionField (computer science)Web pageBitData structureMetadataComputer configurationDigital rights managementForm (programming)
17:26
Group actionModemOnline helpMoment (mathematics)StapeldateiWeb pageConfluence (abstract rewriting)Data structureAnalog-to-digital converterTelephone number mappingElectronic visual displayField (computer science)Maxima and minimaComa BerenicesCodeStreaming mediaAsynchronous Transfer ModePresentation of a groupData structureCartesian coordinate systemMobile appData modelContent (media)Computer animation
17:57
Gamma functionSynchronizationData structureField (computer science)StapeldateiComa BerenicesQuantumCodeConfluence (abstract rewriting)Data modelPresentation of a groupWeb pageEmailOnline helpGroup actionColor managementUniform resource locatorNumberDecimalCartesian coordinate systemComputer configurationSpacetimeOpen sourceData structureCartesian coordinate systemStapeldateiField (computer science)MappingComputer animation
18:33
Uniform resource locatorComputer configurationOpen sourceNumberSpacetimeVoltmeterContent (media)DecimalSymmetry (physics)Inheritance (object-oriented programming)Physical systemGamma functionStapeldateiWeb pageCASE <Informatik>Content (media)Web pageComputer animation
19:04
Group actionWeb pageSharewareDemonGame theoryLattice (order)Menu (computing)Salem, IllinoisContent (media)NavigationOnline helpSlide ruleFile formatView (database)Type theoryForm (programming)ResultantData structureWeb pageComputer animation
19:26
Cartesian coordinate systemHydraulic jumpConfluence (abstract rewriting)Mathematical analysisSpacetimeOrdinary differential equationDigital filterAnalog-to-digital converterOnline helpWeb pageHuman migrationWikiContent (media)DatabaseRelational databaseInstallation artComputer hardwareSoftwareRevision controlCode refactoringGroup actionText editorRange (statistics)Windows RegistryElectric currentMereologyRight angleStreaming mediaRow (database)INTEGRALPhysical systemDependent and independent variablesArithmetic progressionDifferent (Kate Ryan album)Human migrationExtension (kinesiology)StapeldateiComputer configurationWeb pageOffice suiteContent (media)Confluence (abstract rewriting)Group actionMoment (mathematics)Type theoryAdditionMathematicsScripting languageDirectory serviceCASE <Informatik>LoginTraffic reportingBitData conversionMetadataComputer fileMacro (computer science)QuicksortMappingRevision controlExterior algebraComputer animation
26:12
Link (knot theory)Confluence (abstract rewriting)Mathematical analysisSpacetimeDiagramDigital filterOnline helpHuman migrationWeb pageWikiHierarchyIntranetType theoryMetadataStreaming mediaComputer animation
26:35
Program flowchart
Transcript: English(auto-generated)
00:05
about migrating from proprietary to open-source knowledge management tools. I'll talk a bit in general about the migration process and then demonstrate two migrations, one from Confluence and one from SharePoint with different technologies just to see what's possible on our side and in general.
00:25
So first, why migrate from proprietary to open-source? I'm sure everyone has lots of good reasons for that in mind, but some that we have identified as well would be first for privacy and security concerns. Of course, with open-source software, you have more flexibility in where you host your data,
00:42
how you host it, the environment and access that you set up. Two other concerns that are related and we have also seen some important examples in the recent years are vendor and data lock-in. So when you're using a proprietary software,
01:00
you're a bit more vulnerable to the vendor than when you're using an open-source software in the sense that if they make any significant strategy or pricing changes, you may find yourself in a situation where you need to migrate quickly or you need to quickly adjust to that change. A recent example is Confluence.
01:21
Maybe some of you have stumbled across it. They had a cloud version and a server version and a data center, but that's usually for larger teams. The server version was used by smaller teams that wanted to host the data on their premise. And in late 2020, Confluence decided to stop the server version.
01:44
So because of that, a lot of small companies had to either move to cloud, so host their data in the States, which wasn't possible for a lot of people, or move to data center, which was much more expensive and difficult to handle. And that was an example of how using proprietary software
02:02
can make you quite vulnerable to their changes. One other thing that is important is the data lock-in. So using open-source software usually allows you to migrate easier and integrate with certain tools due to open standards and protocols and formats. So if you're using an open-source tool at some point
02:20
and then you want to migrate to another, it may be easier than migrating from proprietary tools to other proprietary tools or open-source ones. Then, of course, accessibility, having access to the code and all the features may allow us to further extend it or contribute to it and implement other features.
02:41
And finally, from a values or ethical point of view, when we are using open-source, we facilitate the integration and technological growth for everyone rather than focusing on products and standards that just have a small benefit or a benefit for one company.
03:02
Okay, so with all these good reasons, I hope, in mind, I'm going to talk about how to approach migration. So in general, as a plan, when we migrate from one knowledge management tool to another,
03:20
but this can be kind of extended to other software as well, we first need to think about data format and dependencies. What type of data do we have? Do we have flat HTML pages? Do we have structured pages? Do we have any metadata tags? All that needs to be considered. Then we also need to look at the other extensions that we're using. What type of authentication do we have?
03:40
Are we integrating with anything else that needs to be kept? Then once we have that listed, based on that, we can identify and set up the new software. For example, let's say that we have identified that on the Confluent side, we can export to XML. Xfiki supports that and we want to do this migration.
04:04
We need to set up the new software in the environment that works for us. Then we need to make the migration plan and clean up. This is, of course, quite specific to the software that you're migrating to and from. But in general, it's also interesting to note that it can be a good moment to clean up your data.
04:23
For example, if you have a system that you have been using for 10 years, you may have a lot of obsolete data that can be cleaned up at this moment. So aside from the plan of migrating, you can also eliminate some pages or even reorganize the content if possible. Then, of course, we need to execute the migration.
04:43
Again, it depends on each organization or company, if they executed with a team, if they executed with other services. But this is an important step that also needs to be planned and realistically planned in time as well to say so.
05:03
When we migrate itself, we also need to try to map the data and its structure. As I said previously, if you have some type of structured data, you will want to map that. Also, you will want to create the new integrations or dependencies or install them.
05:20
And then finally, once that is done, we need to do a thorough testing, of course, before releasing the new system. Finally, delivering in production and a step that is quite commonly skipped would be the user training. If you have any sort of organization, you may have been in the situation where you had to change the software
05:41
and people may have been resistant to using the new software or became a bit less efficient or didn't really know how to handle certain aspects. A bit of user training can be very helpful as well if you're changing your knowledge management tools.
06:00
Now we know the general plan and we can see two examples in action. So for the Confluence Tweekzviki migration, I'm going to demonstrate the XML export way. And for the SharePoint, we're going to see an example of CSV export. So let me...
06:22
Okay, go to this one. Okay, so on the migration from Confluence Tweekzviki, we did have... So we already had a lot of tools available to migrate.
06:40
However, in recent years, we've dedicated a bit more efforts to trying to make them as easy to use as possible. We had before the filter streams converter that also supported the Confluence format. Nowadays, we have an integrated migrator that has a couple of steps.
07:00
We'll see them right away. And for the macros, so in Confluence, we have some macros like we have in other wiki systems or knowledge management tools. One concern while migrating were the macros that had to be supported
07:22
or migrated properly. So because of that, we also developed a package of kind of bridge macros, how we call them, or macros that were identical to what exists in Confluence and that would support migration. Of course, we don't have all the macros that exist in Confluence
07:40
because both Ixviki and Confluence are products that have been around for a really long time and they have their own specificities. But that is a concern to keep in mind, not only when migrating from Confluence to Ixviki but from any software. Again, this fits into the dependencies or applications that you're using part.
08:03
Okay, so let's see how a migration works. Again, I'm going to use the integrated one, the integrated migrator, which reuses the original filter streams. So what we did for this migrator itself to make it, again, a bit nicer to use,
08:21
we created this concept of a profile in which you can basically import each space separately and you'll have a bit of an analysis on what pages were imported, if you had any pages that caused issues and you're able to kind of follow the process a bit better. So you can name your profile however you want.
08:42
If you want, you can also connect to the Confluence instance. This is not mandatory, but if you connect to the Confluence instance, it also gives you a pre-analysis of the pages that were already imported into Ixviki, so that can be useful. If you're having Confluence cloud,
09:02
your username and token would be a username and the token that you get, the API token, sorry. And if you're running Confluence server, the username would be your administrator username and the token will be your password. So we have here a, of course, a demo purpose Confluence instance.
09:23
It's not what we use internally. And the token, I just saved it here. Oops, sorry. Okay. And then we need to put the URL as well in order to map the URL schema.
09:42
So let's take that as well. We don't have a custom URL path. This is important, again, for the URL schema and for the patient. If you have Wiki in the path, that will be a standard URL.
10:00
And if you don't, if you have things like display or something else, that would be a custom URL. In this case, it's not applied. And then we need to mention the name, the key of the space that we're migrating. In this case, I made this dummy text demo space called F-O-S. So this is the space that I want to import.
10:21
Okay, so now I have my profile. Let's see if I'm connected. Yes, I can start the migration. And the way in which the migration works is that you have a couple of steps. The first one is just a prerequisites. It would tell you what would be the requirements for migration.
10:41
They usually apply for larger imports. In our case, we're just going to do like a seven megabyte zip, so it's not that large. We don't need to adapt everything. But of course, in general, when you're running a migration, you need to have enough resources on the machine, enough memory, disk space, and all that.
11:01
And on the, specifically on the Exfiki side, you can also disable notifications in the Exfiki properties file. And you can also disable some listeners if you know that you will be importing very large packages. Okay, the second step that I told you about previously with that analysis, if you are connected to the Confidence instance,
11:23
you can see if you have on Exfiki any pages that already exist. So that if you have in the package that you're trying to import from Confidence, pages that exist on Exfiki. So we have some logs here. We can see that it looked at all the pages. And we don't have on Exfiki pages
11:42
that we're trying to import right now. In the third step, it's just to tell you to go to Confidence and export. It depends on what server version or Confidence or cloud version you have. In this case, it's a cloud version with XML, full or custom export.
12:04
You can choose again between those two. I already have it downloaded. So I'm not going to download it again. And at this step, you just have to import your export file. So let me show you the example to import.
12:28
Or if you have it on the same server, you can also specify the source in the server. So if you have Exfiki running on the same server that you have the files in,
12:42
you can also specify directly. All of this configuration is the filter streams configuration that you can adapt. It has some fields that are prefilled, but there's also a lot of power in other things that you can configure. For example, you can also import users.
13:02
And you can do user ID mapping. For example, if you have an LDAP that has generated on the Confluence side some random number IDs, and you want to map those to the new users that you have created on Exfiki, that's something you can do. Also, you can choose if you want to keep the author
13:21
and history metadata and so on. So you have some nice configuration that is quite granular. Once the configuration is done, you would import. And this is the point where our documents are getting created. And because I configured it so, we also have the history.
13:41
For example, here you see this was created and then updated because on the Confluence side, I had multiple changes on those pages. And now we see that we have the pages imported with no error. With no error is quite...
14:02
It's, of course, a great thing, but you can also have errors, of course. In our experience, the most common errors are caused by unsupported characters or corrupted pages on the Confluence side. So if you are trying this out and you have some errors, the logs should tell you
14:21
what is the page that is causing the issue, and you can then fix it on the Confluence side and then re-import or fix it manually next week. Whatever suits you best. Okay, so now we have the pages imported. This is a post import fixes check that we can also perform
14:41
in case we have pages that were imported that don't have a parent or pages that have corrupted parents. Both in Confluence and Xfiki, we have the hierarchy system. In Xfiki, we have nested pages. And in Confluence, you may have situations where the parent pages are corrupted.
15:01
So if you would have had that, you would see it in these reports. It's not the case here. And finally, we would need to recreate the hierarchy that we had in Confluence. You can see now that the pages that I have imported are flat. So we have just one level hierarchy here.
15:20
So now I'm going to execute the nested pages migration tool that we also have at Xfiki. And the pages will be moved into their parents according to the hierarchy that they had in Confluence. As you see, it's converting all the pages and they will be moved in the right place.
15:43
Okay, cool. So now we have a migration done. You can look at the pages to see all of your content. You also have, again, a lot of the macros that are also installed and can be reused. For the macros,
16:02
the pro macros that I told you about, the bridge macros on our side are packaged. So they are open source. They are public here if you want to check them out or repackage them. On our side, they are packaged under a license
16:20
to be able to further support the development of the product. But if you want to check them out or contribute to them, you can see them on our Git. And yeah, we have here a Confluence migration done very quickly and without much hassle.
16:41
Okay, we saw the Confluence migration. Now let's see the SharePoint one. So the way in which we migrate from Confluence is based on the XML export. From SharePoint, it's very different. In SharePoint, you have the option to export to CSV.
17:01
So if you're using SharePoint as a knowledge management tool and you have your documents with a bit of metadata, so like we have in this case, department could be considered metadata or structured data, a field that you can check or uncheck and change. And the pages have a form structure.
17:23
So if you have this type of data, one thing that you can do is to export to CSV, then create the same data structure on XWiki. So on XWiki, we have an application that is called App Within Minutes that allows you to create structured data systems.
17:41
Here I already have an example made, but we can look at the structure. Basically, I just created the same structure that I had in the SharePoint example. So title, department, reviewed, and finally the content of my documents. And then once I have that structure done,
18:03
I can use the batch import application. Sorry, not here. Okay. And with the batch import application, okay, I would import the CSV
18:21
that I have just got from SharePoint. And I'm able to map the columns from the CSV to fields in XWiki that I have just created. Here's the mapping that I just did before. You can choose whatever you want, even exclude some columns if it's the case.
18:41
Then we preview the mapping, and this is what they would look like on the XWiki side. So you can see that all the content is getting migrated. Let's just see a page. Here you can say what you want to happen if you have duplicates.
19:04
And then we do the import. And the final result is something like this. So all the pages were imported. And if you go to a page, you can see that you have this structured form type. And you can further edit it.
19:20
Okay, that's all for the two examples. Sorry, I had to go through them very quickly. There are a lot of things that you can do to migrate. And of course, we're very happy to facilitate any migration from any other proprietary tool to get more users to open source. Thank you if you have any questions.
19:49
No questions? That clear? Yes. I have one question. Yes, please. How would you deal a migration from basically just a directory with all of these documents?
20:01
So how do we deal with migration from directory old Office documents? So two things that we can do. If you have... So when you import Office documents into XWiki, we do have abundant integration with LibreOffice that allows you to convert Office pages into XWiki pages.
20:24
But that's page by page. Or if you have any sort of directory of Office files, what you can do is to actually create manually this type of a CSV where you put in a row the content. And in this way, you can also add some metadata, for example, if you want to organize them in some departments
20:41
or responsible persons, so on. You can do that and then still use the batch import. At the moment, we don't have an existing tool for just feeding some files. We have something in progress also with batch import. But yeah, the one option is to either convert them one by one or use the batch import,
21:01
but you would still need to organize them in a list. That answers it. Yes, please. My experience with migration is that the technical migration is the easiest part. It's all the clean-up and the preparation by the users
21:21
before the migration that is the tough part. And I see two things. One is really clean-up, like deciding what to migrate and what to migrate, and the other is improving the quality of the data. Because the situation you often have in business is that there's a backing metadata, so you want to enrich the documents in the migration.
21:43
Is this something that your tools can use? So adding extra metadata or adding metadata values where they are missing, for example. Yeah, so thank you for the question. Just to repeat it, if it's the case, the question is if we facilitate in any way
22:02
the addition of metadata or the clean-up, I would assume. So on the metadata, as just mentioned now, for the Office part, if you have Office documents in any way, you can adapt that CSV file before migrating. So for example, if you have Office files,
22:22
or if you have an export from SharePoint, but not all documents have metadata, you can add them manually in the CSV that you do. On the Confluence side, not that much. You can, of course, so the labels and everything are imported,
22:42
but to be straight here, it depends more on what you have on Confluence. Because basically with the migration from Confluence, we just take everything that you have and put it into Xfikie. We don't really facilitate any clean-up. But we allow you to migrate labels and macros that also do reports and all that.
23:03
But for Confluence specifically, it's a bit difficult to add metadata. Sorry? For SharePoint, we also migrate pages, lists, so not only documents. From SharePoint, at the moment, we migrate documents,
23:21
so Word documents. There are other tools that we're working on with Office integrations and Microsoft integration, but at the moment, we only import documents. Thank you. Maybe you told it. What about the user's permission,
23:40
right to view a document or part of the document? Are they integrated too? Thank you. That's a really good question. So the question is for user rights or permissions. That's in the part of the dependencies or integrations that we need to mind. At Xfikie, if you migrate from Confluence, for example,
24:02
and you have native Confluence users, yes, we have the option to import them. You just need to configure that in the filter streams and you can import the users, but not the permissions. The issue with the permissions is that the systems are very different. In Confluence, you have a very different system
24:21
of access permissions compared to Xfikie. You can do that custom. If you do a script that maps the rights and tries to set up some rights, we can imagine that, but at the moment, it's not possible. It's very difficult to do it generically. The alternative or the best case scenario is if you have something like an LDAP or even an NSSO system that you have connected
24:42
to your current tool, and when you migrate, you connect that same user directory to the new tool, such as Xfikie, and you just have the users created at the first login. That's, of course, the best case scenario. It's also actually possible to migrate users with the batch import. So you can do a bit of a hack there
25:02
and import users as well. But for permissions, it's generally very complicated, and it's a case-by-case situation. You can import permissions. You can import groups from LDAP. We're also working on importing groups from Azure SSO, but permissions, it's not yet generic enough
25:21
done in our extensions. So thank you. Also a great question. If the history of additions is kept,
25:41
for the conference migration, yes, or for the XML migrations in general, yes. We do have that, and you can also see in our example here, I'm not sure if this one has enough history, but yeah, okay. So just a quick example. The history is retained, again,
26:01
if you configure the filter to do so, and if you have this history retained, you can also see the changes within the versions. So that's something very nice. For SharePoint, we don't have that at the moment because we're not taking all the metadata from the documents.
26:20
And also on other tools that support this type of filter streams migration, you may also get the history. Okay. Thank you very much. Thanks.