The development infrastructure of the TYPO3 project
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 90 | |
| Author | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/40330 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 201311 / 90
2
5
8
10
12
13
14
15
17
19
21
24
25
28
29
31
32
34
36
39
40
43
44
46
50
51
52
54
55
57
58
62
65
66
67
78
79
87
88
00:00
Projective planeBitSoftware developerType theoryGoodness of fitLecture/Conference
00:28
Multiplication signProjective planeOpen sourceConnected spaceCASE <Informatik>Software developerSoftware repositoryWordPerfect groupSoftwareBitTouch typingXMLComputer animation
01:34
TwitterServer (computing)Student's t-testComputerSystem administratorSoftware developerCanadian Mathematical SocietyServer (computing)Type theoryCore dumpWordEmailStudent's t-testComputer scienceAddress spaceMultiplication signConstraint (mathematics)Process (computing)Touch typingMeeting/Interview
02:00
Multiplication signSlide ruleType theoryXML
02:13
Product (business)DataflowSoftware frameworkAlpha (investment)Source codeOpen sourceProduct (business)Canadian Mathematical SocietyContent management systemProjective planeType theoryCodeVideoconferencingMultiplication signState of matterSoftware developerAlpha (investment)Web-DesignerDataflowSoftware frameworkFreewareBasis <Mathematik>Source codeXML
03:06
Core dumpDataflowOpen sourceTranslation (relic)Source codeSoftware testingDynamische GeometrieData managementService (economics)CollaborationismTelecommunicationGroup actionInternet forumSynchronizationDirection (geometry)Electronic mailing listMaxima and minimaMagneto-optical driveExtension (kinesiology)CodeTwitterKey (cryptography)Product (business)Repository (publishing)Electronic mailing listInterface (computing)SoftwareMultiplication signSource codeCollaborationismDecision theoryCanadian Mathematical SocietyTelecommunicationGroup actionSinc functionEmailSoftware testingCategory of beingCore dumpInternet forumTask (computing)1 (number)Element (mathematics)Type theoryProjective planeClosed setMathematicsCodeHomographyData managementComputing platformMereologyProjektiver RaumDependent and independent variablesPoint (geometry)Touch typingVolumenvisualisierungAssociative propertyOpen sourceCausalityLevel (video gaming)Server (computing)Extension (kinesiology)Entire functionLine (geometry)Event horizonSynchronizationSpacetimeWikiBitSpring (hydrology)Plug-in (computing)Web browserData mining3 (number)Link (knot theory)Peer-to-peerSoftware bugRevision controlTranslation (relic)Limit (category theory)Lecture/ConferenceXMLComputer animation
09:47
CodeData typeServer (computing)Mean free pathSource codeCoding theoryElectronic program guideLine (geometry)MassExtension (kinesiology)Software testingJava appletExecution unitIntegrated development environmentTranslation (relic)File formatCore dumpInternet service providerFormal languageHuman migrationSimilarity (geometry)ParsingArtistic renderingChainDataflowElectronic mailing listInternet forumSoftware testingSoftware metricCombinational logicUnit testingCore dumpTranslation (relic)Software developerCodePatch (Unix)Virtual machineContinuous integrationProjective planeAuthorizationExecution unitElectronic mailing listFreewareOpen sourceServer (computing)Set (mathematics)Point (geometry)Extension (kinesiology)SoftwareLine (geometry)MathematicsMultilaterationMultiplication signHomographyInstallation artVotingRevision controlSlide ruleBranch (computer science)Integrated development environmentMessage passingView (database)File formatFormal languageMetric systemDifferenz <Mathematik>DatabaseMatrix (mathematics)Different (Kate Ryan album)Machine codeCoefficient of determinationWeb pageLoginElectronic signatureArrow of timeVideoconferencingComplex (psychology)Type theoryTraffic reportingCellular automatonPiState of matterComputer animation
16:20
Slide ruleLecture/Conference
Transcript: English(auto-generated)
00:01
Good afternoon everybody. The next talk is Development Infrastructure of the TYPO3 project by Stefan Gebet. So, thanks for the introduction and thanks for listening to my talk. I'm going to tell you in the next couple of minutes a bit how TYPO3 is developed and of course what TYPO3 is if you don't know it.
00:23
So, just to give you an overview what you can expect from this talk, you will learn how to set up a project infrastructure. Unfortunately, the time is a bit short to do that. Maybe to see a perfect development infrastructure in the open source ecosphere.
00:40
Well, I think we're pretty good, but we are by far not perfect. So, you will gain some impressions I think. But at least you will learn how people in another open source project work. And you can see our infrastructure that's built from 100% open source software.
01:01
Maybe you find some inspiration for your project and maybe we can just get in touch together and discuss some things and say, why don't you use this tool? Are we using that? And I already got in touch with Media Wiki guys, so that would be really nice to have more connections to other open source project.
01:21
So, us in this case is admin at typosuite.org and most of our infrastructure is public somewhere in Git repos. In doubt, if you have interest in something, just ask us, we are happy to share with you. Some words about myself. I used to be a type three CMS core developer from 2010 until like a month ago
01:42
where I decided to focus more on the infrastructure work in the server administration team and in my day to day job, I'm a PhD student in computer science in Wurzburg in the center of Germany. That's my email address and my Twitter handle if you want to get in touch with me. I'm very happy to hear from you.
02:00
Due to the time constraints, we might not be able to answer questions. The slides are already on SlideShare, slideshare.net, that's the Stephen King and I also added some comments to them. So what's type of three? The most known or the most well-known project or product of the type of three project
02:20
is type of three CMS. It's a content management system on PHP basis, has more than five million downloads and it was started in 1997 so it has a big code base and a long time of development and our community also brought two new products.
02:42
One is a PHP framework, type of three flow and if this sounds interesting to you, there was a talk by the lead developer, Robert Lemke. I guess you can watch it in the videos after this conference and also there's type of three Neos, a new content management system which is currently in alpha state based on type of three flow.
03:02
So about the persons of type of three. Type of three is community-driven open source. There is, that was the wrong key, I'm sorry. Ah, that was Twitter. I'm very sorry for that. Type of three is community-driven open source. There's no company behind only the type of three association
03:21
for financial support but it's not about paid work. It's more about enabling costs to bring people to code sprints, et cetera. We have core teams with like 10 to 25 people who you have to commit access and a lot of contributors.
03:41
So while preparing this talk, I saw the bit what requirements does an open source project have? And I identified these categories. One is collaboration and communication. The other one is source code and versioning. Then quality assurance and testing and documentation and translation.
04:01
And in the following, I will present you the solutions we're using for these tasks. So you might also ask, why are you using your own infrastructure and why don't you use GitHub? Some points about this. Of course, often people think about just migrating to GitHub but one reason would be
04:22
we have been there before there was GitHub. In the beginning, when type of three was smaller, SourceForge was used for CVS hosting. It started being hacked, sometimes it was unreliable. So yeah, in the subversion times, everything was started on own infrastructure.
04:41
The most important point, I think, is that we own our data. In the times of Git, it's not so critical because everybody has the copy of the source code. But things like all the issues which are in the issue tracker are, yeah, really worth some time and work. And we were also able to just switch
05:00
the issue tracker like one year ago because we have access to all data. And we have an own account management with all the teams. That wouldn't be just so easy to migrate to totally different platforms where we don't have backend access to. And the other thing is that we want to provide a central place for the whole community.
05:24
Nevertheless, to admit it, it costs, sorry, we open towards other services which are freely available and we use them for the not so critical things too. And one thing to admit, of course, this costs some efforts to have an own infrastructure.
05:50
So the first point, communication and collaboration in Title III. Many things are communicated through mailing list and decided also through them. And they are used since 2003
06:02
and it took 10 years to build a human accessible, for the younger people accessible interface which is a forum which was launched last Monday. And it's based on FUD forum and it provides a bi-directional synchronization of mailing lists and use groups in the forum.
06:22
So we can now also link to mailing lists, threads, et cetera. So that's a pretty nice software. And that was pretty important for us to finally set such an interface up because the decisions take place in the list and this is the transparency that the Title III,
06:41
let's say, deciders bring towards the other world and people that don't read it, they feel offended because they say, what the shit have you been deciding there and they don't see the discussion before. So that was important and on the other hand, we're trying to really avoid internal discussions. So behind closed doors or in internal lists
07:03
and try to communicate things clearly. Nevertheless, we often still fail at this, unfortunately. So another thing for collaboration is our Forge platform on forge.title3.org. We have a red mine running there and this offers project space for more than 2,400 projects.
07:25
So it's not for the top level products but also for community extensions which are the plugins for Title III CMS and also for other teams like design team or the event team. They all have their collaboration space there and yeah, one interesting feature
07:42
or one important feature I would say is a membership listing for the different teams because when you say, I want to contribute to this part of Title III, who's responsible for that? With whom should I get in touch? So the good thing with the red mine is that it lists just all the memberships
08:01
for the different projects and you can just go there and see that guys are responsible and I can get in touch with them and ask them what I can do for them. And it's the issue tracker for all the projects and that's a tiny render. So we are not so extremely happy with the red mine as issue tracker.
08:21
The search is a bit limited and especially when you are not so familiar with it. So if you have ideas how to improve red mine search, we're really open towards that. Other features are like wikis per project, a repository browser, news, time tracking, et cetera.
08:42
Search code and versioning, that's the next topic. We're using Git since mid-2000s and for one product and other products since the beginning of 2011. We're using a special flavor of Git. I will come to that very soon.
09:03
One disadvantage though, because of us admins having limited time, there's not yet official Git support for all community extensions. So many of them are still limited to subversion and we are about to change that hopefully within the next few weeks.
09:21
So search code and versioning was pretty short. Quality assurance and testing covers this topic now but again through code reviews. So Title III is doing code reviews and the motivation behind that is that unfortunately bugs do exist.
09:41
It's pretty sad. We are ensuring our code quality through peer review. So every change that has to go in there which is not totally trivial has to be peer reviewed by at least two other people. Discussing a code improves it. So when it's easy that you improve
10:00
a suggested code change, then it leads to just a better quality instead of saying, okay, we merge it and afterwards we make it clean. And so that doesn't work usually. And so we're doing the reviews before the merge. When we do it, once we have done it,
10:20
others, it never worked really. And the good thing also, when you're reviewing stuff from other people, you learn from them. You learn how they code and they can say, why don't you use just this method? It's way cleaner or way faster and so on. And although these reviews make the development transparent to the outside world,
10:43
people can just go there and say, and see why it was implemented this way maybe or why it was changed or not changed. And which brings us back to the special flavor of Git is that we're using the Gerrit code review software for these code reviews.
11:01
And Gerrit is also our Git server. Just to give you a small impression how Gerrit looks like, you can go to review.typosuite.org and you just see a list of all the pending patches, who the author is, and to what project it
11:24
refers to. Once you zoom in into such an issue, you see, again, the owner and to which branch and the state, so it's still in review. You see the commit message and you see the votings by different people. And yeah, you need some positive votings
11:42
to be able to really merge that change. There are also some more nice things. You have directly a diff view online in Gerrit and you can comment on single lines and say, please do it this way or do it that way and people can communicate and then just upload a new patch set.
12:02
If Gerrit is interesting to you and we are by far not the only project using Gerrit, I have a second talk here at 15.14 and you're invited to just come back there and see in detail how Gerrit works.
12:25
So we're also doing continuous integration and we of course have running a Jenkins server which does the thing which are just the usual choose for PHP project and interesting thing is that Gerrit automatically votes in Gerrit one.
12:41
There is a violation of the coding guidelines, et cetera. But we don't do unit tests in Jenkins. But I come to that later. Then we have metrics.typosuite.org, a big sonar installation with 5,600 projects. So all the core projects plus the extensions which are listed there
13:01
and that presents you code metrics like cyclomatic, complexity, et cetera, lines per method, and so on. So we have Travis also as continuous integration software. Travis is a hosted service so we don't have it on our own infrastructure
13:21
and it's built for the open source community and yeah, well what does Travis, other than Jenkins? It still just executes tests. But it does it in a very interesting way. First it fetches code from GitHub, only from GitHub. That's a bit problematic because yeah,
13:40
I said we're hosting our own Git server so we are replicating just to GitHub. And Travis also supports plenty of languages so it's not limited to PHP. So Travis CI just spins up virtual machines to run your unit tests so you always have a clean environment.
14:00
You don't have headaches because your virtual machine got hacked or so by unit tests. And it runs them in different environments. That's interesting thing. For example, you could say, I want PHP 5.2 to 5.4 in different databases that builds up this metrics of test combinations.
14:22
And Travis runs all of them and how would you as a single developer find out that this combination fails? Travis does that for you and it launches all different versions that you're saying it should test and then it reports back if all pass or what fails.
14:42
Documentation and translation in TYPO3. To translate TYPO3 and the extensions we're having Pootle installed. Pootle is a Python-based project. And we're using XLIF format like two years ago or so for the translations.
15:05
Every night language packages are built out of the translations which are entered into Pootle and people can go into the TYPO3 installation and say fetch translations and then these packages are downloaded. Pootle is actually pretty nice.
15:21
It also offers for everybody to suggest translation. So it's not a single team that does all the translations but everybody can just come here with a TYPO3.org account, log in and suggest a translation and team members can really set this translation active and everybody can just suggest new translations.
15:45
Documentation in open source project. Well, it was a raw point for a long time. Probably not the only reason but one reason, sorry.
16:01
Feel free to, we're using restructured text. Feel free to watch the slides online. It's a really nice format. It's built for Python initially and I'm out there for questions. Nice that you attended.
16:20
Please see through my slides and also the comments online. Thanks for attending. Thank you.