Challenges and Opportunities of Open-Source Software: the case of SU2
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 60 | |
| Author | ||
| Contributors | ||
| License | CC Attribution 3.0 Germany: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/42515 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Open sourceSpecial unitary groupOpen sourceCASE <Informatik>Open setXMLComputer animation
00:17
Open sourceSoftwareDifferential (mechanical device)Partial differential equationMathematical analysisPartial derivativeMathematical optimizationNumerical analysisSpecial unitary groupMathematical analysisMathematical optimizationOpen sourcePartial differential equationMereologyNumeral (linguistics)Nichtlineares GleichungssystemState of matterXMLUML
00:39
Special unitary groupStokes' theoremInfinityDataflowAverageComputational physicsFluidGoogolPower (physics)Uniformer RaumSoftware developerUniverse (mathematics)Disk read-and-write headGroup actionData structureDifferent (Kate Ryan album)Constraint (mathematics)Dynamical systemProof theoryReliefError messageFreezingPhysicistFluidRepository (publishing)ImplementationDirection (geometry)BitEntropiecodierungSpacetimeElement (mathematics)Formal verificationPower (physics)Game theoryDampingEuler anglesMathematical optimizationValidity (statistics)Open sourceSoftware developer1 (number)Latin squareInformationDifferential (mechanical device)Figurate numberCircleVaporVideo gameMereologyDataflowData compressionComputerKey (cryptography)TouchscreenVolume (thermodynamics)Revision controlInteractive televisionCodeAlgorithmAerodynamicsProjective planeFinitismusShape (magazine)Point (geometry)Order (biology)Analytic continuationExpandierender GraphComputational scienceXML
04:22
CodeComputational fluid dynamicsCloningSoftware developerSpecial unitary groupLine (geometry)Product (business)Self-organizationDigital rights managementSoftware maintenanceWebsiteINTEGRALForestAverageMachine codeArchitectureMereologyLogical constantCloningDivision (mathematics)Software repositoryProduct (business)CodeSoftware developerEntropiecodierungDifferent (Kate Ryan album)Computer architectureLine (geometry)Direction (geometry)BitResultantSelf-organizationCore dumpPoint (geometry)UsabilityParallel portLattice (order)Group actionCentralizer and normalizerDivisorRevision controlSoftware engineeringVideo gameOpen sourceMomentumUniverse (mathematics)Software maintenanceDigital rights managementPressureMultiplication signTouch typingNumbering schemeMereologyProcess (computing)outputWave packetGoodness of fitStudent's t-testFunction (mathematics)AreaCASE <Informatik>Peer-to-peerTheoryEuler anglesCivil engineeringArithmetic meanMessage passingLogical constantContinuum hypothesisFamilyXMLUML
10:58
Self-organizationSoftwareCollaborationismInternet forumEvent horizonEngineering physicsCellular automatonInternet forumSoftware developerCollaborationismAdaptive behavior
11:26
Group actionWhiteboardMoment (mathematics)Special unitary groupSoftware developerDirectory servicePhysical systemOffice suiteOnline chatSoftware developerWhiteboardLattice (order)BitOffice suiteQuicksortStreaming mediaComputing platformPhysical systemPublic key certificatePoint (geometry)Process (computing)CodeNumbering schemeMoment (mathematics)Decision theoryVideo gameWebsiteProgram flowchart
13:30
Sign (mathematics)SoftwareWeb pageOpen sourceProjective plane
13:52
FeedbackSpecial unitary groupMultiplication signFeedbackForm (programming)SoftwareQuicksortOpen sourceInstance (computer science)Lecture/Conference
14:57
Software developerSpecial unitary groupLine (geometry)CodeINTEGRALGroup actionRevision controlCodeRight angleForestSet (mathematics)Point (geometry)Online helpFunction (mathematics)Computational fluid dynamicsEntropiecodierungXMLUMLComputer animationDiagram
16:38
Point (geometry)Projective planeSoftware developerCycle (graph theory)Protein3 (number)BootingLecture/Conference
17:26
Software developerSpecial unitary groupLine (geometry)CodeFundamental theorem of algebraSurvival analysisAnalytic continuationProjective planeTerm (mathematics)Multiplication signCodeComputing platformPolar coordinate systemRight angleComputer animationDiagram
18:29
Data miningStandard deviationCodeRight angleOpen sourceBlock (periodic table)Video gameTerm (mathematics)AuthorizationLecture/Conference
19:01
Open sourceSpecial unitary groupGoogolCodeXML
19:33
Error messagePhase transitionSoftwareLattice (order)Software developerProjective planeLecture/Conference
20:18
Software developerCodeLine (geometry)Special unitary groupProjective planeInformationLattice (order)SoftwareDifferent (Kate Ryan album)Open setSoftware developerAreaError messageForm (programming)Theory of relativityComputer animationDiagram
21:31
TelecommunicationLevel (video gaming)Dimensional analysisLecture/Conference
22:10
INTEGRALProduct (business)InformationComputing platformCategory of beingTelecommunicationMultiplication signSet (mathematics)Cone penetration testCodeComputer animation
22:49
Lecture/Conference
Transcript: English(auto-generated)
00:01
Okay, I'm here to talk about the challenges of opportunities of open source, and I will particularize it for the case of SC2. So I guess maybe someone in the audience may know about SC2, many won't. So first question that comes is what's SC2? So SC2 is an open source collection of C++ and Python tools for the analysis of
00:26
partial differential equations and also for the optimization of PDE-based problems. And it's using state-of-the-art numerical techniques and also joint technology for optimization. So to explain a little bit more what all of that meant, let me go back to how we started.
00:46
So we are an open source project, as I was introducing. So we started at the Airspace Design Laboratory at Stanford University, so in the group of Professor Juan Alonso and Dr. Francisco Palacios, who's now at Boeing, working.
01:02
We are a competition of fluid dynamics code. So some of you may know OpenFoam. We are actually living proof that there is life outside of OpenFoam in the open source community. We are mainly focused on compressible flows at the beginning, particularly Reynolds-Abert
01:20
Navier-Stokes flows. And we have some starter capabilities, so finite volume and structural multigrade. The big important key point for SC2 is that from scratch it was formulated for design. So it included the first versions, included a continuous adjoint formulation intended for
01:41
aerodynamic shape optimization. So we have some examples here. What happened is on the initial steps, SC2 first released was in 2012, version 1.0. And we have a 2013 conference paper that details the first implementation. Back then the code was called Stanford University and Structure, hence the SC2.
02:03
And this conference paper has already over 250 citations, I checked them last week, which for a conference paper I would say is quite a few over six years. There was another SC2 paper in 2014 on the validation and verification.
02:22
In 2016 we have the journal paper where the name is already SC2. How did we get to the name of SC2? Well from 2013 to 2016 a lot of things happened. What happened was that SC2 was released under LGPL 2.1 on the repository that you see on
02:44
the screen. And a lot of groups started contributing. So some examples I'll give now. So the chair of propulsion and power at TU Delft decided, okay, we want to contribute in turbo machinery and non-ideal compressible fluid dynamics.
03:04
The chair of scientific computing at TU Casus Latin, which is where I sit now, contributed with a joint based on algorithm differentiation, also acoustics, and a lot of other capabilities. Then there was a group from the Poltanglio Milano that said, okay, we also have expertise
03:22
with NAFCD. We will also work on rotorcraft icing, many other features. A group at Imperial College, which is actually where I did my Ph.D. thesis, started contributing on fluid structure interaction. And I have some other examples. So a group of University of Liège, they wrapped the code using SWIC, so it's accessible
03:45
from Python. A group at the University of Twente from Professor van der Weide implemented high order discontinuous alerting into SC2. Laboratory of new concepts at ETA Brazil, so we're expanding. implemented the DDS solver, and there were many others.
04:04
What happened is that at the last developers meeting that we had last month in Italy, we were all of those groups that I just presented, and many others, including the ones that you see on the screen, and we have here a picture. We chose a pretty nice place, I'll say.
04:21
So what happened, I was saying, we went from a one institution code to an open source project which has around 10,000 repo visits per month, around 1,000 clones per month, and around 400 active forks.
04:40
That of course means the lines of code grow. So version one in 2012, it was under 50,000 lines of C++ code. We are now almost on 200,000. After 2016, you can see we started having more momentum, we started having some annual meetings, and they are actually getting faster.
05:02
We're also talking about having user meetings next year. So some additional idea of how we are growing, well, that was number one was the first developers meeting, number four was the last one. You can tell that our community is growing slowly but steadily.
05:22
Also, you can tell that we need to work a bit more on parity, but as a result, SC2, I can say it's a global product by now, and that has some consequences. So first thing, there are many agents involved. We have a lot of organizations.
05:41
We have academia and we have industry. We have a lot of people at industry, different companies that are interested in SC2, that are using SC2, or are actively developing. We also have a lot of users and a lot of developers, a big community of developers. All of this needs management. So if we take it from SC2 as a product, there are a lot of aspects that need to
06:05
be managed, from administrative issues, getting resources, dealing with legal aspects, marketing, and a lot of technical management, so maintenance. How do we handle 200,000 lines of code, training, support, and also defining strategic
06:23
directions or new features. So regarding the strategic directions of new features, as I was saying earlier, we have a lot of factors involved, and a lot of factors that have different interests. So for example, industry wants a fast solution, they want a product that is easily integrated into their processes.
06:42
Basically, they want a reliable product. Academia, they want a product that gives them the chance to publish. So of course, everyone wants to leverage on the community expertise. Everyone wants something with the latest methods. But what everyone wants at the end in academia is to be able to publish.
07:02
Users say, I want a product that is easy to use. I want a product that is well documented. But basically, I want something that is free and that fits my needs. And the developers say, okay, I want my life to be easy. And I also want to develop my career. I also want to get help from more experienced developers.
07:25
So I've talked a bit about the opportunities. We have gained a lot of momentum. They're having a lot of publications going on, a lot of happy groups with all the features that we have, and a lot of people that are now using SC2. But of course, we have encountered challenges along the road.
07:43
So far, most of the development of SC2 has been carried out at universities. What this means is in many cases, our developers are PhD students. There's always a pressure to implement novel methods, to get it fast, get results. Code quality might not be a priority for a PhD student, and I have been there.
08:04
Also, developers are not necessarily experienced software engineers. So we have aeronautical engineers, main core of developers, but we also have mechanical engineers. We also have civil engineers, or I should say civil engineer, which is probably me.
08:21
We have mathematicians. We have a lot of community who are not necessarily experienced developers. What happens is there's very little motivation to do some non-glamorous work, or non-publishable work, which includes documentation, which takes up a lot of time, includes code usability.
08:45
So state-of-the-art features might not be obvious how you use them for everyone, includes code architecture. So as there are more features coming and coming into the code, we need to be able to manage them correctly. We need to make sure they are compatible with each other.
09:01
It involves code maintenance, so there are a lot of features like input, output, parallelization. That doesn't give you a paper, but it requires constant maintenance, and it requires involvement to get code reviews done on GitHub. Also there are some career challenges for developers. So there's little, if any, public funding to do maintenance of open source codes.
09:29
So if you're an experienced developer, so you started your Ph.D., you knew nothing, now you're an experienced developer, you want to forward your career, you can go in the industry, and an industrial partner might want you because you have insight of the
09:44
code, but they might not be so keen for you to release your contributions. On the other side, if you want to follow the academic career path, what you will need is to be able to publish as much as possible, so you might not have time to spend on input, on output, on a lot of things that are necessary for the code.
10:03
So what happens is that continuing developers, at the end of the day, are investing big parts of their personal time in maintaining and supporting a C2. Another challenge that we have encountered is that we have a lack of centralized infrastructure. What it means is, how do we define our strategic direction?
10:23
At the beginning, when I first started in the code, we got in touch with the core developers at Stanford, which were at that point the central point of contact, but now there are a lot of groups contributing. How do you define the code priorities?
10:41
Who decides what gets integrated or not? Also there's no clear point of contact. So who do you reach out if you are interested in developing? And support is very complicated to give a volunteer best. So how we have approached this problem?
11:02
We have recently created a foundation, a nonprofit mission-driven foundation, which is called the SAG Foundation. The idea is to promote global software development and education to increase the pace of innovation in engineering sciences, and also provide a forum, a neutral forum for community collaboration.
11:21
So we're mission-driven. This is what we're targeting to do. So as I was saying, it was recently, very recently incorporated last month, and it was proposed as a nonprofit entity in California. The governance of the foundation will be carried out by the board of the directors, which are five at the moment, and the directors will not just take the decision
11:45
from everyone, but what we will do is basically to empower the community to keep contributing to the code, and also to decide on the future, working through groups and committees. So we really want to have the community engaged into the foundation processes.
12:04
And finally, we will undertake the activities that we find appropriate to further or follow the mission. So this is a little bit how we have approached this problem. So a bit of the board of directors and how we want to define committees and how
12:25
we want to define our way forward. And then finally, as I was saying, there's a lot of non-glamorous work, and we want to see how we can encourage volunteer development or how we can encourage the developers to
12:43
really engage. So some proposals on last month's meeting were to issue certificates or awards or to gamify this, to define a point system with a top developers list, for example.
13:02
Some other, another challenge is to interact with developers efficiently. So we had some additional proposals, which were maybe to have some office hours where all the developers are available and newer developers can directly ask questions to them, or to record and live stream the meetings that we have as developers, or to provide
13:24
some sort of social platform. The big question here is that we really don't know how we want to do this. So all we have done so far is we have put together a splash page where people can register, and we want to really leverage on the knowledge of the RSC community and also
13:45
on the knowledge of all the other open source projects to help SC2 thrive from now. So as I was saying, if you have any questions, please go ahead. And I would really like to hear some feedback. Thank you.
14:06
Thank you very much. We have a lot of time for questions, so don't hesitate to ask one. Thank you. You've asked what you can do to incentivize so-called non-publishable work.
14:25
And I'm wondering in what respect do you think that the software, which is open source, is a non-publishable work? Is it something you can perhaps tackle through better workflows, through implementing some sort of software citation or publication workflow for at least some of the instances
14:45
and forks of the software? So yeah, that's a very good question. So what I mean by non-publishable work is, of course, and I will go back to what – the idea here is we get to a point in which what ends up being a publishable
15:05
work is a new feature, right? So you have a CFD code that does its job, and you add a new feature, that new feature is published, and that new feature, you can get a paper out of it. But that new feature needs to merge into the other features. Then somebody else does another feature, and they don't really match up.
15:26
There's someone that has to do that work. There's someone that has to decide how you integrate things, especially if you want this to keep growing and not collapse. Because of course you could integrate things on standalone versions, but that at the end
15:43
will just be a set of in-house codes from each of the developing groups, and no one will really benefit from this, because you will have a feature, another feature, they are not compatible. Another thing that I will consider non-publishable work is you – of course, you have a CFD
16:02
code, it throws a lot of output, right? You add new features, you need to take care of that. Maybe you need to completely rework the output. That work may take you six months, but you cannot publish a paper on a new output for the C2 solver, basically, because the reviewer will just tell you, well, this is not new.
16:22
This is just an output. So is this kind of work that the one I was considering as non-publishable work? I don't know if that answers your question. Does anybody else have questions, otherwise – okay. Any other questions?
16:43
Okay. Sure if it's needed, but you mentioned that most of the development has been done through research, I guess, through academia, through – I guess through from one research project
17:02
to the next to the next to the next, and I guess you are at the point in which those research projects will end, and then there's the big question. If we run out of funding, what happens with SU2? So that's a very good question. So exactly the cycles, you can more or less tell that we are already at the third cycle
17:26
of projects, if you want to call that. So of course, there's a big question. You have a research project, you hire two PhD students, maybe a postdoc, that people finishes their roles and they go somewhere else. You have lost all of that.
17:42
So exactly that's what we're trying to tackle here, being able to somehow give continuity over the research projects which are at three, four years' time. So something I forgot to mention, of course, is the foundation. That's one of the goals of the foundation, right? Being able to somehow provide a platform that people can do some extra work for the code,
18:08
and eventually maybe get paid out of that, because that's fundamental work that is needed for the survivability of the code in the long term. So I don't know if that answers your question.
18:22
I guess it's kind of difficult to tell them you should have someone employed who sustains the helps to maintain the code in the long term.
18:41
Of course, yeah, that's very difficult to convince a company to put someone to provide a long life to an open source code because they will see it as, well, then somebody else will benefit from what I'm paying, right? So just to give an example, here I have Tom as the fourth author.
19:03
He's at Bosch Research and Technology. So he is in one of these first papers. He's actually leading the journal paper. He did his PhD in Stanford on the code. So he finalized, continued there, and now Bosch has hired him, and he's working mainly
19:22
on SC2. But it's difficult to convince a lot of companies to do that, of course. Okay, further questions? Yep, thank you.
19:44
And creating a foundation to support this kind of software in the long run is a good idea, I think, and I want to know where you drew your inspirations from, how to organize and how to incentivize the further development, like from other projects, what has worked
20:04
and what has not worked. So we're just starting. So actually we're on the trial and error phase still. So we thought, or the foundation is something that was first mentioned at the first meeting
20:20
which was back in 2016, and it only got incorporated last month. So of course, there's a lot of trial and error, a lot of things that are involved, a lot of, of course, building up a foundation needs some investment. There are a lot of things around it. How we can incentivize people, that's actually one of the things that we don't know and
20:43
we want to learn about. We want to see how people can be incentivized or what we can do to incentivize people. So the answer to that is really with, I don't know. Well, let me be more specific. I mean, like, I think there are also other, maybe not research related projects who have
21:04
created a foundation to support their software in the long run. And whether you have looked at that and what they did to incentivize developers. Yeah, of course. I mean, we have based ourselves on a lot of foundations that are there. There's also for, I was mentioning earlier, OpenFoam.
21:22
They have a foundation, Linux has a foundation, Eclipse has a foundation. There are a lot of different approaches out there. So we're trying to gather the information from as much as we can from all of them and then try to come up with our own approach to the problem. But we're still in the very early stages. So if you have actually any idea on that, it will be great to learn.
21:45
Maybe we're doing the conference dinner. A last question? Someone? Actually, yes. Okay. Just a small minor question.
22:01
You mentioned that you're using Splash for communication? No, I haven't mentioned what we're using. I think, so we very recently, and Tim there, who is also an author, can give some more information. We have very recently agreed with RocketChat to have some platform.
22:23
So they have kindly told us that we will be able to use their platform for some time. But yeah, we are still setting up the channel of communication. So we tried it a couple of months ago. We liked it, and we're trying to go towards that.
22:41
But we also, of course, we have the code on GitHub. So there's a lot of issues discussed there. Okay, let's thank, again, the speaker.