We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Status Quo in Requirements Engineering: A Theory and a Global Family of Surveys

00:00
subtitles
off
playback rate
1
subtitles
off
English (auto-generated)
playback rate
0.25
0.5
0.75
1
1.25
1.5
1.75
2
quality
576p
9
 Cite
 Share
 Related Material
 Download Purchase
Logo of Universität StuttgartUniversität Stuttgart
 Wagner, Stefan

Formal Metadata

Title
Status Quo in Requirements Engineering: A Theory and a Global Family of Surveys
Subtitle
Presentation at the International Conference on Software Engineering 2020
Author
License
CC Attribution - NonCommercial 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language
Producer
Production Year2020
Production PlaceStuttgart

Content Metadata

Subject Area
Genre
Abstract
Requirements Engineering (RE) has established itself as a software engineering discipline over the past decades. While researchers have been investigating the RE discipline with a plethora of empirical studies, attempts to systematically derive an empirical theory in context of the RE discipline have just recently been started. However, such a theory is needed if we are to define and motivate guidance in performing high quality RE research and practice. We aim at providing an empirical and externally valid foundation for a theory of RE practice, which helps software engineers establish effective and efficient RE processes in a problem-driven manner. We designed a survey instrument and an engineer-focused theory that was first piloted in Germany and, after making substantial modifications, has now been replicated in 10 countries worldwide. We have a theory in the form of a set of propositions inferred from our experiences and available studies, as well as the results from our pilot study in Germany. We evaluate the propositions with bootstrapped confidence intervals and derive potential explanations for the propositions. In this presentation, I summarise the design of the family of surveys, its underlying theory, and the results obtained from the replication studies conducted in 10 countries with participants from 228 organisations. Our results represent a substantial step forward towards developing an empirical theory of RE practice. The results reveal, for example, that there are no strong differences between organisations in different countries and regions, that interviews, facilitated meetings and prototyping are the most used elicitation techniques, that requirements are often documented textually, that traces between requirements and code or design documents are common, that requirements specifications themselves are rarely changed and that requirements engineering (process) improvement endeavours are mostly internally driven. Our study establishes a theory that can be used as starting point for many further studies for more detailed investigations. Practitioners can use the results as theory-supported guidance on selecting suitable RE methods and techniques.
Keywords
00:00
Archaeological field surveyFamilyClosed set1 (number)TrailTheoryArchaeological field surveySoftware engineeringMeeting/Interview
00:15
Requirements engineeringPoint (geometry)Projective planeState of matterMeeting/Interview
00:29
State of matterMeeting/Interview
00:42
Software testingFormal grammarMathematicsProduct (business)PrototypeProcess (computing)Data modelData managementEndliche ModelltheorieTheoryObservational studyCore dumpSoftware testingParameter (computer programming)TheoryPropositional formulaVideoconferencingArchaeological field surveyPositional notationStandard deviationSet (mathematics)Distribution (mathematics)Confidence intervalWeb 2.0Sampling (statistics)Statistical hypothesis testingPosition operatorCausalityInformation technology consultingStudent's t-testOpen setQuicksortOperator (mathematics)Roundness (object)Meeting/Interview
03:11
TelecommunicationProcess (computing)Enterprise architectureFood energyTotal S.A.Data modelMixed realityMultiplication signRange (statistics)Enterprise architectureEndliche ModelltheorieSoftware developerPosition operatorBitMixed realityPlanningProcess modelingResultantMeeting/Interview
03:45
PrototypeArchaeological field surveyStructured programmingElectronic mailing listEndliche ModelltheorieProcess (computing)Constraint (mathematics)FreewareFormal grammarTime domainFunctional (mathematics)PiPropositional formulaMultiplication signDiagramData modelDifferent (Kate Ryan album)1 (number)Archaeological field surveyElectronic mailing listEndliche ModelltheorieMathematicsObservational studyBitResultantData structureTheoryConfidence intervalFormal grammarSeries (mathematics)FreewareContext awarenessForm (programming)Roundness (object)Position operatorMeeting/Interview
05:56
CollaborationismArchaeological field surveyElectric currentCausalityTheoryRight angleOpen sourcePublic key certificateMultiplication signResultantProjective planeArchaeological field surveyGraph coloringState observerWave packetMeeting/Interview
06:34
State of matterPairwise comparisonMeeting/Interview
06:46
SoftwareExecution unitTwitterEmailGroup actionCore dumpMeeting/Interview
Transcript: English(auto-generated)
00:00
Hi, I'm Stefan Wagner, Professor for Imperial Software Engineering at the University of Stuttgart in Germany. This is our journal first track talk of our paper at TOSEM, a status quo in requirements engineering, a theory and a global family of surveys. And our starting point for this whole project was that we were looking at requirements engineering research
00:24
and we saw that there are mostly isolated investigations of methods there. And especially that research is not always driven by problems from industry. And so we set out to get an empirical understanding of the state of the practice and of problems in requirements engineering. And we think this is necessary for relevant research.
00:42
So the core ideas in the design of the study that we then set up is that we've described it in these five research questions. How are requirements elicited and documented? How are requirements changed and aligned with tests? Why and how is requirements engineering improved? Is there a requirements engineering standard and how it is applied?
01:02
And what contemporary problems exist in requirements engineering and how do they manifest themselves? The last question is something that's out of scope of this video because there's a separate paper on this. It was clear from the beginning that this is not something we could do only in Germany and only once. And then we have a good answer. But we set out from the beginning that we want to do this regularly every other year
01:24
and strive at least to go worldwide. And in this paper we reported on the second run of this survey. These are the countries that participated initially. We don't have data from all of them. But we had 10 countries and 228 different companies participating in our second run that was open 2014 and 15.
01:45
And it was important for us that we not only have a survey and some empirical data, but to relate this and aggregate this into a theory. And we used the approach by Zoidberg et al. to describe it graphically in a kind of UML notation.
02:04
So we have a set of technologies like requirements elicitation techniques, activities like requirements elicitation and actors that participate like requirements engineers. And then we have propositions describing relationships, for example, between technologies and activities. This is then a set of propositions in this respect.
02:25
These are the activities for requirements elicitation from this web book. And we had corresponding items in the questionnaire. So we refrain from using hypothesis testing because we don't have a clear understanding how representative our sample is.
02:41
And what the theoretical distributions in our populations are. So we calculated a 95 percent confidence intervals using bootstrapping to reduce that problem. So we want to understand if things are commonly used and commonly used. We define then that our confidence interval is above 20 percent. So if we are 95 percent confident that more than 20 percent of our
03:03
population would give a corresponding answer, we accepted it and accepted the corresponding proposition. So how's the sample? As I said, we had 228 companies from a wide range of domains, as you see here. We had small, medium and large enterprises, a little bit of an emphasis on large enterprises,
03:25
which is, of course, probably not very representative, but it's usually where we had more contacts. And we had a mix of agile plan driven or mixed process models with an emphasis on agile development, which might actually be representative.
03:45
Some results, of course, I cannot show you a lot of results here, but if you go back to our propositions that we just saw, we only had this one that was supported already in the last run. And the other ones were new. This is what we found. You see here the percentage of people giving the corresponding answer.
04:03
Of course, they could check several of them because you can use, of course, different elicitation techniques to elicit requirements. And you also see the confidence interval here depicted as well. And we do see that all of the confidence intervals are above 20 percent. So we can accept all our propositions for the theory.
04:23
We then added explanations why we think the propositions might hold. This is based on other study results that relate to this. Answers to open questions we also had in the survey and our own reasoning where we think this might be useful. Another example, we asked for the first time how requirements are documented, so it was not in the first run.
04:45
And I want to highlight some. I think the most obvious is that we included freeform textual requirements lists, something we would expect in practice to be used a lot. Other things that we expected were, for example, data models that are documented semi-formally in something like UML or entity relationship diagrams.
05:06
And from research, we also expected that goal models are usually documented semi-formally or formally. And these are the results. It's a bit small, probably, as you read. So I zoom in and we see that freeform textual structural requirements lists are actually on the top.
05:22
So 42 percent said they use this as documentation. We also see that semi-formal, for example, UML data models are very commonly used. More to our surprise, semi-formal and formal goal models were actually at the end of our list. So they are rarely used. They are not a common occurrence in practice.
05:44
And this, of course, led to several changes in our propositions. So, for example, for the next round, we propose that goal models are not documented semi-formally or formally. And again, we added explanations for this. To sum up, we want to do, of course, more empirical research to better understand where the problems and the results come from.
06:04
By further surveys, we have a third and fourth run that we work on. But also use other techniques like interviews and observations to better understand aspects that we cannot understand by surveys. We collaborate with IRAP to include our findings in their trainings and certifications.
06:23
For the third run, we right now are still analyzing the data and preparing publications. So look out for further results from this project and this endeavor. And I hope I could convince you that an improved understanding of the state of the practice is necessary and important for our e-research to make it relevant and make it comparable.
06:46
I want to end with a big thanks to the whole NEPAYA team. There's a big group of people contributing to this research. And I want to especially mention the core team, Daniel Mendez, Michael Felder, and Markus Kalinowski. Thanks. This has been a great research endeavor.