Empirical Software Engineering
In 2002, we conducted a controlled experiment to compare pair programming with single developers. The single developers were assisted by an additional review of their program code. The main incentive of the study was to find a development technique which exploits only 20% of the cost of developer pairs but delivers 80% of their quality. Inspections are an accepted technique for quality assurance. Thus, reviews used during the preparation of an inspection meeting seemed to be a reasonable candidate. The study was conducted with 20 participants of the Extreme Programming course.
Two preliminary results could be observed. First, single programmers are nearly as expensive as developer pairs, if they have to produce the same code quality as the programmer pairs do. Second, if equal code quality is of no concern, developer pairs produce on average 7 - 13% more reliable programs with about 24% higher cost than single developers. However, both results are far from being statistically significant.
Scheduling of Software Projects
To cut development costs and meet tight deadlines in short staffed software projects, it is essential that software project managers optimize the project plan and schedule. Good software project scheduling is an extremely hard task in practice, though. The time needed to complete a software development activity is difficult to estimate since it depends not only on technical factors, but also on human factors such as the experience of the developers. Even worse, it is typical for software projects that the completion of tasks is delayed by unanticipated rework. Such rework is caused by feedback in the development process.
To support software project managers in scheduling, we have developed a simulator which represents key factors in the dynamics of software projects: rework caused by design changes, varying staff skill levels, component coupling,and changing task assignments. The simulator takes as input statistical data collected during past projects and high-level design data about the current project. In addition, the simulator explicitly takes a scheduling strategy as input. Using the simulator, a manager can compare different strategies and choose the one which he thinks is best for the next project setting.
As a first application, we have systematically studied the performance of the so-called list policies for a sample project. Our computations clearly show that the choice of the scheduling strategy has a strong impact on the progress and completion time of a project. Using the simulation traces, we have also provided a detailed analysis of why the list policies perform as observed.
Our research is funded by the Deutsche Forschungsgemeinschaft DFG.
Lightweight Software Processes
Lightweight development processes such as Extreme Programming (XP) are still widely discussed. However, there is a lack of models to evaluate the cost and benefits of agile methods. To start a comparison, the individual techniques of lightweight processes, as for example Pair Programming, have to be analyzed. During Pair programming two developers work on the same task, such that the personal cost is almost doubled as compared to conventional single developers. However, a developer pair is quicker than a single programmer and, most often, the pairs develop higher quality code.
The main question is, if the benefit of programmer pairs outweighs their additional personal cost. We developed a cost-benefit model for Pair Programming to answer this question. It turns out that the economic context to which a project is related, is the decisive factor: if the market pressure is high, Pair Programming can be profitable. However, if the market pressure is low, the cost of Pair Programming will outweigh its benefit.
Methods of Software Reliability
Inspections are a successful technique to detect defects in software documents. Inspections can be applied to all kinds of documents, such as designs, specifications, or code. Usually, not all the defects contained in a document are detected during an inspection. Thus, management must decide whether to re-inspect the document to find additional defects before passing the document on to the next development phase. As a basis for this decision, management must have a reliable estimate for the number of defects which have escaped the inspection.
Empirical studies show that the existing methods for estimating the defect content after an inspection are much too unreliable to be used in practice. The methods show extreme outliers and a high variation in the estimation error. The reason is that the existing methods do not take into account the experience made in past inspections; their only input are the results of the inspection to be estimated.
We have developed a novel method which is much different from the existing approaches. Besides data about the inspection to be estimated, we also use data from past inspections as input. Our method uses the empirical data to compute a relation between the number of defects detected and the number of defects which escaped. For the standard benchmark in the field, our new method outperforms the existing methods by a factor of 4 to 7.