Improving Software Engineering experimental knowledge to improve decision making in development projects

This project aims to improve the technology selection process used by organizations. To be able to select one of two competitive technologies, we first need to characterize these technologies, know their strengths and weaknesses and their applicability limits. This project will draw on the discipline of Empirical Software Engineering to find the best way of doing these tasks.

The key goal of this project is to improve the procedures in place in the software development community for characterizing the technologies in use and help industry to select which technologies are best for each application setting. This will enable industry to improve decision making on the type of technology to be used during development (which techniques to use, which tools, etc.).

To do this, it is necessary to improve the way researchers run and report experiments about those technologies, targeting these activities towards the use of experiment results in the software industry. The discipline that deals with experimentation on software development is called Experimental Software Engineering and dates back to the late 70s. By experimenting at different levels (with pilot or real projects, in different settings, etc.) information can be gathered about the causal relationships between the different factors involved in software development. Nowadays those experiments are generally done by independent experimenters checking different and no related objectives related. This avoids the definition of a solid empirical body of knowledge built upon the results of consistent and related experiments about software development. On the other hand, there are any guidelines that help experimenters to report the results of their studies in a way that can be useful neither to other researchers to keep on working on the experimental chain, nor to software practitioners that can use the results of those experiments.

Therefore, the aim of this project is systematizing experimentation on Software Engineering by dealing with three issues:
* Definition of guidelines for reporting experiments
* Definition of procedures for replicating experiments
* Definition of mechanisms for aggregating/combining experiment results.

On one hand, those achievements will clearly improve the state of the practice. On the other hand, industry decision making about the best technologies for each setting will benefit from experiments being properly run and reported.

Decision-making; Experimental Software Engineering; Aggregation of Experiments, Experimental Packages

The consortium is looking for an industrial partner, which could be either a big or a small organization, interested in improving its decision-making process about what type of technology to use in development projects.

The industrial partners will need to achieve the following tasks:

* Regarding guidelines for reporting experiments: read some of the experiments the research partners will
re-write following the guidelines developed in the project to check if the information you get from those well-written experiments is useful to you in making decisions about what technology to use.

* Regarding replications: as in other disciplines, experiments in SE need to move from the laboratory (experiments at university with students) to field studies (experiments in industry). Industrial partners will need to replicate one or two experiments that the universities will have performed. As part of the experiment, industrial partners will need to have some practitioners (preferably, 10, 20 or more) perform a development task (for instance, test code) in a controlled way (we will take the measurements) over one or two days. Another option for industrial experiments (called case studies) is for industrial partners to apply the techniques researchers have experimented on in the laboratory in industrial developments. The consortium will decide which type of experiment industries are to perform when we get to that project workpackage.

* Regarding the combination of experiments: university partners will gather items of SE knowledge from the combination of experiments. Industries will check how those items of knowledge should be used in decision making.

Sofware development companies

No