Software Experiment To draw meaningful conclusions from a software experiment, we apply statistical analysis methods on the collected data to interpret the results. To get the most out of the experiment, it must be carefully planned and designed. Which statistical analyses we can apply depend on the chosen design, and the used measurement scales. An experiment consists of a series of tests of the treatments also known as "Trails". Software Experimental Design Design of an experiment describes how the tests are organized and run. We can define an experiment as a set of tests. General design principles of any experiment consist of: Randomization All statistical methods used for analyzing the data require that the observations be from independent random variables. Randomization applies on the allocation of the objects, subjects and in which order the tests are performed. It averages out the effect of a factor that may otherwise be present and is also used to select sub
Empirical Software Engineering Software engineering requires a cycle of model building, experimentation, learning, and re-modeling. Researcher's role is to understand the nature of the processes, products, and their relationship in the context: They (often) use laboratory settings to observe and manipulate the variables What is the effect?, Why is this so?,... Practitioner's role is to build "improved" systems using available knowledge They need to better understand how to build better systems What is the problem?, What are the potential solutions?, What is the cost?, To what extent do they solve the problem?,... Empirical software engineering provides methods, techniques, and tools to systematically obtain relevant information. Empirical software engineering considers the systematic application of scientific (research) methods to understand , evaluate , and model software engineering phenomenon e.g.: Something is won with software development - what