What Makes Good Research in Software Engineering? - Key Points

The main objective of this research is to devise an efficient research strategy in the field of Software Engineering. Another main object of the project is to identify the kinds of contributions that have a substantial impact and the types of research that are successful.
Understanding of research strategies can help design research plan and reports in software engineering. Software and engineering research can be characterized on the basis of:

• Kinds of interesting research questions.
• Research methodology adopted.
• Criteria for evaluation of research results.

Software engineering researchers rarely write explicitly about their paradigms of research and their standards for judging the quality of the research. In 1984-85, Redwine, Riddle, and others proposed a model for the way software engineering technology evolves from research ideas to widespread practice. Software engineering researchers recently criticized common practices in the field for failing to collect, analyze and report experimental measurements in research reports. Some of the successful paradigms for software engineering research have been drawn mainly from Software Architecture examples. The following are used in scientific and engineering research fields:

• What kind of questions are "interesting"?
• What kinds of results help to answer these questions, and what research methods can produce these results?
• What kinds of evidence can demonstrate the validity of a result, and how are good results distinguished from the bad ones?

The main focus of research and the software technology evolution phases are:

Software engineering research is intended to improve the practices for software development, thus research practices should make provisions for transitions. Around 10 of the 15-20 years of evolution is spent on 'concept formulization' phase, then development and extension phases. Even more, time is spent on the 'basic research' phase but cannot be quantized due to difficulty in identifying the phase during the evolution cycle.
Understanding of research strategies must account for the accumulation of evidence over time, and for this an empirical research had also being conducted. The majority of the paper produced design and modeling results which were then evaluated on the basis of the article's text devoted to evaluation. Hypothesis testing was found to be rare in the field of Computer Science, and worse for Software Engineering research. In another evaluation, too many papers contained no experimental validations  or were with informal validations. This gave rise to a new question:
What are the characteristics of the software engineering research that the field recognizes as being of high quality?
The types of contributions made in the field of research are as follow:

• EM - Enhanced Analytical Modeling Technique:
• Based on Relevant Theory.
• Used in the identification of practical designs.
• Used in making performance predictions.
• ES - Enhanced Solutions:
• Help solve insoluble aspects of the problem.
• Help solve problems that are easier to analyze using existing modeling techniques.
• ET - Enhanced Tools and Methods:
• Used to apply analytical models.
• Used to build functional models or prototypes.

Newman created a "pro-forma abstract" that can be used to describe a research problem:

SYNTAX:                         <mode-type> <properties> <solution-strategy>

Pro-forma abstract was first to be used for:

1. Existing Data.
2. Enhanced Data.

Pro-forma abstract can be used to help:

• Identify the kind of research reported in the research paper.
• Focus on researched content.

Mainly presented truths in the research papers consisted of:

• Narrow Truth:
• It is proved using statistically sound experiments.
• Although satisfy the needs of a scientific research but are to be only a few and narrow compared to the decisions designers make on the daily basis.

• Broad Truth:
• It is an applicable solution (in most of the cases).
• It is supported only by possibly unrepresentative observations.
• It provides a pragmatic guidance.
• It involves a high risk of over-generalization.

Brooks proposed a solution to the above by introducing "Certainty Shell Structure" with the primary objective of classifying the results of the research conducted:

The key objectives of research focus on the trade of triangle.

Most common papers produced were of methods or means of software development, testing, and verification. According to conferences and journal papers there are three important aspects of an individual's research:

• Research questions:

These research questions evolved with the maturity of the field itself. 

• Research results:

• Research validation: Result itself is not enough for good research instead a clear and convincing evidence in favor of the results is also required. The evidence should be based upon experience and systematic analysis and not only on persuasive arguments or textbook examples. Following are the types of most accepted research validations:

Good research requires experimental computer science. Most common software engineering research strategy involves solving software development problems i.e. designing of a new procedure or technique with validation by analysis or discussion of an example. Another common research strategy in software engineering is the development of a way to analyze some of the aspects of software development such as the analytic, often formal, models and validating it through formal analysis or an experience with its use.