Course syllabus

Utvärdering av programvarusystem
Evaluation of Software Systems

EDAA35, 7 credits, G1 (First Cycle)

Valid for: 2016/17
Decided by: Education Board A
Date of Decision: 2016-04-05

General Information

Main field: Technology.
Compulsory for: D1
Language of instruction: The course will be given in Swedish

Aim

The aim of the course is to provide an understanding for how scientific studies are conducted, and give knowledge about scientific writing and oral presentations of scientific work. More specifically, the course is about software systems can be evaluated, and how this type of evaluations should be planned, executed, and reported.

Learning outcomes

Knowledge and understanding
For a passing grade the student must

 

• be able to explain the typical steps of a scientific study, such as planning and design, execution and data collection, data analysis, validity evaluation, and formulation of conclusions.
• be able to explain how a runtime system works with respect to compilation, interpretation, garbage collection, etc.
• be able to explain how measurements are conducted on software systems, and have knowledge about dynamic and static metrics that can be defined for quality attributes, mainly with respect to performance and static metrics.
• be able to explain how measurements are conducted on stochastic processes and have knowledge about statistics such as mean and variance.

 

Competences and skills
For a passing grade the student must

 

• be able to plan and execute a study which involves measurements on software systems.
• be able to carry out execution time measurements for software that is executed with a modern virtual machine. This includes understanding the difference between start-up and steady-state execution time.
• be able to carry out measurement of static code metrics.
• be able to analyse metrics in order to decide mean value, confidence intervals, etc, and to be able to interpret these statistics.
• be able to analyse conducted studies and decide if observed differences are statistically significant
• be able to report this type of studies in written reports. This includes that the student should be able to refer to work of others in a correct way.
• be able to evaluate the quality of a written report that presents the results of an evaluation of a software system
• be able to present this type of study orally.

 

Judgement and approach
For a passing grade the student must

• understand how the design of a study affects the validity
• understand which parts of a report that are required in order to give the reader an understanding of the execution, results, validity etc.
• understand the importance of statistical data analysis before the students studies statistical theory in future courses.

Contents

 

• planning and design of scientific studies
• written and oral presentation of scientific studies
• introduction to statistical analysis, including mean value and variation
• runtime systems, virtual machines
• measurement of dynamic properties, e.g. execution time in Java
• measurement and definition of static metrics
• descriptive statistics of measurement data and hypothesis tests
• basic statistics with tool support
• report structure and language in scientific reports
• Latex, bibtex
• evaluation of scientific studies, review of reports

 

Examination details

Grading scale: UG
Assessment: Assessment is based on submitted written reports, and oral presentations. In order to be pass the course, lab assignments and the project must be approved.

Admission

Admission requirements:

• Approved laboratories or written exam in EDA016

Required prior knowledge: Relevant parts of Programming second course, relevant parts of Linear algebra
The number of participants is limited to: No

Reading list

• Material. Booklet, articles, and other material presented by the department.

Contact and other information

Course coordinator: Martin Höst, Martin.Host@cs.lth.se