Course syllabus

# Markovprocesser

Markov Processes

## FMSF15, 7,5 credits, G2 (First Cycle)

## General Information

## Aim

## Learning outcomes

## Contents

## Examination details

## Admission

Admission requirements:## Reading list

## Contact and other information

Markov Processes

Valid for: 2023/24

Faculty: Faculty of Engineering, LTH

Decided by: PLED I

Date of Decision: 2023-04-14

Elective for: BME4, C4-sec, D4-ns, E4-ae, F4, F4-bg, F4-bm, I4, Pi4-ssr, MMSR2

Language of instruction: The course will be given in English

Markov chains and processes are a class of models which, apart from a rich mathematical structure, also has applications in many disciplines, such as telecommunications and production (queue and inventory theory), reliability analysis, financial mathematics (e.g., hidden Markov models), automatic control, and image processing (Markov fields).

The aim of this course is to give the student the basic concepts and methods for Poisson processes, discrete Markov chains and processes, and also the ability to apply them. The course presents examples of applications in different fields, in order to facilitate the use of the knowledge in other courses where Markov models appear.

Knowledge and understanding

For a passing grade the student must

- explain the Markov property and the intensity concept, as well as the concepts of recurrence, communication, stationary distribution, and how they relate to each other,
- perform calculations of stationary distributions and absorption times for discrete Markov chains and processes,
- explain the suitability of the Poisson process as a model for rare events and perform calculations of probabilities using the properties of the Poisson process in one and several dimensions.

Competences and skills

For a passing grade the student must

- construct a model graph for a Markov chain or process describing a given system, and use the model for studying the system,
- in connection with problem solving, show ability to integrate knowledge from the different parts of the course,
- read and interpret easier literature with elements of Markov models and their applications.

Judgement and approach

For a passing grade the student must

- identify problems that can be solved using Markov models, and choose an appropriate method,
- use knowledge of Markov models in other courses, as transfer concepts, tools, and knowledge between different courses where Markov models are used.

Markov chains: model graphs, Markov property, transition probabilities, persistent and transient states, positive and null persistent states, communication, existence and uniqueness of stationary distribution, and calculation thereof, absorption times.

Poisson process: Law of small numbers, counting processes, event distance, non-homogeneous processes, diluting and super positioning, processes on general spaces.

Markov processes: transition intensities, time dynamic, existence and uniqueness of stationary distribution, and calculation thereof, birth-death processes, absorption times.

Introduction to renewal theory and regenerative processes.

Grading scale: TH - (U,3,4,5) - (Fail, Three, Four, Five)

Assessment: Written exam and compulsory computer exercises.

The examiner, in consultation with Disability Support Services, may deviate from the regular form of examination in order to provide a permanently disabled student with a form of examination equivalent to that of a student without a disability.

Parts

Code: 0115. Name: Examination.

Credits: 6,5. Grading scale: TH. Assessment: Written examination.

Code: 0215. Name: Laboratory Work Part 1.

Credits: 0,5. Grading scale: UG. Assessment: The first computer exercise

Code: 0315. Name: Laboratory work Part 2.

Credits: 0,5. Grading scale: UG. Assessment: The rest of the computer exercises

- FMAA20 Linear Algebra with Introduction to Computer Tools or FMAA21 Linear Algebra with Numerical Applications or FMAB20 Linear Algebra or FMSF20 Mathematical Statistics, Basic Course or FMSF25 Mathematical Statistics - Complementary Project or FMSF32 Mathematical Statistics or FMSF45 Mathematical Statistics, Basic Course or FMSF50 Mathematical Statistics, Basic Course or FMSF55 Mathematical Statistics, Basic Course or FMSF70 Mathematical Statistics or FMSF75 Mathematical Statistics, Basic Course or FMSF80 Mathematical Statistics, Basic Course
- FMAB30 Calculus in Several Variables or FMAB35 Calculus in Several Variables or FMSF20 Mathematical Statistics, Basic Course or FMSF25 Mathematical Statistics - Complementary Project or FMSF32 Mathematical Statistics or FMSF45 Mathematical Statistics, Basic Course or FMSF50 Mathematical Statistics, Basic Course or FMSF55 Mathematical Statistics, Basic Course or FMSF70 Mathematical Statistics or FMSF75 Mathematical Statistics, Basic Course or FMSF80 Mathematical Statistics, Basic Course

Assumed prior knowledge: A basic course in mathematical statistics.

The number of participants is limited to: No

The course overlaps following course/s: FMS180, MASC03, MASC13

- Lindgren, G. & Rydén, T.: Markovprocesser. KFS, 2002. One of the books is enough. Recommended for Swedish students.
- Mark Pinsky and Samuel Karlin: An Introduction to Stochastic Modeling. Academic Press Inc, 2011, ISBN: 9780123814166. One of the books is enough. Recommended for exchange students.

Director of studies: Johan Lindström, studierektor@matstat.lu.se

Course administrator: Susann Nordqvist, expedition@matstat.lu.se

Course homepage: https://www.maths.lu.se/utbildning/civilingenjoersutbildning/matematisk-statistik-paa-civilingenjoersprogram/

Further information: The course is also given at the faculty of science with the code MASC13.