Practical Parallel Programming - Main

practica   -    project

Motivation. With the end of the ever-increasing clock frequency, parallel processing has become the only option to expand processing power. Moore's law now applies for the number of on-chip processors. The course focuses on how the software developer can exploit concurrent programming.

Goal. The student will understand the architecture of the various modern parallel systems. He will be able to conceive, implement and analyze parallel algorithms. He will understand why the utilization of parallel resources is not for free, that parallel processing out-of-the-box does not exist. The student will be able to utilize modern parallel technology autonomously, in the design of efficient parallel solutions for all kinds of sequential algorithms.
He will be able to apply his understanding of low-level processes and hardware characteristics to devise high-performant, scalable solutions. Since the quality of parallel programs is also determined by their genericity, usability and cost he will have learned to employ advanced principles of software engineering to design flexible and reusable parallel software.
The student will have learned to look for twofold-generic parallel solutions. On the one hand, the solution should be appropriate to a wide variety of parallel systems, such as a dedicated cluster, a GRID environment or a multiprocessor. On the other hand, it should be easy-to-use and efficient for a broad class of similar algorithms. The student will understand the trade-off between genericity and performance. This integration of high-level software issues and low-level hardware characteristics will have sharpened his general IT-skills.
By the practicum, the student will have demonstrated that he can make good judgments about complex situations and communicate his conclusions. Specific or complex parallel solutions are possible, but these are difficult to maintain and less generic. Only simple, clever solutions are feasible. The student will be able to participate to discussions about exploiting parallelism and the proper use of modern technology.

This course is an elective course of the following VUB Master programs: Computerwetenschappen, Toegepaste Computerwetenschappen, Computer Sciences en Applied Computers Sciences. And also of the ULB Master programs in Computer Science and Engineering.
The course is given in the first semester.


The course is assessed with an oral exam, which counts for 40%, a mini-project (20%) and a project which counts for the remaining 40%.
  1. Oral exam: open book, a few theoretical questions and more practical (how do you parallelize algorithm x). Written preparation of max 2.5 hours of which during the first 20 minutes you may consult the course material ('open book'), followed by an oral defense.
  2. Mini-project
  3. Parallel project


Slides   (a lecturer who wants to use my slides can always contact me)

            Chapter 0: Course Organization           Chapter 1: Introduction: What is a Parallel System? Why are they harder to program than sequential computers?

  Chapter 2: Speeding up the sequential (SIMD & Vectorization).
  Chapter 3: Shared-memory paradigm: Multithreading.
  Chapter 4: Message-passing paradigm.
    1. MPI Quick Reference
    2. The MPI API
  Chapter 5: Performance Analysis.

- Back to the top -