Parallel Systems - Main

practica   -    project

Motivation. With the end of the ever-increasing clock frequency, parallel processing has become the only option to expand processing power. Moore's law now applies for the number of on-chip processors. All big players, such as Intel and Microsoft, have jumped on parallelism. A novel trend is to open the cheap commodity graphics cards for general-purpose programming. A clash between NVIDIA (with CUDA) and Intel (with their Xeon Phi) is happening. The course focusses on how the software developer can exploit concurrent programming.

Goal. The student will understand the architecture of the various modern parallel systems. He will be able to conceive, implement and analyze parallel and distributed algorithms. He will understand why the utilization of parallel resources is not for free, that parallel processing out-of-the-box does not exist. The student will be able to utilize modern parallel technology autonomously, in the design of efficient parallel solutions for all kinds of sequential algorithms.
He will be able to apply his understanding of low-level processes and hardware characteristics to devise high-performant, scalable solutions. Since the quality of parallel programs is also determined by their genericity, usability and cost he will have learned to employ advanced principles of software engineering to design flexible and reusable parallel software.
The student will have learned to look for twofold-generic parallel solutions. On the one hand, the solution should be appropriate to a wide variety of parallel systems, such as a dedicated cluster, a GRID environment, a multiprocessor or a Graphical Processing Unit (GPU). On the other hand, it should be easy-to-use and efficient for a broad class of similar algorithms. The student will understand the trade-off between genericity and performance. This integration of high-level software issues and low-level hardware characteristics will have sharpened his general IT-skills.
By the practicum, the student will have demonstrated that he can make good judgments about complex situations and communicate his conclusions. Specific or complex parallel solutions are possible, but these are difficult to maintain and less generic. Only simple, clever solutions are feasible. The student will be able to participate to discussions about exploiting parallelism and the proper use of modern technology.


This course is an elective course of the following VUB Master programs of the Faculty of Engineering: Computerwetenschappen, Toegepaste Computerwetenschappen, Computer Sciences en Applied Computers Sciences.
The course is given in the first semester.

Assessment

The course is assessed with an oral exam, which counts for 50%, and a project which counts for the other 50%.
  1. Oral exam: open book (all material may be used), 2 to 3 questions about the theory. Written preparation, oral defense.
  2. Parallel project

Until 2016 practicum exam was part of the examination. 2 programs had to be parallelized, one with multithreading, one with MPI. Here is an example.

 References


Slides   (a lecturer who wants to use my slides can always contact me)

Handouts will be distributed at the start of each lesson.

          Chapter 1: Introduction: What is a Parallel System? Why are they harder to program than sequential computers?

  Chapter 2: Shared-memory paradigm: Multithreading.
  Chapter 4: Programming GPUs. (We start as soon as possible with chapter 4 so that the project can be initiated.)

  Chapter 3: Message-passing paradigm.
    1. The MPI API
    2. The MPJ API
          Chapter 5: Performance Analysis.
Chapter 6: Dense Matrix Algorithms.
  Chapter 7: Discrete Optimization Problems.
  Chapter 8: Sorting.
  Chapter 9: Conclusions
Old stuff, from past years

Slides 2015 - 2016


      
  Short introduction
  Chapter 4: Programming GPUs. (We started with chapter 4 so that the project could be initiated.)

          Chapter 1: Introduction: What is a Parallel System? Why are they harder to program than sequential computers?

  Chapter 2: Shared-memory paradigm: Multithreading.
  Chapter 3: Message-passing paradigm.
    1. The MPI API
    2. The MPJ API
          Chapter 5: Performance Analysis.
Chapter 6: Dense Matrix Algorithms.
  Chapter 7: Discrete Optimization Problems.
  Chapter 8: Sorting.
  Chapter 9: Conclusions

Scientific study

    Students choose one of the following subjects or propose a subject of their own. Let the professor know which subject got your preference. They will read the given material and give a 15-minute presentation. In order to prepare the presentation, the student meets the professor to discuss what he learned. In the presentation they demonstrate their understanding of the subject by clearly defining the problem, the proposed solutions and articulate a personal opinion. After the presentation a small discussion with the teacher and fellow students of 5 minutes will follow. It is allowed to cooperate and tackle the same subject together. Don't forget to list all the consulted resources!
The topics marked with a are highly-appreciated topics.
Take the following points into consideration:

The Java memory model. How is it defined? What is its purpose? What are the main issues? What should (parallel) programmers know about it?
Multicore Panic. Will we be able to program the future hardware?
The Future of Parallel Computing.
New trends in processor technology: multi- and manycores.

Parallel computing for Signal Processing.


Parallel Programming Languages. Discuss one of them.

User-Transparant Parallel Image Processing

After Amdahl: Gustafson

- Back to the top -