Connect to Moodle site for this course

 

Brief Description                                                                          Credits: 4

​

• Objective:  This course is proposed for  M.Tech.(CS/AI/IT), IM.Tech-VII. and MCA-V students. By the end of the semester, students should be able to develop the following skills:
   

  • Thinking in parallel for the solution of a given problem. Should be able to understand parallel algorithm paradigms 

  • Analyze parallel time and processor’s complexity with different models of Parallel Algorithms, and design efficient parallel algorithms for given problems (PRAM, BSP)

  • Apply the Message Passing Interface/Compiler directives for getting solution (MPI, pthread, OpenMP)

  • Real implementation issues and algorithms for the target machine (Multiprocessors/Multicomputer/Multicores) 

  • For a given problem justify the use of certain parallel design paradigm and architecture, using different parameters

  • Develop the skill to learn how to write a parallel library 

  • Develop skills to learn how to write parallel modules on GPUs.

​

• Prerequisite: Knowledge of Introductory Algorithms, Networks, Java/C/C++, and Unix/Linux, good if you know socket programming.

​

Preferred Books

1. Quinn, M. J., Parallel Computing: Theory and Practice (McGraw-Hill Inc.).  

2. Bary Wilkinson and Michael Allen: Parallel Programming Techniques using Networked of workstations and Parallel Computers, Prentice-Hall, 1999.

3. W. Gropp, E. Lusk, N. Doss, A. Skjellum, A high-performance portable implementation of the message passing Interface (MPI) standard, Parallel Computing 22 (6), Sep 1996.

4. Gibbons, A., W. Rytter, Efficient Parallel Algorithms (Cambridge Uni. Press).

5. Programming Massively Parallel Processors: A Hands-on Approach, authors, David B. Kirk, Wen-mei W. Hwu, Morgan Kaufmann, 2010 (This book is only on NVIDIA GPUs and CUDA programming despite its title)

6. CUDA by Example: An Introduction to General‐Purpose GPU Programming by Jason Sanders and Edwards Kandrot Addison‐Wesley, 2011.

7. CUDA Programming A Developer’s Guide to Parallel Computing with GPUs Shane Cook, Morgan Kaufmann

 

​Course Outline: 

 

Here is a preliminary and non-exhaustive list of topics we will be or might be covering. This is subject to change with advanced notice, partly based on the understanding of the students.

 

Lecture Notes:

​

1. Introduction to Parallel Computing: 

Why Parallel Computing & Scope of Parallel Computing, Sieve of Eratosthenes, Control and Data Approach, PRAM model of parallel computation, Design paradigms of Parallel Computing, examples, Bulk Synchronous Parallel (BSP) model, algorithms on PRAM and BSP model.
 

[Part-I pdf] [Part-II pdf] 

​

2. Practical Parallel Programming Paradigms:
 

Foster's design paradigm for Multi computing programming, Programmability Issues, Programming Models: Message passing, Message passing standards: PVM (Parallel Virtual Machine), MPI (Message Passing Interface) and its routines, Advanced Features of MPI, Load balancing techniques. Programming on Multiprocessors:  Introduction to OpenMP (History, Overview, Programming Model, OpenMP Constructs, Performance Issues, and examples, Explicit Parallelism: Advanced Features of OpenMP). 
 

[Part-I pdf] [Part-II pdf] 

​

3. Threading on Intel Multi-Core Processors 

​

Hardware-based Threading, Hyper-Threading Technology, Difference between Multiprocessor and Hyper-Threading, Technology, Hyper-Threading Technology Architecture, Multi-Core Processors, Architectural Details, Comparison between Multiprocessors and Multi-Core, Processors, Multiple Processor Interaction, Inter-Processor Communication, and Multi-threaded Programming, Power Consumption, Power Metrics.

​

[Part-I pdf] [Part-II pdf] [Part-III pdf] [Part-IV pdf] [Part-V pdf] [Part-VI pdf] [Part-VII pdf] [Part-VIII pdf] [Part-XI pdf]

 

4. Introduction to Heterogeneous Multi-Core Processors 

​

Introduction to Many cores Programming, Cell Processor Multinode Computing, The Early Days of GPGPU Coding, GPU Hardware, Alternatives to CUDA, OpenCL, Direct Compute 
CPU alternatives, Directives and libraries, Understanding Parallelism with GPUs.

 

[Part-I pdf] [Part-II pdf] [Part-III pdf] [Part-IV pdf] [Part-V pdf] [Part-VI pdf] [Part-VII pdf] [Part-VIII pdf] [Part-XI pdf]

 

5. Heterogeneous Multi-Core Programming with CUDA

​

Introduction to GPU Computing, CUDA Programming Model, CUDA API, Simple Matrix Multiplication in CUDA, CUDA Memory Model, Shared Memory Matrix Multiplication, Additional CUDA API Features, Threading Hardware, Memory Hardware, Memory Bank Conflicts, Parallel Thread Execution, Control Flow, Precision.

 

[Part-I pdf] [Part-II pdf] [Part-III pdf] [Part-IV pdf] [Part-V pdf] [Part-VI pdf] [Part-VII pdf] [Part-VIII pdf] [Part-XI pdf]
 

​

​​

Assessments: Internal: 40 Marks

Three internals of 30 marks each (best two will be selected) Lab assignments (10) marks

 

​