**PARALLEL COMPUTING:**

Lectures (28h) - topics:

- Lecture 1.
*Introduction: Parallel computers, why parallel computing, application examples, short history, to port or not to port. Performance: overhead, performance metrics for parallel systems* - Lecture 2.
*Performance Metrics for Parallel Programs: analystic modeling, execution time, overhead, speedup, efficiency, cost, granularity, scalability, roadblocks, asymptotic analysis* - Lecture 3.
*Architecture: logical organization - Flynn taxonomy, SIMD, MIMD, communication; physical organization - historical context, shared memory versus distributed memory* - Lecture 4.
*Architecture and Models: physical organization - radius-based classification, multicore, clusters, grids, trends; early models, PRAM* - Lecture 5.
*Models: dataflow and systolic architectures, circuit model, graph model, LogP and LogGP; message-passing paradigm; levels of parallelism* - Lecture 6.
*Implicit Parallelism - Instruction Level Parallelism. Pipeline, Vector and Superscalar Processors* - Lecture 7.
*Cache coherence in multiprocessor systems. Interconnection Networks - classification, topologies, evaluating static and dysnamic interconnection networks* - Lecture 8.
*Communication costs, routing mechanism, mapping techniques, cost-performance tradeoffs* - Lecture 9.
*Concurrency and Steps in Parallel Algoritm Design:concurrency in parallel programs, approaches to achieve concurrency, basic leyers of software concurrency; tasks, processes and processors, design steps,decomposition - simple examples and classification* - Lecture 10.
*Decomposition and Orchestration: recursive, data, exploratoty, speculative and hybrid decompositions, ochestration under the data parallel, shared-address space and message passing model* - Lecture 11.
*Mapping Techniques for Load Balancing and Methods for Containing Interaction Overheads: mapping classification, schemes for static mapping, schemes for dynamic mapping, maximizing data locality, overlapping computations with interactions, replication, optimized collective interactions* - Lecture 12.
*Emulations, Scheduling and Patterns: emulations among architectures, task scheduling problem, scheduling algorithms, load balancing; patterns - task decomposition, data decomposition, group tasks, order tasks, data sharing, design evaluation* - Lecture 13.
*Models of Parallel Algoritms and Simple Parallel Algorithms: models - data parallel, task graph, work pool, master-slave, pipeline, hybrids; applying data parallel model, bilding-block computations; sorting networks* - Lecture 14.
*Parallel computations in numerical analysis: linear equations, nonlinear equations, ordinary differential equations, computational fluid dynamic (slides only in Romanian, book in English).*

Labs (14h) - topics:

- Lab 1
*OpenMP - generalities, simple examples, matrix operations* - Lab 2
*OpenMP - performance studies* - Lab 3
*MPI - generalities and simple examples* - Lab 4
*MPI - matrix operations* - Lab 5
*MPI - performance studies* - Lab 6
*CUDA - generalities, simple examples (examples here)* - Lab 7
*CUDA - performance studies*

Lectures & Labs - Schedule in Winter Semester of 2015-2016 (subject of changes):

Teaching hours

- Monday 18:00 Lecture in English (AIDC)
- Monday 19:40 Lab in English (AIDC)

- 05 Oct 2015: Lecture + Lab 1
- 12 Oct 2015: Lecture + Lab 2
- 02 Nov 2015: Lecture + Lab 3
- 09 Nov 2015: Lecture + Lab 4
- 16 Nov 2015: Lecture + Lab 5
- 23 Nov 2015: Lecture + Lab 6
- 07 Dec 2015: Lecture + Lab 7
- 04 Jan 2016: 2 x Lecture
- 11 Jan 2016: 2 x Lecture
- 18 Jan 2015: 2 x Lecture

Previous lectures only in Romanian:

- parallel computing lectures for master students
- parallel computer architecture for master students
- parallel computing in numerical analysis for master students
- short version for post-graduate students
- same as above in a printer-friedndly version
- old version for post-graduate students
- short PVM presentation

Textbooks:

- D.Petcu, Parallel computing in English - 2009
- D.Petcu, Parallel processing in Romanian - 2001
- D.Petcu, Parallel algorithms in Romanian - 1994

Links for labs:

- Infrastructure to use: (InfraGrid, GPU Cluster, Blue Gene)
- Connection to InfraGrid and OpenMP
- MPI jobs execution using LoadLeveler

References:

- Bahi Jacques M., Contassot-Vivier Sylvain, Couturier Raphaël eds. Parallel iterative algorithms: from sequential to grid computing, Chapman & Hall/CRC, Taylor & Francis Group, 2008
- Bruaset, Are Magnus, Tveito Aslak Eds.. Numerical Solution of Partial Differential Equations on Parallel Computers, Springer, 2006
- Culler David, Singh Jaswinder Pal, Gupta Anoop. Parallel Computer Architecture. A Hardware/Software Approach, Morgan Kaufmann Publishers, 1997.
- Grama Ananth, Gupta Anshul, Karypis George, Kumar Vipin. Introduction to Parallel Computing, Second Edition, Addison Wesley, 2003
- Hughes Cameron, Hughes Tracey. Parallel and Distributed Programming Using C++, Addison Wesley, 2003.
- Karniadakis George E., Kirby Robert M. Parallel Scientific Computing in C++ and MPI, Cambridge University Press, 2003.
- Kontoghiorghes Erricos J. ed. Handbook of Parallel Computing and Statistics, Chapman & Hall/CRC, Taylor & Francis Group, 2006
- Lastovetsky Alexey L. Parallel Computing on Heterogeneous Networks, John Wiley & Sons, 2003
- Mattson Timothy G., Sanders Beverly A., Massingill Berna L.Patterns for Parallel Programming, Addison- Wesley Professional, 2004
- Parhami Behrooz. Introduction to Parallel Processing. Algorithms and Architectures, Kluwer Academic Publishers, 2002
- Dana Petcu. Parallel Numerical Algorithms. Part I: Solving systems of linear, nonlinear or differential equations, Mathematical Monographs 60, Printing House of University of Timisoara, 1996.
- Dana Petcu. Parallel Numerical Algorithms. Part II: Solving partial differential equations, Mathematical Monographs 61, TPrinting House of University of Timisoara, 1996.
- Dana Petcu. Parallelism in solving ordinary differential equations, Mathematical Monographs 64, Printing House of University of Timisoara, 1998.
- Petersen W. P., Arbenz P., Introduction to Parallel Computing, Oxford University Press, 2004
- Wittwer Tobias. An Introduction to Parallel Programming, VSSD, Netherlands, 2006

Last modification: April, 2015