**PARALLEL COMPUTING:**

Lectures (28h) - topics:

- Lecture 1.
*Introduction: Parallel computers, why parallel computing, application examples, short history, to port or not to port. Performance: overhead, performance metrics for parallel systems* - Lecture 2.
*Performance Metrics for Parallel Programs: analystic modeling, execution time, overhead, speedup, efficiency, cost, granularity, scalability, roadblocks, asymptotic analysis* - Lecture 3.
*Architecture: logical organization - Flynn taxonomy, SIMD, MIMD, communication; physical organization - historical context, shared memory versus distributed memory* - Lecture 4.
*Architecture and Models: physical organization - radius-based classification, multicore, clusters, grids, trends; early models, PRAM* - Lecture 5.
*Models: dataflow and systolic architectures, circuit model, graph model, LogP and LogGP; message-passing paradigm; levels of parallelism* - Lecture 6.
*Implicit Parallelism - Instruction Level Parallelism. Pipeline, Vector and Superscalar Processors* - Lecture 7.
*Cache coherence in multiprocessor systems. Interconnection Networks - classification, topologies, evaluating static and dysnamic interconnection networks* - Lecture 8.
*Communication costs, routing mechanism, mapping techniques, cost-performance tradeoffs* - Lecture 9.
*Concurrency and Steps in Parallel Algoritm Design:concurrency in parallel programs, approaches to achieve concurrency, basic leyers of software concurrency; tasks, processes and processors, design steps,decomposition - simple examples and classification* - Lecture 10.
*Decomposition and Orchestration: recursive, data, exploratoty, speculative and hybrid decompositions, ochestration under the data parallel, shared-address space and message passing model* - Lecture 11.
*Mapping Techniques for Load Balancing and Methods for Containing Interaction Overheads: mapping classification, schemes for static mapping, schemes for dynamic mapping, maximizing data locality, overlapping computations with interactions, replication, optimized collective interactions* - Lecture 12.
*Emulations, Scheduling and Patterns: emulations among architectures, task scheduling problem, scheduling algorithms, load balancing; patterns - task decomposition, data decomposition, group tasks, order tasks, data sharing, design evaluation* - Lecture 13.
*Models of Parallel Algoritms and Simple Parallel Algorithms: models - data parallel, task graph, work pool, master-slave, pipeline, hybrids; applying data parallel model, bilding-block computations; sorting networks* - Lecture 14.
*Parallel computations in numerical analysis: linear equations, nonlinear equations, ordinary differential equations, computational fluid dynamic (slides only in Romanian, book in English).*

Labs (14h) - topics:

- Lab 1
*OpenMP - generalities, simple examples, matrix operations* - Lab 2
*OpenMP - performance studies* - Lab 3
*MPI - generalities and simple examples* - Lab 4
*MPI - matrix operations* - Lab 5
*MPI - performance studies* - Lab 6
*CUDA - generalities, simple examples (examples here)* - Lab 7
*CUDA - performance studies*

Lectures & Labs - Schedule in Summer Semester of 2015-2016 (subject of changes):

Teaching hours

- Friday 18:00 Lecture
- Friday 19:40 Lab

- 04 Mar 2016: Lecture 1 + Lab 1-IS+IASTE
- 11 Mar 2016: Lecture 2 + Lab 1-IACD
- 18 Mar 2016: Lecture 3 + Lab 2-IS+IASTE
- 25 Mar 2016: Lecture 4 + Lab 2-IACD
- 01 Apr 2016: Lecture 5 + Lab 3-IS+IASTE
- 08 Apr 2016: Lecture 6 + Lab 3-IACD
- 15 Apr 2016: Lecture 7 + Lab 4-IS+IASTE
- 22 Apr 2016: Lecture 8 + Lab 4-IACD
- 29 Apr 2016: Lecture 9 + Lab 5-IS+IASTE
- 13 May 2016: Lecture 10 + Lab 5-IACD
- 20 May 2016: Lecture 11 (14:40)+ Lecture 12 (18:00)+ Lab 6-IS+IASTE (19:40)
- 24 May 2016: Lab 7-IS+IASTE (19:40)
- 25 May 2016: Lab 6-IACD (19:40)
- 27 May 2016: Lecture 13 (14:40)+ Lecture 14 (18:00)+ Lab 7-IACD (19:40)

Previous lectures only in Romanian:

- parallel computing lectures for master students
- parallel computer architecture for master students
- parallel computing in numerical analysis for master students
- short version for post-graduate students
- same as above in a printer-friedndly version
- old version for post-graduate students
- short PVM presentation

Textbooks:

- D.Petcu, Parallel computing in English - 2009
- D.Petcu, Parallel processing in Romanian - 2001
- D.Petcu, Parallel algorithms in Romanian - 1994

Links for labs:

- Infrastructure to use: (InfraGrid, GPU Cluster, Blue Gene)
- Connection to InfraGrid and OpenMP
- MPI jobs execution using LoadLeveler

References:

- Bahi Jacques M., Contassot-Vivier Sylvain, Couturier Raphaël eds. Parallel iterative algorithms: from sequential to grid computing, Chapman & Hall/CRC, Taylor & Francis Group, 2008
- Bruaset, Are Magnus, Tveito Aslak Eds.. Numerical Solution of Partial Differential Equations on Parallel Computers, Springer, 2006
- Culler David, Singh Jaswinder Pal, Gupta Anoop. Parallel Computer Architecture. A Hardware/Software Approach, Morgan Kaufmann Publishers, 1997.
- Grama Ananth, Gupta Anshul, Karypis George, Kumar Vipin. Introduction to Parallel Computing, Second Edition, Addison Wesley, 2003
- Hughes Cameron, Hughes Tracey. Parallel and Distributed Programming Using C++, Addison Wesley, 2003.
- Karniadakis George E., Kirby Robert M. Parallel Scientific Computing in C++ and MPI, Cambridge University Press, 2003.
- Kontoghiorghes Erricos J. ed. Handbook of Parallel Computing and Statistics, Chapman & Hall/CRC, Taylor & Francis Group, 2006
- Lastovetsky Alexey L. Parallel Computing on Heterogeneous Networks, John Wiley & Sons, 2003
- Mattson Timothy G., Sanders Beverly A., Massingill Berna L.Patterns for Parallel Programming, Addison- Wesley Professional, 2004
- Parhami Behrooz. Introduction to Parallel Processing. Algorithms and Architectures, Kluwer Academic Publishers, 2002
- Dana Petcu. Parallel Numerical Algorithms. Part I: Solving systems of linear, nonlinear or differential equations, Mathematical Monographs 60, Printing House of University of Timisoara, 1996.
- Dana Petcu. Parallel Numerical Algorithms. Part II: Solving partial differential equations, Mathematical Monographs 61, TPrinting House of University of Timisoara, 1996.
- Dana Petcu. Parallelism in solving ordinary differential equations, Mathematical Monographs 64, Printing House of University of Timisoara, 1998.
- Petersen W. P., Arbenz P., Introduction to Parallel Computing, Oxford University Press, 2004
- Wittwer Tobias. An Introduction to Parallel Programming, VSSD, Netherlands, 2006

Last modification: February, 2016