**PARALLEL COMPUTING:**

Lectures (28h) - topics:

- Lecture 1.
*Introduction: Parallel computers, why parallel computing, application examples, short history, to port or not to port. Performance: overhead, performance metrics for parallel systems* - Lecture 2.
*Performance Metrics for Parallel Programs: analystic modeling, execution time, overhead, speedup, efficiency, cost, granularity, scalability, roadblocks, asymptotic analysis* - Lecture 3.
*Architecture: logical organization - Flynn taxonomy, SIMD, MIMD, communication; physical organization - historical context, shared memory versus distributed memory* - Lecture 4.
*Architecture and Models: physical organization - radius-based classification, multicore, clusters, grids, trends; early models, PRAM* - Lecture 5.
*Models: dataflow and systolic architectures, circuit model, graph model, LogP and LogGP; message-passing paradigm; levels of parallelism* - Lecture 6.
*Implicit Parallelism - Instruction Level Parallelism. Pipeline, Vector and Superscalar Processors* - Lecture 7.
*Cache coherence in multiprocessor systems. Interconnection Networks - classification, topologies, evaluating static and dysnamic interconnection networks* - Lecture 8.
*Communication costs, routing mechanism, mapping techniques, cost-performance tradeoffs* - Lecture 9.
*Concurrency and Steps in Parallel Algoritm Design:concurrency in parallel programs, approaches to achieve concurrency, basic leyers of software concurrency; tasks, processes and processors, design steps,decomposition - simple examples and classification* - Lecture 10.
*Decomposition and Orchestration: recursive, data, exploratoty, speculative and hybrid decompositions, ochestration under the data parallel, shared-address space and message passing model* - Lecture 11.
*Mapping Techniques for Load Balancing and Methods for Containing Interaction Overheads: mapping classification, schemes for static mapping, schemes for dynamic mapping, maximizing data locality, overlapping computations with interactions, replication, optimized collective interactions* - Lecture 12.
*Emulations, Scheduling and Patterns: emulations among architectures, task scheduling problem, scheduling algorithms, load balancing; patterns - task decomposition, data decomposition, group tasks, order tasks, data sharing, design evaluation* - Lecture 13.
*Models of Parallel Algoritms and Simple Parallel Algorithms: models - data parallel, task graph, work pool, master-slave, pipeline, hybrids; applying data parallel model, bilding-block computations; sorting networks* - Lecture 14.
*Parallel computations in numerical analysis: linear equations, nonlinear equations, ordinary differential equations, computational fluid dynamic (only in Romanian).*

Labs (14h) - topics:

- Lab 1
*OpenMP - generalities, basic mechanisms and simple examples* - Lab 2
*OpenMP - Matrix operations* - Lab 3
*OpenMP - Performance studies with OpenMP* - Lab 4
*MPI - generalities, basic mechanisms and simple examples* - Lab 5
*MPI - Matrix operations* - Lab 6
*MPI - Performance studies with MPI* - Lab 7
*MPI vs. OpenMP*

Lectures & Labs in Romanian - Schedule in Spring Semester of 2014 (subject of changes):

- 28 Feb (with Lab 1)
- 7 Mar (with Lab 1)
- 14 Mar (with Lab 2)
- 21 Mar (with Lab 2)
- 11 Apr (with Lab 3)
- 18 Apr (with Lab 3)
- 9 May (with Lab 4)
- 16 May (with Lab 5)
- 30 May (with Lab 6)
- 6 June (with Lab 7)

Previous lectures only in Romanian:

- parallel computing lectures for master students
- parallel computer architecture for master students
- parallel computing in numerical analysis for master students
- short version for post-graduate students
- same as above in a printer-friedndly version
- old version for post-graduate students
- short PVM presentation

Textbooks:

- D.Petcu, Parallel computing in English - 2009
- D.Petcu, Parallel processing in Romanian - 2001
- D.Petcu, Parallel algorithms in Romanian - 1994

References:

- Bahi Jacques M., Contassot-Vivier Sylvain, Couturier Raphaël eds. Parallel iterative algorithms: from sequential to grid computing, Chapman & Hall/CRC, Taylor & Francis Group, 2008
- Bruaset, Are Magnus, Tveito Aslak Eds.. Numerical Solution of Partial Differential Equations on Parallel Computers, Springer, 2006
- Culler David, Singh Jaswinder Pal, Gupta Anoop. Parallel Computer Architecture. A Hardware/Software Approach, Morgan Kaufmann Publishers, 1997.
- Grama Ananth, Gupta Anshul, Karypis George, Kumar Vipin. Introduction to Parallel Computing, Second Edition, Addison Wesley, 2003
- Hughes Cameron, Hughes Tracey. Parallel and Distributed Programming Using C++, Addison Wesley, 2003.
- Karniadakis George E., Kirby Robert M. Parallel Scientific Computing in C++ and MPI, Cambridge University Press, 2003.
- Kontoghiorghes Erricos J. ed. Handbook of Parallel Computing and Statistics, Chapman & Hall/CRC, Taylor & Francis Group, 2006
- Lastovetsky Alexey L. Parallel Computing on Heterogeneous Networks, John Wiley & Sons, 2003
- Mattson Timothy G., Sanders Beverly A., Massingill Berna L.Patterns for Parallel Programming, Addison- Wesley Professional, 2004
- Parhami Behrooz. Introduction to Parallel Processing. Algorithms and Architectures, Kluwer Academic Publishers, 2002
- Dana Petcu. Parallel Numerical Algorithms. Part I: Solving systems of linear, nonlinear or differential equations, Mathematical Monographs 60, Printing House of University of Timisoara, 1996.
- Dana Petcu. Parallel Numerical Algorithms. Part II: Solving partial differential equations, Mathematical Monographs 61, TPrinting House of University of Timisoara, 1996.
- Dana Petcu. Parallelism in solving ordinary differential equations, Mathematical Monographs 64, Printing House of University of Timisoara, 1998.
- Petersen W. P., Arbenz P., Introduction to Parallel Computing, Oxford University Press, 2004
- Wittwer Tobias. An Introduction to Parallel Programming, VSSD, Netherlands, 2006

Last modification: February, 2013