**PARALLEL COMPUTING:**

Lectures (28h) - topics:

- Lecture 1.
*Introduction: Parallel computers, why parallel computing, application examples, short history, to port or not to port. Performance: overhead, performance metrics for parallel systems* - Lecture 2.
*Performance Metrics for Parallel Programs: analystic modeling, execution time, overhead, speedup, efficiency, cost, granularity, scalability, roadblocks, asymptotic analysis* - Lecture 3.
*Architecture: logical organization - Flynn taxonomy, SIMD, MIMD, communication; physical organization - historical context, shared memory versus distributed memory* - Lecture 4.
*Architecture and Models: physical organization - radius-based classification, multicore, clusters, grids, trends; early models, PRAM* - Lecture 5.
*Models: dataflow and systolic architectures, circuit model, graph model, LogP and LogGP; message-passing paradigm; levels of parallelism* - Lecture 6.
*Implicit Parallelism - Instruction Level Parallelism. Pipeline, Vector and Superscalar Processors* - Lecture 7.
*Cache coherence in multiprocessor systems. Interconnection Networks - classification, topologies, evaluating static and dysnamic interconnection networks* - Lecture 8.
*Communication costs, routing mechanism, mapping techniques, cost-performance tradeoffs* - Lecture 9.
*Concurrency and Steps in Parallel Algoritm Design:concurrency in parallel programs, approaches to achieve concurrency, basic leyers of software concurrency; tasks, processes and processors, design steps,decomposition - simple examples and classification* - Lecture 10.
*Decomposition and Orchestration: recursive, data, exploratoty, speculative and hybrid decompositions, ochestration under the data parallel, shared-address space and message passing model* - Lecture 11.
*Mapping Techniques for Load Balancing and Methods for Containing Interaction Overheads: mapping classification, schemes for static mapping, schemes for dynamic mapping, maximizing data locality, overlapping computations with interactions, replication, optimized collective interactions* - Lecture 12.
*Emulations, Scheduling and Patterns: emulations among architectures, task scheduling problem, scheduling algorithms, load balancing; patterns - task decomposition, data decomposition, group tasks, order tasks, data sharing, design evaluation* - Lecture 13.
*Models of Parallel Algoritms and Simple Parallel Algorithms: models - data parallel, task graph, work pool, master-slave, pipeline, hybrids; applying data parallel model, bilding-block computations; sorting networks* - Lecture 14.
*Parallel computations in numerical analysis: linear equations, nonlinear equations, ordinary differential equations, computational fluid dynamic (slides only in Romanian, book in English).*

Labs (14h) - topics:

- Lab 1
*OpenMP - generalities, simple examples, matrix operations* - Lab 2
*OpenMP - performance studies* - Lab 3
*OpenMP - sorting* - Lab 4
*MPI - generalities and simple examples* - Lab 5
*MPI - matrix operations* - Lab 6
*MPI - performance studies* - Lab 7
*MPI - solving linear systems*

Schedule in Spring semester of 2017/2018

Weekly meetings:

- Monday, in English, 18:00-19:30 Lecture in 031, 19:40-21:10 Lab in 045C
- Friday, in Romanian, 18:00-19:30 Lecture in 045C, 19:40-21:10 Lab in 045C
- Friday, 16:20-17:50, Recover lecture or Lab in 045C

- 26 Feb 2018 (week 01): 18:00 Lecture 1 (EN), 19:40 Lab 1 (AIDC)
- 02 Mar 2018 (week 01): 18:00 Lecture 1 (RO), 19:40 Lab 1 (IS+IASTE)
- 05 Mar 2018 (week 02): 18:00 Lecture 2 (EN)
- 09 Mar 2018 (week 02): 16:20 Lab 2 (IS+IASTE), 18:00 Lecture 2 (RO), 19:40 Lab 1 (IACD)
- 12 Mar 2018 (week 03): 18:00 Lecture 3 (EN), 19:40 Lab 2 (AIDC)
- 19 Mar 2018 (week 04): 18:00 Lecture 4 (EN)
- 23 Mar 2018 (week 04): 16:20 Lecture 3 (RO), 18:00 Lecture 4 (RO), 19:40 Lab 2 (IACD)
- 26 Mar 2018 (week 05): 18:00 Lecture 5 (EN), 19:40 Lab 3 (AIDC)
- 30 Mar 2018 (week 05): 16:20 Lecture 5 (RO), 18:00 Lecture 6 (RO), 19:40 Lab 3 (IS+IASTE)
- 02 Apr 2018 (week 06): 18:00 Lecture 6 (EN)
- 06 Apr 2018 (week 06): 16:20 Lecture 7 (EN), 18:00 Lecture 7 (RO), 19:40 Lab 4 (IACD)
- 20 Apr 2018 (week 07): 16:20 Lecture 8 (EN), 18:00 Lecture 8 (RO), 19:40 Lab 4 (IS+IASTE)
- 23 Apr 2018 (week 08): 18:00 Lab 4 (AIDC)
- 27 Apr 2018 (week 08): 16:20 Lecture 9 (EN), 18:00 Lecture 9 (RO), 19:40 Lab 4 (IACD)
- 04 May 2018 (week 09): 16:20 Lecture 10 (EN), 18:00 Lecture 10 (RO), 19:40 Lab 5 (IS+IASTE)
- 07 May 2018 (week 10): 18:00 Lab 5 (AIDC)
- 11 May 2018 (week 10): 16:20 Lecture 11 (EN), 18:00 Lecture 11 (EN), 19:40 Lab 4 (IACD)
- 14 May 2018 (week 11): 18:00 Lecture 12 (EN), 19:40 Lab 6 (AIDC)
- 18 May 2018 (week 11): 16:20 Lecture 13 (EN), 18:00 Lecture 12 (RO), 19:40 Lab 6 (IS+IASTE)
- 21 May 2018 (week 12): 18:00 Lab 7 (AIDC)
- 25 May 2018 (week 12): 16:20 Lab 7 (IS+IASTE), 18:00 Lecture 13 (RO), 19:40 Lab 6 (IACD)
- 04 June 2018 (week 14): 18:00 Lecture 14 (AIDC)
- 08 June 2018 (week 14): 16:20 Recover any lab, 18:00 Lecture 14 (RO), 19:40 Lab 7 (IACD)

Previous lectures only in Romanian:

- parallel computing lectures for master students
- parallel computer architecture for master students
- parallel computing in numerical analysis for master students
- short version for post-graduate students
- same as above in a printer-friedndly version
- old version for post-graduate students
- short PVM presentation

Textbooks:

- D.Petcu, Parallel computing in English - 2009
- D.Petcu, Parallel processing in Romanian - 2001
- D.Petcu, Parallel algorithms in Romanian - 1994

Links for labs:

- Infrastructure to use: (InfraGrid, GPU Cluster, Blue Gene)
- Connection to InfraGrid and OpenMP
- MPI jobs execution using LoadLeveler

References:

- Bahi Jacques M., Contassot-Vivier Sylvain, Couturier Raphaël eds. Parallel iterative algorithms: from sequential to grid computing, Chapman & Hall/CRC, Taylor & Francis Group, 2008
- Bruaset, Are Magnus, Tveito Aslak Eds.. Numerical Solution of Partial Differential Equations on Parallel Computers, Springer, 2006
- Culler David, Singh Jaswinder Pal, Gupta Anoop. Parallel Computer Architecture. A Hardware/Software Approach, Morgan Kaufmann Publishers, 1997.
- Grama Ananth, Gupta Anshul, Karypis George, Kumar Vipin. Introduction to Parallel Computing, Second Edition, Addison Wesley, 2003
- Hughes Cameron, Hughes Tracey. Parallel and Distributed Programming Using C++, Addison Wesley, 2003.
- Karniadakis George E., Kirby Robert M. Parallel Scientific Computing in C++ and MPI, Cambridge University Press, 2003.
- Kontoghiorghes Erricos J. ed. Handbook of Parallel Computing and Statistics, Chapman & Hall/CRC, Taylor & Francis Group, 2006
- Lastovetsky Alexey L. Parallel Computing on Heterogeneous Networks, John Wiley & Sons, 2003
- Mattson Timothy G., Sanders Beverly A., Massingill Berna L.Patterns for Parallel Programming, Addison- Wesley Professional, 2004
- Parhami Behrooz. Introduction to Parallel Processing. Algorithms and Architectures, Kluwer Academic Publishers, 2002
- Dana Petcu. Parallel Numerical Algorithms. Part I: Solving systems of linear, nonlinear or differential equations, Mathematical Monographs 60, Printing House of University of Timisoara, 1996.
- Dana Petcu. Parallel Numerical Algorithms. Part II: Solving partial differential equations, Mathematical Monographs 61, TPrinting House of University of Timisoara, 1996.
- Dana Petcu. Parallelism in solving ordinary differential equations, Mathematical Monographs 64, Printing House of University of Timisoara, 1998.
- Petersen W. P., Arbenz P., Introduction to Parallel Computing, Oxford University Press, 2004
- Wittwer Tobias. An Introduction to Parallel Programming, VSSD, Netherlands, 2006

Last modification: February, 2018