Go to Dana Petcu's homepage


PARALLEL COMPUTING:


Lectures (28h) - topics:

  1. Lecture 1. Introduction: Parallel computers, why parallel computing, application examples, short history, to port or not to port. Performance: overhead, performance metrics for parallel systems
  2. Lecture 2. Performance Metrics for Parallel Programs: analystic modeling, execution time, overhead, speedup, efficiency, cost, granularity, scalability, roadblocks, asymptotic analysis
  3. Lecture 3. Architecture: logical organization - Flynn taxonomy, SIMD, MIMD, communication; physical organization - historical context, shared memory versus distributed memory
  4. Lecture 4. Architecture and Models: physical organization - radius-based classification, multicore, clusters, grids, trends; early models, PRAM
  5. Lecture 5. Models: dataflow and systolic architectures, circuit model, graph model, LogP and LogGP; message-passing paradigm; levels of parallelism
  6. Lecture 6. Implicit Parallelism - Instruction Level Parallelism. Pipeline, Vector and Superscalar Processors
  7. Lecture 7. Cache coherence in multiprocessor systems. Interconnection Networks - classification, topologies, evaluating static and dysnamic interconnection networks
  8. Lecture 8. Communication costs, routing mechanism, mapping techniques, cost-performance tradeoffs
  9. Lecture 9. Concurrency and Steps in Parallel Algoritm Design:concurrency in parallel programs, approaches to achieve concurrency, basic leyers of software concurrency; tasks, processes and processors, design steps,decomposition - simple examples and classification
  10. Lecture 10. Decomposition and Orchestration: recursive, data, exploratoty, speculative and hybrid decompositions, ochestration under the data parallel, shared-address space and message passing model
  11. Lecture 11. Mapping Techniques for Load Balancing and Methods for Containing Interaction Overheads: mapping classification, schemes for static mapping, schemes for dynamic mapping, maximizing data locality, overlapping computations with interactions, replication, optimized collective interactions
  12. Lecture 12. Emulations, Scheduling and Patterns: emulations among architectures, task scheduling problem, scheduling algorithms, load balancing; patterns - task decomposition, data decomposition, group tasks, order tasks, data sharing, design evaluation
  13. Lecture 13. Models of Parallel Algoritms and Simple Parallel Algorithms: models - data parallel, task graph, work pool, master-slave, pipeline, hybrids; applying data parallel model, bilding-block computations; sorting networks
  14. Lecture 14. Parallel computations in numerical analysis: linear equations, nonlinear equations, ordinary differential equations, computational fluid dynamic (slides only in Romanian, book in English).

Labs (14h) - topics:

  1. Lab 1 OpenMP - generalities, simple examples, matrix operations
  2. Lab 2 OpenMP - performance studies
  3. Lab 3 OpenMP - sorting
  4. Lab 4 MPI - generalities and simple examples
  5. Lab 5 MPI - matrix operations
  6. Lab 6 MPI - performance studies
  7. Lab 7 MPI - solving linear systems

Schedule in Spring semester of 2017/2018

Weekly meetings:

  1. Monday, in English, 18:00-19:30 Lecture in 031, 19:40-21:10 Lab in 045C
  2. Friday, in Romanian, 18:00-19:30 Lecture in 045C, 19:40-21:10 Lab in 045C
  3. Friday, 16:20-17:50, Recover lecture or Lab in 045C
Dates:
  1. 26 Feb 2018 (week 01): 18:00 Lecture 1 (EN), 19:40 Lab 1 (AIDC)
  2. 02 Mar 2018 (week 01): 18:00 Lecture 1 (RO), 19:40 Lab 1 (IS+IASTE)
  3. 05 Mar 2018 (week 02): 18:00 Lecture 2 (EN)
  4. 09 Mar 2018 (week 02): 16:20 Lab 2 (IS+IASTE), 18:00 Lecture 2 (RO), 19:40 Lab 1 (IACD)
  5. 12 Mar 2018 (week 03): 18:00 Lecture 3 (EN), 19:40 Lab 2 (AIDC)
  6. 19 Mar 2018 (week 04): 18:00 Lecture 4 (EN)
  7. 23 Mar 2018 (week 04): 16:20 Lecture 3 (RO), 18:00 Lecture 4 (RO), 19:40 Lab 2 (IACD)
  8. 26 Mar 2018 (week 05): 18:00 Lecture 5 (EN), 19:40 Lab 3 (AIDC)
  9. 30 Mar 2018 (week 05): 16:20 Lecture 5 (RO), 18:00 Lecture 6 (RO), 19:40 Lab 3 (IS+IASTE)
  10. 02 Apr 2018 (week 06): 18:00 Lecture 6 (EN)
  11. 20 Apr 2018 (week 07): 16:20 Lab 3 (IACD), 18:00 Lecture 7 (RO), 19:40 Lab 4 (IS+IASTE)
  12. 23 Apr 2018 (week 08): 18:00 Lab 4 (AIDC)
  13. 27 Apr 2018 (week 08): 16:20 Lecture 7 (EN), 19:40 Lab 4 (IACD) in 003
  14. 04 May 2018 (week 09): 14:40 Lecture 8 (EN+RO) in 103, 16:20 Lab 5 (IS+IASTE)
  15. 07 May 2018 (week 10): 18:00 Lab 5 (AIDC)
  16. 11 May 2018 (week 10): 14:40 Lecture 9 (EN) in 103, 16:20 Lecture 9 (RO), 18:00 Lecture 10 (RO), 19:40 Lab 5 (IACD)
  17. 14 May 2018 (week 11): 18:00 Lecture 10 (EN), 19:40 Lab 6 (AIDC)
  18. 18 May 2018 (week 11): 16:20 Lecture 11 (EN), 18:00 Lecture 11 (RO), 19:40 Lab 6 (IS+IASTE)
  19. 21 May 2018 (week 12): 18:00 Lab 7 (AIDC)
  20. 25 May 2018 (week 12): 14:40 Lecture 12 (EN) in 103, 16:20 Lab 7 (IS+IASTE), 18:00 Lecture 12 (RO), 19:40 Lab 6 (IACD)
  21. 04 June 2018 (week 14): 18:00 Lecture 13 (EN)
  22. 08 June 2018 (week 14): 14:40 Lecture 14 (EN) in 103, 16:20 Lecture 13 (RO), 18:00 Lecture 14 (RO), 19:40 Lab 7 (IACD)


Previous lectures only in Romanian:

  1. parallel computing lectures for master students
  2. parallel computer architecture for master students
  3. parallel computing in numerical analysis for master students
  4. short version for post-graduate students
  5. same as above in a printer-friedndly version
  6. old version for post-graduate students
  7. short PVM presentation

Textbooks:

  1. D.Petcu, Parallel computing in English - 2009
  2. D.Petcu, Parallel processing in Romanian - 2001
  3. D.Petcu, Parallel algorithms in Romanian - 1994

Links for labs:

  1. Infrastructure to use: (InfraGrid, GPU Cluster, Blue Gene)
  2. Connection to InfraGrid and OpenMP
  3. MPI jobs execution using LoadLeveler

References:

  1. Bahi Jacques M., Contassot-Vivier Sylvain, Couturier RaphaŽl eds. Parallel iterative algorithms: from sequential to grid computing, Chapman & Hall/CRC, Taylor & Francis Group, 2008
  2. Bruaset, Are Magnus, Tveito Aslak Eds.. Numerical Solution of Partial Differential Equations on Parallel Computers, Springer, 2006
  3. Culler David, Singh Jaswinder Pal, Gupta Anoop. Parallel Computer Architecture. A Hardware/Software Approach, Morgan Kaufmann Publishers, 1997.
  4. Grama Ananth, Gupta Anshul, Karypis George, Kumar Vipin. Introduction to Parallel Computing, Second Edition, Addison Wesley, 2003
  5. Hughes Cameron, Hughes Tracey. Parallel and Distributed Programming Using C++, Addison Wesley, 2003.
  6. Karniadakis George E., Kirby Robert M. Parallel Scientific Computing in C++ and MPI, Cambridge University Press, 2003.
  7. Kontoghiorghes Erricos J. ed. Handbook of Parallel Computing and Statistics, Chapman & Hall/CRC, Taylor & Francis Group, 2006
  8. Lastovetsky Alexey L. Parallel Computing on Heterogeneous Networks, John Wiley & Sons, 2003
  9. Mattson Timothy G., Sanders Beverly A., Massingill Berna L.Patterns for Parallel Programming, Addison- Wesley Professional, 2004
  10. Parhami Behrooz. Introduction to Parallel Processing. Algorithms and Architectures, Kluwer Academic Publishers, 2002
  11. Dana Petcu. Parallel Numerical Algorithms. Part I: Solving systems of linear, nonlinear or differential equations, Mathematical Monographs 60, Printing House of University of Timisoara, 1996.
  12. Dana Petcu. Parallel Numerical Algorithms. Part II: Solving partial differential equations, Mathematical Monographs 61, TPrinting House of University of Timisoara, 1996.
  13. Dana Petcu. Parallelism in solving ordinary differential equations, Mathematical Monographs 64, Printing House of University of Timisoara, 1998.
  14. Petersen W. P., Arbenz P., Introduction to Parallel Computing, Oxford University Press, 2004
  15. Wittwer Tobias. An Introduction to Parallel Programming, VSSD, Netherlands, 2006

Last modification: February, 2018