Go to Dana Petcu's homepage


PARALLEL COMPUTING:


Lectures (28h) - topics:

  1. Lecture 1. Introduction: Parallel computers, why parallel computing, application examples, short history, to port or not to port. Performance: overhead, performance metrics for parallel systems
  2. Lecture 2. Performance Metrics for Parallel Programs: analystic modeling, execution time, overhead, speedup, efficiency, cost, granularity, scalability, roadblocks, asymptotic analysis
  3. Lecture 3. Architecture: logical organization - Flynn taxonomy, SIMD, MIMD, communication; physical organization - historical context, shared memory versus distributed memory
  4. Lecture 4. Architecture and Models: physical organization - radius-based classification, multicore, clusters, grids, trends; early models, PRAM
  5. Lecture 5. Models: dataflow and systolic architectures, circuit model, graph model, LogP and LogGP; message-passing paradigm; levels of parallelism
  6. Lecture 6. Implicit Parallelism - Instruction Level Parallelism. Pipeline, Vector and Superscalar Processors
  7. Lecture 7. Cache coherence in multiprocessor systems. Interconnection Networks - classification, topologies, evaluating static and dysnamic interconnection networks
  8. Lecture 8. Communication costs, routing mechanism, mapping techniques, cost-performance tradeoffs
  9. Lecture 9. Concurrency and Steps in Parallel Algoritm Design:concurrency in parallel programs, approaches to achieve concurrency, basic leyers of software concurrency; tasks, processes and processors, design steps,decomposition - simple examples and classification
  10. Lecture 10. Decomposition and Orchestration: recursive, data, exploratoty, speculative and hybrid decompositions, ochestration under the data parallel, shared-address space and message passing model
  11. Lecture 11. Mapping Techniques for Load Balancing and Methods for Containing Interaction Overheads: mapping classification, schemes for static mapping, schemes for dynamic mapping, maximizing data locality, overlapping computations with interactions, replication, optimized collective interactions
  12. Lecture 12. Emulations, Scheduling and Patterns: emulations among architectures, task scheduling problem, scheduling algorithms, load balancing; patterns - task decomposition, data decomposition, group tasks, order tasks, data sharing, design evaluation
  13. Lecture 13. Models of Parallel Algoritms and Simple Parallel Algorithms: models - data parallel, task graph, work pool, master-slave, pipeline, hybrids; applying data parallel model, bilding-block computations; sorting networks
  14. Lecture 14. Parallel computations in numerical analysis: linear equations, nonlinear equations, ordinary differential equations, computational fluid dynamic (only in Romanian).

Labs (14h) - topics:

  1. Lab 1 OpenMP - generalities, basic mechanisms and simple examples
  2. Lab 2 OpenMP - Matrix operations
  3. Lab 3 OpenMP - Performance studies with OpenMP
  4. Lab 4 MPI - generalities, basic mechanisms and simple examples
  5. Lab 5 MPI - Matrix operations
  6. Lab 6 MPI - Performance studies with MPI
  7. Lab 7 MPI vs. OpenMP

Lectures & Labs in Romanian - Schedule in Spring Semester of 2014 (subject of changes):

  1. 28 Feb (with Lab 1)
  2. 7 Mar (with Lab 1)
  3. 14 Mar (with Lab 2)
  4. 21 Mar (with Lab 2)
  5. 11 Apr (with Lab 3)
  6. 18 Apr (with Lab 3)
  7. 2 May (with Lab 4)
  8. 9 May (with Lab 4)
  9. 16 May (with Lab 5)
  10. 23 May (with Lab 5)
  11. 30 May (with Lab 6)
  12. 6 June (with Lab 7)

Previous lectures only in Romanian:

  1. parallel computing lectures for master students
  2. parallel computer architecture for master students
  3. parallel computing in numerical analysis for master students
  4. short version for post-graduate students
  5. same as above in a printer-friedndly version
  6. old version for post-graduate students
  7. short PVM presentation

Textbooks:

  1. D.Petcu, Parallel computing in English - 2009
  2. D.Petcu, Parallel processing in Romanian - 2001
  3. D.Petcu, Parallel algorithms in Romanian - 1994

References:

  1. Bahi Jacques M., Contassot-Vivier Sylvain, Couturier RaphaŽl eds. Parallel iterative algorithms: from sequential to grid computing, Chapman & Hall/CRC, Taylor & Francis Group, 2008
  2. Bruaset, Are Magnus, Tveito Aslak Eds.. Numerical Solution of Partial Differential Equations on Parallel Computers, Springer, 2006
  3. Culler David, Singh Jaswinder Pal, Gupta Anoop. Parallel Computer Architecture. A Hardware/Software Approach, Morgan Kaufmann Publishers, 1997.
  4. Grama Ananth, Gupta Anshul, Karypis George, Kumar Vipin. Introduction to Parallel Computing, Second Edition, Addison Wesley, 2003
  5. Hughes Cameron, Hughes Tracey. Parallel and Distributed Programming Using C++, Addison Wesley, 2003.
  6. Karniadakis George E., Kirby Robert M. Parallel Scientific Computing in C++ and MPI, Cambridge University Press, 2003.
  7. Kontoghiorghes Erricos J. ed. Handbook of Parallel Computing and Statistics, Chapman & Hall/CRC, Taylor & Francis Group, 2006
  8. Lastovetsky Alexey L. Parallel Computing on Heterogeneous Networks, John Wiley & Sons, 2003
  9. Mattson Timothy G., Sanders Beverly A., Massingill Berna L.Patterns for Parallel Programming, Addison- Wesley Professional, 2004
  10. Parhami Behrooz. Introduction to Parallel Processing. Algorithms and Architectures, Kluwer Academic Publishers, 2002
  11. Dana Petcu. Parallel Numerical Algorithms. Part I: Solving systems of linear, nonlinear or differential equations, Mathematical Monographs 60, Printing House of University of Timisoara, 1996.
  12. Dana Petcu. Parallel Numerical Algorithms. Part II: Solving partial differential equations, Mathematical Monographs 61, TPrinting House of University of Timisoara, 1996.
  13. Dana Petcu. Parallelism in solving ordinary differential equations, Mathematical Monographs 64, Printing House of University of Timisoara, 1998.
  14. Petersen W. P., Arbenz P., Introduction to Parallel Computing, Oxford University Press, 2004
  15. Wittwer Tobias. An Introduction to Parallel Programming, VSSD, Netherlands, 2006

Last modification: February, 2013