This course will consolidate material presented in the beginner cluster course and expand on the concepts to be aware of when trying to optimize use of the cluster.
The main message of the course is to embrace the parallelism available within the cluster and that pipelines should be made from lots of small independent pieces that are spread throughout the cluster rather than large monolithic long jobs that run on a single node. The course will show why this should be done and how to achieve it.
Number of course hours : 9h
Date : 9th, 10th and 16th of April 2024
Level: Medium
Topics Covered:
- Supercomputers, beowulf clusters
- Horizontal v vertical scaling
- Hardware considerations
- Multithreaded jobs, parallelism, Amdahl's Law
- Job arrays & job dependencies
- Building a pipeline
- Storage issues, treemap
- Job stats, resource estimation
- Scaling analysis