Efficient multi-GPU and multi-node execution of AI applications and frameworks on the GPU nodes of Karolina supercomputer

This half-day course is dedicated to learning how to efficiently use the GPU accelerated part of Karolina for Deep and Machine Learning.

Schedule

13:00 - 14:00 Access to Karolina's GPU accelerated part

1. Short introduction of the Karolina supercomputer 
2. How to access the Karolina GPU nodes
3. First login
4. Computing environment and available software libraries and tools
5. HPC resources allocation, PBS
6. Scratch and Project storages
7. Special tools (Nodes availability overview, ...)

14:15 - 15:15 Efficient multi-GPU and multi-node execution of Deep and Machine Learning frameworks

1. Introduction to Data Parallel Deep Learning with Horovod
2. Multi-node/-GPU aware Data Processing Pipelines
3. Demonstration of Multi-node/-GPU Examples using Tensorflow
4. Multi-node/-GPU Machine Learning with scikit-learn

15:15 - 16:00 Introduction to HyperQueue

1. Efficient execution of a large number of small tasks transparently over HPC schedulers (SLURM/PBS) using HyperQueue
2. Guided examples

Type of methodology: Combination lecture with live demonstrations.

Participants receive the certificate of attendance: Yes, if requested.

Paid training activity for participants: No, it is free of charge.

Participants prerequisite knowledge: Experience using GPU accelerated systems.