Introduction to Scalable Deep Learning (ONLINE)

Course/Event Essentials

Event/Course Start
Event/Course End
Event/Course Format
Online
Live (synchronous)

Venue Information

Country: Germany
Venue Details: Click here

Training Content and Scope

Scientific Domain
Level of Instruction
Intermediate
Advanced
Sector of the Target Audience
Research and Academia
Industry
HPC Profile of Target Audience
Application Users
Application Developers
Data Scientists
Language of Instruction

Other Information

Organiser
Supporting Project(s)
PRACE
Event/Course Description

In this course, we will cover machine learning and deep learning and how to achieve scaling to high performance computing systems. The course aims at covering all levels, from fundamental software design to specific compute environments and toolkits. We want to enable the participants to unlock the resource of machines like the JUWELS booster for their machine learning workflows. Different from previous years we assume that the participants have a background from a university level introductory course to machine learning. Suggested options for self-teaching are given below.

We will start the course with a presentation of high performance computing system architectures and the design paradigms for HPC software. In the tutorial, we familiarize the users with the environment. Furthermore, we give a recap of important machine learning concepts and algorithms and the participants will train and test a reference model. Afterwards, we introduce how deep learning algorithms can be parallelized for supercomputer usage with Horovod. Furthermore, we discuss best practicies and pitfalls in adopting deep learning algorithms on supercomputers and learn to test their function and performance. Finally we apply the gained expertise to large scale unsupervised learning, with a particular focus on Generative Adversarial Networks (GANs).

This course is a PRACE training course.