Workflows on HPC

Course/Event Essentials

Event/Course Start
Event/Course End
Event/Course Format
Mixed
Live (synchronous)
Primary Event/Course URL
https://admin.kuleuven.be/icts/opleidin…

Venue Information

Country: Belgium
Venue Details: Click here

Training Content and Scope

Scientific Domain
Not scientific domain specific
Technical Domain
General HPC systems and architectures
Level of Instruction
Intermediate
Sector of the Target Audience
Research and Academia
Industry
Public Sector
HPC Profile of Target Audience
Application Users
Language of Instruction
English

Other Information

Organiser
Vlaams Computer Centrum (VSC)
Supporting Project(s)
EuroCC2/CASTIEL2
Event/Course Description

Running a job on an HPC system is typically not that hard, but  executing a non-trivial workflow that involves both data movement  and various computations is not so trivial. In this  training you will get an overview of tools that can help you to  efficiently and conveniently run workflows on supercomputers. 

When you complete this training you will:

  • be able to schedule tasks that run on a login node at given time(s);
  • be able to run SLURM jobs at given time(s);
  • know how to break down a computation that consists of multiple tasks that require specific resources into a set of SLURM  jobs using job dependencies;
  • understand how to use job dependencies to restart jobs that checkpoint their state;
  • understand how to SLURM job arrays to run multiple jobs with similar resource requirements;
  • be able to use either worker-ng or atools to simplify managing such as parallel workflows;
  • use a workflow manager to run a workflow that consists of multiple tasks that require specific resources.