Worker and atools

Course/Event Essentials

Event/Course Start
Event/Course End
Event/Course Format
In person

Venue Information

Country: Belgium
Venue Details: Click here

Training Content and Scope

Scientific Domain
Level of Instruction
Beginner
Intermediate
Sector of the Target Audience
Research and Academia
HPC Profile of Target Audience
Application Users
Data Scientists
System Administrators
Language of Instruction

Other Information

Supporting Project(s)
EuroCC/CASTIEL
Event/Course Description

Topics covered
In many scientific domains, experiments are set up to explore parameter spaces. This type of workload can easily be parallelized since the individual computations are independent. However, a considerable amount of bookkeeping may be required to perform such experiments effectively.

Two frameworks have been developed to make this type of task trivial.
Worker was developed in 2010, and has been improved over the years.
Atools is a recent development to accommodate use cases that Worker is not optimal for.

Both frameworks have a fairly large feature set, including monitoring of running jobs to keep track of the tasks that are completed/failed/pending,  resuming interrupted tasks, merging output from the tasks, and so on.

This lunchbox session will introduce you to these tools, and illustrate the more advanced features.
You will also get tips on how to use the tools efficiently.

Target audience
This info session is primarily targeted at VSC-users, although other interested parties are welcome as well.

Prerequisite skills 
Participants have a working knowledge of the Linux command shell.
Level: introductory

Result/Objectives
Participants know how to efficiently run embarrassingly parallel workloads on a cluster, and can select the best tool for the task at hand.