This course will take place as an online event. The link to the streaming platform will be provided to the registrants only. The course will be held in English.
Course Content:
Unreal Engine is a powerful 3D rendering tool widely used in game development. In recent years, it has become increasingly popular in science and industry. This course teaches you how to use Unreal Engine to generate synthetic data for machine learning projects.
You will learn to create 3D assets using AI tools (ComfyUI), build large virtual worlds automatically using Unreal's Procedural Content Generation (PCG) framework, export video footage from these worlds via Network Device Interface (NDI), and prepare the data for training video generation models. By the end of the course, you will have built a complete working pipeline and understand how to use it for machine learning workflows.
What you will learn:
- Generate 3D assets using AI (ComfyUI) and import them into Unreal Engine
- Build scalable virtual worlds with Unreal's PCG framework
- Export camera footage and metadata from virtual scenes using NDI (network video over IP)
- Getting to know city sample and how NDI outputs data streams for training data
- Organise and prepare synthetic datasets for machine learning
- Understand training pipelines for video generation models
Prerequisites:
Basic familiarity with 3D/graphics concepts. Understanding of meshes, materials, or 3D modeling is helpful but not required. Basic Python or scripting experience is useful for data pipeline automation, but not essential.
Familiarity with machine learning concepts is recommended such as a general understanding of datasets, train/validation splits, and model inference. We assume participants are familiar with general concepts of machine learning and deep learning. For an introduction to these topics, we refer to open resources:
Previous knowledge of Unreal Engine is essential, at least knowledge of viewport and blueprints.
To be well prepared, install in advance the Unreal Engine 5.1.* and a C++ Development IDE such as Visual Studio 2022, as well as ComfyUI with Trellis support. Details are available on https://gitlab.jsc.fz-juelich.de/hedgedoc/24tF29QsTe-ijNBHU1I5VQ?both where we are updating information as problems and questions arise.
A personal institutional email address (university/research institution, government agency, organisation, or company) is required to register for JSC training courses. If you don't have an institutional email address, please get in touch with the contact person for this course.
Target Audience:
- Scientists and engineers building synthetic datasets for computer vision, 3D reconstruction, or video generation research
- Developers of analysis pipelines that consume synthetic camera data or structured 3D scene parameters
- Researchers interested in controlled, reproducible virtual scene generation for model evaluation or training
- Engineers deploying Unreal Engine as a visualization or data-generation tool in scientific and industrial settings
- Anyone interested in understanding the complete pipeline from 3D scene creation to machine learning model training setup
Language:
This course is taught in English.
Duration:
2 days
Dates:
22-23 September 2026, 09:00-12:00, 13:00-16:00 each day
Venue:
Online via Zoom
Number of Participants:
Minimum 5, maximum 30
Instructor:
Thomas George, JSC
Fees
Free of charge
Pre-required logistics
Basic familiarity with 3D/graphics concepts. Understanding of meshes, materials, or 3D modeling is helpful but not required. Basic Python or scripting experience is useful for data pipeline automation, but not essential.
Familiarity with machine learning concepts is recommended such as a general understanding of datasets, train/validation splits, and model inference. We assume participants are familiar with general concepts of machine learning and deep learning.
Previous knowledge of Unreal Engine is essential, at least knowledge of viewport and blueprints.