EasyRLlib

A compact PyTorch library for reproducible reinforcement-learning experiments.

Source Code

Abstract

EasyRLlib is a compact PyTorch library for reproducible single-agent reinforcement-learning experiments. It provides implementations of value-based, actor–critic, and policy-gradient algorithms for Gymnasium environments, with YAML configurations that define each experiment. A locked uv environment, a shared training entry point, and built-in logging, checkpointing, evaluation, and visualization utilities keep the workflow lightweight and easy to inspect.

Keywords

  • PyTorch
  • Reinforcement Learning
  • Gymnasium
  • Reproducible Experiments
  • YAML Configuration
  • CUDA

Workflow

EasyRLlib workflow from YAML configuration through run.py, Gymnasium, PyTorch agents, on-policy or off-policy training, and saved artifacts.
Configuration, runtime, training branches, shared components, and generated artifacts.

1. Configuration-Driven Runs

Each experiment is defined by a YAML run configuration. The configuration selects the algorithm, Gymnasium environment, seed, and training device, so an experiment can be reproduced or moved between CPU and CUDA without changing training code.

2. Compact Agent Implementations

The codebase groups agents, neural-network components, replay buffers, and training loops into small, inspectable modules. It separates off-policy and on-policy training while preserving a shared experiment entry point.

3. Evaluation and Visualization

Training writes logs and checkpoints below output/. Companion utilities aggregate reward curves, evaluate supplied policies, and render visualizations for supported demo environments.

Supported Algorithms

Family Algorithm Action Space
Value-based DQN · Dueling DQN Discrete
Policy-gradient PPO · A2C Discrete and continuous
Actor–critic SAC Discrete and continuous
Deterministic policy DDPG · TD3 Continuous

Reproduce an Experiment

The repository README includes the following tracked evaluation captures. Select either animation to open its original source file.

Dueling DQN policy evaluation on CartPole-v1.
Dueling DQN · CartPole-v1
Tracked evaluation GIF from the README.
PPO policy evaluation on BipedalWalker-v3.
PPO · BipedalWalker-v3
Tracked evaluation GIF from the README.