|

Skyfall AI Releases MORPHEUS: A Persistent Enterprise Simulation Benchmark That Makes Continual Reinforcement Learning Necessary Under Structured Non-Stationarity

ByRicardo

Most reinforcement studying benchmarks reset the world after each episode. Real operations by no means reset. Skyfall AI’s MORPHEUS targets that hole. It is a persistent enterprise simulation platform for continuous reinforcement studying (CRL).

What is MORPHEUS?

MORPHEUS is grounded within the Big World Hypothesis (Javed & Sutton, 2024). It says the world’s complexity exceeds any agent’s representational capability. As a outcome, the atmosphere seems non-stationary even beneath fastened dynamics.

To power continuous studying, MORPHEUS requires three properties: persistence, non-stationarity, and operational complexity. Persistence means previous choices compound into future dynamics. Non-stationarity means any fastened coverage ultimately turns into suboptimal. Operational complexity means no fastened optimum coverage exists.

Each atmosphere is a self-contained TypeScript world plugin. It exports Operational Descriptors (ODs), a simulation scheduler, seed knowledge, and documentation. An OD defines the step-by-step execution plan for a functionality. Agents act via a functionality API, and every name triggers an OD execution.

How the Platform Works?

Building on that structure, non-stationarity comes from two engines. First, a failure injection engine inserts typed disruptions between OD steps. It attracts from eleven failure sorts, together with missing_data, dependency_failure, and rate_limit. It runs at 4 preset charges: mild (5%), reasonable (8%), reasonable (15%), and aggressive (30%).

Second, an asynchronous configuration shift controller adjustments failure presets and demand at fastened timestamps. It runs independently of the coaching loop, so shifts by no means align with gradient updates. This stops the agent from utilizing replace periodicity as a proxy clock.

Alongside these engines, reward comes from three operational verifiers logged natively by the platform. These are failure occasion indicators, monetary ledger standing, and useful resource throughput. The composite reward combines them. Default weights are w_f = 0.5 and w_l = w_p = 0.25.

Copy Code
# Composite reward — MORPHEUS, Appendix C (default weights).
def clip(x, lo, hello):
    return max(lo, min(hello, x))

def composite_reward(tickets, actual_cost, planned_cost, items, capability,
                     w_f=0.5, w_l=0.25, w_p=0.25):
    r_f = -sum(t["severity"] for t in tickets)          # failure occasion sign
    r_l = clip(1 - actual_cost / planned_cost, -1, 1)   # monetary ledger
    r_p = clip(items / capability, 0, 1)                  # useful resource throughput
    return w_f * r_f + w_l * r_l + w_p * r_p

Under the upper-bound assumptions (zero failures, minimal value, full throughput), the certain per configuration equals 0.50.

Policy Initialisation

Because the motion house is massive, pure RL from scratch is impractical. Therefore MORPHEUS makes use of a two-stage pipeline. A frontier mannequin (Gemini 3.1 professional) collects trajectories utilizing the ReAct framework. These traces then fine-tune Qwen3-14B by way of supervised fine-tuning (SFT).

Consequently, each RL run begins from this shared SFT checkpoint. This isolates continuous studying behaviour from primary operational competence. All baselines then use PPO as the bottom optimizer for on-line post-training.

The Six-Metric Evaluation Protocol

With coaching outlined, cumulative reward alone isn’t sufficient. A scalar sum hides efficiency throughout a non-stationary horizon. So the analysis workforce suggest six metrics as a substitute. These are per-configuration reward, adaptation pace, forgetting, restoration time, stability, and efficiency hole.

Among these, adaptation pace is the headline metric. It counts steps till the running-average reward reaches half the higher certain. Two supplementary diagnostics additionally monitor relative adaptation benefit (RAA) and plasticity by way of efficient rank.

Baseline Results

Using this protocol, the analysis workforce exams 4 algorithm households from the shared SFT checkpoint. Two duties are outlined. Task 1 is dynamic useful resource allocation beneath structured drift. Task 2 is scheduling beneath drift with delayed results.

Family Mechanism Outbound Task 1 Outbound Task 2 Inbound Task 2
PPO No CL mechanism Failure baseline Adapts solely early Baseline reward
HER Hindsight replay Mid reward Best reward Best reward, prime rank
EWC Weight consolidation Best reward Best adaptation Weakest reward
LCM Latent context mannequin Fastest adaptation No benefit Best adaptation

Across these outcomes, no single household dominates. On process-outbound Task 1, EWC leads reward and LCM adapts quickest. On Task 2, HER leads reward whereas LCM loses its edge beneath delayed reward. Meanwhile, imply efficiency gaps sit close to 1.0 for each methodology. That indicators a big settled-state deficit, not a minor tuning hole.

Notably, PPO and HER typically adapt solely within the first configuration. They then fail to adapt in later regimes, even with out label indicators.

Use Cases with Examples

In follow, MORPHEUS fits a number of reader roles. For AI engineers, it exams whether or not an agent detects regime shifts with out labels. For instance, demand switches from low to bursty, and the coverage should adapt with no sign.

For knowledge scientists, it stresses delayed credit score project. For instance, On-Time In-Full (OTIF) supply is observable solely days after the dispatch determination. For software program engineers, the TypeScript plugin format permits swapping rewards or toggling observability with out altering dynamics.

Strengths and Weaknesses

Strengths:

  • Persistent worlds with no resets, matching deployed enterprise methods.
  • Parameterisable, reproducible regime shifts for honest cross-algorithm comparability.
  • Rewards from native operational verifiers, needing no exterior annotation.
  • Open-sourced analysis code (Skyfall-Research/morpheus-evals).

Weaknesses:

  • Only two of 5 environments are evaluated thus far.
  • The higher certain assumes zero failures, so it stays optimistic.
  • Shifts are externally triggered, not pushed by compounding choices.
  • Reward weights are analysis variables, not validated trade goals.

Key Takeaways

  • MORPHEUS runs persistent enterprise worlds that by no means reset, not like episodic RL benchmarks.
  • It ships 5 environments; two are evaluated right here: process-outbound and process-inbound.
  • A six-metric protocol scores per-configuration reward, adaptation, forgetting, restoration, stability, and gap-to-upper-bound.
  • Four baselines (PPO, HER, EWC, LCM) all sit far beneath the theoretical higher certain.
  • No single algorithm wins; reward and adaptation pace choose completely different winners.

Check out the Paper and Project Page. Also, be at liberty to observe us on Twitter and don’t neglect to affix our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

Need to companion with us for selling your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar and many others.? Connect with us

The publish Skyfall AI Releases MORPHEUS: A Persistent Enterprise Simulation Benchmark That Makes Continual Reinforcement Learning Necessary Under Structured Non-Stationarity appeared first on MarkTechPost.

Similar Posts