Risk-aware reinforcement learning beyond average return

The decision: ask about the loss tail

Two distributions can have the same mean and very different extremes. The lab uses ten equally likely invented losses to isolate this point. Both presets average 13.2 loss units. For the spread-out preset, the worst two losses are 25 and 34; for the concentrated preset, they are 26 and 65.

With a 20% upper tail, their CVaR values are therefore 29.5 and 45.5. The difference is a property of these explicit synthetic losses. It is not a portfolio return, a trading recommendation, or a reproduced backtest.

The consequence: forecasting and deciding stay distinct

The published architecture combines LSTM–GRU forecasting, sequential allocation through deep reinforcement learning, a CVaR-oriented risk objective, and meta-optimization. A forecast enters the decision process; it is not itself an allocation policy.

The browser experiment isolates only a loss-plus-CVaR teaching objective. At λ = 0.5, the two objectives are 27.95 and 35.95. Setting λ to zero removes the tail penalty and returns both objectives to the same mean. This makes the preference encoded by the objective inspectable without implying that the trained agent uses this exact simplified expression.

What must be held constant in a comparison

Changing a tail size changes which losses enter the estimate. Changing a penalty changes how that estimate is valued. These are different sensitivity questions, so the lab exposes separate controls and records both in a shared experiment link.

An evaluation of the full forecasting-and-control pipeline would also need to inspect temporal splits, information available at each decision, relevant baselines, transaction assumptions, and stability under changed conditions. The teaching example supplies no evidence that those conditions have been met.

• Keep chronological evaluation separate from training and parameter selection.
• Inspect risk estimation across more than one tail size or penalty.
• Distinguish a favorable simulated outcome from evidence of deployment reliability.

The boundary: arithmetic, not a trading system

The small Python package checks the invented losses, their mean, an empirical left-quantile VaR convention, tail-average CVaR, and the illustrative objective. The convention is explicit because finite-sample quantile definitions can differ.

No public full-paper implementation is linked from this portfolio. The package does not claim to reproduce trained forecasters, reinforcement-learning agents, meta-optimization, or market results. It is an educational instrument for discussing how a risk objective changes a decision criterion.