feat(trading): self-improving MetaTrader 5 trading bot with risk-first execution - #3216
Open
gareth-prog wants to merge 2 commits into
Open
feat(trading): self-improving MetaTrader 5 trading bot with risk-first execution#3216gareth-prog wants to merge 2 commits into
gareth-prog wants to merge 2 commits into
Conversation
Adds a standalone systematic trading bot under ai-trading-bot/. It generates signals from a strategy book, learns which strategies work in which market regimes, sizes every position from a fixed risk budget, and can ingest externally-described strategies through a validation gauntlet. Why the design looks like this: A dollar-per-week profit target is only meaningful relative to account size, so goal/controller.py derives the required weekly return from live equity and classifies it (comfortable / aggressive / unlikely / infeasible). The load-bearing property is that the goal can only scale risk DOWN -- `scale` is clamped to <= 1.0 -- so falling behind target can never talk the bot into betting bigger. That feedback loop is the usual way accounts die, and here it is structurally impossible rather than discouraged. Risk decisions live in exactly one place (risk/manager.py) and it is the only component that can approve a trade. Sizing is risk-first: a wider stop buys a smaller position, so the dollar loss on a stop-out is invariant to instrument and volatility. Circuit breakers cover daily loss, weekly loss, max drawdown (hard halt, manual reset) and a file-based kill switch; guard state persists so a crash loop cannot reset the daily limit. Learning runs on two independent mechanisms. A discounted Thompson-sampling bandit over (strategy, regime) learns what works and suppresses what stops working, using R-multiple rather than profit so statistics survive changes in account size. A gradient-boosting meta-model gates individual setups -- and is refused outright if it cannot beat AUC 0.55 on a forward-chained holdout, since a filter with no demonstrated skill would just be an expensive random number generator. The research pipeline converts strategy descriptions into testable specs. It never eval()s extracted text: research/safe_expr.py parses to an AST and rejects any node off an allowlist (no attribute access, subscripting, imports, comprehensions or string literals). Extracted strategies are inert until they clear backtest, walk-forward, cost-shock and cross-instrument robustness, plus forward paper testing and a human ack. Martingale and grid sizing are refused at ingest rather than merely unsupported. The backtester is deliberately pessimistic: stops win intrabar ties, signals act on the next bar's open, and costs are charged both ways. A regression test asserts it LOSES money on a random walk -- if that ever passes, it is cheating. Live safety: real-money trading needs two independent keys (mode: live AND allow_live: true) plus interactive confirmation. Paper mode is the default and runs the identical code path through the same Broker protocol. Credentials come from the environment or a gitignored file, never from config.yaml. Verification: 265 tests pass, including end-to-end integration tests that drive the real TradingEngine (signals -> orders -> closes -> learning), property sweeps asserting the per-trade risk ceiling and the goal-scale invariant across thousands of input combinations, indicator causality checks, and 22 sandbox escape attempts. Three bugs found and fixed by those tests: - a position exactly at +1R computed r=0.9999999999999556 from float subtraction, so the breakeven stop never applied and the trade could reverse into a full loss; stops and targets both derive from the same ATR, so hitting the threshold exactly is routine - strategy extraction mirrored the non-directional filter "ADX > 25" into "ADX < 25" for the short leg, silently inverting a trend filter into a chop filter; a leg of pure filters with no entry trigger would also have fired on most bars - walk-forward laid folds out from bar 0, silently dropping the first window for lack of training data, so a caller asking for 4 windows got 3 and read it as a failure rather than a data limit Also fixed an O(n^2) backtest: strategies recomputed indicators over full history each bar. Now linear at ~320 bars/sec with byte-identical results. Co-Authored-By: RuFlo <ruv@ruv.net>
The MT5 Python bridge reports (-6, 'Terminal: Authorization failed') for every
authorization problem. An expired demo, a wrong server, an investor password
and a terminal with no account logged in are indistinguishable from that
string, so doctor was telling people to "start MetaTrader 5 and log in" when
the terminal was already running and they had already tried.
The terminal itself knows more -- it writes the broker's actual words to its
log ("Invalid account", "Invalid password", "no connection to <server>") -- but
those logs are UTF-16-LE and effectively unreadable by hand.
Adds ops/mt5_diagnostics.py to parse them and report the real cause, plus the
likely fix. Against the local install this turns an unactionable error code
into: "account 124578369 on ICMarketsSC-Demo: authorization failed (Invalid
account)" with an explanation that this usually means an expired demo or the
right login pointed at the wrong server.
Also:
- Pin mt5.terminal_path in config.yaml. With several terminals installed,
mt5.initialize() attaches to whichever one the OS hands it, so the bot could
silently trade a different account than intended. Two are running on this
machine, which is how the ambiguity surfaced.
- Fix the doctor failure count. Explanatory rows are now marked ok=None so they
render without being counted as separate failures (it reported "3 checks
failed" for one underlying problem).
- Document the failure modes and the multi-terminal hazard in the README.
Diagnostics are defensive by design: unreadable, corrupt or absent logs degrade
to the previous generic message rather than breaking the check that is meant to
be diagnosing the problem.
15 new tests covering UTF-16 decoding, each rejection kind, most-recent-outcome
precedence, and corrupt input. Full suite: 280 passing.
Note this does not by itself make doctor pass here -- the account is rejected
by the broker as invalid, which needs valid credentials rather than a code
change. It makes the reason legible.
Co-Authored-By: RuFlo <ruv@ruv.net>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a standalone systematic trading bot for MetaTrader 5 under
ai-trading-bot/. It generates signals from a strategy book, learns which strategies work in which market regimes, sizes every position from a fixed risk budget, and can ingest externally-described strategies through a validation gauntlet.Self-contained: no dependency on the ruflo framework, and nothing outside
ai-trading-bot/is touched.Why the design looks like this
A dollar-per-week target is only meaningful relative to account size.
goal/controller.pyderives the required weekly return from live equity and classifies it (comfortable / aggressive / unlikely / infeasible). $1000/wk is 0.5%/wk on $200k and 20%/wk on $5k — the second is not a strategy, it's a wish.The load-bearing property: the goal can only scale risk DOWN.
scaleis clamped to<= 1.0, so falling behind target can never talk the bot into betting bigger. That feedback loop — lose, size up to catch up, lose bigger — is the usual way accounts die, and here it is structurally impossible rather than discouraged.test_scale_never_exceeds_oneasserts it across thousands of input combinations.Risk decisions live in exactly one place (
risk/manager.py) and it is the only component that can approve a trade. Sizing is risk-first: a wider stop buys a smaller position, so the dollar loss on a stop-out is invariant to instrument and volatility. Circuit breakers cover daily loss, weekly loss, max drawdown (hard halt, manual reset only) and a file-based kill switch. Guard state persists, so a crash loop cannot be used to reset the daily limit.Learning runs on two independent mechanisms. A discounted Thompson-sampling bandit over
(strategy, regime)learns what works and suppresses what stops working, using R-multiple rather than profit so statistics survive changes in account size. A gradient-boosting meta-model gates individual setups — and is refused outright if it cannot beat AUC 0.55 on a forward-chained holdout, since a filter with no demonstrated skill would just be an expensive random number generator.The research pipeline never
eval()s extracted text. It converts strategy descriptions into testable specs;research/safe_expr.pyparses to an AST and rejects any node off an allowlist (no attribute access, subscripting, imports, comprehensions, or string literals). This matters because extracted rules would otherwise run on a machine logged into a brokerage account. Extracted strategies stay inert until they clear backtest → walk-forward → cost-shock → cross-instrument robustness, plus forward paper testing and a human ack. Martingale and grid sizing are refused at ingest rather than merely unsupported.The backtester is deliberately pessimistic: stops win intrabar ties, signals act on the next bar's open, costs are charged both ways. A regression test asserts it loses money on a random walk — if that ever passes, it's cheating.
Live-trading safety
Real-money trading requires two independent keys (
mode: liveandallow_live: true) plus interactive confirmation. Paper mode is the default and runs the identical code path through the sameBrokerprotocol. Credentials come from the environment or a gitignored file, never fromconfig.yaml.Verification
265 tests pass, including end-to-end integration tests that drive the real
TradingEngine(signals → orders → closes → learning), property sweeps on the per-trade risk ceiling and the goal-scale invariant, indicator causality checks, and 22 sandbox escape attempts.Three bugs found and fixed by those tests:
r = 0.9999999999999556from float subtraction, so the breakeven stop never applied and the trade could reverse into a full loss. Stops and targets both derive from the same ATR, so hitting the threshold exactly is routine, not a corner case.ADX > 25intoADX < 25for the short leg, silently inverting a trend filter into a chop filter. A leg of pure filters with no entry trigger would also have fired on most bars.Also fixed an O(n²) backtest (strategies recomputed indicators over full history each bar). Now linear at ~320 bars/sec with byte-identical results.
Reviewer notes
MetaTrader5is Windows-only and marked as an optional extra, so it does not affect CI on other platforms.rufloat all? It's fully standalone and might be better as its own repository — happy to relocate it.🤖 Generated with RuFlo