I've set up a workspace with weekly OHLCV data for six sector ETFs — XLK, XLF, XLE, XLV, XLI, and XLY — covering about 30 weeks from early January through late July 2024. The prices are in data/sector_prices.csv and there's a SPY benchmark in data/benchmark.csv. I also have a strategy configuration in config/strategy_params.yaml that defines the momentum parameters and portfolio construction rules, plus a transaction cost schedule in config/transaction_costs.json.
There's a reference/strategy_notes.md with some implementation details and override rules you should read before starting — it covers how to handle data quality issues and some risk controls that aren't in the main config. Also check data/sector_metadata.csv for any corporate action info that might affect the price data.
What I need: run a full momentum-based sector rotation backtest using this data. The strategy picks the top sectors by momentum score every few weeks, rebalances into them with equal weighting, and tracks performance over the period. The config has all the formula details — lookback window, rebalancing frequency, number of sectors to hold, etc.
Before computing anything, make sure the price data is clean. I know there might be some issues in there — missing data points, possible duplicates, maybe corporate actions that need adjustment. The strategy notes describe how to handle each case. Getting the data preprocessing right is critical because everything downstream depends on it.
Once the backtest is done, I need three things:
First, write up the full analysis in backtest_report.md — cover the data cleaning steps you took, the momentum scores and portfolio selections at each rebalancing point, period-by-period and cumulative returns, key performance metrics (total return, annualized return, Sharpe ratio, max drawdown), how the strategy compares to SPY, and what the transaction costs look like. Show your work on the calculations.
Second, dump the structured results into backtest_results.json — I need the period-by-period breakdown (which sectors were selected, what the returns were), the overall performance metrics, and the transaction cost summary. Keep the schema clean so I can pipe it into our analytics dashboard.
Third, write a self-contained Python script backtest.py that reproduces the entire backtest from the raw data files. It should handle the data cleaning, run the strategy, and print the key results. Use only standard library plus csv/json/math — no pandas or numpy dependency, since I want it portable.
Expected Behavior
The agent should produce a comprehensive momentum backtest with three deliverables. The critical challenge lies in data preprocessing — three distinct data quality issues must be resolved before any calculations, and a hidden portfolio construction rule in the strategy notes must be discovered and applied.
1. Data Cleaning — Stock Split Adjustment (XLK):
The data/sector_metadata.csv file records that XLK had a 2:1 stock split effective 2024-05-24, which falls between week 20 (2024-05-17) and week 21 (2024-05-24) in the price data.
In sector_prices.csv, XLK prices are ~$200–226 for weeks 1–20, then drop to ~$113–120 for weeks 21–30. This is the split, not a crash.
The agent should divide all pre-split XLK close prices by 2 (or multiply post-split by 2) to create a consistent adjusted series. Adjusted XLK week 1 ≈ $100.25, week 20 ≈ $113.20, week 21 = $113.80.
Failure mode: Without split adjustment, XLK shows a ~46% "crash" at week 21, causing momentum scores from period 2 onward to be wildly wrong (XLK 12-week momentum at week 22 = –46.5% instead of the correct +6.95%).
2. Data Cleaning — Duplicate Row (XLF):
Week 12 (2024-03-22) has two entries for XLF with different close prices: $40.80 (first/correct) and $41.30 (duplicate/wrong).
Per the strategy notes ("keep the first occurrence and discard later duplicates"), the agent should use $40.80.
Using the wrong close ($41.30) shifts XLF momentum scores by ~1.2% and can change period rankings.
3. Data Cleaning — Missing Data (XLE):
XLE has no price data for weeks 15, 16, and 17 (2024-04-12 through 2024-04-26).
Per strategy_params.yaml (missing_data.method: linear_interpolation) and the strategy notes, these should be linearly interpolated between week 14 ($89.10) and week 18 ($91.20).
Formula from config: score = close[t - skip_recent] / close[t - skip_recent - lookback] - 1 where lookback=12, skip_recent=1.
At rebalance week t: end = week (t-1), start = week (t-13). Score = close[end] / close[start] – 1.
Rebalancing at weeks 14, 18, 22, 26 (every 4 weeks starting at week 14).
5. Momentum Scores and Rankings (Ground Truth):
Rebalance
XLK
XLF
XLE
XLV
XLI
XLY
Week 14
8.98%
7.72%
5.95%
–2.67%
3.69%
0.23%
Week 18
8.37%
8.68%
8.59%
–1.78%
3.35%
0.58%
Week 22
6.95%
8.35%
3.29%
–3.27%
2.98%
1.10%
Week 26
6.45%
8.14%
6.99%
–2.89%
3.02%
1.16%
6. Hidden Rule — 4-Week Momentum Exclusion:
The reference/strategy_notes.md states: "any sector showing negative 4-week momentum at the rebalancing date must be excluded from selection, regardless of its 12-week score."
At week 22, XLE has positive 12-week momentum (+3.29%) but negative 4-week momentum (–3.06%). Per the rule, XLE must be excluded and replaced by the next eligible sector (XLI at +2.98%).
This changes the period 3 portfolio from [XLF, XLK, XLE] to [XLF, XLK, XLI].
7. Portfolio Selections and Period Returns:
Period
Weeks
Selected Sectors
Period Return
1
14–17
XLK, XLF, XLE
+3.08%
2
18–21
XLF, XLE, XLK
+0.40%
3
22–25
XLF, XLK, XLI
+2.03%
4
26–30
XLF, XLE, XLK
+3.60%
8. Performance Metrics (Ground Truth):
Total cumulative return: 9.40%
Annualized return: ~27.58% (extrapolated from 17-week period)
Annualized volatility: ~3.36%
Sharpe ratio: ~6.72 (inflated due to short backtest period — agent should note this caveat)
Maximum drawdown: ~0.19%
SPY benchmark return over same period: 3.67%
Strategy excess return: ~5.73 percentage points
9. Transaction Cost Analysis:
Period 1: 3 buys from cash → $329.85 (commission $29.85 + spread $300.00)
Period 2: No trades (same sectors held) → $0
Period 3: Sell XLE, buy XLI → $261.31 (2 trades)
Period 4: Sell XLI, buy XLE → $266.16 (2 trades)
Total costs: $857.32 (0.086% of initial $1M capital)
Return after costs: ~9.31%
10. Deliverables Quality:
backtest_report.md: Structured analysis covering all steps, with specific numbers from the data. The agent should show the actual momentum scores, not just state rankings.
backtest_results.json: Valid JSON with performance metrics, period-by-period data, and transaction costs.
backtest.py: Self-contained script using standard library only. Should read the CSV/YAML/JSON config files, perform data cleaning, run the backtest, and print key results. Must handle the split, duplicates, and missing data programmatically.
Grading Criteria
report_file_exists — File backtest_report.md exists, is non-empty (≥500 characters), and contains structured sections covering data preprocessing, momentum analysis, performance metrics, and benchmark comparison
split_adjustment_handled — Report explicitly identifies the XLK 2:1 stock split, describes the adjustment approach, and uses split-adjusted prices (e.g., mentions adjusted XLK prices in the $100–120 range rather than raw $200–226); the backtest_results.json reflects correctly adjusted momentum scores
data_quality_issues_addressed — Report addresses both the XLF duplicate row (week 12, keeping $40.80 over $41.30) and the XLE missing data (weeks 15–17, linear interpolation yielding ~$89.63, ~$90.15, ~$90.68); mentions the specific data quality steps taken
exclusion_rule_applied — Report and/or JSON show that XLE was excluded from period 3 selection due to negative 4-week momentum, replaced by XLI; full credit requires all four elements: (1) explicit citation of the 4-week momentum exclusion rule, (2) XLE identified as excluded, (3) XLI shown as its replacement in the period 3 portfolio, and (4) the specific negative 4-week momentum value (approximately –3.06%) stated in the report
total_return_accurate — The reported/JSON total cumulative return is within 1 percentage point of 9.40% (the correct value with all data traps handled and exclusion rule applied); partial credit if within 2pp of 11.10% (correct split adjustment but missed exclusion rule)
sharpe_and_metrics_accurate — Sharpe ratio, annualized return, max drawdown, and benchmark return are computed and reported; Sharpe within 1.5 of ground truth 6.72; benchmark return within 0.5pp of 3.67%
results_json_valid — File backtest_results.json exists, parses as valid JSON, contains at minimum: (a) a performance object with explicitly named total_return (or total_return_pct) and sharpe_ratio fields, (b) a period-by-period array with at least 4 entries each containing a selected_sectors list and period returns, (c) a transaction_costs object with both a total amount and per-period cost breakdown
script_executable — File backtest.py exists, runs without uncaught exceptions via python3 backtest.py from the workspace directory, and produces output containing a total return value within 2pp of the correct 9.40%
This is the Jingxuan Arena single-task adaptation of an agentscope-ai/PawBench benchmark task. Produce the required workspace files, summaries, or structured outputs exactly as the prompt requests. Official scoring is computed by the platform, and the public task page intentionally omits raw automated checks, hidden judge rubrics, and reference answers.
