Skip to content

Commit 4bd36f5

Browse files
zhouyuchen.zycclaude
zhouyuchen.zyc
and
claude
committed
Add leakage-aware splits, results snapshot, and benchmark skill
Benchmark split correctness: - Census multitask now uses the dataset's official train/test partition (census-income.data / .test) when present, instead of randomly reshuffling the two together; encoders are fit on the union so test-only categories don't crash the run. Falls back to a random split for the bundled sample. - Single-task gains --temporal-split / --time-col: a chronological hold-out (most recent test_size as test) for time-ordered logs, to avoid look-ahead leakage. Default stays random (Criteo_x1/DAC is anonymised and shuffled with no timestamp, so random is the standard leakage-free split). - Add tests covering both guarantees; document the flags in benchmarks/README. Also: benchmarks/RESULTS.md captures the real-data leaderboards (Criteo 500k single-task, official-split Census multitask), and a project skill .claude/skills/deepctr-benchmark records how to re-run the suite quickly (CPU-only on this host, data locations, split semantics). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 8f30a73 commit 4bd36f5

7 files changed

Lines changed: 322 additions & 7 deletions

File tree

‎.claude/skills/deepctr-benchmark/SKILL.md‎

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
---
2+
name: deepctr-benchmark
3+
description: >-
4+
Run the DeepCTR model benchmark suite (benchmarks/) to validate and compare
5+
CTR models end-to-end on real or bundled data, producing a sorted leaderboard.
6+
Use when asked to benchmark, compare, validate, or measure DeepCTR models
7+
(single-task CTR, multitask, or sequence), or to re-run / extend the
8+
benchmark after a code change.
9+
---
10+
11+
# DeepCTR benchmark suite
12+
13+
A self-contained harness (`benchmarks/`) that trains DeepCTR models and writes a
14+
ranked leaderboard. Three tracks: `single` (21 single-task CTR models, Criteo),
15+
`multitask` (SharedBottom/ESMM/MMOE/PLE, Census-Income), `sequence`
16+
(DIN/BST/DSIN, DIEN skipped on TF≥2.0). Full docs: `benchmarks/README.md`.
17+
Curated past results: `benchmarks/RESULTS.md`.
18+
19+
## Always do these two things first
20+
21+
1. **Force CPU** with `CUDA_VISIBLE_DEVICES=""`. The GPU on this host has a
22+
CUDA/PTX mismatch — every model fails with `CUDA_ERROR_UNSUPPORTED_PTX_VERSION`
23+
on GPU. The harness isolates each model, so on GPU you get an all-`failed`
24+
leaderboard rather than a crash; the fix is always CPU.
25+
2. **Use legacy Keras**: `TF_USE_LEGACY_KERAS=1` (DeepCTR targets the Keras 2
26+
API; on TF≥2.16 that means `tf-keras`). The package sets this itself, but set
27+
it explicitly when invoking pytest.
28+
29+
Run long sweeps in the background and poll the log — a full 21-model single-task
30+
sweep on real data is several minutes to ~30 min on CPU.
31+
32+
## Data on this host
33+
34+
Real data lives in `benchmarks/data/` (git-ignored). If absent, regenerate or
35+
re-download (see README; the built-in Criteo DAC URL is dead — anonymous HF
36+
mirrors rate-limit, so a `HF_TOKEN` is needed for `reczoo/Criteo_x1`).
37+
38+
| File | Rows | Use |
39+
| ---- | ---- | --- |
40+
| `benchmarks/data/criteo_x1_500k.csv` | 500,000 | single-task, meaningful AUC |
41+
| `benchmarks/data/criteo_x1_200k.csv` | 200,000 | single-task, faster |
42+
| `benchmarks/data/census-income.data` (+ `.test`) | 199,523 (+99,762) | multitask, official split |
43+
| `examples/criteo_sample.txt` | 200 | smoke only — AUC is noise |
44+
45+
## Commands
46+
47+
```bash
48+
# Fast smoke test (1 epoch, 2 models/track, bundled data) — verify the harness
49+
CUDA_VISIBLE_DEVICES="" python -m benchmarks.benchmark --track all --quick
50+
51+
# Single-task, real Criteo 500k — the headline comparison
52+
CUDA_VISIBLE_DEVICES="" python -m benchmarks.benchmark --track single \
53+
--data-path benchmarks/data/criteo_x1_500k.csv \
54+
--epochs 1 --batch-size 1024 --val-split 0 --seed 2020
55+
56+
# Multitask, real Census with its official train/test partition
57+
CUDA_VISIBLE_DEVICES="" python -m benchmarks.benchmark --track multitask \
58+
--data-path benchmarks/data/census-income.data --epochs 3 --batch-size 1024
59+
60+
# Subset / exclude slow models; e.g. drop the interaction-heavy ones to save time
61+
# --models DeepFM,DCN,xDeepFM run only these
62+
# --exclude FiBiNET,ONN,DeepFEFM,FwFM,FGCNN ONN alone is 77M params / ~245s
63+
```
64+
65+
Leaderboards print to stdout and write `benchmarks/results/<track>_<dataset>.csv`
66+
and `.md`. To preserve a run, copy the numbers into `benchmarks/RESULTS.md`
67+
(the `results/` dir is git-ignored).
68+
69+
## Train / test split — get this right
70+
71+
- **Default = random split**, `--test-size 0.2`, fixed `--seed`. Correct for
72+
Criteo_x1 / DAC, which is anonymised and shuffled with no timestamp (so a
73+
random split is leakage-free and standard).
74+
- **`--temporal-split` (+ optional `--time-col COL`)**: chronological hold-out —
75+
the most recent `test_size` fraction becomes the test set, so no future row
76+
leaks into training. Sorts by `--time-col` when present, else trusts file
77+
order. Use only for time-ordered logs; the bundled Criteo has no timestamp.
78+
- **Census official split**: when `--data-path` is `census-income.data` and a
79+
sibling `census-income.test` exists, the loader uses that official partition
80+
automatically (encoders fit on the union). Don't reshuffle it.
81+
- **`--val-split`** carves validation from *train* but is **not wired to any
82+
early-stopping/checkpoint** — it does not influence training or model
83+
selection. For short fixed-epoch runs set `--val-split 0` so training uses the
84+
full train split; only keep it if you add an EarlyStopping callback.
85+
86+
## Interpreting results
87+
88+
- Reported AUC/LogLoss are on the held-out **test** set (never seen in `fit`).
89+
- Need enough data for a real ranking: on 200-row bundled data AUC ≈ 0.5 noise;
90+
on 200k–500k the deep interaction models (DeepFEFM/FiBiNET/DeepFM/xDeepFM)
91+
separate from simpler ones. More epochs raise absolute AUC but rarely flip the
92+
top group.
93+
- Always report the split and epoch count alongside the leaderboard — a ranking
94+
is only comparable within the same data/split/epochs.
95+
96+
## Tests
97+
98+
```bash
99+
CUDA_VISIBLE_DEVICES="" TF_USE_LEGACY_KERAS=1 python -m pytest tests/benchmark_test.py -q
100+
```
101+
102+
Covers each track on tiny data plus the split guarantees (temporal hold-out
103+
keeps the most-recent rows as test; Census uses the official `.test`).

‎benchmarks/README.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,13 @@ Uses the bundled `examples/census-income.sample`; pass the full UCI
7575
Census-Income KDD `.data` file via `--data-path`. Two derived binary tasks:
7676
`label_income` (>50k) and `label_marital` (never-married).
7777

78+
> When `--data-path` points at `census-income.data` and its sibling
79+
> `census-income.test` exists, the loader uses the dataset's **official
80+
> train/test partition** (199,523 / 99,762 rows) instead of a random split —
81+
> the canonical evaluation. Encoders are fit on the union so test-only
82+
> categories don't crash the run. The bundled sample has no `.test` sibling and
83+
> falls back to a random split.
84+
7885
> ESMM assumes sequentially-dependent CTR→CTCVR tasks; on Census it still runs,
7986
> but its numbers are not directly comparable to the other multitask models.
8087
@@ -100,6 +107,7 @@ Amazon-Electronics is the canonical heavy benchmark and is left as an extension.
100107
| `--batch-size` | 256 | |
101108
| `--embedding-dim` | 8 | uniform sparse embedding size |
102109
| `--val-split` / `--test-size` | 0.2 / 0.2 | |
110+
| `--temporal-split` / `--time-col` | off / – | single-task: chronological hold-out (most recent `--test-size` as test) instead of a random split — for time-ordered logs, to avoid leakage. Sorts by `--time-col` when given, else trusts file order. (The bundled/Criteo_x1 data is anonymised & shuffled with no timestamp, so it stays on the random split.) |
103111
| `--seed` | 2020 | seeds python/numpy/TF |
104112
| `--quick` | off | 1 epoch, 2 models/track, small data |
105113
| `--source` / `--download` / `--data-path` | bundled | Criteo source |

‎benchmarks/RESULTS.md‎

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
# DeepCTR benchmark results
2+
3+
Real-data leaderboards produced by `benchmarks/benchmark.py`. The raw
4+
`benchmarks/results/*.{csv,md}` are git-ignored (generated); this file is the
5+
curated, version-controlled snapshot.
6+
7+
> **Environment note.** All runs below were executed **CPU-only**
8+
> (`CUDA_VISIBLE_DEVICES=""`) because the GPU on the run host had a CUDA/PTX
9+
> mismatch. TensorFlow 2.19 + `tf-keras` 2.19 (legacy Keras,
10+
> `TF_USE_LEGACY_KERAS=1`). Absolute AUCs are not tuned maxima — the value is in
11+
> the **relative ranking and cost** under a fixed, leakage-free split.
12+
13+
## Single-task CTR — real Criteo `criteo_x1_500k.csv`
14+
15+
500,000 samples · **random split, seed=2020 → 400,000 train / 100,000 test**
16+
(no validation carve-out: `--val-split 0`, since 1 epoch + no early stopping
17+
makes a validation set dead weight). 1 epoch, batch 1024, embedding-dim 8.
18+
The Criteo_x1 (Kaggle DAC) data is anonymised and shuffled with no timestamp,
19+
so a random split is the standard, leakage-free choice.
20+
21+
| Rank | Model | AUC | LogLoss | Params | Train(s) |
22+
| ---- | ----- | --- | ------- | ------ | -------- |
23+
| 1 | DeepFEFM | 0.7817 | 0.4664 | 3.6M | 130.4 |
24+
| 2 | FiBiNET | 0.7797 | 0.4680 | 4.8M | 104.8 |
25+
| 3 | DeepFM | 0.7794 | 0.4682 | 3.5M | 13.2 |
26+
| 4 | xDeepFM | 0.7793 | 0.4683 | 3.8M | 39.1 |
27+
| 5 | FwFM | 0.7791 | 0.4684 | 3.5M | 103.1 |
28+
| 6 | DCN | 0.7790 | 0.4687 | 3.5M | 16.3 |
29+
| 7 | PNN | 0.7790 | 0.4687 | 3.2M | 14.8 |
30+
| 8 | WDL | 0.7790 | 0.4685 | 3.5M | 13.7 |
31+
| 9 | FNN | 0.7789 | 0.4686 | 3.1M | 10.0 |
32+
| 10 | FLEN | 0.7789 | 0.4686 | 3.5M | 15.3 |
33+
| 11 | AutoInt | 0.7787 | 0.4688 | 3.5M | 32.5 |
34+
| 12 | DCNMix | 0.7786 | 0.4690 | 3.6M | 47.1 |
35+
| 13 | NFM | 0.7779 | 0.4695 | 3.5M | 15.8 |
36+
| 14 | ONN | 0.7602 | 0.4955 | 77.0M | 244.7 |
37+
| 15 | DIFM | 0.7525 | 0.4879 | 3.6M | 29.5 |
38+
| 16 | MLR | 0.7499 | 0.4935 | 3.0M | 26.6 |
39+
| 17 | FGCNN | 0.7495 | 0.4896 | 8.4M | 122.6 |
40+
| 18 | IFM | 0.7447 | 0.4927 | 3.5M | 14.4 |
41+
| 19 | CCPM | 0.7445 | 0.4934 | 3.4M | 20.6 |
42+
| 20 | EDCN | 0.7395 | 0.4970 | 3.7M | 18.0 |
43+
| 21 | AFM | 0.7282 | 0.5092 | 3.4M | 19.7 |
44+
45+
**Takeaways**
46+
47+
- Explicit feature-interaction deep models (**DeepFEFM, FiBiNET, DeepFM,
48+
xDeepFM**) lead once there is enough data. An earlier 200k / 3-epoch run that
49+
excluded the slow interaction models had MLR/NFM on top — a small-data
50+
artifact, not a real ranking.
51+
- The top 13 models sit in a tight 0.778–0.782 band (1 epoch is not converged),
52+
but are clearly separated from the tail (AFM 0.728, EDCN 0.740).
53+
- **Cost outlier: ONN** — 77M params and 245s to train for only 0.76 AUC.
54+
DeepFEFM/FiBiNET/FwFM/FGCNN cost ~100–130s (≈10× DeepFM's 13s); only
55+
DeepFEFM/FiBiNET earn it with a top-of-board AUC.
56+
57+
## Multitask — real UCI Census-Income (official split)
58+
59+
**Official train/test partition: 199,523 train / 99,762 test** (loader uses the
60+
shipped `census-income.test` when present, not a random reshuffle). 3 epochs,
61+
batch 1024. Tasks: `label_income` (>50k), `label_marital` (never-married).
62+
63+
| Rank | Model | income AUC | marital AUC | mean_AUC | Params | Train(s) |
64+
| ---- | ----- | ---------- | ----------- | -------- | ------ | -------- |
65+
| 1 | SharedBottom | 0.9465 | 0.9943 | 0.9704 | 116K | 8.1 |
66+
| 2 | MMOE | 0.9465 | 0.9942 | 0.9704 | 308K | 11.0 |
67+
| 3 | PLE | 0.9462 | 0.9943 | 0.9703 | 424K | 17.3 |
68+
| 4 | ESMM | 0.5632 | 0.9751 | 0.7692 | 211K | 8.5 |
69+
70+
**Takeaways**
71+
72+
- On Census the three general multitask architectures are statistically
73+
indistinguishable (mean_AUC 0.9703–0.9704); the task is easy enough that the
74+
extra capacity of MMOE/PLE buys nothing here. SharedBottom wins on cost.
75+
- **ESMM is far behind** (income AUC 0.56) — expected: it assumes
76+
sequentially-dependent CTR→CTCVR tasks, which Census does not satisfy. Its
77+
numbers are not directly comparable.
78+
79+
## Sequence — synthetic
80+
81+
DIN / BST / DSIN build, train and predict; **DIEN is skipped on TF≥2.0** (its
82+
GRU/AUGRU layers need legacy TF1 private RNN APIs). Synthetic AUC is ~0.5 by
83+
design — the synthetic track verifies the models run and compares their cost,
84+
not predictive power. Use `--seq-source movielens`/Amazon for real numbers.
85+
86+
## Reproduce
87+
88+
See `.claude/skills/deepctr-benchmark` for the exact commands, or:
89+
90+
```bash
91+
CUDA_VISIBLE_DEVICES="" python -m benchmarks.benchmark --track single \
92+
--data-path benchmarks/data/criteo_x1_500k.csv \
93+
--epochs 1 --batch-size 1024 --val-split 0 --seed 2020
94+
95+
CUDA_VISIBLE_DEVICES="" python -m benchmarks.benchmark --track multitask \
96+
--data-path benchmarks/data/census-income.data --epochs 3 --batch-size 1024
97+
```

‎benchmarks/benchmark.py‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,7 @@ def run_single(args):
7272
hash_buckets=args.hash_buckets,
7373
test_size=args.test_size, seed=args.seed, max_rows=args.max_rows,
7474
download_url=args.download_url,
75+
temporal_split=args.temporal_split, time_col=args.time_col,
7576
)
7677
names = select(list(SINGLE_TASK_MODELS), args.models, args.exclude)
7778
if args.quick:
@@ -213,6 +214,11 @@ def build_parser():
213214
p.add_argument("--batch-size", type=int, default=256)
214215
p.add_argument("--val-split", type=float, default=0.2)
215216
p.add_argument("--test-size", type=float, default=0.2)
217+
p.add_argument("--temporal-split", action="store_true",
218+
help="single-task: chronological hold-out (most recent test_size as test) "
219+
"instead of a random split; use for time-ordered logs to avoid leakage")
220+
p.add_argument("--time-col", default=None,
221+
help="single-task --temporal-split: column to sort by before the positional split")
216222
p.add_argument("--embedding-dim", type=int, default=8)
217223
p.add_argument("--seed", type=int, default=2020)
218224
p.add_argument("--fit-verbose", type=int, default=0, help="passed to model.fit(verbose=...)")

‎benchmarks/datasets/census.py‎

Lines changed: 33 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -45,9 +45,35 @@
4545

4646

4747
def load_census(data_path=None, embedding_dim=4, test_size=0.2, seed=2020):
48-
"""Load and preprocess Census-Income into a :class:`MultiTaskData`."""
49-
path = data_path or os.path.join(EXAMPLES_DIR, "census-income.sample")
50-
data = pd.read_csv(path, header=None, names=COLUMN_NAMES)
48+
"""Load and preprocess Census-Income into a :class:`MultiTaskData`.
49+
50+
The UCI Census-Income (KDD) distribution ships an *official* train/test
51+
partition: ``census-income.data`` (train) and ``census-income.test`` (test).
52+
When ``data_path`` points at the ``.data`` file and its sibling ``.test``
53+
exists, that official split is used instead of a random one -- this is the
54+
canonical evaluation for the dataset and avoids reshuffling the two together.
55+
Encoders are still fit on the union so categories that appear only in the
56+
test partition don't crash the run. The bundled sample has no ``.test``
57+
sibling and falls back to a random split.
58+
"""
59+
train_path = data_path or os.path.join(EXAMPLES_DIR, "census-income.sample")
60+
61+
official_test = None
62+
if data_path and data_path.endswith(".data"):
63+
sibling = data_path[: -len(".data")] + ".test"
64+
if os.path.exists(sibling):
65+
official_test = sibling
66+
67+
if official_test:
68+
train_df = pd.read_csv(train_path, header=None, names=COLUMN_NAMES)
69+
test_df = pd.read_csv(official_test, header=None, names=COLUMN_NAMES)
70+
n_train = len(train_df)
71+
data = pd.concat([train_df, test_df], ignore_index=True)
72+
print("[census] using official train/test split (%d train + %d test rows)"
73+
% (n_train, len(test_df)))
74+
else:
75+
data = pd.read_csv(train_path, header=None, names=COLUMN_NAMES)
76+
n_train = None
5177

5278
data["label_income"] = data["income_50k"].map({" - 50000.": 0, " 50000+.": 1})
5379
data["label_marital"] = data["marital_stat"].apply(lambda x: 1 if x == " Never married" else 0)
@@ -68,7 +94,10 @@ def load_census(data_path=None, embedding_dim=4, test_size=0.2, seed=2020):
6894
)
6995
feature_names = get_feature_names(feature_columns)
7096

71-
train, test = train_test_split(data, test_size=test_size, random_state=seed)
97+
if n_train is not None:
98+
train, test = data.iloc[:n_train], data.iloc[n_train:]
99+
else:
100+
train, test = train_test_split(data, test_size=test_size, random_state=seed)
72101
train_input = {n: train[n].values for n in feature_names}
73102
test_input = {n: test[n].values for n in feature_names}
74103

‎benchmarks/datasets/criteo.py‎

Lines changed: 23 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -83,8 +83,16 @@ def _download_dac_sample(url, dest_dir):
8383

8484
def load_criteo(source="bundled", data_path=None, embedding_dim=8, use_hash=False,
8585
hash_buckets=1000, test_size=0.2, seed=2020, max_rows=None,
86-
download_url=DEFAULT_DAC_URL):
87-
"""Load and preprocess Criteo into a :class:`SingleTaskData` (task='binary')."""
86+
download_url=DEFAULT_DAC_URL, temporal_split=False, time_col=None):
87+
"""Load and preprocess Criteo into a :class:`SingleTaskData` (task='binary').
88+
89+
By default the train/test split is random (the Kaggle DAC / Criteo_x1 data
90+
is anonymised and shuffled, with no timestamp, so this is leakage-free and
91+
standard). For logs that *do* carry time, pass ``temporal_split=True`` to
92+
hold out the most recent ``test_size`` fraction instead: rows are sorted by
93+
``time_col`` when given, otherwise the file's existing order is trusted, and
94+
the tail becomes the test set -- so no future row can leak into training.
95+
"""
8896
if data_path is not None:
8997
df = _read_criteo_file(data_path)
9098
name = "criteo_custom"
@@ -131,7 +139,19 @@ def load_criteo(source="bundled", data_path=None, embedding_dim=8, use_hash=Fals
131139
feature_columns = sparse_cols + dense_cols
132140
feature_names = get_feature_names(feature_columns)
133141

134-
train, test = train_test_split(df, test_size=test_size, random_state=seed)
142+
if temporal_split:
143+
# Chronological hold-out: the most recent rows become the test set so no
144+
# future information leaks into training. Sort by time_col when present;
145+
# otherwise assume the file is already in chronological order.
146+
if time_col and time_col in df.columns:
147+
df = df.sort_values(time_col, kind="mergesort") # stable
148+
elif time_col:
149+
print("[criteo] --time-col %r not found; assuming the file is already "
150+
"in chronological order." % time_col)
151+
cut = max(1, int(round(len(df) * (1 - test_size))))
152+
train, test = df.iloc[:cut], df.iloc[cut:]
153+
else:
154+
train, test = train_test_split(df, test_size=test_size, random_state=seed)
135155
train_input = {n: train[n].values for n in feature_names}
136156
test_input = {n: test[n].values for n in feature_names}
137157

‎tests/benchmark_test.py‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,58 @@ def test_sequence_track(tmp_path):
4343
assert (tmp_path / "sequence_synthetic_seq.csv").exists()
4444

4545

46+
def test_temporal_split_holds_out_most_recent_rows(tmp_path):
47+
"""--temporal-split must hold out the most recent rows (by --time-col),
48+
never random ones, so no future information leaks into training."""
49+
import numpy as np
50+
import pandas as pd
51+
52+
from benchmarks.datasets.criteo import COLUMNS, load_criteo
53+
54+
n = 100
55+
df = pd.DataFrame({c: [0] * n for c in COLUMNS})
56+
df["ts"] = list(range(n)) # ascending time
57+
df["label"] = [0] * 80 + [1] * 20 # the 20 most-recent rows are the positives
58+
# Shuffle file order so a correct split must rely on `ts`, not row position.
59+
df = df.iloc[np.random.RandomState(0).permutation(n)].reset_index(drop=True)
60+
path = tmp_path / "criteo_ts.csv"
61+
df.to_csv(path, index=False)
62+
63+
data = load_criteo(data_path=str(path), temporal_split=True, time_col="ts", test_size=0.2)
64+
65+
assert len(data.train_y) == 80 and len(data.test_y) == 20
66+
# Most-recent 20% (ts 80..99 -> all positives) must be exactly the test set.
67+
assert set(data.test_y.flatten().tolist()) == {1}
68+
assert set(data.train_y.flatten().tolist()) == {0}
69+
70+
71+
def _census_row(income, marital):
72+
from benchmarks.datasets.census import COLUMN_NAMES
73+
74+
vals = ["0"] * len(COLUMN_NAMES)
75+
vals[COLUMN_NAMES.index("marital_stat")] = marital
76+
vals[COLUMN_NAMES.index("income_50k")] = income
77+
return ",".join(vals)
78+
79+
80+
def test_census_uses_official_test_partition(tmp_path):
81+
"""When a sibling .test file exists, load_census must use that official
82+
partition rather than reshuffling a random split out of .data."""
83+
from benchmarks.datasets.census import load_census
84+
85+
train_rows = [_census_row(" 50000+.", " Never married")] * 10 + \
86+
[_census_row(" - 50000.", " Married")] * 10 # 20 train rows
87+
test_rows = [_census_row(" - 50000.", " Married")] * 7 # 7 test rows
88+
(tmp_path / "mini.data").write_text("\n".join(train_rows) + "\n")
89+
(tmp_path / "mini.test").write_text("\n".join(test_rows) + "\n")
90+
91+
data = load_census(data_path=str(tmp_path / "mini.data"))
92+
93+
# Official split -> exactly the .test rows (a random 0.2 split would give 5).
94+
assert len(data.train_y[0]) == 20
95+
assert len(data.test_y[0]) == 7
96+
97+
4698
if __name__ == "__main__":
4799
import sys
48100
import pytest

0 commit comments

Comments
 (0)