Design and evaluate computational workflows in Python to discover candidate disease-gene associations de novo from single-cell RNA-seq (scRNA-seq) data — not to retrieve associations that are already known.
Your selected workflow will be scored by running its frozen predict() function, unmodified, on a set of held-out indications you will not see and have no information about — different diseases, different tissues, different datasets than Indications A and B.
Performance on Indications A and B is not itself the grading criterion; it is only a proxy you use during development to compare and select a workflow. A workflow that performs well on A and B but fails to generalize will score poorly.
Design accordingly: every methodological choice should be one you'd defend as sound for an indication you've never seen, not one justified by how well it happens to score on the two datasets in front of you.
Please confirm the following before running unattended. If any is not available (e.g. no internet, no execution environment, read-only filesystem), say so now and the task will be adjusted — for example, producing unexecuted code plus a written description of expected output, rather than an executed CSV.
| Permission | Scope | Why it's needed |
|---|---|---|
| Read access to input files | {PATH_TO_INPUT_DATA} (dataset files for Indications A and B) |
Loading the two real AnnData objects and understanding their structure |
| Write access to a workspace | The same directory this prompt.md file is located in, and nowhere else (full scope in §4) |
Keeps all work products in one predictable, inspectable location and prevents accidental modification of source data or anything outside scope |
| Code execution | Python, with ability to install packages | Running and testing workflows, not just writing code that's never executed |
| Package installation | scanpy, anndata, scikit-learn, numpy, pandas, statsmodels, and any others you choose, including any pretrained model packages and weights your approach requires | Standard single-cell / stats stack, plus whatever is needed for the approach(es) you select |
| Internet access | Unrestricted, but usage restricted by the rules in §8 — permitted only inside evaluation code, scoped only to Indications A and B; downloading pretrained model weights for use inside a workflow is permitted since weights are general-purpose, not disease-specific | Querying disease-gene databases/literature as ground truth during evaluation; downloading pretrained model weights, if used, for workflow code |
| Memory / conversation history | None — do not draw on memory of past conversations or sessions when designing or justifying this workflow | Keeps the exercise self-contained and evaluable on its own terms, independent of anything discussed previously |
Once access is confirmed, run the full task (workflow design, evaluation, report) without further checkpoints, and surface only the final deliverables listed in §11.
All workflow, evaluation, and supporting code must be entirely self-contained with respect to the filesystem. You may create temporary files and subfolders in the workspace as needed for your own working process; all required deliverables must be written directly into the workspace directory (not nested arbitrarily) with clear, self-explanatory filenames. Do not write, move, or delete anything outside the workspace, and do not write anything into {PATH_TO_INPUT_DATA} itself.
Code must not search, walk, list, or otherwise probe the filesystem outside of:
- (a) the exact
data_pathpassed intopredict()for reading input, and - (b) the single workspace directory containing this
prompt.md, for writing deliverables and temporary files.
This means:
- No directory traversal, globbing, or recursive search (e.g.
os.walk,glob.glob("**/..."),Path.rglob, shelling out tofind/locate, etc.) against any path other than the two explicitly permitted locations above. - No reading of, or reference to, files elsewhere on disk — including sibling directories, home directories, temp directories outside the workspace, other datasets, cached files from prior runs elsewhere, or environment/config files not required for the two permitted locations.
- No hardcoded or discovered paths outside the workspace and the provided
data_path. If a workflow needs auxiliary files (e.g. downloaded model weights), they must be fetched via network/package-manager mechanisms and cached inside the workspace directory, not searched for or written elsewhere. - No reuse of any pre-existing virtual environment, conda environment, or system Python installation already present on the machine. Create and use your own fresh, self-contained environment (e.g. a new venv or conda environment created from scratch for this task) rather than activating, installing into, or otherwise relying on any environment that already exists. Do not search for, list, or inspect existing environments (e.g.
conda env list, scanning for other venv/site-packages directories) as a way of finding packages or shortcuts — all dependencies must be installed fresh into your own environment. Record every package and version installed into this environment in arequirements.txt(orenvironment.yml) file written to the workspace, so the environment can be reconstructed exactly later, including whenpredict()is eventually run against held-out indications outside this session.
This restriction applies to every function in every workflow and in the evaluation code, not just predict() itself. No filesystem access beyond this scope is permitted regardless of how it might be justified (e.g. "checking for cached files," "looking for a config").
Two full scRNA-seq datasets (as AnnData .h5ad objects) under {PATH_TO_INPUT_DATA}. Inspect the directory and each file's structure directly to determine disease/control label columns, cell-type annotation columns, and gene ID format — do not assume a specific schema without checking. This inspection should be limited to the files at that path itself, not a broader scan of parent or sibling directories.
Metadata overviews for the two provided objects and the holdout datasets are available under dataset_metadata_overview.md and dataset_metadata_holdout_overview.md.
Expect real-world variability across indications in cell count, genes profiled, sequencing depth, dropout rate, and cell-type composition — including in the held-out indications your workflow will ultimately be judged on, about which you are given no further information beyond what's stated in this document. Workflows must not assume fixed values for any of these, and must not assume the held-out indications resemble A and B in any particular way.
This exercise has three phases with different rules for what you're allowed to look up. Read carefully; do not blend these phases.
Before writing any workflow code, research and identify multiple diverse strategies to identify de novo disease-gene associations from scRNA-seq data. Prioritize breadth of methodology — statistical, embedding-based, ML-based, pretrained-model-based, or otherwise — subject to the evidence-source constraints in §8 ("What's fair game"). Define as many orthogonal approaches as possible, then prioritize from the list. Compute requirements should not be a major limitation.
During Phase 0, you may freely search and read literature, model papers, documentation, and benchmarks. What you may not do during Phase 0, even though you have internet access:
Look up or record what genes are already known to be associated with Indication A, Indication B, or any held-out indication. Phase 0 is about method design, not answer-finding. If a paper you're reading for methodology reasons happens to mention a disease-gene association relevant to one of your indications, do not carry that fact forward into your workflow design or note it as something to "check for" later.
Decide on or hint at your final ranked list based on anything read here.
End Phase 0 by writing a short planning note (part of Deliverable 1) stating: which methods you'll use and why; how they'll be integrated into the pipeline; and what you found in the literature that supports this choice. Cite and delineate your rationale for what informed your decisions.
Do not use only standard differential-expression testing as your method for scoring genes — this includes per-gene marginal tests such as the Wilcoxon rank-sum test, Student's/Welch's t-test, or DESeq2/edgeR-style negative-binomial models comparing case vs. control. Explore an alternative methodology instead — e.g. multivariate or embedding-based approaches, pretrained single-cell/biological foundation models, network- or graph-based inference, or another approach of your choosing — provided it still derives signal only from the input expression data. If you implement more than one workflow, treat the simplest one as your reference point: do not select a more complex workflow over a simpler one unless it demonstrates a real, defensible improvement — added complexity is not itself evidence of a better workflow.
Once Phase 0 is complete, all workflow code you write is bound by the constraints in §8 ("What's fair game, and where the line is").
After completing Phase 1, run your evaluation code against each workflow you built. Use the results — not just aggregate scores, but where in the ranked list known associations land, which genes are recovered and which are missed — to identify where each approach has room to improve. Then revise: adjust preprocessing, scoring, aggregation, or model usage, and re-evaluate to measure whether the change actually made the underlying scheme stronger. Repeat this evaluate → revise → re-evaluate cycle, using each round's results to inform the next.
Iteration is expected, and you are not limited in how many workflows or revisions you try — but each should represent a meaningfully different idea, not a cosmetic variation. Use this phase to explore approaches beyond your Phase 1 starting points — embedding-based methods, pretrained foundation models, network- or graph-based strategies, ensembles — rather than converging on a single paradigm early. Anchor against your simplest working workflow throughout: added complexity must earn its place through demonstrated improvement, not be assumed superior by default.
Every change must remain a general-purpose methodological improvement — something you'd make for any indication — rather than one motivated by how it happened to score on Indication A or B. If a revision is hard to justify without reference to the specific evaluation numbers it improved, treat that as a sign you're overfitting rather than genuinely strengthening the workflow. Document each iteration and its rationale in APPROACH.md so the evolution of the approach is traceable.
For each distinct approach you try, implement it as a workflow: a function, or a small collection of functions working together (e.g., a preprocessing step, a scoring step, an aggregation step). Each workflow must expose a single frozen entrypoint with this signature:
def predict(data_path: str, params: dict) -> pd.DataFrame:
"""Return ranked candidate disease-gene associations for one indication.
params: indication-specific settings, fully documented by you
(e.g. disease_key, control_key, celltype_keys, and anything
else your workflow needs). No indication-specific values may
be hardcoded in the code itself — everything that varies by
indication goes through params. Logic must not depend on
what the disease/control labels are named or what values they
take — only on which cells each label marks.
Returns a DataFrame with exactly two columns:
gene (HGNC gene symbol)
score (float, higher = stronger evidence of association)
sorted descending by score. Also writes this table to a clearly
named CSV inside the workspace directory described above (e.g.
ranked_targets_<workflow_name>.csv).
Subject to the filesystem and environment scope defined above.
"""This exact signature is what will be called against held-out indications later — treat it as a fixed contract, not a convention you can adapt per dataset. Internally, predict may call helper functions that make up the workflow — the constraints in §8 apply to all of them, not just to predict itself.
Build multiple, genuinely diverse workflows — this is a core part of the exercise, not optional polish. Implement at least four workflows, each reflecting a meaningfully different methodological strategy (e.g., a differential-expression/statistical approach, an embedding-based approach, a network/GRN-based approach, a pretrained-model-based approach) rather than minor variations on the same idea (e.g., the same test with different thresholds does not count as a second workflow). For each, justify why it's a methodologically sound, orthogonal way to surface disease-relevant signal — not just a different knob on the same underlying method.
Treat the simplest workflow as your reference point: do not select a more complex workflow over a simpler one unless it demonstrates a real, defensible improvement — added complexity is not itself evidence of a better workflow.
A separate function (or set of functions), clearly decoupled from all workflow code, used only to validate a workflow's output table and select the best-performing workflow. This evaluation code is called on the final gene, score table a workflow produces; no function belonging to a workflow may call, import, or otherwise have knowledge of the evaluation code or the ground-truth data it uses.
Ground truth. The gold-standard ground truth for evaluation is: for each of Indication A and Indication B, the set of genes whose Open Targets aggregate association score for that indication places them in the top 5% of all genes with a nonzero score for that indication. Evaluate a workflow's ranked output by how well it recovers this gold-standard set (e.g., precision/recall or rank-based enrichment at the top of the workflow's list — you choose and justify the specific metric). This leeway does not extend to any workflow function, and does not extend to held-out indications, for which no ground-truth checking is available to you at all.
Reproducibility requirement. The evaluation code must record and report, for each of Indication A and Indication B:
- the exact Open Targets EFO/MONDO identifier used to query the indication,
- the Open Targets data release/version queried (e.g., API version or release date),
- the query date, and
- the resulting gold-standard gene count and score threshold (i.e., the actual aggregate-score value at the 95th percentile cutoff, not just "top 5%" as a description).
Persist this alongside the evaluation output (e.g., a small ground_truth_manifest.json or equivalent) so the gold-standard set used for a given evaluation run can be reconstructed later even if Open Targets' underlying data changes.
Covering: which workflows you tried, how you evaluated them, what you observed, which one you selected, and the reasoning behind each choice — including any close calls, tradeoffs, and specifically how you guarded against overfitting to Indications A and B given that generalization to unseen indications is the actual grading criterion. Explicitly compare all workflows tried against each other and justify your final selection in light of that comparison. Include the full iteration history from Phase 2: each cycle's diagnosis, the changes it motivated, and their measured effect.
Inside a workflow (across all of its functions): any single-cell methodology or tool is permitted — QC, normalization, batch correction, clustering, cell-type annotation, differential expression, trajectory/pseudotime, cell-cell communication inference, GRN inference, eQTL-style methods, pretrained-model-derived embeddings/representations, etc. — as long as it derives signal only from the input expression data itself. Structural or functional gene annotation that carries no disease information — GO terms, Reactome pathways, protein family/domain membership, subcellular localization — is also permitted, since it describes what a gene is, not which diseases it's linked to. No function in a workflow may access the internet, query or hardcode from external databases, consult literature, or use genetic/GWAS or pathway/target-disease resources — whether for discovery, filtering, or "sanity-checking" — with the exception of downloading general-purpose pretrained model weights, which is permitted since weights are not disease-specific. None may branch on which indication they're being run on. This is what makes it a general-purpose workflow rather than a lookup dressed up as one, and it is also what makes generalization to indications you've never seen possible at all.
No filesystem access beyond scope (see §4) — this is a hard boundary regardless of how it might be justified (e.g. "checking for cached files," "looking for a config").
Inside the evaluation code, and only there: full internet access may be used — to query external disease-gene databases, pull literature, check GWAS/pathway resources, or anything else — as ground truth to score how well a workflow's ranked list recovers known associations, scoped strictly to Indication A and Indication B. This leeway does not extend to any workflow function, and does not extend to filesystem access outside the workspace directory.
The final ranked list a workflow produces must come entirely from your code — no manual curation, reordering, or hand-editing of ranked_targets.csv based on background knowledge or anything looked up.
If any step in a workflow could plausibly be seen as smuggling in outside disease-gene knowledge (internet-derived or otherwise), flag it explicitly and explain why it's still principled.
Compliance will be checked by reading the code, not just by inspecting outputs. A workflow found to violate these constraints — including via a hardcoded gene name, gene set, or disease-specific constant embedded anywhere in its code, or via any out-of-scope filesystem access — fails regardless of how good its output looks, and will also fail immediately on held-out indications where no such shortcut is available.
Validate and compare your workflows on Indications A and B, but treat strong performance on these two with real skepticism — a workflow that works well by exploiting quirks of those two datasets (sample size, batch structure, cell-type composition) is not what's being asked for, and will not transfer to the held-out indications it will actually be judged on. You will not get feedback on held-out performance during development, so your only defense against overfitting is principled design and honest self-scrutiny. If you can't articulate why a methodological choice would help on an indication you've never seen, it doesn't belong in the workflow regardless of what it does to the A/B numbers.
- Each workflow must be reproducible — set and report all random seeds.
- Filesystem and environment scope as defined in §4 (workspace directory +
data_pathonly; fresh environment, no reuse of existing venvs/conda envs) applies throughout. predict()(for the selected workflow) must be directly runnable, unmodified, on a new indication given only a newdata_pathandparamsdict — this will in fact happen, on indications you don't have access to.
Written into the workspace directory, per the filesystem scope in §4.
- Planning note (Phase 0 output) plus workflow outlines — one per approach tried, with purpose of each step/function.
- Python code for each workflow, exposing the
predict(data_path, params)entrypoint. - Python code for the evaluation function(s), clearly separated from all workflow code.
requirements.txt(orenvironment.yml) documenting every package and version installed into the fresh environment.PARAMS.mddocumenting, for the selected workflow, every key itsparamsdict expects, its type, and what it controls — precise enough that someone who has never seen this session could construct a validparamsdict for a new indication.APPROACH.md(per §7.3).