Data curation, download, and organization utilities for the Liu Lab.
Import name: labdata.
Early scaffolding. The first feature is the GEO part: labdata.Series, a lazy handle
on a GEO Series (GSE000000) that resolves its linked publication, samples (GSMs), and
SRA experiments (SRXs) through NCBI Entrez. From there, labdata geo download fetches
the FASTQs — orchestrated as a Snakemake DAG (prefetch → extract → compress) with
resource-aware concurrency: a dedicated ncbi resource caps concurrent NCBI network
jobs while --cores saturates the rest of the machine, so the download link stays busy
while cores extract and gzip. Zenodo and other databases come later.
This project uses pixi with conda-forge + bioconda channels.
All Python tooling is managed by pixi.
pixi install # solve & install the default env (resolves from pixi.lock if present)
pixi shell # activate the env
pixi run check # lint + fmt-check + typecheck + test (the CI gate)To use labdata from a Jupyter notebook, register this env's Python as a kernel
once, then pick it in Jupyter:
pixi run register-kernel # creates the "Python (liulab-data)" kernel (idempotent)Entrez requests need a contact email (and accept an optional API key that raises the rate limit). Configure once:
labdata ncbi configure # interactive prompt, cached to ~/.cache/liulab-data/
# or set env vars (take precedence over the cache file):
export NCBI_EMAIL="you@lab.org"
export NCBI_API_KEY="..." # optionalSee AGENTS.md for the full contributor/agent working agreement.
MIT — see LICENSE.