Appendix C — Bundled datasets

Every example in this book runs on the files listed here. They are downloaded from public archives by scripts/fetch_data.py and verified against data/MANIFEST.tsv, which records the exact size and SHA-256 checksum of each file. Nothing in the book depends on data that is not in this list.

python scripts/fetch_data.py          # fetch everything (~12 MB)
python scripts/fetch_data.py --check  # verify against the manifest

Reach these files from chapter code with the book_data() helper rather than a literal path, so that code works from any chapter directory:

from pybfx import book_data
book_data("proteins", "globins.fasta")

Licensing

Each file remains under the licence of the archive it came from. The sequence, structure and variant records are public-domain or openly licensed research data from NCBI, UniProt, RCSB PDB, the AlphaFold Protein Structure Database and STRING. The expression counts come from a Bioconductor data package distributed under the Artistic-2.0 licence. The simulated read files are generated by this book’s own script and carry the same licence as the book’s code.

The read files are simulated, and the book says so

reads/mgen_sim_R1.fastq.gz and its mate are not real sequencing output. They are generated from the bundled Mycoplasmoides genitalium reference by scripts/fetch_data.py with a fixed random seed, using a position-dependent quality decay and matching base-call errors. They behave like real reads for the purposes of the exercises — and because their true origin on the genome is encoded in each read name, you can check your own alignments against the truth, which real data never lets you do. Conclusions about sequencing chemistry should not be drawn from them.

C.1 Genome annotation and ontology

annotation/human_go_annotations.tsv — 5.1 MB: 20,431 reviewed human proteins with primary gene symbol and GO term identifiers - background universe for enrichment analysis

Source: UniProtKB REST (reviewed human proteome, CC BY 4.0)

SHA-256: 3f9757dd8ced605a7b7187a228fe79c8…

annotation/mycoplasma-genitalium.gff — 339.4 kB: RefSeq GFF3 annotation for the bundled M. genitalium G37 genome

Source: NCBI Datasets, assembly GCF_000027325.1 (ASM2732v1), RefSeq annotation

SHA-256: 332b945d89ed6280c375eef0badef306…

C.2 Expression counts

expression/pasilla_gene_counts.tsv — 486.7 kB: Drosophila RNAi knockdown of pasilla: 7 samples, used for the differential expression walk-through

Source: Bioconductor pasilla 1.40.0 (Artistic-2.0); counts from Brooks et al. 2011, GEO GSE18508

SHA-256: ea0dafbfcc600559644cfe7dd5cc8de8…

expression/pasilla_sample_annotation.csv — 527 B: Drosophila RNAi knockdown of pasilla: 7 samples, used for the differential expression walk-through

Source: Bioconductor pasilla 1.40.0 (Artistic-2.0); counts from Brooks et al. 2011, GEO GSE18508

SHA-256: 1844f871ca5d750f41685c73b87bd7dd…

C.3 Interaction networks

networks/p53_string_network.tsv — 18.2 kB: High-confidence (score >= 0.700) human interactions among 30 cell-cycle and DNA-damage genes

Source: STRING v12 API (string-db.org), CC BY 4.0

SHA-256: e7f0d3abe5351c4bddadfc087ea7fd71…

C.4 Protein sequences

proteins/P04637.fasta — 490 B: Human cellular tumour antigen p53, canonical sequence

Source: UniProtKB P04637 (P53_HUMAN)

SHA-256: 59aace7211b64e5918ef6547fc48d641…

proteins/P04637.txt — 436.1 kB: Full UniProt flat-file record used for annotation parsing

Source: UniProtKB P04637 (P53_HUMAN)

SHA-256: 8373a03cbdbc75ba2c90b2d24ed0400b…

proteins/globins.fasta — 935 B: Human alpha/beta haemoglobin and two myoglobins - pairwise alignment examples

Source: UniProtKB: P69905, P68871, P02144, P02185

SHA-256: e61d0f68348256404b88c8f6cf437f04…

proteins/p53-orthologs.fasta — 3.8 kB: p53 orthologues across eight vertebrates for multiple alignment and tree building

Source: UniProtKB: P04637, P02340, P10361, P41685, P67939, Q9TTA1, P79734, P79820

SHA-256: f15e8035718fa865648f1ce1da07c8fa…

C.5 Sequencing reads

reads/mgen_sim_R1.fastq.gz — 2.9 MB: 20,000 paired 150 bp reads sampled from NC_000908.2 with position-dependent quality decay and matching sequencing errors; mate R1

Source: Simulated in silico by scripts/fetch_data.py (seed 20250101)

SHA-256: a1bf6d7e833e8f6b0addc7e68aab11eb…

reads/mgen_sim_R2.fastq.gz — 2.9 MB: 20,000 paired 150 bp reads sampled from NC_000908.2 with position-dependent quality decay and matching sequencing errors; mate R2

Source: Simulated in silico by scripts/fetch_data.py (seed 20250101)

SHA-256: 414f29abfa2da9edcf9cb5bce9523289…

C.6 Nucleotide sequences

sequences/TP53-NM_000546.6.fasta — 2.6 kB: Homo sapiens tumour protein p53 (TP53), transcript variant 1, mRNA

Source: NCBI Nucleotide NM_000546.6

SHA-256: b29a603f664c8853c80d7550de1db1cf…

sequences/TP53-NM_000546.6.gb — 36.9 kB: Same TP53 transcript in GenBank format (CDS feature used in translation examples)

Source: NCBI Nucleotide NM_000546.6

SHA-256: c14355676c0755f9e552a398f2545c2a…

sequences/mycoplasma-genitalium-NC_000908.2.fasta — 574.6 kB: Mycoplasmoides genitalium G37 complete genome (580,076 bp) - compact genome for annotation work

Source: NCBI Nucleotide NC_000908.2

SHA-256: cc21ace789ae96213ab10a595b5f52c8…

sequences/sars-cov-2-NC_045512.2.fasta — 29.7 kB: SARS-CoV-2 isolate Wuhan-Hu-1, complete genome (29,903 bp)

Source: NCBI Nucleotide NC_045512.2

SHA-256: 0891c00cc2d503f58e0c600ce5ee64e9…

sequences/sars-cov-2-NC_045512.2.gb — 76.7 kB: Same record in GenBank flat-file format, used for feature-table examples

Source: NCBI Nucleotide NC_045512.2

SHA-256: 87744b176aa9ce71350c8becf0d06295…

C.7 Protein structures

structures/1mbn.pdb — 144.0 kB: Sperm whale myoglobin (Watson 1969) - the first protein structure solved

Source: RCSB PDB entry 1MBN

SHA-256: e4635115be5a605e58f449f82824e151…

structures/1tup.pdb — 518.0 kB: p53 core domain bound to DNA (Cho et al. 1994) - structural analysis examples

Source: RCSB PDB entry 1TUP

SHA-256: e4fe9eaa33d05b46e627444ab152f1b2…

structures/AF-P04637-F1-confidence.json — 412.4 kB: Predicted aligned error matrix accompanying the p53 AlphaFold model

Source: AlphaFold Protein Structure Database, AF-P04637-F1 (v6)

SHA-256: 9cfbaa7522ce1aecf0caa3b66bb41481…

structures/AF-P04637-F1-model.pdb — 248.4 kB: Predicted structure of human p53; per-residue pLDDT occupies the B-factor column

Source: AlphaFold Protein Structure Database, AF-P04637-F1 (v6)

SHA-256: 5a7051e8f97848d43b3747268825b29c…

C.8 Variants

variants/tp53_clinvar.vcf — 1.1 MB: 3,888 ClinVar records across TP53 with clinical significance, review status and molecular consequence

Source: NCBI ClinVar GRCh38 VCF (ftp.ncbi.nlm.nih.gov/pub/clinvar), remote tabix slice

SHA-256: fc076e32797ee8f7638b3e1ca48beddd…

# Bundled datasets {#sec-appendix-data} Every example in this book runs on the files listed here. They are downloaded from public archives by `scripts/fetch_data.py` and verified against `data/MANIFEST.tsv`, which records the exact size and SHA-256 checksum of each file. Nothing in the book depends on data that is not in this list. ```{.bash} python scripts/fetch_data.py # fetch everything (~12 MB) python scripts/fetch_data.py --check # verify against the manifest ``` Reach these files from chapter code with the `book_data()` helper rather than a literal path, so that code works from any chapter directory: ```{.python} from pybfx import book_data book_data("proteins", "globins.fasta") ``` ::: {.callout-note title="Licensing"} Each file remains under the licence of the archive it came from. The sequence, structure and variant records are public-domain or openly licensed research data from NCBI, UniProt, RCSB PDB, the AlphaFold Protein Structure Database and STRING. The expression counts come from a Bioconductor data package distributed under the Artistic-2.0 licence. The simulated read files are generated by this book's own script and carry the same licence as the book's code. ::: ::: {.callout-warning title="The read files are simulated, and the book says so"} `reads/mgen_sim_R1.fastq.gz` and its mate are **not** real sequencing output. They are generated from the bundled *Mycoplasmoides genitalium* reference by `scripts/fetch_data.py` with a fixed random seed, using a position-dependent quality decay and matching base-call errors. They behave like real reads for the purposes of the exercises — and because their true origin on the genome is encoded in each read name, you can check your own alignments against the truth, which real data never lets you do. Conclusions about sequencing chemistry should not be drawn from them. ::: ## Genome annotation and ontology {#sec-data-annotation} **`annotation/human_go_annotations.tsv`** — 5.1 MB : 20,431 reviewed human proteins with primary gene symbol and GO term identifiers - background universe for enrichment analysis *Source:* UniProtKB REST (reviewed human proteome, CC BY 4.0) *SHA-256:* `3f9757dd8ced605a7b7187a228fe79c8…` **`annotation/mycoplasma-genitalium.gff`** — 339.4 kB : RefSeq GFF3 annotation for the bundled M. genitalium G37 genome *Source:* NCBI Datasets, assembly GCF_000027325.1 (ASM2732v1), RefSeq annotation *SHA-256:* `332b945d89ed6280c375eef0badef306…` ## Expression counts {#sec-data-expression} **`expression/pasilla_gene_counts.tsv`** — 486.7 kB : Drosophila RNAi knockdown of pasilla: 7 samples, used for the differential expression walk-through *Source:* Bioconductor pasilla 1.40.0 (Artistic-2.0); counts from Brooks et al. 2011, GEO GSE18508 *SHA-256:* `ea0dafbfcc600559644cfe7dd5cc8de8…` **`expression/pasilla_sample_annotation.csv`** — 527 B : Drosophila RNAi knockdown of pasilla: 7 samples, used for the differential expression walk-through *Source:* Bioconductor pasilla 1.40.0 (Artistic-2.0); counts from Brooks et al. 2011, GEO GSE18508 *SHA-256:* `1844f871ca5d750f41685c73b87bd7dd…` ## Interaction networks {#sec-data-networks} **`networks/p53_string_network.tsv`** — 18.2 kB : High-confidence (score >= 0.700) human interactions among 30 cell-cycle and DNA-damage genes *Source:* STRING v12 API (string-db.org), CC BY 4.0 *SHA-256:* `e7f0d3abe5351c4bddadfc087ea7fd71…` ## Protein sequences {#sec-data-proteins} **`proteins/P04637.fasta`** — 490 B : Human cellular tumour antigen p53, canonical sequence *Source:* UniProtKB P04637 (P53_HUMAN) *SHA-256:* `59aace7211b64e5918ef6547fc48d641…` **`proteins/P04637.txt`** — 436.1 kB : Full UniProt flat-file record used for annotation parsing *Source:* UniProtKB P04637 (P53_HUMAN) *SHA-256:* `8373a03cbdbc75ba2c90b2d24ed0400b…` **`proteins/globins.fasta`** — 935 B : Human alpha/beta haemoglobin and two myoglobins - pairwise alignment examples *Source:* UniProtKB: P69905, P68871, P02144, P02185 *SHA-256:* `e61d0f68348256404b88c8f6cf437f04…` **`proteins/p53-orthologs.fasta`** — 3.8 kB : p53 orthologues across eight vertebrates for multiple alignment and tree building *Source:* UniProtKB: P04637, P02340, P10361, P41685, P67939, Q9TTA1, P79734, P79820 *SHA-256:* `f15e8035718fa865648f1ce1da07c8fa…` ## Sequencing reads {#sec-data-reads} **`reads/mgen_sim_R1.fastq.gz`** — 2.9 MB : 20,000 paired 150 bp reads sampled from NC_000908.2 with position-dependent quality decay and matching sequencing errors; mate R1 *Source:* Simulated in silico by scripts/fetch_data.py (seed 20250101) *SHA-256:* `a1bf6d7e833e8f6b0addc7e68aab11eb…` **`reads/mgen_sim_R2.fastq.gz`** — 2.9 MB : 20,000 paired 150 bp reads sampled from NC_000908.2 with position-dependent quality decay and matching sequencing errors; mate R2 *Source:* Simulated in silico by scripts/fetch_data.py (seed 20250101) *SHA-256:* `414f29abfa2da9edcf9cb5bce9523289…` ## Nucleotide sequences {#sec-data-sequences} **`sequences/TP53-NM_000546.6.fasta`** — 2.6 kB : Homo sapiens tumour protein p53 (TP53), transcript variant 1, mRNA *Source:* NCBI Nucleotide NM_000546.6 *SHA-256:* `b29a603f664c8853c80d7550de1db1cf…` **`sequences/TP53-NM_000546.6.gb`** — 36.9 kB : Same TP53 transcript in GenBank format (CDS feature used in translation examples) *Source:* NCBI Nucleotide NM_000546.6 *SHA-256:* `c14355676c0755f9e552a398f2545c2a…` **`sequences/mycoplasma-genitalium-NC_000908.2.fasta`** — 574.6 kB : Mycoplasmoides genitalium G37 complete genome (580,076 bp) - compact genome for annotation work *Source:* NCBI Nucleotide NC_000908.2 *SHA-256:* `cc21ace789ae96213ab10a595b5f52c8…` **`sequences/sars-cov-2-NC_045512.2.fasta`** — 29.7 kB : SARS-CoV-2 isolate Wuhan-Hu-1, complete genome (29,903 bp) *Source:* NCBI Nucleotide NC_045512.2 *SHA-256:* `0891c00cc2d503f58e0c600ce5ee64e9…` **`sequences/sars-cov-2-NC_045512.2.gb`** — 76.7 kB : Same record in GenBank flat-file format, used for feature-table examples *Source:* NCBI Nucleotide NC_045512.2 *SHA-256:* `87744b176aa9ce71350c8becf0d06295…` ## Protein structures {#sec-data-structures} **`structures/1mbn.pdb`** — 144.0 kB : Sperm whale myoglobin (Watson 1969) - the first protein structure solved *Source:* RCSB PDB entry 1MBN *SHA-256:* `e4635115be5a605e58f449f82824e151…` **`structures/1tup.pdb`** — 518.0 kB : p53 core domain bound to DNA (Cho et al. 1994) - structural analysis examples *Source:* RCSB PDB entry 1TUP *SHA-256:* `e4fe9eaa33d05b46e627444ab152f1b2…` **`structures/AF-P04637-F1-confidence.json`** — 412.4 kB : Predicted aligned error matrix accompanying the p53 AlphaFold model *Source:* AlphaFold Protein Structure Database, AF-P04637-F1 (v6) *SHA-256:* `9cfbaa7522ce1aecf0caa3b66bb41481…` **`structures/AF-P04637-F1-model.pdb`** — 248.4 kB : Predicted structure of human p53; per-residue pLDDT occupies the B-factor column *Source:* AlphaFold Protein Structure Database, AF-P04637-F1 (v6) *SHA-256:* `5a7051e8f97848d43b3747268825b29c…` ## Variants {#sec-data-variants} **`variants/tp53_clinvar.vcf`** — 1.1 MB : 3,888 ClinVar records across TP53 with clinical significance, review status and molecular consequence *Source:* NCBI ClinVar GRCh38 VCF (ftp.ncbi.nlm.nih.gov/pub/clinvar), remote tabix slice *SHA-256:* `fc076e32797ee8f7638b3e1ca48beddd…`