# place the two tarballs (no extraction):
#   ~/tensorflow_datasets/downloads/manual/ILSVRC2012_img_train.tar
#   ~/tensorflow_datasets/downloads/manual/ILSVRC2012_img_val.tar
python -c "import tensorflow_datasets as tfds; \
           tfds.builder('imagenet2012').download_and_prepare()"

That validates, shuffles, and reshards into version 5.1.0 under ~/tensorflow_datasets/imagenet2012/5.1.0/: 1,281,167 training images across 1024 tfrecord shards (147.90 GB) and 50,000 validation images across 64 shards (6.74 GB) — the exact layout every ImageNet trainer in this book consumes. Budget disk accordingly: the raw tarballs (\(\sim \)144 GB) plus the 5.1.0 output (\(\sim \)155 GB) plus TFDS’s intermediate extraction mean you want roughly 500 GB of free space to prepare it in one pass. From there the JAX bridge does no image-loading of its own: it calls tfds.load(’imagenet2012’, split=…, data_dir=…) and streams through tfds.as_numpy, so TFDS’s tf.data pipeline — parallel shard reads, JPEG decode, prefetch, all engineered to scale — carries the loading, and we inherit that performance work for free.

Demo datasets. The demos/ trainers step past image classification into the three other task families in the book — detection, segmentation, and language — and each pulls a dataset of its own. Oxford-IIIT Pets backs both the UNet segmenter (unet-pets-train, via trimaps) and the YOLO detector (yolov1-pets-train-bootstrap, via head boxes tiled into \(2 \times 2\) mosaics); MSD Task01 (BraTS) carries segmentation to real medical imaging — four-modality brain-tumour MRI, where the thin-class collapse a Pets trimap merely tolerates becomes the entire problem (enhancing tumour is \(\sim \)0.5 % of voxels). Two trainers read it: unet-brats-train (a from-scratch UNet at native \(240 \times 240\)) and unet-brats-r34 (the ResNet-34 encoder of §10.2.3, which needs the \(224\) centre-cropped build since its stride-32 backbone does not divide 240); tiny Shakespeare — a single 1 MB character stream — is what the TinyGPT char-level transformer (tinygpt-shakespeare) trains on; and TinyStories — Eldan & Li’s synthetic short-story corpus, \(\sim \)2 GB — is its scaled-up sibling, byte-level-BPE tokenized to a deliberately small 4096-token vocabulary for the \(\sim \)8.5 M-parameter decoder-only GPT (tinystories). Detection then leaves the household frame entirely: VisDrone-DET2019 is drone-altitude imagery where the objects are the problem — a median box is \(20 \times 25\) px in the source frame, some 70 of them per image, so the single \(7\times 7\) grid that suffices for Pets head boxes collapses outright and the multi-scale FPN head of Definition 95 becomes necessary rather than decorative. MS-COCO 2017 is the standard detection benchmark alongside it, at the opposite end of the same axis (photographic scale, 80 classes, 118k training images); the repository ships its loader and the two standard subsets described below, though no trainer in this book consumes it yet. Same fetch-and-unpack story as the table above, with one extra step: each is tokenized or tiled into the trainer’s input format by the matching preprocess_*.py script before the first lake exe run.

The demo datasets, in the same columns as above. Each also has a preprocess_*.py step (e.g. preprocess_shakespeare.py builds train.bin/val.bin/vocab.txt) between download and training. For detection, preprocess_pets_mosaic.py tiles the Pets head boxes into class-balanced \(2 \times 2\) mosaics (data/pets_mosaic_bal/); the same download_pets.sh download feeds both the segmenter and the detector. For TinyStories, preprocess_tinystories.py trains a byte-level BPE tokenizer on a slice of the corpus and emits it in HuggingFace format (vocab.json/merges.txt) alongside the flat int32-LE token streams (train.bin/val.bin); the small vocabulary keeps the in-graph one-hot embedding and head matmuls cheap and the tokenizer auditable. The two detection sets follow the same pattern: preprocess_visdrone.py and preprocess_coco.py write the same three record formats — single-grid, single-scale anchor, and multi-scale FPN — so one set of Lean loaders and one codegen serve both. VisDrone’s official Google-Drive/Baidu links are large-file-flaky, so download_visdrone.sh pulls the byte-identical Ultralytics release-asset mirror over plain HTTPS; COCO’s own hosting needs no such workaround.

Building the BraTS dataset. BraTS 2021 proper (1,251 cases) is gated behind a free Synapse account and a signed research-use agreement, so no script can fetch it. The Medical Segmentation Decathlon’s Task01_BrainTumour is the same task from open storage: 484 labelled 4D volumes built from the BraTS 2016/2017 cases, the same four co-registered MRI modalities (FLAIR / T1w / T1gd / T2w) and the same glioma sub-regions, as a single 7.6 GB tarball over S3. Thus we have fewer volumes and numbers that are not comparable to a BraTS 2021 leaderboard — but this is also a real segmentation result on real medical data that a reader can reproduce.