opentau.datasets.grounding

Utilities for encoding spatial outputs as PaliGemma-style location tokens.

PaliGemma reserves 1024 single-token IDs <loc0000>..`<loc1023>` that quantize a coordinate axis into 1024 bins. Bounding boxes and points are emitted as plain strings (<locYMIN><locXMIN><locYMAX><locXMAX> label), which the standard tokenizer turns into a single integer per <locNNNN>.

Two helpers live here:

• loc_codec: pure functions to convert pixel coordinates to/from the loc-token string format. No torch dependency.

• tokenizer_utils.ensure_loc_tokens: makes the loc strings available on any HuggingFace tokenizer. A no-op for PaliGemma (already shipped). For Gemma 3 (and any other base tokenizer) it appends them as special tokens and, when given a model handle, resizes the embedding table to match.

Concrete grounding datasets (PixMo-points, RefCOCO, …) are NOT yet shipped under this package — see the follow-up tracking the configurable response formatter that will make them config-driven rather than one class per source.

Modules

loc_codec	Codec between pixel coordinates and PaliGemma <locNNNN> strings.
tokenizer_utils	Tokenizer-side support for PaliGemma <loc0000>..`<loc1023>` tokens.