opentau.policies.pi07_paligemma.high_level_planner.configuration_pi07_high_level

Configuration module for the PI07 high level planner Policy.

This module defines the PI07HighLevelPlannerConfig class, which handles the configuration parameters for the PI07 high level planner. It includes settings for the model architecture, optimization, scheduling, and data processing.

Classes

PI07HighLevelPlannerConfig(n_obs_steps, ...)

Configuration for the π07 high-level planner policy.

class opentau.policies.pi07_paligemma.high_level_planner.configuration_pi07_high_level.PI07HighLevelPlannerConfig(n_obs_steps: int = 1, normalization_mapping: dict[str, ~opentau.configs.types.NormalizationMode] = <factory>, input_features: dict[str, ~opentau.configs.types.PolicyFeature] = <factory>, output_features: dict[str, ~opentau.configs.types.PolicyFeature] = <factory>, device: str | None = None, use_amp: bool = False, use_torch_compile: bool = False, torch_compile_mode: str = 'default', pretrained_path: str | None = None, skip_normalization_weights: bool = False, skip_input_resolution_check: bool = False, save_normalization_stats: bool = True, dataset_names: list[str] | None = None, dataset_to_norm_index: dict[str, int] | None = None, cloud_vlm_latency_mean: float = 0.0, cloud_vlm_latency_std: float = 0.0, cloud_vlm_latency_lower: float = 0.0, cloud_vlm_latency_upper: float = 0.0, action_decoder_latency_mean: float = 0.0, action_decoder_latency_std: float = 0.0, action_decoder_latency_lower: float = 0.0, action_decoder_latency_upper: float = 0.0, max_state_dim: int = 32, resize_imgs_with_padding: tuple[int, int] = (224, 224), empty_cameras: int = 0, prompt_max_length: int = 256, memory_max_length: int = 52, response_max_length: int = 52, metadata_max_length: int = 52, subtask_indicator_max_length: int = 4, memory_indicator_max_length: int = 4, discrete_action_tokenizer_path: str = 'physical-intelligence/fast', dropout: float = 0.1, attention_implementation: str = 'eager', freeze_vision_encoder: bool = True, optimizer_lr: float = 2.5e-05, optimizer_betas: tuple[float, float] = (0.9, 0.95), optimizer_eps: float = 1e-08, optimizer_weight_decay: float = 1e-10, scheduler_warmup_steps: int = 1000, scheduler_decay_steps: int = 30000, scheduler_decay_lr: float = 2.5e-06)

Bases: PreTrainedConfig

Configuration for the π07 high-level planner policy.

The high-level planner takes images, a language instruction, robot state, and past memory, then autoregressively predicts updated memory and the next subtask string. This config controls model architecture, tokenizer limits, initialization, and optimizer/scheduler presets.

Parameters:

• n_obs_steps – Number of observation steps to use. Only 1 is currently supported. Defaults to 1.

• normalization_mapping – Mapping from feature type names to normalization modes. Defaults to identity for visual features and mean-std for state.

• max_state_dim – Maximum dimension for state vectors. Shorter vectors are zero-padded. Defaults to 32.

• resize_imgs_with_padding – Target (height, width) for image resizing with aspect-ratio-preserving padding. Must match the resolution of the input images — training fails fast on a mismatch (skip_input_resolution_check=true downgrades it to a warning for legacy checkpoints); null passes frames through at the input resolution. Non-224 values run natively (pad to the next patch multiple + interpolated position embeddings), with no letterboxing to 224x224. Defaults to (224, 224).

• empty_cameras – Number of empty (zero-filled) camera inputs to add. Defaults to 0.

• prompt_max_length – Maximum token length for the composite language prompt (task + past memory + state). Defaults to 256.

• memory_max_length – Maximum token length for the updated memory sequence. Defaults to 52.

• response_max_length – Maximum token length for the subtask response sequence. Defaults to 52.

• metadata_max_length – Maximum token length for episode metadata strings. Defaults to 52.

• subtask_indicator_max_length – Number of tokenizer pieces for the fixed "Subtask: " span (encode(..., add_special_tokens=False)). Used to align CE slices with the prefix layout; for google/paligemma-3b-pt-224 this is 4. Defaults to 4.

• memory_indicator_max_length – Number of tokenizer pieces for the fixed "Updated Memory: " span. Used for documentation and layout checks; for google/paligemma-3b-pt-224 this is 4. Defaults to 4.

• dropout – Dropout rate applied in the transformer expert. Defaults to 0.1.

• attention_implementation – Attention backend — "eager" or "fa2" (Flash Attention 2). Defaults to "eager".

• freeze_vision_encoder – Whether to freeze the SigLIP vision encoder during fine-tuning. Defaults to True.

• optimizer_lr – Peak learning rate for AdamW. Defaults to 2.5e-5.

• optimizer_betas – Beta parameters for AdamW. Defaults to (0.9, 0.95).

• optimizer_eps – Epsilon for AdamW. Defaults to 1e-8.

• optimizer_weight_decay – Weight decay for AdamW. Defaults to 1e-10.

• scheduler_warmup_steps – Linear warmup steps. Defaults to 1_000.

• scheduler_decay_steps – Cosine decay steps. Defaults to 30_000.

• scheduler_decay_lr – Final learning rate after decay. Defaults to 2.5e-6.

__init__(n_obs_steps: int = 1, normalization_mapping: dict[str, ~opentau.configs.types.NormalizationMode] = <factory>, input_features: dict[str, ~opentau.configs.types.PolicyFeature] = <factory>, output_features: dict[str, ~opentau.configs.types.PolicyFeature] = <factory>, device: str | None = None, use_amp: bool = False, use_torch_compile: bool = False, torch_compile_mode: str = 'default', pretrained_path: str | None = None, skip_normalization_weights: bool = False, skip_input_resolution_check: bool = False, save_normalization_stats: bool = True, dataset_names: list[str] | None = None, dataset_to_norm_index: dict[str, int] | None = None, cloud_vlm_latency_mean: float = 0.0, cloud_vlm_latency_std: float = 0.0, cloud_vlm_latency_lower: float = 0.0, cloud_vlm_latency_upper: float = 0.0, action_decoder_latency_mean: float = 0.0, action_decoder_latency_std: float = 0.0, action_decoder_latency_lower: float = 0.0, action_decoder_latency_upper: float = 0.0, max_state_dim: int = 32, resize_imgs_with_padding: tuple[int, int] = (224, 224), empty_cameras: int = 0, prompt_max_length: int = 256, memory_max_length: int = 52, response_max_length: int = 52, metadata_max_length: int = 52, subtask_indicator_max_length: int = 4, memory_indicator_max_length: int = 4, discrete_action_tokenizer_path: str = 'physical-intelligence/fast', dropout: float = 0.1, attention_implementation: str = 'eager', freeze_vision_encoder: bool = True, optimizer_lr: float = 2.5e-05, optimizer_betas: tuple[float, float] = (0.9, 0.95), optimizer_eps: float = 1e-08, optimizer_weight_decay: float = 1e-10, scheduler_warmup_steps: int = 1000, scheduler_decay_steps: int = 30000, scheduler_decay_lr: float = 2.5e-06) → None

property action_delta_indices: None

Returns None; action deltas are not used by this planner.

attention_implementation: str = 'eager'

discrete_action_tokenizer_path: str = 'physical-intelligence/fast'

dropout: float = 0.1

empty_cameras: int = 0

freeze_vision_encoder: bool = True

get_optimizer_preset() → AdamWConfig

Returns the default AdamW optimizer configuration.

Returns:

An AdamWConfig populated from this config’s optimizer_* fields.

get_scheduler_preset() → LRSchedulerConfig

Returns the default cosine-decay-with-warmup scheduler configuration.

Returns:

A CosineDecayWithWarmupSchedulerConfig populated from this config’s scheduler_* and optimizer_lr fields.

max_state_dim: int = 32

memory_indicator_max_length: int = 4

memory_max_length: int = 52

metadata_max_length: int = 52

n_obs_steps: int = 1

normalization_mapping: dict[str, NormalizationMode]

property observation_delta_indices: None

Returns None; observation deltas are not used by this planner.

optimizer_betas: tuple[float, float] = (0.9, 0.95)

optimizer_eps: float = 1e-08

optimizer_lr: float = 2.5e-05

optimizer_weight_decay: float = 1e-10

prompt_max_length: int = 256

resize_imgs_with_padding: tuple[int, int] = (224, 224)

response_max_length: int = 52

property reward_delta_indices: None

Returns None; reward deltas are not used by this planner.

scheduler_decay_lr: float = 2.5e-06

scheduler_decay_steps: int = 30000

scheduler_warmup_steps: int = 1000

subtask_indicator_max_length: int = 4

validate_features() → None

Adds placeholder camera features for empty camera slots.

Dynamically inserts zero-filled camera entries into self.input_features for each configured empty camera, so the model receives a fixed number of image inputs regardless of which cameras are physically present.