Hugging Face's logo

Diffusers documentation

Pipeline

You are viewing main version, which requires installation from source. If you'd like regular pip install, checkout the latest stable version (v0.36.0).
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Pipeline

ModularPipeline

class diffusers.ModularPipeline

< >

( blocks: diffusers.modular_pipelines.modular_pipeline.ModularPipelineBlocks | None = None pretrained_model_name_or_path: str | os.PathLike | None = None components_manager: diffusers.modular_pipelines.components_manager.ComponentsManager | None = None collection: str | None = None modular_config_dict: dict[str, typing.Any] | None = None config_dict: dict[str, typing.Any] | None = None **kwargs )

Parameters

  • blocks — ModularPipelineBlocks, the blocks to be used in the pipeline

Base class for all Modular pipelines.

> This is an experimental feature and is likely to change in the future.

from_pretrained

< >

( pretrained_model_name_or_path: str | os.PathLike | None trust_remote_code: bool | None = None components_manager: diffusers.modular_pipelines.components_manager.ComponentsManager | None = None collection: str | None = None **kwargs )

Parameters

  • pretrained_model_name_or_path (str or os.PathLike, optional) — Path to a pretrained pipeline configuration. It will first try to load config from modular_model_index.json, then fallback to model_index.json for compatibility with standard non-modular repositories. If the pretrained_model_name_or_path does not contain any pipeline config, it will be set to None during initialization.
  • trust_remote_code (bool, optional) — Whether to trust remote code when loading the pipeline, need to be set to True if you want to create pipeline blocks based on the custom code in pretrained_model_name_or_path
  • components_manager (ComponentsManager, optional) — ComponentsManager instance for managing multiple component cross different pipelines and apply offloading strategies.
  • collection (str, optional) —` Collection name for organizing components in the ComponentsManager.

Load a ModularPipeline from a huggingface hub repo.

get_component_spec

< >

( name: str )

load_components

< >

( names: list[str] | str | None = None **kwargs )

Parameters

  • names — list of component names to load. If None, will load all components with default_creation_method == “from_pretrained”. If provided as a list or string, will load only the specified components.
  • **kwargs — additional kwargs to be passed to from_pretrained().Can be:
    • a single value to be applied to all components to be loaded, e.g. torch_dtype=torch.bfloat16
    • a dict, e.g. torch_dtype={“unet”: torch.bfloat16, “default”: torch.float32}
    • if potentially override ComponentSpec if passed a different loading field in kwargs, e.g. pretrained_model_name_or_path, variant, revision, etc.
    • if potentially override ComponentSpec if passed a different loading field in kwargs, e.g. pretrained_model_name_or_path, variant, revision, etc.

Load selected components from specs.

register_components

< >

( **kwargs )

Parameters

  • **kwargs — Keyword arguments where keys are component names and values are component objects. E.g., register_components(unet=unet_model, text_encoder=encoder_model)

Register components with their corresponding specifications.

This method is responsible for:

  1. Sets component objects as attributes on the loader (e.g., self.unet = unet)
  2. Updates the config dict, which will be saved as modular_model_index.json during save_pretrained (only for from_pretrained components)
  3. Adds components to the component manager if one is attached (only for from_pretrained components)

This method is called when:

  • Components are first initialized in init:
    • from_pretrained components not loaded during init so they are registered as None;
    • non from_pretrained components are created during init and registered as the object itself
  • Components are updated with the update_components() method: e.g. loader.update_components(unet=unet) or loader.update_components(guider=guider_spec)
  • (from_pretrained) Components are loaded with the load_components() method: e.g. loader.load_components(names=[“unet”]) or loader.load_components() to load all default components

Notes:

  • When registering None for a component, it sets attribute to None but still syncs specs with the config dict, which will be saved as modular_model_index.json during save_pretrained
  • component_specs are updated to match the new component outside of this method, e.g. in update_components() method

save_pretrained

< >

( save_directory: str | os.PathLike push_to_hub: bool = False **kwargs )

Parameters

  • save_directory (str or os.PathLike) — Path to the directory where the pipeline will be saved.
  • push_to_hub (bool, optional) — Whether to push the pipeline to the huggingface hub.
  • **kwargs — Additional arguments passed to save_config() method

Save the pipeline to a directory. It does not save components, you need to save them separately.

to

< >

( *args **kwargs ) → DiffusionPipeline

Parameters

  • dtype (torch.dtype, optional) — Returns a pipeline with the specified dtype
  • device (torch.Device, optional) — Returns a pipeline with the specified device
  • silence_dtype_warnings (str, optional, defaults to False) — Whether to omit warnings if the target dtype is not compatible with the target device.

Returns

DiffusionPipeline

The pipeline converted to specified dtype and/or dtype.

Performs Pipeline dtype and/or device conversion. A torch.dtype and torch.device are inferred from the arguments of self.to(*args, **kwargs).

> If the pipeline already has the correct torch.dtype and torch.device, then it is returned as is. Otherwise, > the returned pipeline is a copy of self with the desired torch.dtype and torch.device.

Here are the ways to call to:

  • to(dtype, silence_dtype_warnings=False) → DiffusionPipeline to return a pipeline with the specified dtype
  • to(device, silence_dtype_warnings=False) → DiffusionPipeline to return a pipeline with the specified device
  • to(device=None, dtype=None, silence_dtype_warnings=False) → DiffusionPipeline to return a pipeline with the specified device and dtype

update_components

< >

( **kwargs )

Parameters

  • **kwargs — Component objects or configuration values to update:
    • Component objects: Models loaded with AutoModel.from_pretrained() or ComponentSpec.load() are automatically tagged with loading information. ConfigMixin objects without weights (e.g., schedulers, guiders) can be passed directly.
    • Configuration values: Simple values to update configuration settings (e.g., requires_safety_checker=False)

Update components and configuration values and specs after the pipeline has been instantiated.

This method allows you to:

  1. Replace existing components with new ones (e.g., updating self.unet or self.text_encoder)
  2. Update configuration values (e.g., changing self.requires_safety_checker flag)

In addition to updating the components and configuration values as pipeline attributes, the method also updates:

  • the corresponding specs in _component_specs and _config_specs
  • the config dict, which will be saved as modular_model_index.json during save_pretrained

Examples:

# Update pre-trained model
pipeline.update_components(unet=new_unet_model, text_encoder=new_text_encoder)

# Update configuration values
pipeline.update_components(requires_safety_checker=False)

Notes:

  • Components with trained weights should be loaded with AutoModel.from_pretrained() or ComponentSpec.load() so that loading specs are preserved for serialization.
  • ConfigMixin objects without weights (e.g., schedulers, guiders) can be passed directly.
Update on GitHub

←Parallel inference Blocks→