Diffusers documentation

HiDreamImageTransformer2DModel

You are viewing main version, which requires installation from source. If you'd like regular pip install, checkout the latest stable version (v0.43.0).
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

HiDreamImageTransformer2DModel

用于 HiDream-I1 中类图像数据的 Transformer 模型。

可以使用以下代码片段加载该模型。

from diffusers import HiDreamImageTransformer2DModel

transformer = HiDreamImageTransformer2DModel.from_pretrained("HiDream-ai/HiDream-I1-Full", subfolder="transformer", dtype=torch.bfloat16)

加载 HiDream-I1 的 GGUF 量化检查点

HiDreamImageTransformer2DModel 的 GGUF 检查点可以使用 ~FromOriginalModelMixin.from_single_file 进行加载

import torch
from diffusers import GGUFQuantizationConfig, HiDreamImageTransformer2DModel

ckpt_path = "https://huggingface.co/city96/HiDream-I1-Dev-gguf/blob/main/hidream-i1-dev-Q2_K.gguf"
transformer = HiDreamImageTransformer2DModel.from_single_file(
    ckpt_path,
    quantization_config=GGUFQuantizationConfig(compute_dtype=torch.bfloat16),
    dtype=torch.bfloat16
)

HiDreamImageTransformer2DModel

class diffusers.HiDreamImageTransformer2DModel

< >

( patch_size: int | None = Nonein_channels: int = 64out_channels: int | None = Nonenum_layers: int = 16num_single_layers: int = 32attention_head_dim: int = 128num_attention_heads: int = 20caption_channels: list[int] = Nonetext_emb_dim: int = 2048num_routed_experts: int = 4num_activated_experts: int = 2axes_dims_rope: tuple[int, int] = (32, 32)max_resolution: tuple[int, int] = (128, 128)llama_layers: list[int] = Noneforce_inference_output: bool = False )

forward

< >

( hidden_states: Tensortimesteps: LongTensor = Noneencoder_hidden_states_t5: Tensor = Noneencoder_hidden_states_llama3: Tensor = Nonepooled_embeds: Tensor = Noneimg_ids: typing.Optional[torch.Tensor] = Noneimg_sizes: list[tuple[int, int]] | None = Nonehidden_states_masks: typing.Optional[torch.Tensor] = Noneattention_kwargs: dict[str, typing.Any] | None = Nonereturn_dict: bool = True )

参数

  • hidden_states(形状为 (batch_size, in_channels, height, width) 或 (batch_size, patch_height * patch_width, patch_size * patch_size * channels) 的 torch.Tensor)— 输入的 hidden_states。
  • timesteps (torch.LongTensor) — 用于指示去噪步骤。
  • encoder_hidden_states_t5 (torch.Tensor) — 由 T5 文本编码器计算得出的条件嵌入。
  • encoder_hidden_states_llama3 (torch.Tensor) — 由 Llama3 文本编码器计算得出的条件嵌入。
  • pooled_embeds (torch.Tensor) — 用于额外条件控制的池化文本嵌入。
  • img_ids (torch.Tensor, 可选) — 用于补丁隐藏状态(patched hidden states)的图像位置 ID。
  • img_sizes (list of tuple of int,可选) — 用于对输出进行 unpatchify 的每个样本的 patch 网格大小。
  • hidden_states_masks (torch.Tensor,可选) — 针对分块后的 hidden_states 的掩码。
  • attention_kwargs (dict, 可选) — 一个 kwargs 字典,如果指定,将传递给 diffusers.models.attention_processor 中定义的 AttentionProcessor,该处理器在 self.processor 下定义。
  • return_dict (bool,可选,默认为 True) — 是否返回 ~models.transformer_2d.Transformer2DModelOutput 而不是普通的元组。

HiDreamImageTransformer2DModel 的前向方法。

Transformer2DModelOutput

类 diffusers.models.modeling_outputs.Transformer2DModelOutput

< >

( sample: torch.Tensor )

参数

  • sample(形状为 (batch_size, num_channels, height, width) 的 torch.Tensor,或者如果 Transformer2DModel 是离散的,则为 (batch size, num_vector_embeds - 1, num_latent_pixels))— 以 encoder_hidden_states 输入为条件的隐藏状态输出。如果是离散的,则返回未加噪潜在像素的概率分布。

Transformer2DModel 的输出。

Update on GitHub