TrainLoraNode - ComfyUI Built-in Node Documentation

Complete documentation for the TrainLoraNode node in ComfyUI. Learn its inputs, outputs, parameters and usage.

This documentation was AI-generated. If you find any errors or have suggestions for improvement, please feel free to contribute! Edit on GitHub

The TrainLoraNode creates and trains a LoRA (Low-Rank Adaptation) model on a diffusion model using provided latents and conditioning data. It allows you to fine-tune a model with custom training parameters, optimizers, and loss functions. The node outputs the trained LoRA weights, a loss history map, and the total training steps completed.

Inputs

Parameter	Data Type	Required	Range	Description
model	MODEL	Yes	-	The model to train the LoRA on.
latents	LATENT	Yes	-	The Latents to use for training, serve as dataset/input of the model.
positive	CONDITIONING	Yes	-	The positive conditioning to use for training.
batch_size	INT	Yes	1-10000	The batch size to use for training (default: 1).
grad_accumulation_steps	INT	Yes	1-1024	The number of gradient accumulation steps to use for training (default: 1).
steps	INT	Yes	1-100000	The number of steps to train the LoRA for (default: 16).
learning_rate	FLOAT	Yes	0.0000001-1.0	The learning rate to use for training (default: 0.0005).
rank	INT	Yes	1-128	The rank of the LoRA layers (default: 8).
optimizer	COMBO	Yes	”AdamW" "Adam" "SGD" "RMSprop”	The optimizer to use for training (default: “AdamW”).
loss_function	COMBO	Yes	”MSE" "L1" "Huber" "SmoothL1”	The loss function to use for training (default: “MSE”).
seed	INT	Yes	0-18446744073709551615	The seed to use for training (used in generator for LoRA weight initialization and noise sampling) (default: 0).
training_dtype	COMBO	Yes	”bf16" "fp32" "none”	The dtype to use for training. ‘none’ preserves the model’s native compute dtype instead of overriding it. For fp16 models, GradScaler is automatically enabled (default: “bf16”).
lora_dtype	COMBO	Yes	”bf16" "fp32”	The dtype to use for lora (default: “bf16”).
quantized_backward	BOOLEAN	Yes	-	When using training_dtype ‘none’ and training on quantized model, doing backward with quantized matmul when enabled (default: False).
algorithm	COMBO	Yes	Multiple options available	The algorithm to use for training.
gradient_checkpointing	BOOLEAN	Yes	-	Use gradient checkpointing for training (default: True).
checkpoint_depth	INT	Yes	1-5	Depth level for gradient checkpointing (default: 1).
offloading	BOOLEAN	Yes	-	Offload model weights to CPU during training to save GPU memory (default: False).
existing_lora	COMBO	Yes	Multiple options available	The existing LoRA to append to. Set to None for new LoRA (default: “[None]”).
bucket_mode	BOOLEAN	Yes	-	Enable resolution bucket mode. When enabled, expects pre-bucketed latents from ResolutionBucket node (default: False).
bypass_mode	BOOLEAN	Yes	-	Enable bypass mode for training. When enabled, adapters are applied via forward hooks instead of weight modification. Useful for quantized models where weights cannot be directly modified (default: False).

Note: The number of positive conditioning inputs must match the number of latent images. If only one positive conditioning is provided with multiple images, it will be automatically repeated for all images.

Note on training_dtype: When set to “none”, the model’s native compute dtype is preserved. For fp16 models, GradScaler is automatically enabled to prevent underflow during gradient computation. If fp16_accumulation is also enabled (via --fast flags), this combination can be numerically unstable and may cause NaN values.

Note on quantized_backward: This parameter is only relevant when training_dtype is set to “none” and the model is a quantized model. It enables quantized matrix multiplication during the backward pass.

Note on bypass_mode: When enabled, adapters are applied via forward hooks instead of modifying the model weights directly. This is particularly useful for quantized models where weights cannot be directly modified.

Outputs

Output Name	Data Type	Description
lora	LORA_MODEL	The trained LoRA weights that can be saved or applied to other models.
loss_map	LOSS_MAP	A dictionary containing the training loss values over time.
steps	INT	The total number of training steps completed (including any previous steps from existing LoRA).

---

Source fingerprint (SHA-256): e145145306330099293c8b4bb344ec0a38de690040bd796b84225f54638900c3