Hugging Face cheat sheet

$ pip install "transformers[torch]" datasets huggingface_hub

$ hf auth login

Sources: transformers installation · transformers on PyPI · transformers v5 migration guide

import os
from huggingface_hub import login, hf_hub_download, snapshot_download

# The token comes from the environment; never paste it into the file.
if os.environ.get("HF_TOKEN"):
    login(token=os.environ["HF_TOKEN"])

repo = "Qwen/Qwen2.5-0.5B-Instruct"

# One file, cached under $HF_HOME/hub
config_path = hf_hub_download(repo, "config.json", revision="main")

# A filtered snapshot into a plain folder
folder = snapshot_download(
    repo,
    allow_patterns=["*.json", "*.safetensors"],
    local_dir="./qwen-0.5b",
)
print(config_path, folder)

• There are three token types. read downloads, write can also push, and fine-grained is scoped to chosen resources. Use read unless you must write.
• huggingface-cli is gone. Use hf. In hub 2.0 hf repo became hf repos.
• The datasets cache is separate, at HF_DATASETS_CACHE. To move everything, set HF_HOME.
• hf_transfer is dead. The hub package installs hf-xet for you, so downloads are fast by default.

Sources: hf CLI guide · Environment variables · User access tokens

Eight defaults that bite, most of them changed in 2026, each with the one line fix.

1. transformers 5Jan 26, 2026

   01

   torch_dtype is now dtype

   In transformers 5 the argument is dtype, and the default is "auto", so a model loads in the dtype it was saved in. torch_dtype still works but warns.

   Do thisWrite dtype="auto" or a real dtype, and stop passing torch_dtype.

   transformers v5 migration guide

2. transformers 5

   02

   load_in_4bit=True raises TypeError

   Version 5 removed load_in_4bit and load_in_8bit from from_pretrained.

   Do thisPass quantization_config=BitsAndBytesConfig(load_in_4bit=True, ...).

   transformers v5 migration guide · Quantization overview

3. transformers

   03

   generate() stops at 20 tokens

   Unless the model's generation config sets more, generate() returns up to 20 tokens. For a decoder only model that count includes your prompt, so answers look cut off.

   Do thisAlways set max_new_tokens, and do_sample=True if you want temperature or top_p to matter.

   Text generation tutorial

4. Install

   04

   The wrong Python installs an old stack

   On Python 3.9, pip install transformers gives 4.57.6 and the 5.x examples fail. Plain pip install transformers also has no PyTorch.

   Do thisUse Python 3.11 or newer and pip install "transformers[torch]".

   transformers installation · transformers on PyPI

5. huggingface_hubSep 24, 2026

   05

   huggingface-cli no longer exists

   It was deleted in huggingface_hub 1.0 and replaced by hf. Hub 2.0 also renamed hf repo to hf repos and moved HTTP to httpx2.

   Do thisUse hf auth login, hf cache ls and hf repos create.

   huggingface_hub 2.0 migration · huggingface_hub 1.0.0 release

6. Trainer

   06

   Trainer arguments were renamed

   evaluation_strategy, warmup_ratio, no_cuda and Trainer(tokenizer=) all raise TypeError now. report_to defaults to "none".

   Do thisUse eval_strategy, a float warmup_steps, use_cpu and processing_class=.

   transformers v5 migration guide · Trainer

7. Security

   07

   trust_remote_code runs code from the Hub

   It executes whatever code sits in the repo on your machine. Pickle weights can run code the moment you load them.

   Do thisKeep it False, prefer model.safetensors, and pin revision to a commit hash.

   Pickle security

8. TRL 1.x

   08

   SFTConfig is not TrainingArguments

   max_seq_length is now max_length. SFTConfig turns on gradient_checkpointing and bf16 for you, and a model passed as a string loads in float32.

   Do thisSet max_length, bf16=False on CPU, and model_init_kwargs={"dtype": ...}.

   TRL SFT trainer

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Qwen/Qwen2.5-0.5B-Instruct"

# Pin revision to a commit hash in production; "main" moves.
tok = AutoTokenizer.from_pretrained(model_id, revision="main", padding_side="left")
if tok.pad_token is None:
    tok.pad_token = tok.eos_token

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    revision="main",
    dtype=torch.bfloat16,
    device_map="auto",
    attn_implementation="sdpa",
)

prompts = ["The capital of France is", "A haiku about caches:"]
inputs = tok(prompts, return_tensors="pt", padding=True).to(model.device)
out = model.generate(
    **inputs, max_new_tokens=30, do_sample=True, temperature=0.7, top_p=0.9
)
print(tok.batch_decode(out, skip_special_tokens=True))

• from_pretrained takes dtype, device_map, attn_implementation, quantization_config and trust_remote_code as keyword arguments. revision can be a branch, a tag or a commit hash.
• generate() returns up to 20 tokens by default, unless the model's generation config sets more. Set max_new_tokens. max_length counts the prompt too.
• temperature and top_p only matter when do_sample=True.
• For a batch of prompts on a decoder only model, use padding_side="left". Many tokenizers have no pad token, so set tokenizer.pad_token = tokenizer.eos_token when it is None.
• attn_implementation takes "sdpa", "flash_attention_2", "flash_attention_3", "flex_attention" or "eager". The paged| prefix is deprecated since 5.19.0.
• Weights load from safetensors when they exist. weights_only is True by default, and torch.load needs torch 2.6 or newer.

Sources: Text generation tutorial · transformers v5 migration guide

from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
messages = [
    {"role": "system", "content": "You answer in one sentence."},
    {"role": "user", "content": "What is Xet?"},
]

# Inspect the exact prompt text the model will see
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
print(text)

# v5 returns a BatchEncoding (input_ids and attention_mask), not a bare tensor
batch = tok.apply_chat_template(
    messages, add_generation_prompt=True, return_dict=True, return_tensors="pt"
)
print(batch["input_ids"].shape)
# model.generate(**batch, max_new_tokens=64)

from threading import Thread
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    TextIteratorStreamer,
    TextStreamer,
)

model_id = "Qwen/Qwen2.5-0.5B-Instruct"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
inputs = tok.apply_chat_template(
    [{"role": "user", "content": "Count to five."}],
    add_generation_prompt=True, return_dict=True, return_tensors="pt",
).to(model.device)

# 1. Print tokens to stdout as they arrive
model.generate(**inputs, streamer=TextStreamer(tok, skip_prompt=True), max_new_tokens=60)

# 2. Iterate over chunks (for a web response); generate runs in a thread
streamer = TextIteratorStreamer(tok, skip_prompt=True, skip_special_tokens=True)
Thread(target=model.generate, kwargs={**inputs, "streamer": streamer, "max_new_tokens": 60}).start()
for chunk in streamer:
    print(chunk, end="", flush=True)

• apply_chat_template(..., add_generation_prompt=True) adds the tokens that open the assistant turn. Without them the model may carry on the user message instead of replying.
• In version 5 it returns a BatchEncoding with input_ids and attention_mask, so pass it to model.generate(**batch). Use tokenize=False to get the prompt string.
• TextStreamer prints to stdout. TextIteratorStreamer yields chunks and needs generate in a threading.Thread.

Sources: Chat templating · transformers v5 migration guide

import torch
from transformers import AutoModelForCausalLM, BitsAndBytesConfig

# pip install bitsandbytes  (v5 removed load_in_4bit as a from_pretrained argument)
bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-7B-Instruct",
    quantization_config=bnb,
    device_map="auto",
)
print(f"{model.get_memory_footprint() / 1e9:.1f} GB")

• The weights take parameters times bytes per parameter. The activations and the KV cache come on top, so leave headroom. The sizes above are weights only.
• accelerate estimate-memory MODEL estimates a model without downloading it.
• bitsandbytes gives on the fly 4 and 8 bit loading. NF4 is the type to use for training 4 bit base models. Other methods in the docs include AWQ, GGUF and compressed tensors.
• On Apple Silicon do not promise bfloat16. A tiny model crashed with bf16 and device_map="auto" on MPS, while float32 and CPU bf16 were fine.

Sources: Quantization overview

• Pickle files can run code when you load them. safetensors removes that risk, but a safetensors model from a stranger can still be badly behaved, so it is not proof of trust.
• trust_remote_code=True runs code from the repo on your machine whatever the weight format. Leave it off unless you have read the code.
• Pin revision to a commit hash. A branch name or tag can move.
• Never run torch.load on a downloaded .bin, .pt or .pth file with weights_only=False.
• The Hub scans every file under 2 GB for malware at each commit. The docs say that is not foolproof, and checking is your job.
• Gated models need an approved account. Without a token, hf_hub_download raises GatedRepoError with a 401.

Sources: Pickle security · Gated models · safetensors

from datasets import load_dataset

ds = load_dataset("rajpurkar/squad", split="train")  # cached after the first run
ds = ds.filter(lambda r: len(r["context"]) > 200)
ds = ds.map(
    lambda batch: {"q_words": [len(q.split()) for q in batch["question"]]},
    batched=True,
    num_proc=4,
)
ds = ds.remove_columns(["id", "title"]).shuffle(seed=42).select(range(1000))
splits = ds.train_test_split(test_size=0.1, seed=42)
print(splits)

# Streaming: no full download, you get an IterableDataset
stream = load_dataset("rajpurkar/squad", split="train", streaming=True)
for row in stream.shuffle(seed=42, buffer_size=1_000).take(2):
    print(row["question"])

• streaming=True returns an IterableDataset and downloads nothing up front. It has map, filter, shuffle, take, skip and batch, but no select, train_test_split or num_proc.
• Streaming shuffle uses a buffer of 1,000 rows. Since datasets 5.0 it draws from several shards, and max_buffer_input_shards=1 gives the old behavior.
• Transforms cache to Arrow files. Pass load_from_cache_file=False to map to skip the cache.
• datasets 4.0 removed dataset loading scripts and trust_remote_code. Use Parquet or another packaged format.

Sources: Load a dataset · Stream a dataset · Process a dataset · Datasets cache

import os
from datasets import Dataset, load_from_disk

ds = Dataset.from_dict({"text": ["hello", "world"], "label": [0, 1]})
# Dataset.from_pandas(df) works the same way

ds.save_to_disk("./my_ds")          # Arrow folder, reload with load_from_disk
ds = load_from_disk("./my_ds")
ds.to_parquet("./my_ds.parquet")    # single parquet file

# Needs a write token and a repo you own
ds.push_to_hub(
    "your-username/my-dataset",
    private=True,
    token=os.environ["HF_TOKEN"],
)

• push_to_hub also works on a streaming IterableDataset since datasets 4.0, so you can map, filter and push without writing to disk.
• Pass private=True for a private repo and num_proc to speed up the upload.

from datasets import load_dataset
from transformers import (AutoModelForSequenceClassification, AutoTokenizer,
                          DataCollatorWithPadding, Trainer, TrainingArguments)

name = "distilbert/distilbert-base-uncased"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForSequenceClassification.from_pretrained(name, num_labels=2)

ds = load_dataset("stanfordnlp/imdb")
ds = ds.map(lambda b: tok(b["text"], truncation=True), batched=True)

args = TrainingArguments(
    output_dir="imdb-distilbert",
    eval_strategy="epoch",
    save_strategy="epoch",
    per_device_train_batch_size=16,
    gradient_accumulation_steps=2,
    warmup_steps=0.1,  # warmup_ratio was removed; warmup_steps accepts a float
    report_to="none",
    push_to_hub=False,
)
trainer = Trainer(
    model=model,
    args=args,
    train_dataset=ds["train"],
    eval_dataset=ds["test"],
    data_collator=DataCollatorWithPadding(tok),
    processing_class=tok,
)
trainer.train()

• report_to defaults to "none" in version 5. Set it to "wandb", "tensorboard" or "mlflow" to log.
• load_best_model_at_end=True needs eval_strategy set and a matching save_strategy.
• The defaults are 3 epochs, a learning rate of 5e-05, a batch size of 8 per device and bf16 off. The Trainer sets use_cache=False on the model.
• Seq2SeqTrainingArguments adds predict_with_generate and the generation settings, for use with Seq2SeqTrainer.

Sources: Trainer · transformers v5 migration guide

import torch
from datasets import load_dataset
from peft import LoraConfig
from trl import SFTConfig, SFTTrainer

dataset = load_dataset("trl-lib/Capybara", split="train")

args = SFTConfig(
    output_dir="qwen3-sft",
    max_length=2048,
    packing=True,
    per_device_train_batch_size=2,
    gradient_accumulation_steps=8,
    model_init_kwargs={"dtype": torch.bfloat16},  # a model id string loads in float32 by default
)
trainer = SFTTrainer(
    model="Qwen/Qwen3-0.6B",
    args=args,
    train_dataset=dataset,
    peft_config=LoraConfig(r=16, lora_alpha=32, target_modules="all-linear"),
)
trainer.train()
trainer.save_model()

• TRL 1.0 shipped on Mar 31, 2026. The stable trainers are SFT, DPO, KTO, GRPO, RLOO, Reward and Distillation. Many others live in trl.experimental.
• SFT takes four dataset shapes: plain {"text": ...}, conversational {"messages": [...]}, and prompt and completion pairs in either style. Conversational data gets the chat template applied for you.
• SFTConfig differs from TrainingArguments: logging_steps 10, gradient_checkpointing on, bf16 on unless fp16 is set, and a learning rate of 2e-5. max_seq_length is now max_length.
• A model passed as a string loads in float32 unless you set model_init_kwargs={"dtype": ...}.
• Accelerate launches multi GPU runs: accelerate config, then accelerate launch train.py. torchrun still works.

Sources: TRL SFT trainer · TRL docs · Accelerate launch

import os

from huggingface_hub import InferenceClient

client = InferenceClient(provider="auto", api_key=os.environ["HF_TOKEN"])

completion = client.chat.completions.create(
    model="openai/gpt-oss-120b",  # add :cheapest, :fastest or :groq to pick a provider
    messages=[{"role": "user", "content": "Explain LoRA in two sentences."}],
    max_tokens=300,
)
print(completion.choices[0].message.content)

stream = client.chat_completion(
    messages=[{"role": "user", "content": "Count to five."}],
    model="openai/gpt-oss-120b",
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

import os

from openai import OpenAI

client = OpenAI(
    base_url="https://router.huggingface.co/v1",
    api_key=os.environ["HF_TOKEN"],  # a Hugging Face token, not an OpenAI key
)

completion = client.chat.completions.create(
    model="openai/gpt-oss-120b:cheapest",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
)
print(completion.choices[0].message.content)

• The router base URL is https://router.huggingface.co/v1, and it speaks the OpenAI chat completions protocol. Other tasks need the Hugging Face clients.
• Routed through Hugging Face, you pay the provider rate with no markup. Included credits apply to that mode only: free accounts get none, PRO gets $2.00 a month, and Team or Enterprise gets $2.00 per seat. A key from the provider is billed by the provider.
• Choose a provider with a suffix on the model name: :fastest (the default), :cheapest, :preferred, or a provider name such as :groq. In InferenceClient use provider="auto" or a provider name.
• Team and Enterprise organizations can bill an org with InferenceClient(bill_to="my-org").
• Text Generation Inference is in maintenance mode and its repo is archived. The docs point to vLLM, SGLang, llama.cpp and MLX for self hosting.
• transformers serve starts a small OpenAI compatible server at localhost:8000 after pip install "transformers[serving]". The docs call it a lightweight option, so use vLLM or SGLang at scale.

Sources: Inference Providers pricing · Inference Providers · transformers serve · Text Generation Inference

import os

import gradio as gr
from huggingface_hub import InferenceClient

client = InferenceClient(api_key=os.environ["HF_TOKEN"])  # set as a Space secret


def respond(message, history):
    messages = [*history, {"role": "user", "content": message}]
    out = client.chat_completion(messages, model="openai/gpt-oss-120b", max_tokens=512)
    return out.choices[0].message.content


demo = gr.ChatInterface(respond, title="My chat demo")

if __name__ == "__main__":
    demo.launch()

• A Space is a git repo. Every push rebuilds it. The README header holds sdk, app_file, python_version and suggested_hardware.
• python_version defaults to 3.10, and transformers 5, TRL 1.x and diffusers 0.42 need 3.11 or newer. Set python_version: "3.12" in the header.
• Secrets and variables live in the Space settings. Both arrive as environment variables. Secrets cannot be read back and are not copied to duplicates.
• Static Spaces are free. Gradio and Docker Spaces need a paid plan to create, except that free accounts in good standing can host up to 2 Gradio Spaces on ZeroGPU. PRO can host 10.
• @spaces.GPU allocates half an RTX Pro 6000 Blackwell by default. size="xlarge" is a full GPU and costs 2x the daily quota.
• Gradio 6 moved theme and css from Blocks to launch(), and removed the tuple chat history. ChatInterface history is a list of {"role", "content"} dicts.

Sources: Spaces overview · Spaces configuration · ZeroGPU · Gradio ChatInterface · Gradio 6 migration guide

import torch
from diffusers import DiffusionPipeline

pipe = DiffusionPipeline.from_pretrained(
    "Qwen/Qwen-Image",
    dtype=torch.bfloat16,
    device_map="cuda",  # or "mps", "xpu", "cpu"
)

prompt = "A lighthouse on a cliff at dawn, film still, soft fog"
image = pipe(prompt, num_inference_steps=30).images[0]
image.save("lighthouse.png")

• diffusers 0.42 loads with dtype= and device_map="cuda", not torch_dtype. Use "mps", "xpu" or "cpu" on other hardware.
• Gated models such as FLUX need an account, an accepted license and a login.
• Check each model card for its license before you ship anything built on it.

Sources: diffusers quickstart

import evaluate

accuracy = evaluate.load("accuracy")
print(accuracy.compute(predictions=[0, 1, 1, 0], references=[0, 1, 0, 0]))


# pass this to Trainer(compute_metrics=...)
def compute_metrics(eval_pred):
    logits, labels = eval_pred
    return accuracy.compute(predictions=logits.argmax(-1), references=labels)

• evaluate.load("accuracy") needs scikit-learn, and it downloads the metric script from the Hub when you call it.
• The evaluate repo is not archived, but its recent commits are CI upkeep and the last release was 0.4.6 on Sep 18, 2025. The official docs call LightEval more actively maintained for newer evaluation work.

Sources: evaluate docs