Granite Switch 4.1 3B Preview loads a single checkpoint that combines the Granite 4.1 3B base model with twelve task-specific adapters for RAG, safety, and core capabilities. Pass adapter_name="answerability" in the chat template to activate the answerability adapter, or omit it to run the base model. The checkpoint works with both Hugging Face Transformers and vLLM.
How Granite Switch selects an adapter
Granite Switch is IBM Research's composition toolkit that embeds multiple LoRA and activated-LoRA (aLoRA) adapters into one deployable model. Each adapter is trained for a narrow task with a defined input/output contract. The official Granite Switch 4.1 3B Preview combines the Granite 4.1 3B base model (3 billion parameters, 128K context) with twelve adapters from three libraries: Core (Requirement Check, Context Attribution, Uncertainty), RAG (Query Rewrite, Query Clarification, Answerability, Hallucination Detection, Citation Generation), and Guardian (safety risks, Factuality Detection, Factuality Correction, Policy Guardrails). The checkpoint is licensed under Apache 2.0.
When you pass adapter_name="answerability" to the chat template, the template inserts the adapter's control token into the rendered prompt. Granite Switch's switching layer detects that token and produces per-position adapter indices. Every decoder layer then applies the selected LoRA weights from that position forward.
Activated LoRA (aLoRA) adapters apply their weights only to tokens generated after the control token, letting adapters reuse the base model's preceding KV cache instead of recomputing the entire conversation. With standard LoRA, switching adapters in a multi-step pipeline often forces the model to discard and recalculate the cache. The default single switch mode holds one adapter per sequence; if your workflow needs to route to different adapters within one prompt, use the multi switch mode.
Check the software and checkpoint for a first test
The Granite Switch 4.1 3B Preview model card states these dependencies for its examples:
| Dependency | Version |
|---|---|
| Python | >= 3.9 |
| PyTorch | >= 2.0.0 |
| Transformers | >= 5.5.1 |
| vLLM (optional) | >= 0.19.1, < 0.21.0 |
Clone the Granite Switch repository and install the Hugging Face or vLLM extra:
git clone https://github.com/generative-computing/granite-switch.git
cd granite-switch
# Pick what you need:
pip install -e ".[
hf]" # HuggingFace inference
pip install -e ".[
vllm]" # vLLM inferencepip install mellea
python -m vllm.entrypoints.openai.api_server \
--model ibm-granite/granite-switch-4.1-3b-preview \
--port 8000The checkpoint also runs directly through Hugging Face Transformers by importing granite_switch.hf to register the architecture.
Try the documented answerability adapter
The model card includes a Hugging Face example that activates the answerability adapter. The code below preserves the model card's message structure and its illustrated output:
import granite_switch.hf # Register the Granite Switch architecture
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "ibm-granite/granite-switch-4.1-3b-preview"
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained(model_id)
messages = [
{"role": "assistant", "content": "Hello there, how can I help you?"},
{"role": "user", "content": "What is the square root of 4?"},
]
documents = [
{"doc_id": "1", "text": "The square root of 4 is 2."}
]
prompt = tokenizer.apply_chat_template(
messages,
documents=documents,
adapter_name="answerability",
add_generation_prompt=True,
tokenize=False,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=10)
# Decode only the newly generated tokens
generated_text = tokenizer.decode(
outputs[0][inputs.input_ids.shape[-1]:],
skip_special_tokens=True
)
print(generated_text)
# Model card illustrated output: "answerable"Omitting adapter_name runs the base model without an adapter. The answerability adapter returns answerable or unanswerable rather than a prose answer.
To call the same adapter through vLLM's HTTP API:
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:8000/v1",
api_key="unused",
)
response = client.chat.completions.create(
model="ibm-granite/granite-switch-4.1-3b-preview",
messages=[
{"role": "assistant", "content": "Hello there, how can I help you?"},
{"role": "user", "content": "What is the square root of 4?"},
],
extra_body={
"documents": [
{"doc_id": "1", "text": "The square root of 4 is 2."}
],
"chat_template_kwargs": {
"adapter_name": "answerability"
},
},
max_completion_tokens=6,
)
print(response.choices[0].message.content)
# Model card illustrated output: "answerable"When run through Mellea, adapters expose Python functions with typed inputs and outputs. Mellea's constrained decoding enforces the output schema at the token level; the raw Transformers and vLLM calls shown here return strings and require the caller to validate the format.
Choose routing for multiple tasks
The default single switch mode holds one adapter per sequence: once activated, that adapter stays active. If your application needs to route different parts of a single prompt to different adapters (for example, an agent that picks a safety adapter, then an answerability adapter, then a citation adapter within one request), compose a checkpoint with --switch-type multi. The Hello MultiSwitch and Multi-turn MultiSwitch tutorials demonstrate when and how to enable mid-sequence routing.
For production workflows that chain several adapter calls (such as query rewrite → answerability → hallucination check → citation → safety check), the RAG 101 (fifteen minutes) and Full RAG Flow (thirty minutes) tutorials demonstrate multi-step pipelines and Guardian checks with Mellea.
Judge fit against your application's task boundaries
Granite Switch does not replace domain validation or general reasoning. Guardian and factuality adapters add safety checks but are not substitutes for application-level safety testing. Test adapter behavior on representative data before deployment, especially for safety-critical applications. The uncertainty adapter can signal when the model's confidence is low and human review is warranted.
The documented examples invoke adapters through control tokens but do not establish output quality, latency, memory use, or serving cost for a production workload. The sources do not provide a measured comparison with deployments using separate models, so you'll need to benchmark the multi-adapter checkpoint against your current setup to quantify the advantage for your task mix.
Next step
Read the 3B preview model card for the full list of twelve adapters and their input/output contracts, or open the Hello Mellea notebook (five minutes) to call adapters through Mellea's typed Python API.
Verification status: Supported by official documentation only under the conditions below; not reproduced in a test environment by IGPROD. Checked 2026-10-17.
Applies to: Granite Switch 4.1 3B Preview; granite-switch-4.1-3b-preview; Granite 4.1.
Conditions:
- The model card lists Python >= 3.9, PyTorch >= 2.0.0, and Transformers >=5.5.1; its optional vLLM range is >= 0.19.1 and < 0.21.0.
- The documented first example uses the ibm-granite/granite-switch-4.1-3b-preview checkpoint and an answerability adapter.
- The default single switch engine holds one adapter per sequence; routing to multiple adapters within a sequence uses the multi engine.
- Guardian adapters require application-level safety testing before production use.
Related guides
- gpu-unlock for Nvidia 3000-series GPUs: doubles the FP16/BF16 instruction issue rate for local AI work.
- MOSS-Image: test text rendering and layout control: how to evaluate an image model before you fine-tune it.
- Clairic extraction models on Workers AI: typed values from text and images, with confidence scores.
Sources:
