Supervised Fine-Tuning: From SFT to LoRA Practice

After pretraining, large language models already have strong language understanding and generation abilities. But if we want a model to follow instructions more consistently, answer in a specific format, or adapt to the expression style of a domain, we usually need one more layer of post-training.

Supervised Fine-Tuning, or SFT, is the process of teaching a model with many labeled examples: user input plus a reference answer. In plain terms, SFT teaches the model how we want it to respond.

This article uses a Llama 3.1 8B + Google Colab + Unsloth + LoRA experiment to explain the core ideas of SFT, the data format, the training workflow, and why LoRA makes large-model fine-tuning practical in free or low-cost GPU environments.

1. What is SFT?

SFT stands for Supervised Fine-Tuning. Its goal is not to train a model from scratch, but to adapt a pretrained language model to a downstream task.

During training, the model sees the user task and context, then learns to generate the reference answer. After seeing many similar examples, the model gradually learns:

how to follow instructions;
how to use a required response format;
how to keep a stable answer style;
how to handle a recurring task type;
how to express existing knowledge in a domain-specific way.

In a common post-training pipeline, SFT usually sits after pretraining and before preference optimization: Pretraining -> SFT -> RLHF / DPO / Preference Learning.

2. Why use LoRA in this experiment?

Fully fine-tuning a Llama 3.1 8B model requires a lot of GPU memory and compute. The model has roughly 8 billion parameters, and updating all of them is difficult on a standard Colab T4 runtime.

LoRA, short for Low-Rank Adaptation, is a parameter-efficient fine-tuning method. Its core idea is to freeze most of the original model weights, insert a small number of trainable adapter parameters near key layers, and update only those LoRA adapters during training.

In this experiment, only about 0.52% of the parameters are trainable. That is the key learning point: we are not retraining the whole Llama model; we are training a compact adapter beside it.

Fine-tuning targetSFT defines the behavior the model should learn.

Efficient methodLoRA makes that training possible with much lower memory.

Practical valueAdapters are lightweight, easy to save, replaceable, and simple to iterate.

Freeze most of the original model weights
Insert a small number of trainable adapter parameters
Update only the LoRA adapters during training

3. Colab environment

This experiment ran in a Google Colab GPU runtime, which can be understood as a temporary cloud Linux virtual machine. The successful runtime looked like this:

Current directory: /content
Python: 3.12.13
Platform: Linux-6.6.122+-x86_64-with-glibc2.35
Torch: 2.11.0+cu128
CUDA available: True
GPU: Tesla T4
GPU memory: 14.563 GB

4. Alpaca data format: the training material for SFT

The dataset used in this experiment is unsloth/alpaca-cleaned. It is a cleaned Alpaca instruction-tuning dataset, and the loaded training split contains 51,760 examples.

instructionThe user task or command.

inputOptional extra context. It can be empty.

outputThe reference answer the model should learn to generate.

textThe formatted training text passed to SFTTrainer.

5. Prompt formatting: turning data into learnable text

Raw Alpaca records cannot be sent directly into training. They first need to be formatted into a complete prompt. The official notebook template combines instruction, input, and output into a new text field. That text field is what SFTTrainer actually trains on.

instruction
input
output
text

6. Training result: 60 steps is a smoke test

This run completed 60 training steps. The important point is that 60 steps is a smoke test: it verifies that the model loads, the data formats correctly, the LoRA adapter trains, and the checkpoint saves successfully. It does not prove that the final model quality is fully optimized.

For a serious evaluation, we would need longer training, a validation set, stable generation settings, and a systematic before-and-after comparison.

7. The most important files in the training checkpoint

After training, the Trainer created /content/outputs/checkpoint-60 in Colab. The most important files are:

adapter_model.safetensors
adapter_config.json

adapter_model.safetensors: the trained LoRA adapter weights;
adapter_config.json: the adapter configuration, used to attach the adapter back to the base model;
optimizer.pt, scheduler.pt, trainer_state.json, and similar files: mainly useful for resuming training.

Because LoRA trains only the adapter, the saved result is not a full Llama 3.1 8B model. It is a lightweight adapter. For inference, the base model and the adapter need to be loaded together.

8. How SFT, LoRA, QLoRA, and full fine-tuning relate

A compact comparison makes the relationship easier to see:

Method	Core idea	Resource need	Best fit
SFT	Fine-tune with labeled examples	Depends on training method	Instruction following, domain tasks, format learning
Full Fine-Tuning	Update all model weights	Highest	When resources are sufficient and maximum behavior change is needed
LoRA	Freeze the base model and train adapters	Lower	Learning experiments, low-cost fine-tuning, fast iteration
QLoRA	Load quantized 4-bit weights and train LoRA adapters	Even lower	Training larger models on smaller GPUs

SFT is the training objective. LoRA and QLoRA are implementation methods. The key combination in this experiment is:

Llama 3.1 8B + 4-bit loading + LoRA adapter + Alpaca SFT data + SFTTrainer

9. Summary

At its core, SFT teaches a model how it should answer by showing it reference answers. It helps a general-purpose language model become better suited to a specific task, response format, and domain style.

Connect to a Colab GPU
Install Unsloth / Transformers / TRL / Datasets
Load Llama 3.1 8B
Use 4-bit loading to reduce memory use
Attach a LoRA adapter
Load the Alpaca dataset
Format instruction / input / output
Add EOS_TOKEN
Save a baseline before training
Train with SFTTrainer for 60 steps
Save the checkpoint
Run a before / after comparison

The 60-step run is not meant to produce a final production model. Its value is building a repeatable learning path: first run the official notebook, understand SFT, LoRA, data formatting, and checkpoints, then replace the dataset with your own domain data.

SFT teaches the model how to answer. LoRA makes that training lightweight. The Colab demo gives learners a practical path they can actually run.