Optional Guide: Model Training and Adaptation

← Back to Stage 1

This is a path-selection card, not a training course. It helps you tell which methods change a model and which methods simply help a model do a task. Beginners can read the tables first, then return to Stage 1 to call a ready-made model.

Checked: 2026-08-31 UTC; scope: model training, adaptation, and serving methods.

🧭 See the whole path first

Open full-size image (new tab)

1. Pre-training: large amounts of data are used to build a Base Model.
2. Post-training: demonstrations, preferences, or feedback teach the model to follow instructions better.
3. Inference: after training, the model receives one input and produces one result.
4. Agent system: a model is connected to Prompt, RAG, Memory, Tools, and Harness to complete work.

🧩 Learn the methods without implementing them first

Goal	Method	Plain meaning	Does it change weights?
Teach the model how to act	SFT (Supervised Fine-Tuning)	Show the model good questions and answers so it can imitate them.	Yes
	DPO (Direct Preference Optimization)	Show two answers and indicate which one is preferred.	Yes
	RLHF/RL	Use human or rule-based feedback to help the model get better results.	Yes
	GRPO	Compare several answers to one question, then learn from their relative results.	Yes
Adapt with fewer changes	PEFT	Train only a small part of the parameters.	Only selected or added parameters
	LoRA	Freeze the original weights and train a smaller low-rank matrix.	Original weights no; added parameters yes
Make serving smaller or cheaper	Distillation	Teach a smaller Student Model from a larger Teacher Model.	Trains the Student Model
	Quantization	Store or compute weights with fewer bits, usually using less memory.	Usually no retraining of the original model; some methods add adjustment

Do not mistake external systems for training

Method	What it really does	Usually changes model weights?
Prompt	Tells the model what to do this time.	No
RAG	Finds outside information and puts evidence into this input.	No
Memory	Saves state for later and reads it back when needed.	No
Tools	Lets a program search, calculate, or take another action after checks.	No
Harness	Manages tools, permissions, state, logs, retries, and stop rules.	No

“Usually no” matters. A product may start a separate training job behind the scenes. Check official documentation for training jobs, trainable parameters, or model weights.

📚 Required reading and selected resources

Read the first two to understand the main path. Use the others when you truly need to train or compress a model. Ratings are editorial guidance, not GitHub stars.

Group	Resource	Rating	What you learn
Understand the main path	OpenAI: how models are developed	⭐⭐⭐⭐⭐	How data, training, and models relate.
	Google: LLM tuning	⭐⭐⭐⭐⭐	The boundary between Prompt Engineering, Fine-tuning, and Distillation.
Learn Post-training	OpenAI: gpt-oss	⭐⭐⭐⭐	How one model family describes Pre-training, SFT, and RL.
	Hugging Face TRL	⭐⭐⭐⭐	An entry point for SFT, DPO, GRPO, and other Post-training methods.
Adapt or compress	Hugging Face PEFT	⭐⭐⭐⭐	Approaches that train fewer parameters and their limits.
	Hugging Face LoRA	⭐⭐⭐⭐	Freeze original weights and train a low-rank matrix.
	Hugging Face Quantization	⭐⭐⭐	Use lower precision to reduce memory and compute needs.

🛠 A decision exercise with no GPU

Choose a first path for each case and give one reason:

1. Company rules change every day: try RAG first.
2. Every answer must use a fixed brand voice: start with Prompt and Eval; consider Fine-tuning only if evidence shows it is needed.
3. The model is too large for the device: assess Quantization or a smaller model first.
4. You want to train fewer parameters for a special format: assess LoRA/PEFT first.

These are not permanent answers. Test with your own data, Eval, hardware, and cost limits.

Advanced: what to check before doing real training

• Do you have permission to use the training data, and have you removed sensitive data?
• Does the Base Model license allow your use and distribution method?
• Are training, validation, and test sets separate?
• Did you keep the unadapted model as a baseline?
• After training, did you rerun safety, bias, quality, cost, and latency Evals?
• Can you stop a failed job, keep a checkpoint, and return to the last usable version?

✅ Completion check

• I can say the order of Pre-training, Post-training, and Inference.
• I know Fine-tuning changes model weights, while RAG usually does not.
• I can explain SFT, DPO, RLHF/RL, and GRPO in one sentence each.
• I know LoRA/PEFT, Distillation, and Quantization solve different problems.
• I will not start an expensive training job just because I saw a new term.

← Back to Stage 1 and make your first model call