Model Fine-tuning

This guide is designed to help you understand and make the most of our Model Fine-tuning service — whether you're just getting started or looking to optimize advanced use cases.

1. What is Model Fine-tuning?

Fine-tuning is the process of further training a pre-trained AI model using your own data so that it better understands your specific context, style, or objectives.

2. What do you need to prepare before fine-tuning a model?

You’ll need:

• Strictly follow the expected dataset structure for the model you're fine-tuning. More details about Data sample, visit here: https://fptcloud.com/en/documents/model-fine-tuning/?doc=select-data-format
• Clean, diverse, and non-duplicated data.
• A clear objective for fine-tuning (e.g., tech support, customer service, content writing, etc.).

3. Which model should I choose for fine-tuning?

Depends on your needs:

• Under 1B parameters: fast, cost-efficient, good for lightweight devices.
• 7B – 13B: balanced between quality and performance.
• Over 30B: ideal for demanding, high-quality applications.

4. How long does fine-tuning take?

It depends on:

• Model size.
• Amount of training data.

• Your hardware setup.

  Typically, it ranges from a few hours to several days.

5. How many GPUs do you need to fine-tune a model?

It depends on the model size:

• <1B parameters: 1 GPU (24 GB VRAM) is sufficient
• 7B models: 2–4 GPUs (40 GB VRAM each)
• 13B models: 4–8 GPUs recommended
• 30B+ models: Requires 8+ GPUs and multi-node setup

6. Do I need multiple nodes or just one node?

• For small to medium models (up to 13B), a single node with multiple GPUs is enough.
• For large models (30B+), multi-node setups are recommended for better memory and performance.

7. What is the minimum GPU memory required?

• At least 24 GB per GPU for standard fine-tuning.
• You can fine-tune on GPUs with 8–16 GB VRAM using LoRA or QLoRA methods.

8. Does the size of my training dataset affect hardware needs?

Yes. Larger datasets require:

• More memory and compute power
• Larger batch sizes
• Longer training time → higher GPU and node requirements