- About Us
-
Secure, unlimited storage to ensures efficiency as well as high and continuous data access demand.
Virtual server integration for 3D Rendering, AI or ML
Enhance application capacity and availability.
Secure your exclusive advance order today and gain priority access to an all-inclusive stack for AI development, driven by NVIDIA’s powerful technology!
FPT Web Application Firewall provides powerful protection for web applications
Advanced virtual server with rapid scalability
Backup and restore data instantly, securely and maintain data integrity.
Advanced virtual server with rapid scalability
Secure your exclusive advance order today and gain priority access to an all-inclusive stack for AI development, driven by NVIDIA’s powerful technology!
Enhance application capacity and availability.
Backup and restore data instantly, securely and maintain data integrity.
Recovery, ensuring quick operation for the business after all incidents and disasters.
Diverse throughput and capacity to meet various business workloads.
Secure, unlimited storage to ensures efficiency as well as high and continuous data access demand.
FPT Web Application Firewall provides powerful protection for web applications
Intelligent and Comprehensive Virtual Web Application Firewall - Security Collaboration between FPT Cloud and Penta Security.
The Next generation firewall security service
Easily store, manage, deploy, and secure Container images
Safe, secure, stable, high-performance Kubernetes platform
Provided as a service to deploy, monitor, backup, restore, and scale MongoDB databases on cloud.
Provided as a service to deploy, monitor, backup, restore, and scale Redis databases on cloud.
Provided as a service to deploy, monitor, backup, restore, and scale PostgreSQL databases on cloud.
The service automatically initiates, maintains, manages, and protects APIs of any sizes.
- Pricing
- Partner
- Event
Training configuration
Training configuration
How do I choose the right training configuration?
1. Number of GPUs that need to fine-tune a model
It depends on the model size:
- <1B parameters: 1 GPU (24 GB VRAM) is sufficient
- 7B models: 2–4 GPUs (40 GB VRAM each)
- 13B models: 4–8 GPUs recommended
- 30B+ models: Requires 8+ GPUs and multi-node setup
2. Do I need multiple nodes or just one node?
- For small to medium models (up to 13B), a single node with multiple GPUs is enough.
- For large models (30B+), multi-node setups are recommended for better memory and performance.
3. What is the minimum GPU memory required?
- At least 24 GB per GPU for standard fine-tuning.
- You can fine-tune on GPUs with 8–16 GB VRAM using LoRA or QLoRA methods.
Example:
Model: Llama-3.1-8B-Instruct
-
Training type: Full
- Number of GPUs: can fit into 2 GPUs (nearly 99% usage) -> 4 GPUs for more consistent runtime
- Distributed backend: DeepSeed
- ZeRO Stage: 3
- Batch size per device: 1
- All other parameters can be left as default
-
Training type: LoRA:
- Number of GPUs: can fit into 1 GPU
- LoRA Rank: 16
- Batch size per device: 1
- All other parameters can be left as default
-
To calculate the most suitable training configuration, you can refer here: https://rahulschand.github.io/gpu_poor/ (overhead 10-20%)
© 2025 FPT Cloud. All Rights Reserved.
