What are the common issues when deploying custom models?

Here are key areas to troubleshoot for custom model deployments:

​

1. Deployment hanging or crashing

Common causes:

• Missing model files, especially when using Hugging Face models
• Symlinked files not uploaded correctly
• Outdated firectl version

Solutions:

• Download models without symlinks using:
  Copy

  Ask AI

  huggingface-cli download model_name --local-dir=/path --local-dir-use-symlinks=False
  

• Update firectl to the latest version

​

2. LoRA adapters vs full models

• Compatibility: LoRA adapters work with specific base models.
• Performance: May experience slightly lower speed with LoRA, but quality should remain similar to the original model.
• Troubleshooting quality drops:
  • Check model configuration
  • Review conversation template
  • Add echo: true to debug requests

​

3. Performance optimization factors

Consider adjusting the following for improved performance:

• Accelerator count and accelerator type
• Long prompt settings to handle complex inputs