> ## Documentation Index
> Fetch the complete documentation index at: https://docs.trysixth.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Read Me First

## Running Local Models with Sixth: What You Need to Know 🤖

Sixth is a powerful AI coding assistant that uses tool-calling to help you write, analyze, and modify code. While running models locally can save on API costs, there's an important trade-off: local models are significantly less reliable at using these essential tools.

## Why Local Models Are Different 🔬

When you run a "local version" of a model, you're actually running a drastically simplified copy of the original. This process, called distillation, is like trying to compress a professional chef's knowledge into a basic cookbook – you keep the simple recipes but lose the complex techniques and intuition.

Local models are created by training a smaller model to imitate a larger one, but they typically only retain 1-26% of the original model's capacity. This massive reduction means:

* Less ability to understand complex contexts
* Reduced capability for multi-step reasoning
* Limited tool-use abilities
* Simplified decision-making process

Think of it like running your development environment on a calculator instead of a computer – it might handle basic tasks, but complex operations become unreliable or impossible.

<Frame>
  <img src="https://storage.googleapis.com/sixth_public_images/docs/assets/image%20(4).png" alt="Local model comparison diagram" />
</Frame>

### What Actually Happens

When you run a local model with Sixth:

#### Performance Impact 📉

* Responses are 5-10x slower than cloud services
* System resources (CPU, GPU, RAM) get heavily utilized
* Your computer may become less responsive for other tasks

#### Tool Reliability Issues 🛠️

* Code analysis becomes less accurate
* File operations may be unreliable
* Browser automation capabilities are reduced
* Terminal commands might fail more often
* Complex multi-step tasks often break down

### Hardware Requirements 💻

You'll need at minimum:

* Modern GPU with 8GB+ VRAM (RTX 3070 or better)
* 32GB+ system RAM
* Fast SSD storage
* Good cooling solution

Even with this hardware, you'll be running smaller, less capable versions of models:

| Model Size | What You Get                                            |
| ---------- | ------------------------------------------------------- |
| 7B models  | Basic coding, limited tool use                          |
| 14B models | Better coding, unstable tool use                        |
| 32B models | Good coding, inconsistent tool use                      |
| 70B models | Best local performance, but requires expensive hardware |

Put simply, the cloud (API) versions of these models are the full-bore version of the model. The full version of DeepSeek-R1 is 671B. These distilled models are essentially "watered-down" versions of the cloud model.

### Practical Recommendations 💡

#### Consider This Approach

1. Use cloud models for:
   * Complex development tasks
   * When tool reliability is crucial
   * Multi-step operations
   * Critical code changes
2. Use local models for:
   * Simple code completion
   * Basic documentation
   * When privacy is paramount
   * Learning and experimentation

#### If You Must Go Local

* Start with smaller models
* Keep tasks simple and focused
* Save work frequently
* Be prepared to switch to cloud models for complex operations
* Monitor system resources

### Common Issues 🚨

* **"Tool execution failed":** Local models often struggle with complex tool chains. Simplify your prompt.
* **"No connection could be made because the target machine actively refused it":** This usually means that the Ollama or LM Studio server isn't running, or is running on a different port/address than Sixth is configured to use. Double-check the Base URL address in your API Provider settings.
* **"Sixth is having trouble...":** Increase your model's context length to its maximum size.
* **Slow or incomplete responses:** Local models can be slower than cloud-based models, especially on less powerful hardware. If performance is an issue, try using a smaller model. Expect significantly longer processing times.
* **System stability:** Watch for high GPU/CPU usage and temperature
* **Context limitations:** Local models often have smaller context windows than cloud models. Break tasks down into smaller pieces.

### Looking Ahead 🔮

Local model capabilities are improving, but they're not yet a complete replacement for cloud services, especially for Sixth's tool-based functionality. Consider your specific needs and hardware capabilities carefully before committing to a local-only approach.

### Need Help? 🤝

* Join our [Discord](https://discord.gg/sixth) community and [r/sixth](https://www.reddit.com/r/CLine/)
* Check the latest compatibility guides
* Share your experiences with other developers

Remember: When in doubt, prioritize reliability over cost savings for important development work.