# llama.cpp

## Docs

- [Docker Deployment](https://mintlify.wiki/ggml-org/llama.cpp/advanced/docker.md): Run llama.cpp in Docker containers for easy deployment and isolation
- [Function Calling](https://mintlify.wiki/ggml-org/llama.cpp/advanced/function-calling.md): Use function calling and tool use with llama.cpp models
- [Grammars (GBNF)](https://mintlify.wiki/ggml-org/llama.cpp/advanced/grammars.md): Constrain model outputs with formal grammars using GBNF
- [Performance Tuning](https://mintlify.wiki/ggml-org/llama.cpp/advanced/performance-tuning.md): Optimize llama.cpp for maximum inference speed and efficiency
- [Inference](https://mintlify.wiki/ggml-org/llama.cpp/api/inference.md): Running inference with llama_decode, batching, and KV cache management
- [libllama C API](https://mintlify.wiki/ggml-org/llama.cpp/api/libllama.md): Overview of the llama.cpp C API for LLM inference
- [Model Loading](https://mintlify.wiki/ggml-org/llama.cpp/api/model-loading.md): Loading GGUF models and configuring model parameters
- [Chat Completions](https://mintlify.wiki/ggml-org/llama.cpp/api/rest/chat-completions.md): OpenAI-compatible chat completions endpoint
- [Completions](https://mintlify.wiki/ggml-org/llama.cpp/api/rest/completions.md): OpenAI-compatible text completions endpoint
- [Embeddings](https://mintlify.wiki/ggml-org/llama.cpp/api/rest/embeddings.md): OpenAI-compatible embeddings endpoint for vector generation
- [REST API Overview](https://mintlify.wiki/ggml-org/llama.cpp/api/rest/overview.md): Overview of the llama.cpp OpenAI-compatible REST API
- [Sampling](https://mintlify.wiki/ggml-org/llama.cpp/api/sampling.md): Token sampling strategies and the llama_sampler API
- [llama-bench](https://mintlify.wiki/ggml-org/llama.cpp/api/tools/llama-bench.md): Performance testing and benchmarking tool for llama.cpp
- [llama-cli](https://mintlify.wiki/ggml-org/llama.cpp/api/tools/llama-cli.md): Command-line interface for interacting with LLM models
- [llama-perplexity](https://mintlify.wiki/ggml-org/llama.cpp/api/tools/llama-perplexity.md): Measure perplexity and evaluate language model quality
- [llama-server](https://mintlify.wiki/ggml-org/llama.cpp/api/tools/llama-server.md): HTTP server for serving LLM models with OpenAI-compatible API
- [llama.cpp Architecture](https://mintlify.wiki/ggml-org/llama.cpp/concepts/architecture.md): Understanding the internal architecture and design of llama.cpp
- [Compute Backends](https://mintlify.wiki/ggml-org/llama.cpp/concepts/backends.md): Hardware acceleration options for llama.cpp inference
- [GGUF File Format](https://mintlify.wiki/ggml-org/llama.cpp/concepts/gguf-format.md): Understanding the GGUF binary file format used by llama.cpp
- [Model Quantization](https://mintlify.wiki/ggml-org/llama.cpp/concepts/quantization.md): Understanding quantization methods and performance tradeoffs in llama.cpp
- [Adding New Model Architectures](https://mintlify.wiki/ggml-org/llama.cpp/development/adding-models.md): Step-by-step guide for adding support for new model architectures to llama.cpp
- [CMake Build Options](https://mintlify.wiki/ggml-org/llama.cpp/development/build-options.md): Complete reference for CMake configuration options in llama.cpp
- [Building from Source](https://mintlify.wiki/ggml-org/llama.cpp/development/building.md): Comprehensive guide to building llama.cpp from source on all platforms
- [CMake Presets](https://mintlify.wiki/ggml-org/llama.cpp/development/cmake-presets.md): Using CMake presets for quick and consistent builds across platforms
- [Contributing to llama.cpp](https://mintlify.wiki/ggml-org/llama.cpp/development/contributing.md): Guidelines for contributing code, submitting pull requests, and maintaining code quality
- [Testing llama.cpp](https://mintlify.wiki/ggml-org/llama.cpp/development/testing.md): Comprehensive guide to testing procedures, running tests, and debugging test failures
- [CLI Tool (llama-cli)](https://mintlify.wiki/ggml-org/llama.cpp/inference/cli.md): Command-line interface for LLM inference with llama.cpp
- [Embeddings](https://mintlify.wiki/ggml-org/llama.cpp/inference/embeddings.md): Generate text embeddings with llama.cpp for semantic search and similarity
- [Multimodal Models](https://mintlify.wiki/ggml-org/llama.cpp/inference/multimodal.md): Vision and audio support in llama.cpp for multimodal AI applications
- [Server (llama-server)](https://mintlify.wiki/ggml-org/llama.cpp/inference/server.md): OpenAI-compatible HTTP API server for llama.cpp
- [Speculative Decoding](https://mintlify.wiki/ggml-org/llama.cpp/inference/speculative-decoding.md): Accelerate token generation with draft models and pattern matching in llama.cpp
- [Installation](https://mintlify.wiki/ggml-org/llama.cpp/installation.md): Complete guide to installing llama.cpp - from package managers to building from source with GPU acceleration
- [Introduction](https://mintlify.wiki/ggml-org/llama.cpp/introduction.md): LLM inference in C/C++ with minimal setup and state-of-the-art performance
- [Converting Models](https://mintlify.wiki/ggml-org/llama.cpp/models/converting-models.md): Convert models from PyTorch, SafeTensors, and other formats to GGUF
- [Obtaining Models](https://mintlify.wiki/ggml-org/llama.cpp/models/obtaining-models.md): How to download and acquire models for use with llama.cpp
- [Quantizing Models](https://mintlify.wiki/ggml-org/llama.cpp/models/quantizing-models.md): Reduce model size and improve inference speed through quantization
- [Supported Models](https://mintlify.wiki/ggml-org/llama.cpp/models/supported-models.md): Comprehensive list of LLM architectures supported by llama.cpp
- [Quick Start](https://mintlify.wiki/ggml-org/llama.cpp/quickstart.md): Get started with llama.cpp in minutes - install, download a model, and run your first inference

## OpenAPI Specs

- [openapi](https://mintlify.wiki/ggml-org/llama.cpp/api-reference/openapi.json)