> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/ggml-org/llama.cpp/llms.txt
> Use this file to discover all available pages before exploring further.

# Building from Source

> Comprehensive guide to building llama.cpp from source on all platforms

## Getting the Code

Clone the repository from GitHub:

```bash theme={null}
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
```

## CPU Build

<Steps>
  <Step title="Configure the build">
    Use CMake to configure the build directory:

    ```bash theme={null}
    cmake -B build
    ```
  </Step>

  <Step title="Build the project">
    Compile with CMake:

    ```bash theme={null}
    cmake --build build --config Release
    ```

    <Note>
      For faster compilation, add `-j` to run multiple jobs in parallel:

      ```bash theme={null}
      cmake --build build --config Release -j 8
      ```
    </Note>
  </Step>

  <Step title="Run the binary">
    After building, binaries are located in `build/bin/`:

    ```bash theme={null}
    ./build/bin/llama-cli --help
    ```
  </Step>
</Steps>

### Debug Builds

For debug builds, the process differs based on your generator:

<CodeGroup>
  ```bash Single-config generators (Unix Makefiles) theme={null}
  cmake -B build -DCMAKE_BUILD_TYPE=Debug
  cmake --build build
  ```

  ```bash Multi-config generators (Visual Studio, Xcode) theme={null}
  cmake -B build -G "Xcode"
  cmake --build build --config Debug
  ```
</CodeGroup>

### Static Builds

To build static libraries instead of shared:

```bash theme={null}
cmake -B build -DBUILD_SHARED_LIBS=OFF
cmake --build build --config Release
```

### Performance Tips

* **ccache**: Install [ccache](https://ccache.dev/) for faster repeated compilation
* **Parallel builds**: Use `-j` flag with the number of CPU cores
* **Generators**: Use Ninja generator for automatic parallelization: `cmake -B build -G Ninja`

## Metal Build (macOS)

On macOS, Metal is **enabled by default** for GPU acceleration.

```bash theme={null}
cmake -B build
cmake --build build --config Release
```

<Note>
  Metal makes computations run on the GPU. To disable Metal at compile time:

  ```bash theme={null}
  cmake -B build -DGGML_METAL=OFF
  ```
</Note>

At runtime, you can disable GPU inference with:

```bash theme={null}
./build/bin/llama-cli -m model.gguf --n-gpu-layers 0
```

## CUDA Build (NVIDIA GPU)

For NVIDIA GPU acceleration, ensure you have the [CUDA toolkit](https://developer.nvidia.com/cuda-toolkit) installed.

<Steps>
  <Step title="Install CUDA toolkit">
    Download from the [NVIDIA developer site](https://developer.nvidia.com/cuda-downloads) and follow installation instructions for your platform.
  </Step>

  <Step title="Build with CUDA support">
    ```bash theme={null}
    cmake -B build -DGGML_CUDA=ON
    cmake --build build --config Release
    ```
  </Step>

  <Step title="Verify CUDA is working">
    Run with GPU layers:

    ```bash theme={null}
    ./build/bin/llama-cli -m model.gguf -ngl 99
    ```
  </Step>
</Steps>

### Non-Native CUDA Builds

By default, llama.cpp builds for GPUs connected to your system. For a build covering all CUDA GPUs:

```bash theme={null}
cmake -B build -DGGML_CUDA=ON -DGGML_NATIVE=OFF
cmake --build build --config Release
```

<Warning>
  This results in a larger binary and longer compilation time, but the binary will run on any CUDA GPU.
</Warning>

### Override Compute Capability

If `nvcc` cannot detect your GPU, explicitly specify architectures:

<Steps>
  <Step title="Find your GPU's compute capability">
    Check [NVIDIA's CUDA GPUs page](https://developer.nvidia.com/cuda-gpus) for your GPU's compute capability.

    Examples:

    * GeForce RTX 4090: 8.9
    * GeForce RTX 3080 Ti: 8.6
    * GeForce RTX 3070: 8.6
  </Step>

  <Step title="Build with specific architectures">
    ```bash theme={null}
    cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES="86;89"
    cmake --build build --config Release
    ```
  </Step>
</Steps>

### CUDA Runtime Variables

Control CUDA behavior with environment variables:

```bash theme={null}
# Hide specific devices
CUDA_VISIBLE_DEVICES="-0" ./build/bin/llama-server -m model.gguf

# Increase command buffer for multi-GPU setups
CUDA_SCALE_LAUNCH_QUEUES=4x ./build/bin/llama-cli -m model.gguf

# Enable unified memory (allows RAM fallback)
GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 ./build/bin/llama-cli -m model.gguf
```

## HIP Build (AMD GPU)

For AMD GPU acceleration using ROCm/HIP:

<Steps>
  <Step title="Install ROCm">
    Install ROCm from your Linux distro's package manager or from the [ROCm Quick Start guide](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/tutorial/quick-start.html).
  </Step>

  <Step title="Build with HIP support">
    For a gfx1030-compatible AMD GPU:

    ```bash theme={null}
    HIPCXX="$(hipconfig -l)/clang" HIP_PATH="$(hipconfig -R)" \
        cmake -S . -B build -DGGML_HIP=ON -DGPU_TARGETS=gfx1030 -DCMAKE_BUILD_TYPE=Release
    cmake --build build --config Release -- -j 16
    ```

    <Note>
      `GPU_TARGETS` is optional. Omitting it will build for all GPUs in the current system.
    </Note>
  </Step>

  <Step title="Find your GPU architecture">
    Find your GPU version:

    ```bash theme={null}
    rocminfo | grep gfx | head -1 | awk '{print $2}'
    ```

    Match with [LLVM's processor list](https://llvm.org/docs/AMDGPUUsage.html#processors). For example, `gfx1035` maps to `gfx1030`.
  </Step>
</Steps>

### Windows HIP Build

Using x64 Native Tools Command Prompt for VS:

```bash theme={null}
set PATH=%HIP_PATH%\bin;%PATH%
cmake -S . -B build -G Ninja -DGPU_TARGETS=gfx1100 -DGGML_HIP=ON \
    -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ \
    -DCMAKE_BUILD_TYPE=Release
cmake --build build
```

## Vulkan Build

Vulkan provides cross-platform GPU acceleration.

### Windows

<Tabs>
  <Tab title="w64devkit">
    <Steps>
      <Step title="Install dependencies">
        1. Download and extract [w64devkit](https://github.com/skeeto/w64devkit/releases)
        2. Install the [Vulkan SDK](https://vulkan.lunarg.com/sdk/home#windows)
      </Step>

      <Step title="Copy Vulkan dependencies">
        Launch `w64devkit.exe` and run:

        ```sh theme={null}
        SDK_VERSION=1.3.283.0
        cp /VulkanSDK/$SDK_VERSION/Bin/glslc.exe $W64DEVKIT_HOME/bin/
        cp /VulkanSDK/$SDK_VERSION/Lib/vulkan-1.lib $W64DEVKIT_HOME/x86_64-w64-mingw32/lib/
        cp -r /VulkanSDK/$SDK_VERSION/Include/* $W64DEVKIT_HOME/x86_64-w64-mingw32/include/
        ```
      </Step>

      <Step title="Build">
        ```sh theme={null}
        cmake -B build -DGGML_VULKAN=ON
        cmake --build build --config Release
        ```
      </Step>
    </Steps>
  </Tab>

  <Tab title="Git Bash MINGW64">
    <Steps>
      <Step title="Install dependencies">
        1. Install [Git-SCM](https://git-scm.com/downloads/win)
        2. Install [Visual Studio Community Edition](https://visualstudio.microsoft.com/) with C++
        3. Install [CMake](https://cmake.org/download/)
        4. Install [Vulkan SDK](https://vulkan.lunarg.com/sdk/home#windows)
      </Step>

      <Step title="Build">
        Right-click in llama.cpp directory, select "Open Git Bash Here":

        ```bash theme={null}
        cmake -B build -DGGML_VULKAN=ON
        cmake --build build --config Release
        ```
      </Step>
    </Steps>
  </Tab>

  <Tab title="MSYS2">
    <Steps>
      <Step title="Install dependencies">
        Install [MSYS2](https://www.msys2.org/) and run in UCRT terminal:

        ```sh theme={null}
        pacman -S git \
            mingw-w64-ucrt-x86_64-gcc \
            mingw-w64-ucrt-x86_64-cmake \
            mingw-w64-ucrt-x86_64-vulkan-devel \
            mingw-w64-ucrt-x86_64-shaderc
        ```
      </Step>

      <Step title="Build">
        ```sh theme={null}
        cmake -B build -DGGML_VULKAN=ON
        cmake --build build --config Release
        ```
      </Step>
    </Steps>
  </Tab>
</Tabs>

### Linux

<Tabs>
  <Tab title="System packages">
    Install required dependencies:

    ```bash Debian/Ubuntu theme={null}
    sudo apt-get install libvulkan-dev glslc
    ```

    Then build:

    ```bash theme={null}
    vulkaninfo  # Verify Vulkan is working
    cmake -B build -DGGML_VULKAN=1
    cmake --build build --config Release
    ```
  </Tab>

  <Tab title="LunarG SDK">
    <Steps>
      <Step title="Install Vulkan SDK">
        Follow the [Getting Started with the Linux Tarball Vulkan SDK](https://vulkan.lunarg.com/doc/sdk/latest/linux/getting_started.html) guide.
      </Step>

      <Step title="Source environment">
        ```bash theme={null}
        source /path/to/vulkan-sdk/setup-env.sh
        ```

        <Warning>
          You must source this file in every terminal session where you build or run llama.cpp with Vulkan.
        </Warning>
      </Step>

      <Step title="Build">
        ```bash theme={null}
        vulkaninfo  # Verify setup
        cmake -B build -DGGML_VULKAN=1
        cmake --build build --config Release
        ```
      </Step>
    </Steps>
  </Tab>
</Tabs>

### macOS

<Steps>
  <Step title="Install Vulkan SDK">
    Follow the [Getting Started with the MacOS Vulkan SDK](https://vulkan.lunarg.com/doc/sdk/latest/mac/getting_started.html) guide.

    <Note>
      Check the "KosmicKrisp" box during installation for better performance.
    </Note>
  </Step>

  <Step title="Set environment variables">
    ```bash theme={null}
    source /path/to/vulkan-sdk/setup-env.sh
    ```

    For KosmicKrisp (better performance):

    ```bash theme={null}
    export VK_ICD_FILENAMES=$VULKAN_SDK/share/vulkan/icd.d/libkosmickrisp_icd.json
    export VK_DRIVER_FILES=$VULKAN_SDK/share/vulkan/icd.d/libkosmickrisp_icd.json
    ```
  </Step>

  <Step title="Build">
    ```bash theme={null}
    cmake -B build -DGGML_VULKAN=1 -DGGML_METAL=OFF
    cmake --build build --config Release
    ```
  </Step>
</Steps>

## BLAS Build

BLAS support can improve prompt processing performance for batch sizes > 32.

### Accelerate Framework (macOS)

Enabled by default on macOS. Just build normally:

```bash theme={null}
cmake -B build
cmake --build build --config Release
```

### OpenBLAS (Linux)

Install OpenBLAS and build:

```bash theme={null}
cmake -B build -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS
cmake --build build --config Release
```

### Intel oneMKL

<CodeGroup>
  ```bash Manual installation theme={null}
  source /opt/intel/oneapi/setvars.sh
  cmake -B build -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=Intel10_64lp \
      -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx -DGGML_NATIVE=ON
  cmake --build build --config Release
  ```

  ```bash Docker image theme={null}
  # Use intel/oneapi-basekit Docker image
  cmake -B build -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=Intel10_64lp \
      -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx -DGGML_NATIVE=ON
  cmake --build build --config Release
  ```
</CodeGroup>

## SYCL Build (Intel GPU)

For Intel GPU support (Data Center Max, Flex, Arc series):

```bash theme={null}
cmake -B build -DGGML_SYCL=ON
cmake --build build --config Release
```

See the [SYCL backend documentation](./backend/SYCL.md) for detailed information.

## Platform-Specific Builds

### Windows

<Steps>
  <Step title="Install Visual Studio 2022">
    Install [Visual Studio 2022 Community Edition](https://visualstudio.microsoft.com/vs/community/).

    Select these components:

    * **Workload**: Desktop development with C++
    * **Components**:
      * C++ CMake Tools for Windows
      * Git for Windows
      * C++ Clang Compiler for Windows
      * MS-Build Support for LLVM-Toolset (clang)
  </Step>

  <Step title="Use Developer Command Prompt">
    Always use a Developer Command Prompt or PowerShell for VS2022.
  </Step>

  <Step title="Build">
    For Windows on ARM (WoA):

    ```bash theme={null}
    cmake --preset arm64-windows-llvm-release -D GGML_OPENMP=OFF
    cmake --build build-arm64-windows-llvm-release
    ```

    For x64 with Ninja and clang:

    ```bash theme={null}
    cmake --preset x64-windows-llvm-release
    cmake --build build-x64-windows-llvm-release
    ```
  </Step>
</Steps>

### Android

See the [Android build documentation](./android.md) for detailed instructions.

## Additional Backends

### CANN (Ascend NPU)

```bash theme={null}
cmake -B build -DGGML_CANN=on -DCMAKE_BUILD_TYPE=release
cmake --build build --config release
```

### ZenDNN (AMD EPYC CPUs)

```bash theme={null}
# Automatic build (first time takes 5-10 minutes)
cmake -B build -DGGML_ZENDNN=ON
cmake --build build --config Release

# With custom ZenDNN installation
cmake -B build -DGGML_ZENDNN=ON -DZENDNN_ROOT=/path/to/zendnn/install
cmake --build build --config Release
```

### Arm KleidiAI

Optimized kernels for Arm CPUs:

```bash theme={null}
cmake -B build -DGGML_CPU_KLEIDIAI=ON
cmake --build build --config Release
```

<Note>
  For SME support, set `GGML_KLEIDIAI_SME=1` at runtime.
</Note>

### OpenCL (Adreno GPU)

See the [OpenCL backend documentation](./backend/OPENCL.md) for Android and Windows ARM64 build instructions.

## Multi-Backend Builds

You can build with multiple backends simultaneously:

```bash theme={null}
cmake -B build -DGGML_CUDA=ON -DGGML_VULKAN=ON
cmake --build build --config Release
```

At runtime, specify which device to use:

```bash theme={null}
# List available devices
./build/bin/llama-cli --list-devices

# Use specific device
./build/bin/llama-cli -m model.gguf --device cuda:0

# Disable GPU entirely
./build/bin/llama-cli -m model.gguf --device none
```

## Dynamic Backend Loading

Build backends as dynamic libraries for portability:

```bash theme={null}
cmake -B build -DGGML_BACKEND_DL=ON
cmake --build build --config Release
```

This allows using the same binary on different machines with different GPUs.

## HTTPS/TLS Support

For HTTPS features, install OpenSSL development libraries:

<CodeGroup>
  ```bash Debian/Ubuntu theme={null}
  sudo apt-get install libssl-dev
  ```

  ```bash Fedora/RHEL/Rocky/Alma theme={null}
  sudo dnf install openssl-devel
  ```

  ```bash Arch/Manjaro theme={null}
  sudo pacman -S openssl
  ```
</CodeGroup>

<Note>
  If not installed, llama.cpp will build and run without SSL support.
</Note>