LM/llama.cpp for TurboQuant: Difference between revisions
< LM
Jump to navigation
Jump to search
(→Build) |
(→Build) |
||
| Line 19: | Line 19: | ||
<syntaxhighlight lang="bash"> | <syntaxhighlight lang="bash"> | ||
sudo apt install libvulkan-dev glslc spirv-headers | |||
# set options | # set options | ||
cmake -B build/ | cmake -B build/vulkan -G Ninja \ | ||
-DCMAKE_BUILD_TYPE=Release \ | -DCMAKE_BUILD_TYPE=Release \ | ||
- | -DGGML_VULKAN=ON \ | ||
-DGGML_NATIVE=ON \ | |||
-DGGML_VULKAN_CHECK_RESULTS=OFF | |||
# build | # build | ||
Revision as of 07:31, 21 August 2026
Download source
sudo apt install git cmake
sudo apt install libssl-dev openssl
mkdir -p ~/llama.cpp/src
git clone https://github.com/TheTom/llama-cpp-turboquant ~/llama.cpp/src-tbq
List available options
grep -rn "option(GGML_SYCL" CMakeLists.txt ggml/CMakeLists.txt
grep -rn "SYCL" CMakeLists.txt ggml/CMakeLists.txt
Build
sudo apt install libvulkan-dev glslc spirv-headers
# set options
cmake -B build/vulkan -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DGGML_VULKAN=ON \
-DGGML_NATIVE=ON \
-DGGML_VULKAN_CHECK_RESULTS=OFF
# build
# !!! don't use option --parallel
cmake --build build/sycl --parallel
# install
cmake --install build/sycl --prefix ~/llama.cpp/sycl
# clean
rm -rf build-sycl
# test GPU
cd ~/llama.cpp/sycl/bin
./llama-cli --list-devices
Got It!
Available devices: SYCL0: Intel(R) Arc(TM) Pro B70 Graphics (32656 MiB, 32128 MiB free)