LM/llama.cpp for TurboQuant: Difference between revisions
< LM
Jump to navigation
Jump to search
(→Build) |
(→Build) |
||
| Line 33: | Line 33: | ||
# install | # install | ||
cmake --install build/vulkan --prefix ~/llama.cpp/ | cmake --install build/vulkan --prefix ~/llama.cpp/vulkan | ||
# clean | # clean | ||
| Line 39: | Line 39: | ||
# test GPU | # test GPU | ||
cd ~/llama.cpp/ | cd ~/llama.cpp/vulkan/bin | ||
./llama-cli --list-devices | ./llama-cli --list-devices | ||
</syntaxhighlight> | </syntaxhighlight> | ||
Revision as of 07:36, 21 August 2026
Download source
sudo apt install git cmake
sudo apt install libssl-dev openssl
mkdir -p ~/llama.cpp/src
git clone https://github.com/TheTom/llama-cpp-turboquant ~/llama.cpp/src-tbq
List available options
grep -rn "option(GGML_SYCL" CMakeLists.txt ggml/CMakeLists.txt
grep -rn "SYCL" CMakeLists.txt ggml/CMakeLists.txt
Build
sudo apt install libvulkan-dev glslc spirv-headers
# set options
cmake -B build/vulkan -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DGGML_VULKAN=ON \
-DGGML_NATIVE=ON \
-DGGML_VULKAN_CHECK_RESULTS=OFF
# build
# !!! don't use option --parallel
cmake --build build/vulkan --parallel
# install
cmake --install build/vulkan --prefix ~/llama.cpp/vulkan
# clean
rm -rf build
# test GPU
cd ~/llama.cpp/vulkan/bin
./llama-cli --list-devices
Got It!
Available devices: SYCL0: Intel(R) Arc(TM) Pro B70 Graphics (32656 MiB, 32128 MiB free)