LM/llama.cpp for TurboQuant: Difference between revisions
< LM
Jump to navigation
Jump to search
(→Run) |
(→Build) |
||
| (5 intermediate revisions by the same user not shown) | |||
| Line 19: | Line 19: | ||
<syntaxhighlight lang="bash"> | <syntaxhighlight lang="bash"> | ||
sudo apt install libvulkan-dev glslc spirv-headers | |||
sudo apt install - | |||
# set options | # set options | ||
cmake -B build/ | cmake -B build/vulkan -G Ninja \ | ||
- | -DCMAKE_BUILD_TYPE=Release \ | ||
-DGGML_VULKAN=ON \ | |||
- | -DGGML_NATIVE=ON \ | ||
- | -DGGML_VULKAN_CHECK_RESULTS=OFF \ | ||
- | |||
-DCMAKE_BUILD_TYPE=Release \ | -DCMAKE_BUILD_TYPE=Release \ | ||
-DCMAKE_INSTALL_RPATH="\$ORIGIN/../lib" | -DCMAKE_INSTALL_RPATH="\$ORIGIN/../lib" | ||
# build | # build | ||
cmake --build build/vulkan --parallel | |||
cmake --build build/ | |||
# install | # install | ||
cmake --install build/ | cmake --install build/vulkan --prefix ~/llama.cpp/vulkan | ||
# clean | # clean | ||
rm -rf build | rm -rf build | ||
# test GPU | # test GPU | ||
cd ~/llama.cpp/ | cd ~/llama.cpp/vulkan/bin | ||
./llama-cli --list-devices | ./llama-cli --list-devices | ||
</syntaxhighlight> | </syntaxhighlight> | ||
| Line 61: | Line 50: | ||
SYCL0: Intel(R) Arc(TM) Pro B70 Graphics (32656 MiB, 32128 MiB free) | SYCL0: Intel(R) Arc(TM) Pro B70 Graphics (32656 MiB, 32128 MiB free) | ||
</pre> | </pre> | ||
Latest revision as of 07:37, 21 August 2026
Download source
sudo apt install git cmake
sudo apt install libssl-dev openssl
mkdir -p ~/llama.cpp/src
git clone https://github.com/TheTom/llama-cpp-turboquant ~/llama.cpp/src-tbq
List available options
grep -rn "option(GGML_SYCL" CMakeLists.txt ggml/CMakeLists.txt
grep -rn "SYCL" CMakeLists.txt ggml/CMakeLists.txt
Build
sudo apt install libvulkan-dev glslc spirv-headers
# set options
cmake -B build/vulkan -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DGGML_VULKAN=ON \
-DGGML_NATIVE=ON \
-DGGML_VULKAN_CHECK_RESULTS=OFF \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_RPATH="\$ORIGIN/../lib"
# build
cmake --build build/vulkan --parallel
# install
cmake --install build/vulkan --prefix ~/llama.cpp/vulkan
# clean
rm -rf build
# test GPU
cd ~/llama.cpp/vulkan/bin
./llama-cli --list-devices
Got It!
Available devices: SYCL0: Intel(R) Arc(TM) Pro B70 Graphics (32656 MiB, 32128 MiB free)