{"id":1121,"date":"2026-03-18T15:27:02","date_gmt":"2026-03-18T15:27:02","guid":{"rendered":"https:\/\/techno.slomka.biz\/?p=1121"},"modified":"2026-04-21T09:56:43","modified_gmt":"2026-04-21T09:56:43","slug":"running-ollama-with-nvidia-gpu-inside-wsl-ubuntu-step-by-step-guidese","status":"publish","type":"post","link":"https:\/\/techno.slomka.biz\/?p=1121","title":{"rendered":"Running Ollama with NVIDIA GPU inside WSL (Ubuntu) \u2013 Step-by-Step Guide"},"content":{"rendered":"\n<p>Running large language models locally with GPU acceleration inside WSL2 is not only possible\u2014it\u2019s surprisingly efficient once properly configured. This guide walks through a working setup using Ubuntu, NVIDIA GPU passthrough, and Ollama.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\ud83e\udde9 Target Setup<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Windows host with NVIDIA GPU<\/li>\n\n\n\n<li>WSL2 (Ubuntu)<\/li>\n\n\n\n<li>GPU passthrough via WSL<\/li>\n\n\n\n<li>Ollama using GPU acceleration<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">1. Prepare Windows Host<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Check Windows Version<\/h3>\n\n\n\n<p>Ensure you&#8217;re on a supported version:<\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#EEFFFF;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:flex;align-items:center;padding:10px 0px 10px 16px;margin-bottom:-2px;width:100%;text-align:left;background-color:#304047;color:#d5ffff\">PowerShell<\/span><span role=\"button\" tabindex=\"0\" style=\"color:#EEFFFF;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>winver<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki material-theme\" style=\"background-color: #263238\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #EEFFFF\">winver<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p>Recommended:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Windows 11 (25H2+)<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Enable WSL2<\/h3>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#EEFFFF;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:flex;align-items:center;padding:10px 0px 10px 16px;margin-bottom:-2px;width:100%;text-align:left;background-color:#304047;color:#d5ffff\">PowerShell<\/span><span role=\"button\" tabindex=\"0\" style=\"color:#EEFFFF;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>wsl --install\nwsl --set-default-version 2<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki material-theme\" style=\"background-color: #263238\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #EEFFFF\">wsl <\/span><span style=\"color: #89DDFF\">--<\/span><span style=\"color: #EEFFFF\">install<\/span><\/span>\n<span class=\"line\"><span style=\"color: #EEFFFF\">wsl <\/span><span style=\"color: #89DDFF\">--<\/span><span style=\"color: #82AAFF\">set-default<\/span><span style=\"color: #89DDFF\">-<\/span><span style=\"color: #EEFFFF\">version <\/span><span style=\"color: #F78C6C\">2<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<h3 class=\"wp-block-heading\">Install NVIDIA Driver (with WSL Support) on your Windows machine<\/h3>\n\n\n\n<p>Install a current NVIDIA driver that supports WSL CUDA.<\/p>\n\n\n\n<p>Verify:<\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#EEFFFF;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:flex;align-items:center;padding:10px 0px 10px 16px;margin-bottom:-2px;width:100%;text-align:left;background-color:#304047;color:#d5ffff\">PowerShell<\/span><span role=\"button\" tabindex=\"0\" style=\"color:#EEFFFF;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>nvidia-smi<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki material-theme\" style=\"background-color: #263238\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #EEFFFF\">nvidia<\/span><span style=\"color: #89DDFF\">-<\/span><span style=\"color: #EEFFFF\">smi<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p>If this fails, stop here\u2014GPU passthrough will not work.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">2. Prepare Ubuntu (WSL)<\/h2>\n\n\n\n<p>Start WSL:<\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#EEFFFF;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:flex;align-items:center;padding:10px 0px 10px 16px;margin-bottom:-2px;width:100%;text-align:left;background-color:#304047;color:#d5ffff\">PowerShell<\/span><span role=\"button\" tabindex=\"0\" style=\"color:#EEFFFF;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>wsl<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki material-theme\" style=\"background-color: #263238\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #EEFFFF\">wsl<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p>Update packages:<\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#EEFFFF;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:flex;align-items:center;padding:10px 0px 10px 16px;margin-bottom:-2px;width:100%;text-align:left;background-color:#304047;color:#d5ffff\">Bash<\/span><span role=\"button\" tabindex=\"0\" style=\"color:#EEFFFF;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>sudo apt update &amp;&amp; sudo apt upgrade -y<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki material-theme\" style=\"background-color: #263238\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #FFCB6B\">sudo<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">apt<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">update<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">&amp;&amp;<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #FFCB6B\">sudo<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">apt<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">upgrade<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">-y<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">3. Verify GPU inside WSL<\/h2>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#EEFFFF;--cbp-line-number-width:calc(2 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:flex;align-items:center;padding:10px 0px 10px 16px;margin-bottom:-2px;width:100%;text-align:left;background-color:#304047;color:#d5ffff\">Bash<\/span><span role=\"button\" tabindex=\"0\" style=\"color:#EEFFFF;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>nvidia-smi -L\nGPU 0: NVIDIA GeForce RTX 4070 Ti SUPER (UUID: GPU-0122fdb1-cb26-cf9a-8c28-675c70ee828d)\n\nnvidia-smi\nWed Mar 18 16:02:22 2026\n+-----------------------------------------------------------------------------------------+\n| NVIDIA-SMI 595.54                 Driver Version: 595.79         CUDA Version: 13.2     |\n+-----------------------------------------+------------------------+----------------------+\n| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |\n| Fan  Temp   Perf          Pwr:Usage\/Cap |           Memory-Usage | GPU-Util  Compute M. |\n|                                         |                        |               MIG M. |\n|=========================================+========================+======================|\n|   0  NVIDIA GeForce RTX 4070 ...    On  |   00000000:01:00.0  On |                  N\/A |\n|  0%   38C    P8             17W \/  285W |    1032MiB \/  16376MiB |     11%      Default |\n|                                         |                        |                  N\/A |\n+-----------------------------------------+------------------------+----------------------+<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki material-theme\" style=\"background-color: #263238\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #FFCB6B\">nvidia-smi<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">-L<\/span><\/span>\n<span class=\"line\"><span style=\"color: #FFCB6B\">GPU<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #F78C6C\">0<\/span><span style=\"color: #C3E88D\">:<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">NVIDIA<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">GeForce<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">RTX<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #F78C6C\">4070<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">Ti<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">SUPER<\/span><span style=\"color: #EEFFFF\"> (UUID: <\/span><span style=\"color: #C3E88D\">GPU-0122fdb1-cb26-cf9a-8c28-675c70ee828d<\/span><span style=\"color: #EEFFFF\">)<\/span><\/span>\n<span class=\"line\"><\/span>\n<span class=\"line\"><span style=\"color: #FFCB6B\">nvidia-smi<\/span><\/span>\n<span class=\"line\"><span style=\"color: #FFCB6B\">Wed<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">Mar<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #F78C6C\">18<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #F78C6C\">16<\/span><span style=\"color: #C3E88D\">:02:22<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #F78C6C\">2026<\/span><\/span>\n<span class=\"line\"><span style=\"color: #FFCB6B\">+-----------------------------------------------------------------------------------------+<\/span><\/span>\n<span class=\"line\"><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #FFCB6B\">NVIDIA-SMI<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #F78C6C\">595.54<\/span><span style=\"color: #EEFFFF\">                 <\/span><span style=\"color: #C3E88D\">Driver<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">Version:<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #F78C6C\">595.79<\/span><span style=\"color: #EEFFFF\">         <\/span><span style=\"color: #C3E88D\">CUDA<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">Version:<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #F78C6C\">13.2<\/span><span style=\"color: #EEFFFF\">     <\/span><span style=\"color: #89DDFF\">|<\/span><\/span>\n<span class=\"line\"><span style=\"color: #FFCB6B\">+-----------------------------------------+------------------------+----------------------+<\/span><\/span>\n<span class=\"line\"><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #FFCB6B\">GPU<\/span><span style=\"color: #EEFFFF\">  <\/span><span style=\"color: #C3E88D\">Name<\/span><span style=\"color: #EEFFFF\">                 <\/span><span style=\"color: #C3E88D\">Persistence-M<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #FFCB6B\">Bus-Id<\/span><span style=\"color: #EEFFFF\">          <\/span><span style=\"color: #C3E88D\">Disp.A<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #FFCB6B\">Volatile<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">Uncorr.<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">ECC<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><\/span>\n<span class=\"line\"><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #FFCB6B\">Fan<\/span><span style=\"color: #EEFFFF\">  <\/span><span style=\"color: #C3E88D\">Temp<\/span><span style=\"color: #EEFFFF\">   <\/span><span style=\"color: #C3E88D\">Perf<\/span><span style=\"color: #EEFFFF\">          <\/span><span style=\"color: #C3E88D\">Pwr:Usage\/Cap<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">           <\/span><span style=\"color: #FFCB6B\">Memory-Usage<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #FFCB6B\">GPU-Util<\/span><span style=\"color: #EEFFFF\">  <\/span><span style=\"color: #C3E88D\">Compute<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">M.<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><\/span>\n<span class=\"line\"><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">                                         <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">                        <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">               <\/span><span style=\"color: #FFCB6B\">MIG<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">M.<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><\/span>\n<span class=\"line\"><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">=========================================+========================+======================<\/span><span style=\"color: #89DDFF\">|<\/span><\/span>\n<span class=\"line\"><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">   <\/span><span style=\"color: #FFCB6B\">0<\/span><span style=\"color: #EEFFFF\">  <\/span><span style=\"color: #C3E88D\">NVIDIA<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">GeForce<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">RTX<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #F78C6C\">4070<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">...<\/span><span style=\"color: #EEFFFF\">    <\/span><span style=\"color: #C3E88D\">On<\/span><span style=\"color: #EEFFFF\">  <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">   <\/span><span style=\"color: #FFCB6B\">00000000:01:00.0<\/span><span style=\"color: #EEFFFF\">  <\/span><span style=\"color: #C3E88D\">On<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">                  <\/span><span style=\"color: #FFCB6B\">N\/A<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><\/span>\n<span class=\"line\"><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">  <\/span><span style=\"color: #FFCB6B\">0%<\/span><span style=\"color: #EEFFFF\">   <\/span><span style=\"color: #F78C6C\">38<\/span><span style=\"color: #C3E88D\">C<\/span><span style=\"color: #EEFFFF\">    <\/span><span style=\"color: #C3E88D\">P8<\/span><span style=\"color: #EEFFFF\">             <\/span><span style=\"color: #F78C6C\">17<\/span><span style=\"color: #C3E88D\">W<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">\/<\/span><span style=\"color: #EEFFFF\">  <\/span><span style=\"color: #F78C6C\">285<\/span><span style=\"color: #C3E88D\">W<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">    <\/span><span style=\"color: #FFCB6B\">1032MiB<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">\/<\/span><span style=\"color: #EEFFFF\">  <\/span><span style=\"color: #F78C6C\">16376<\/span><span style=\"color: #C3E88D\">MiB<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">     <\/span><span style=\"color: #FFCB6B\">11%<\/span><span style=\"color: #EEFFFF\">      <\/span><span style=\"color: #C3E88D\">Default<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><\/span>\n<span class=\"line\"><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">                                         <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">                        <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">                  <\/span><span style=\"color: #FFCB6B\">N\/A<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><\/span>\n<span class=\"line\"><span style=\"color: #FFCB6B\">+-----------------------------------------+------------------------+----------------------+<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p>Expected:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Your GPU is listed<\/li>\n\n\n\n<li>Driver info is visible<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">4. (Optional) Install CUDA Toolkit<\/h2>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#EEFFFF;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:flex;align-items:center;padding:10px 0px 10px 16px;margin-bottom:-2px;width:100%;text-align:left;background-color:#304047;color:#d5ffff\">Bash<\/span><span role=\"button\" tabindex=\"0\" style=\"color:#EEFFFF;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>sudo apt install -y nvidia-cuda-toolkit<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki material-theme\" style=\"background-color: #263238\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #FFCB6B\">sudo<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">apt<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">install<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">-y<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">nvidia-cuda-toolkit<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p>Verify:<\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#EEFFFF;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:flex;align-items:center;padding:10px 0px 10px 16px;margin-bottom:-2px;width:100%;text-align:left;background-color:#304047;color:#d5ffff\">Bash<\/span><span role=\"button\" tabindex=\"0\" style=\"color:#EEFFFF;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>nvcc --version<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki material-theme\" style=\"background-color: #263238\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #FFCB6B\">nvcc<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">--version<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">5. Install Ollama<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">\u26a0\ufe0f Required Dependency for Ollama<\/h3>\n\n\n\n<p>Before installing Ollama, install <code>zstd<\/code> (this is required and often missing):<\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#EEFFFF;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:flex;align-items:center;padding:10px 0px 10px 16px;margin-bottom:-2px;width:100%;text-align:left;background-color:#304047;color:#d5ffff\">Bash<\/span><span role=\"button\" tabindex=\"0\" style=\"color:#EEFFFF;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>sudo apt-get install -y zstd<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki material-theme\" style=\"background-color: #263238\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #FFCB6B\">sudo<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">apt-get<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">install<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">-y<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">zstd<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<h3 class=\"wp-block-heading\">Download and Install Ollama<\/h3>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#EEFFFF;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:flex;align-items:center;padding:10px 0px 10px 16px;margin-bottom:-2px;width:100%;text-align:left;background-color:#304047;color:#d5ffff\">Bash<\/span><span role=\"button\" tabindex=\"0\" style=\"color:#EEFFFF;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>curl -fsSL https:\/\/ollama.com\/install.sh | sh<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki material-theme\" style=\"background-color: #263238\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #FFCB6B\">curl<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">-fsSL<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">https:\/\/ollama.com\/install.sh<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #FFCB6B\">sh<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">6. Verify Ollama Installation<\/h2>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#EEFFFF;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:flex;align-items:center;padding:10px 0px 10px 16px;margin-bottom:-2px;width:100%;text-align:left;background-color:#304047;color:#d5ffff\">Bash<\/span><span role=\"button\" tabindex=\"0\" style=\"color:#EEFFFF;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>ollama --version<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki material-theme\" style=\"background-color: #263238\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #FFCB6B\">ollama<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">--version<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">7. Run Your First Model<\/h2>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#EEFFFF;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:flex;align-items:center;padding:10px 0px 10px 16px;margin-bottom:-2px;width:100%;text-align:left;background-color:#304047;color:#d5ffff\">Bash<\/span><span role=\"button\" tabindex=\"0\" style=\"color:#EEFFFF;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>ollama run llama3<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki material-theme\" style=\"background-color: #263238\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #FFCB6B\">ollama<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">run<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">llama3<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">\ud83d\udd0d Verify GPU Usage<\/h2>\n\n\n\n<p>In a second terminal:<\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#EEFFFF;--cbp-line-number-width:calc(2 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:flex;align-items:center;padding:10px 0px 10px 16px;margin-bottom:-2px;width:100%;text-align:left;background-color:#304047;color:#d5ffff\">Bash<\/span><span role=\"button\" tabindex=\"0\" style=\"color:#EEFFFF;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>nvidia-smi\nWed Mar 18 16:08:13 2026\n+-----------------------------------------------------------------------------------------+\n| NVIDIA-SMI 595.54                 Driver Version: 595.79         CUDA Version: 13.2     |\n+-----------------------------------------+------------------------+----------------------+\n| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |\n| Fan  Temp   Perf          Pwr:Usage\/Cap |           Memory-Usage | GPU-Util  Compute M. |\n|                                         |                        |               MIG M. |\n|=========================================+========================+======================|\n|   0  NVIDIA GeForce RTX 4070 ...    On  |   00000000:01:00.0  On |                  N\/A |\n|  0%   58C    P2            240W \/  285W |    6331MiB \/  16376MiB |     91%      Default |\n|                                         |                        |                  N\/A |\n+-----------------------------------------+------------------------+----------------------+<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki material-theme\" style=\"background-color: #263238\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #FFCB6B\">nvidia-smi<\/span><\/span>\n<span class=\"line\"><span style=\"color: #FFCB6B\">Wed<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">Mar<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #F78C6C\">18<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #F78C6C\">16<\/span><span style=\"color: #C3E88D\">:08:13<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #F78C6C\">2026<\/span><\/span>\n<span class=\"line\"><span style=\"color: #FFCB6B\">+-----------------------------------------------------------------------------------------+<\/span><\/span>\n<span class=\"line\"><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #FFCB6B\">NVIDIA-SMI<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #F78C6C\">595.54<\/span><span style=\"color: #EEFFFF\">                 <\/span><span style=\"color: #C3E88D\">Driver<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">Version:<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #F78C6C\">595.79<\/span><span style=\"color: #EEFFFF\">         <\/span><span style=\"color: #C3E88D\">CUDA<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">Version:<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #F78C6C\">13.2<\/span><span style=\"color: #EEFFFF\">     <\/span><span style=\"color: #89DDFF\">|<\/span><\/span>\n<span class=\"line\"><span style=\"color: #FFCB6B\">+-----------------------------------------+------------------------+----------------------+<\/span><\/span>\n<span class=\"line\"><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #FFCB6B\">GPU<\/span><span style=\"color: #EEFFFF\">  <\/span><span style=\"color: #C3E88D\">Name<\/span><span style=\"color: #EEFFFF\">                 <\/span><span style=\"color: #C3E88D\">Persistence-M<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #FFCB6B\">Bus-Id<\/span><span style=\"color: #EEFFFF\">          <\/span><span style=\"color: #C3E88D\">Disp.A<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #FFCB6B\">Volatile<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">Uncorr.<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">ECC<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><\/span>\n<span class=\"line\"><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #FFCB6B\">Fan<\/span><span style=\"color: #EEFFFF\">  <\/span><span style=\"color: #C3E88D\">Temp<\/span><span style=\"color: #EEFFFF\">   <\/span><span style=\"color: #C3E88D\">Perf<\/span><span style=\"color: #EEFFFF\">          <\/span><span style=\"color: #C3E88D\">Pwr:Usage\/Cap<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">           <\/span><span style=\"color: #FFCB6B\">Memory-Usage<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #FFCB6B\">GPU-Util<\/span><span style=\"color: #EEFFFF\">  <\/span><span style=\"color: #C3E88D\">Compute<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">M.<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><\/span>\n<span class=\"line\"><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">                                         <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">                        <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">               <\/span><span style=\"color: #FFCB6B\">MIG<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">M.<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><\/span>\n<span class=\"line\"><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">=========================================+========================+======================<\/span><span style=\"color: #89DDFF\">|<\/span><\/span>\n<span class=\"line\"><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">   <\/span><span style=\"color: #FFCB6B\">0<\/span><span style=\"color: #EEFFFF\">  <\/span><span style=\"color: #C3E88D\">NVIDIA<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">GeForce<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">RTX<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #F78C6C\">4070<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">...<\/span><span style=\"color: #EEFFFF\">    <\/span><span style=\"color: #C3E88D\">On<\/span><span style=\"color: #EEFFFF\">  <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">   <\/span><span style=\"color: #FFCB6B\">00000000:01:00.0<\/span><span style=\"color: #EEFFFF\">  <\/span><span style=\"color: #C3E88D\">On<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">                  <\/span><span style=\"color: #FFCB6B\">N\/A<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><\/span>\n<span class=\"line\"><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">  <\/span><span style=\"color: #FFCB6B\">0%<\/span><span style=\"color: #EEFFFF\">   <\/span><span style=\"color: #F78C6C\">58<\/span><span style=\"color: #C3E88D\">C<\/span><span style=\"color: #EEFFFF\">    <\/span><span style=\"color: #C3E88D\">P2<\/span><span style=\"color: #EEFFFF\">            <\/span><span style=\"color: #F78C6C\">240<\/span><span style=\"color: #C3E88D\">W<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">\/<\/span><span style=\"color: #EEFFFF\">  <\/span><span style=\"color: #F78C6C\">285<\/span><span style=\"color: #C3E88D\">W<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">    <\/span><span style=\"color: #FFCB6B\">6331MiB<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">\/<\/span><span style=\"color: #EEFFFF\">  <\/span><span style=\"color: #F78C6C\">16376<\/span><span style=\"color: #C3E88D\">MiB<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">     <\/span><span style=\"color: #FFCB6B\">91%<\/span><span style=\"color: #EEFFFF\">      <\/span><span style=\"color: #C3E88D\">Default<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><\/span>\n<span class=\"line\"><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">                                         <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">                        <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\">                  <\/span><span style=\"color: #FFCB6B\">N\/A<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><\/span>\n<span class=\"line\"><span style=\"color: #FFCB6B\">+-----------------------------------------+------------------------+----------------------+<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p>You should see:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>A running <code>ollama<\/code> process<\/li>\n\n\n\n<li>Increasing GPU usage (here 91%)<\/li>\n\n\n\n<li>Increasing GPU memory usage (here 6GB compared to 1GB in the screnshot, when we called nvidia-smi for the first time)<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">\u2699\ufe0f Troubleshooting<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Ollama uses CPU instead of GPU<\/h3>\n\n\n\n<p>Try:<\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#EEFFFF;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:flex;align-items:center;padding:10px 0px 10px 16px;margin-bottom:-2px;width:100%;text-align:left;background-color:#304047;color:#d5ffff\">Bash<\/span><span role=\"button\" tabindex=\"0\" style=\"color:#EEFFFF;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>export OLLAMA_USE_GPU=1<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki material-theme\" style=\"background-color: #263238\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #C792EA\">export<\/span><span style=\"color: #EEFFFF\"> OLLAMA_USE_GPU<\/span><span style=\"color: #89DDFF\">=<\/span><span style=\"color: #F78C6C\">1<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\">Model too large<\/h3>\n\n\n\n<p>Test with a smaller model:<\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#EEFFFF;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:flex;align-items:center;padding:10px 0px 10px 16px;margin-bottom:-2px;width:100%;text-align:left;background-color:#304047;color:#d5ffff\">Bash<\/span><span role=\"button\" tabindex=\"0\" style=\"color:#EEFFFF;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>ollama run phi<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki material-theme\" style=\"background-color: #263238\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #FFCB6B\">ollama<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">run<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">phi<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\">GPU not visible in WSL<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Check Windows driver again<\/li>\n\n\n\n<li>Ensure WSL2 is used (<code>wsl -l -v<\/code>)<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">\ud83e\uddea Test Ollama API<\/h2>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#EEFFFF;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:flex;align-items:center;padding:10px 0px 10px 16px;margin-bottom:-2px;width:100%;text-align:left;background-color:#304047;color:#d5ffff\">Bash<\/span><span role=\"button\" tabindex=\"0\" style=\"color:#EEFFFF;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>curl http:\/\/localhost:11434\/api\/generate -d '{\n  \"model\": \"llama3\",\n  \"prompt\": \"Hello\"\n}'<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki material-theme\" style=\"background-color: #263238\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #FFCB6B\">curl<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">http:\/\/localhost:11434\/api\/generate<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">-d<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">&#39;<\/span><span style=\"color: #C3E88D\">{<\/span><\/span>\n<span class=\"line\"><span style=\"color: #C3E88D\">  &quot;model&quot;: &quot;llama3&quot;,<\/span><\/span>\n<span class=\"line\"><span style=\"color: #C3E88D\">  &quot;prompt&quot;: &quot;Hello&quot;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #C3E88D\">}<\/span><span style=\"color: #89DDFF\">&#39;<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<p>Get all running models:<\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#EEFFFF;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:flex;align-items:center;padding:10px 0px 10px 16px;margin-bottom:-2px;width:100%;text-align:left;background-color:#304047;color:#d5ffff\">Bash<\/span><span role=\"button\" tabindex=\"0\" style=\"color:#EEFFFF;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>curl --silent http:\/\/localhost:11434\/api\/ps | python3 -m json.tool<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki material-theme\" style=\"background-color: #263238\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #FFCB6B\">curl<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">--silent<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">http:\/\/localhost:11434\/api\/ps<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #89DDFF\">|<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #FFCB6B\">python3<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">-m<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">json.tool<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"922\" height=\"399\" src=\"https:\/\/techno.slomka.biz\/wp-content\/uploads\/2026\/03\/grafik.png\" alt=\"\" class=\"wp-image-1122\" srcset=\"https:\/\/techno.slomka.biz\/wp-content\/uploads\/2026\/03\/grafik.png 922w, https:\/\/techno.slomka.biz\/wp-content\/uploads\/2026\/03\/grafik-300x130.png 300w, https:\/\/techno.slomka.biz\/wp-content\/uploads\/2026\/03\/grafik-768x332.png 768w\" sizes=\"auto, (max-width: 922px) 100vw, 922px\" \/><\/figure>\n\n\n\n<p>You can access ollama from the windows system, where your wsl runs by using the api. Ollama in wsl automaticall binds to 0.0.0.0. <br><a href=\"http:\/\/localhost:11434\/api\/ps\">http:\/\/localhost:11434\/api\/ps<\/a><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"622\" height=\"445\" src=\"https:\/\/techno.slomka.biz\/wp-content\/uploads\/2026\/03\/grafik-1.png\" alt=\"\" class=\"wp-image-1125\" srcset=\"https:\/\/techno.slomka.biz\/wp-content\/uploads\/2026\/03\/grafik-1.png 622w, https:\/\/techno.slomka.biz\/wp-content\/uploads\/2026\/03\/grafik-1-300x215.png 300w\" sizes=\"auto, (max-width: 622px) 100vw, 622px\" \/><\/figure>\n\n\n\n<p>If this does not work, try to bind ollama to all interfaces:<\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#EEFFFF;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:flex;align-items:center;padding:10px 0px 10px 16px;margin-bottom:-2px;width:100%;text-align:left;background-color:#304047;color:#d5ffff\">Bash<\/span><span role=\"button\" tabindex=\"0\" style=\"color:#EEFFFF;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><pre class=\"code-block-pro-copy-button-pre\" aria-hidden=\"true\"><textarea class=\"code-block-pro-copy-button-textarea\" tabindex=\"-1\" aria-hidden=\"true\" readonly>export OLLAMA_HOST=0.0.0.0\nollama serve<\/textarea><\/pre><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki material-theme\" style=\"background-color: #263238\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #C792EA\">export<\/span><span style=\"color: #EEFFFF\"> OLLAMA_HOST<\/span><span style=\"color: #89DDFF\">=<\/span><span style=\"color: #F78C6C\">0.0<\/span><span style=\"color: #C3E88D\">.0.0<\/span><\/span>\n<span class=\"line\"><span style=\"color: #FFCB6B\">ollama<\/span><span style=\"color: #EEFFFF\"> <\/span><span style=\"color: #C3E88D\">serve<\/span><\/span><\/code><\/pre><\/div>\n","protected":false},"excerpt":{"rendered":"<p>Running large language models locally with GPU acceleration inside WSL2 is not only possible\u2014it\u2019s surprisingly efficient once properly configured. This guide walks through a working setup using Ubuntu, NVIDIA GPU passthrough, and Ollama. \ud83e\udde9 Target Setup 1. Prepare Windows Host Check Windows Version Ensure you&#8217;re on a supported version: Recommended: Enable WSL2 Install NVIDIA Driver (with WSL Support) on your Windows machine Install a current NVIDIA driver that supports WSL CUDA. Verify: If this fails, stop here\u2014GPU passthrough will not work. 2. Prepare Ubuntu (WSL) Start WSL: Update packages: 3. Verify GPU inside WSL Expected: 4. (Optional) Install CUDA Toolkit Verify: 5. Install Ollama \u26a0\ufe0f Required Dependency for Ollama Before installing Ollama, install zstd (this is required and often missing): Download and Install Ollama 6. Verify Ollama Installation 7. Run Your First Model \ud83d\udd0d Verify GPU Usage In a second terminal: You should see: \u2699\ufe0f Troubleshooting Ollama uses CPU instead of GPU Try: Model too large Test with a smaller model: GPU not visible in WSL \ud83e\uddea Test Ollama API Get all running models: You can access ollama from the windows system, where your wsl runs by using the api. Ollama in wsl automaticall binds to 0.0.0.0. http:\/\/localhost:11434\/api\/ps If this does not work, try to bind ollama to all interfaces:<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[141,144,145,143,142],"tags":[147,148,152,149],"class_list":["post-1121","post","type-post","status-publish","format-standard","hentry","category-ai-automation","category-n8n","category-nvidia-cuda","category-ollama","category-wsl","tag-ai","tag-n8n","tag-ollama","tag-wsl"],"_links":{"self":[{"href":"https:\/\/techno.slomka.biz\/index.php?rest_route=\/wp\/v2\/posts\/1121","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/techno.slomka.biz\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/techno.slomka.biz\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/techno.slomka.biz\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/techno.slomka.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1121"}],"version-history":[{"count":6,"href":"https:\/\/techno.slomka.biz\/index.php?rest_route=\/wp\/v2\/posts\/1121\/revisions"}],"predecessor-version":[{"id":1188,"href":"https:\/\/techno.slomka.biz\/index.php?rest_route=\/wp\/v2\/posts\/1121\/revisions\/1188"}],"wp:attachment":[{"href":"https:\/\/techno.slomka.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1121"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/techno.slomka.biz\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1121"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/techno.slomka.biz\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1121"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}