Stuck with new computer (RasPi) getting an LLM installed.

[shykitten55]

I received the parts, assembled them - as best I could - and it works.

But following the steps (many ones tried) and they all fail at one point.

Would/could some one be able to help me if I bring the machine?

I have set it up for headless so don't have a keyboard/mouse with it.

[Lewys Martin]

I'm going to need a bit more information if you want me to help you, what raspberry pi is it, how much ram does it have? Do you have any add-on boards for it and which llms are you trying to run?

[shykitten55]

RasPi 5 8GB ram.

Pi AI Hat2+

This is the link I am using to get it up and working.

https://toucancreator.com/learn/wagner-s-techtalk/tutorial/ai-hat-2-quick-setup-on-a-raspberry-pi-5

Though I have tried others.   I'll get back to that shortly.

So I get t this part:

cd Downloads
sudo dpkg -i hailo_gen_ai_model_zoo_5.1.1_arm64.deb

And it fails.

Directory doesn't exist.

The other links are slightly different in their stuff, but get to the same place and fail in the same way.

Given it is used by a lot of people have used it, I don't understand why/how it is failing when I try it.

Does that help?

[Lewys Martin]

 CAn you run an "ls" in that folder ? Confirm if the file is there 

Regards,

 

Lewys Martin | VK2EVA

https://lewys.eu/ https://vk2eva.com/

--
You received this message because you are subscribed to a topic in the Google Groups "Robots & Dinosaurs" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sydney-hackspace/nw4kBFxwcQU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sydney-hackspa...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/sydney-hackspace/2d3dbc9f-6d7c-4b29-9ee6-156438dc801bn%40googlegroups.com.

[Lewys Martin]

Can you run an "ls" inside that folder to confirm if the file is there ?

[shykitten55]

(Sorry, am trying to rebuild things to make sure there were no cross contaminations of what was installed.)

This is a walk through if me following the commands.

dog@AI-Pi:~ $ hailo-ollama
terminate called after throwing an instance of 'std::runtime_error'
  what():  hailo-ollama directory not found
Aborted
dog@AI-Pi:~ $

So for full disclosure here is the entire flow of commands:

Code: Select all

dog@AI-Pi:~ $ sudo apt update
[sudo] password for dog:
Hit:1 http://deb.debian.org/debian trixie InRelease
Get:2 http://deb.debian.org/debian trixie-updates InRelease [47.3 kB]
Get:3 http://deb.debian.org/debian-security trixie-security InRelease [43.4 kB]              
Get:4 http://deb.debian.org/debian-security trixie-security/main armhf Packages [205 kB]
Get:5 http://archive.raspberrypi.com/debian trixie InRelease [54.8 kB]                    
Get:6 http://deb.debian.org/debian-security trixie-security/main arm64 Packages [224 kB]
Get:7 http://deb.debian.org/debian-security trixie-security/main Translation-en [134 kB]  
Get:8 http://archive.raspberrypi.com/debian trixie/main arm64 Packages [463 kB]                
Fetched 1,171 kB in 7s (178 kB/s)                                                                                                        
All packages are up to date.    
dog@AI-Pi:~ $ sudo apt full-upgrade -y
Summary:                        
  Upgrading: 0, Installing: 0, Removing: 0, Not Upgrading: 0
  1 not fully installed or removed.
  Space needed: 0 B / 49.6 GB available

Setting up hailo-gen-ai-model-zoo (5.1.1) ...
chmod: cannot access '/usr/share/hailo-ollama/models': No such file or directory
dpkg: error processing package hailo-gen-ai-model-zoo (--configure):
 installed hailo-gen-ai-model-zoo package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 hailo-gen-ai-model-zoo
Error: Sub-process /usr/bin/dpkg returned an error code (1)
dog@AI-Pi:~ $

And that is weird.
That didn't happen last time. (Last, the first time)

Ok. So this is where I am now at.

Following on with your requests:

Code: Select all

dog@AI-Pi:~ $ ls -l /usr/share
total 732
drwxr-xr-x    2 root   root    4096 Jul  2 08:36 aclocal
drwxr-xr-x    9 root   root    4096 Jun 18 10:20 alsa
drwxr-xr-x    2 root   root    4096 Jun 18 10:20 apparmor-features
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 applications
drwxr-xr-x    3 root   root    4096 Jun 18 10:20 apport
drwxr-xr-x    2 root   root    4096 Jun 18 10:18 apt
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 avahi
drwxr-xr-x    2 root   root    4096 Jun 18 10:18 base-files
drwxr-xr-x    2 root   root    4096 Jun 18 10:18 base-passwd
drwxr-xr-x    4 root   root    4096 Jun 18 10:21 bash-completion
drwxr-xr-x    2 root   root    4096 Jun 18 10:20 binfmts
drwxr-xr-x   59 root   root    4096 Jul  2 09:17 bug
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 build-essential
drwxr-xr-x    3 root   root    4096 Jun 18 10:19 ca-certificates
drwxr-xr-x    5 root   root    4096 Jul  2 09:06 cmake
drwxr-xr-x    6 root   root    4096 Jul  2 07:32 cmake-3.31
drwxr-xr-x    3 root   root    4096 Jul  2 08:00 color
drwxr-xr-x    2 root   root    4096 Jun 18 10:18 common-licenses
drwxr-xr-x    2 root   root   36864 Jun 18 10:21 consolefonts
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 console-setup
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 consoletrans
drwxr-xr-x    7 root   root    4096 Jul  2 07:30 dbus-1
drwxr-xr-x    2 root   root    4096 Jun 18 10:18 debconf
drwxr-xr-x    3 root   root    4096 Jun 18 10:21 debhelper
drwxr-xr-x    3 root   root    4096 Jun 18 10:18 debianutils
drwxr-xr-x    3 root   root    4096 Jul  2 07:30 defaults
drwxr-xr-x    3 root   root    4096 Jun 18 10:19 dhcpcd
drwxr-xr-x    2 root   root    4096 May  9 02:10 dict
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 distro-info
drwxr-xr-x    4 root   root    4096 Jul  2 09:10 djvu
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 dns
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 dnsmasq-base
drwxr-xr-x 1254 root   root   40960 Jul  2 10:34 doc
drwxr-xr-x    2 root   root    4096 Jul  2 09:10 doc-base
drwxr-xr-x    3 root   root    4096 Jun 18 10:21 dpkg
drwxr-xr-x    2 root   root    4096 Jul  2 09:15 drirc.d
drwxr-xr-x    3 root   root    4096 Jun 18 10:20 dwarves
drwxr-xr-x    2 root   root    4096 Jul  2 07:33 et
drwxr-xr-x    2 root   root    4096 Jul  2 07:59 ffmpeg
drwxr-xr-x    3 root   root    4096 Jun 18 10:21 file
drwxr-xr-x    5 root   root    4096 Jun 18 10:21 fish
drwxr-xr-x    3 root   root    4096 Jul  2 07:23 fontconfig
drwxr-xr-x    8 root   root    4096 Jul  2 07:26 fonts
drwxr-xr-x    3 root   root    4096 Jul  2 07:59 fonts-droid-fallback
drwxr-xr-x    3 root   root    4096 Mar 15  2025 gcc
drwxr-xr-x    3 root   root    4096 Jul  2 07:30 GConf
drwxr-xr-x    2 root   root   12288 Jul  2 07:59 gdal
drwxr-xr-x    6 root   root    4096 Jun 18 10:21 gdb
drwxr-xr-x    3 root   root    4096 Jul  2 08:46 gdcm-3.0
drwxr-xr-x    3 root   root    4096 Jun 18 10:20 gettext
drwxr-xr-x    4 root   root    4096 Jul  2 08:00 ghostscript
drwxr-xr-x    2 root   root    4096 Jul  2 09:05 gir-1.0
drwxr-xr-x    4 root   root    4096 Jul  2 08:09 git-core
drwxr-xr-x    3 root   root    4096 Jul  2 08:09 gitweb
drwxr-xr-x    8 root   root    4096 Jul  2 08:14 glib-2.0
drwxr-xr-x    3 root   root    4096 Jul  2 08:04 glvnd
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 gnupg
drwxr-xr-x    3 root   root    4096 Jul  2 08:12 gobject-introspection-1.0
drwxr-xr-x    3 root   root    4096 Jun 18 10:21 groff
drwxr-xr-x    3 root   root    4096 Jul  2 08:05 gst-plugins-base
drwxr-xr-x    5 root   root    4096 Jul  2 08:35 gstreamer-1.0
drwxr-xr-x    3 root   root    4096 Jul  2 08:19 gtk-3.0
drwxr-xr-x    3 root   root    4096 Jul  2 08:38 gtk-doc
drwxr-xr-x    3 root   root    4096 Jul  2 22:43 hailo-models
drwxr-xr-x    4 root   root    4096 Jun 18 10:19 i18n
drwxr-xr-x    6 root   root    4096 Jul  2 08:26 icons
drwxr-xr-x    2 root   root    4096 Jul  2 07:25 ImageMagick-7
drwxr-xr-x    2 root   root    4096 Jul  2 08:12 info
drwxr-xr-x    7 root   root    4096 Jun 18 10:20 initramfs-tools
drwxr-xr-x    2 root   root    4096 Jun 18 10:19 iproute2
drwxr-xr-x    3 root   root    4096 Jun 18 10:21 iso-codes
drwxr-xr-x    3 root   root    4096 Jul  2 09:12 java
drwxr-xr-x    5 root   root    4096 Jul  2 08:31 javascript
drwxr-xr-x    2 root   root    4096 Jun 18 10:19 keyrings
drwxr-xr-x    3 root   root    4096 Jul  2 08:22 ladspa
drwxr-xr-x    4 root   root    4096 Jun 18 10:21 libcamera
drwxr-xr-x    2 root   root    4096 Jun 18 10:19 libc-bin
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 libdebuginfod-common
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 libdrm
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 libgcrypt20
drwxr-xr-x    2 root   root    4096 Jul  2 08:42 libinput
drwxr-xr-x    2 root   root    4096 Jul  2 07:47 libmysofa
drwxr-xr-x    2 root   root    4096 Jul  2 07:10 libpisp
drwxr-xr-x    2 root   root    4096 Jul  2 07:38 libthai
drwxr-xr-x    3 root   root   40960 Jul  2 08:42 libwacom
drwxr-xr-x    4 root   root    4096 Jun 18 10:18 lintian
drwxr-xr-x  201 root   root    4096 Jul  2 08:20 locale
drwxr-xr-x    3 root   root    4096 Jun 18 10:21 luajit-2.1
drwxr-xr-x   41 root   root    4096 Jun 18 10:21 man
drwxr-xr-x    3 root   root    4096 Jun 18 10:21 maven-repo
drwxr-xr-x    2 root   root    4096 Jun 18 10:18 menu
drwxr-xr-x    3 root   root    4096 Jul  2 08:14 meson
drwxr-xr-x    2 root   root    4096 Jul  2 08:42 metainfo
drwxr-xr-x   16 root   root    4096 Jun 18 10:20 mime
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 misc
drwxr-xr-x    2 root   root    4096 Jul  2 08:49 mysql-common
drwxr-xr-x    4 root   root    4096 Jun 18 10:19 nano
drwxr-xr-x    3 root   root    4096 Jun 18 10:21 netplan
drwxr-xr-x    2 root   root    4096 Jul  2 07:26 numpy3
drwx------    3 ollama ollama  4096 Jul  2 19:18 ollama
drwxr-xr-x    4 root   root    4096 Jul  2 07:56 openal
drwxr-xr-x    5 root   root    4096 Jul  2 09:15 opencv4
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 openssh
drwxr-xr-x    2 root   root    4096 Jun 18 10:18 pam
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 pam-configs
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 pastebin.d
drwxr-xr-x    3 root   root    4096 Jun 18 10:21 perl
drwxr-xr-x    9 root   root    4096 Jul  2 08:09 perl5
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 pixmaps
drwxr-xr-x    3 root   root    4096 Jul  2 09:06 pkgconfig
drwxr-xr-x    3 root   root    4096 Jul  2 09:15 pocketsphinx
drwxr-xr-x    4 root   root    4096 Jun 18 10:21 polkit-1
drwxr-xr-x    6 root   root    4096 Jul  2 07:26 poppler
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 ppp
drwxr-xr-x    2 root   root    4096 Jul  2 08:48 proj
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 publicsuffix
drwxr-xr-x    7 root   root    4096 Jun 18 10:21 python3
drwxr-xr-x    3 root   root    4096 Jun 18 10:21 python-apt
drwxr-xr-x    3 root   root    4096 Jun 18 10:22 python-babel-localedata
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 python-wheels
drwxr-xr-x    3 root   root    4096 Jul  2 09:16 qt5
drwxr-xr-x    2 root   root    4096 Jun 18 10:20 raspi-config
drwxr-xr-x    2 root   root    4096 Jul  2 07:10 raspi-utils
drwxr-xr-x    2 root   root    4096 Jun 18 10:19 readline
drwxr-xr-x    2 root   root    4096 Jul  2 09:09 rpi-camera-assets
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 rpi-keyboard-config
drwxr-xr-x    3 root   root    4096 Jun 18 10:20 rsync
drwxr-xr-x    3 root   root    4096 Jun 18 10:21 runit
drwxr-xr-x    8 root   root    4096 Jul  2 08:32 sgml
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 sgml-base
drwxr-xr-x    5 root   root    4096 Jul  2 09:21 sounds
drwxr-xr-x    2 root   root   12288 Jun 18 10:21 source-highlight
drwxr-xr-x    2 root   root    4096 Jun 18 10:19 systemd
drwxr-xr-x    2 root   root    4096 Jun 18 10:18 tabset
drwxr-xr-x    4 root   root    4096 Jul  2 09:03 tcltk
drwxr-xr-x   44 root   root    4096 Jun 18 10:21 terminfo
drwxr-xr-x    3 root   root    4096 Jul  2 08:56 tesseract-ocr
drwxr-xr-x    4 root   root    4096 Jul  2 08:21 themes
drwxr-xr-x    2 root   root    4096 Jul  2 09:14 thumbnailers
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 ucf
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 usb_modeswitch
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 userconf-pi
drwxr-xr-x    3 root   root    4096 May 10  2025 util-linux
drwxr-xr-x    6 root   root    4096 Jul  2 07:32 vim
drwxr-xr-x    5 root   root    4096 Jul  2 09:14 vulkan
drwxr-xr-x    2 root   root    4096 Jul  2 08:37 wayland
drwxr-xr-x    4 root   root    4096 Jul  2 07:22 X11
drwxr-xr-x    9 root   root    4096 Jul  2 07:23 xml
drwxr-xr-x    2 root   root    4096 Jun 18 10:21 xml-core
drwxr-xr-x   13 root   root    4096 Jun 18 10:19 zoneinfo
drwxr-xr-x    4 root   root    4096 Jun 18 10:19 zsh
dog@AI-Pi:~ $

The "hailo-models" has drwwr-xr-x permissions.
But the problematic path is "hailo-ollama/models"
Just saying.

Going back to check with rpdom's post.

Code: Select all

dog@AI-Pi:~ $ ls /usr/share/hailo-ollama/models
ls: cannot access '/usr/share/hailo-ollama/models': No such file or directory
dog@AI-Pi:~ $ sudo mkdir /usr/share/hailo-ollama
dog@AI-Pi:~ $ ls /usr/share/hailo-ollama/
dog@AI-Pi:~ $ sudo ln -s /usr/share/hailo-models /usr/share/hailo-ollama/models
dog@AI-Pi:~ $ ls /usr/share/hailo-ollama/models
manifests             yolov5n_seg_h10.hef     yolov5s_personface_h8l.hef  yolov8s_h8.hef           yolox_s_leaky_h8l_rpi.hef
resnet_v1_50_h10.hef  yolov5n_seg_h8.hef      yolov6n_h8.hef              yolov8s_h8l.hef
resnet_v1_50_h8l.hef  yolov5n_seg_h8l_mz.hef  yolov6n_h8l.hef             yolov8s_pose_h10.hef
scrfd_2.5g_h8l.hef    yolov5_personface.json  yolov8m_h10.hef             yolov8s_pose_h8.hef
yolov11m_h10.hef      yolov5seg.json          yolov8m_pose_h10.hef        yolov8s_pose_h8l_pi.hef
dog@AI-Pi:~ $

So it does look kind of better/good.

Only because I'm good at shooting myself in th foot......
No. I'd better stop here. I don't want to make a bad situation worse.

So now the link seems to be working.
Though why wouldn't it have worked if I pasted the two commands together before?
(Semi rhetorical)

So, is that helpful?

[shykitten55]

I have completed the hailo part instal.

This is where I'm at:

 hailortcli fw-control identify
Executing on device: 0001:01:00.0
Identifying board
Control Protocol Version: 2
Firmware Version: 5.1.1 (release,app)
Logger Version: 0
Device Architecture: HAILO10H

So that seems to be there.

Though the SOMETHING LIKE on the page is worrying.

This is what the web page shows.

Executing on device: 0000:01:00.0
Identifying board
Control Protocol Version: 2
Firmware Version: 4.17.0 (release,app,extended context switch buffer)
Logger Version: 0
Board Name: Hailo-8
Device Architecture: HAILO8L
Serial Number: HLDDLBB234500054
Part Number: HM21LB1C2LAE
Product Name: HAILO-8L AI ACC M.2 B+M KEY MODULE EXT TMP

Going to the next step/s:

dog@AI-Pi:~ $ cd Downloads
sudo dpkg -i hailo_gen_ai_model_zoo_5.1.1_arm64.deb
[sudo] password for dog:
Selecting previously unselected package hailo-gen-ai-model-zoo.
(Reading database ... 95601 files and directories currently installed.)
Preparing to unpack hailo_gen_ai_model_zoo_5.1.1_arm64.deb ...
Unpacking hailo-gen-ai-model-zoo (5.1.1) ...

Setting up hailo-gen-ai-model-zoo (5.1.1) ...

dog@AI-Pi:~/Downloads $ hailo-ollama
 I |2026-07-03 22:03:28 1783080208514619| MyApp:Server running on port 8000

Now I'm further than I've been before I think.

But I'm now stuck what to do next.

[shykitten55]

Well,

It has decided to work.

Now I'm stuck with which LLM to install.

I play with / use Javascript a lot.

Suggestions?

[shykitten55]

This is where I'm at now.

I'm logged in, but t doesn't seem to like it all.

[shykitten55]

Very slow progress is being made.

Partly because the "example" from whiere I am getting my details is for use on the same machine.   Not making it a network accessible one.

Anyway this is the latest error/problem:

curl http://localhost:8000/api/generate -d '{
  "model": "qwen2.5-coder:1.5b",
  "prompt": "hello",
  "stream": false
}'
server=oatpp/1.4.0
code=500
description=Internal Server Error
stacktrace:
  - No suitable mapper found to deserialize the request body.
  - [ApiController]: Error processing request
  - Error processing request

I am guessing this isn't helping things work smoothly.