I'm working on a free-time project that involves turning a Raspberry Pi 5 into a portable, self-contained LLM.
This project is already quite complicated:
* Raspberry Pi 5 8GB
* Adafruit PiTFT Plus 320x240 capacitive-touch LCD
* Microphone (for receiving spoken queries; mounted on front next to LCD)
* SABRENT USB sound dongle connected to microphone and sound output
* Cable Matters ground-loop isolator (to eliminate hum on the main sound output)
* Adafruit MAX9744 amplifier connected to two 3W speakers
* Adafruit I2C-based MiniGPS breakout board
* Adafruit metal RGB LED button (mounted on front next to microphone)
* Pimoroni I/O Expander (to drive colors to the LED button)
* Metal LED power button
* I'm considering adding the Sony AI camera (https://www.raspberrypi.com/products/ai-camera/) for onboard inference.
The actual LLM pipeline is working really well. When the device is ready for input, the LED button is green. When the user presses and holds the button, the LED button turns blue and the RPi records the user's voice. When the user releases the button, the lED button turns red, the RPi transcribes voice (via whisper), processes it (via an ollama model), and speaks the response (via espeak).
The persistently tricky part is power. I want this to be mobile, I want the RPI to use multiple cores, and I want the MAX9744 amplifier to play sound at a decent volume.
I measured the peak voltage of the RPi, during inference, as about 10 watts. The amplifier isn't running anywhere near 20 watts.
I tried using a 30W power bank (https://www.amazon.com/gp/product/B0C14 ... le_o00_s00). Unfortunately, it wasn't viable: the RPi repeatedly turned off at the start of the second inference attempt. I thought that it might be a memory issue, but when I swapped out the power bank and ran the RPi off of wall power with a 5V / 3.5-amp Argon charger, it was completely stable.
Next, I tried the Anker 737 140-watt power bank (https://www.anker.com/products/a1289?va ... 4285041814), without the amplifier or sound. The RPi ran stably for about 15 minutes, and the Anker reported delivering about 10 watts at peak usage. To my surprise, the RPi occasionally complained of undervoltage, and eventually it crashed. I don't know why a 140-watt power bank is incapable of supplying a clean 5V power at the peak of its battery charge, but that appears to be the case.
I'm now trying the Anker 250-watt power bank (https://www.anker.com/products/a1340-250w-power-bank), which is absolutely overkill for a 10-watt RPi and a 10-watt amplifier. But honestly, I'm just tired of dealing with the issue and figured I'd go big or go home. If that doesn't work and I have some patience left, I will look into power regulation devices.
The other observation is that after running inference for quite a while, vcgencmd reported temps of 85C and peak throttling. The device definitely needs some better ventilation, so I'm buying a couple of 5V Noctua fans and hope to run them off of the I/O expander. It's possible that the last shutdown, while using the Anker power bank, was due to CPU temperature rather than power draw - but we'll see.
If anyone has any ideas I haven't considered yet, I'd be grateful for your suggestions.
This project is already quite complicated:
* Raspberry Pi 5 8GB
* Adafruit PiTFT Plus 320x240 capacitive-touch LCD
* Microphone (for receiving spoken queries; mounted on front next to LCD)
* SABRENT USB sound dongle connected to microphone and sound output
* Cable Matters ground-loop isolator (to eliminate hum on the main sound output)
* Adafruit MAX9744 amplifier connected to two 3W speakers
* Adafruit I2C-based MiniGPS breakout board
* Adafruit metal RGB LED button (mounted on front next to microphone)
* Pimoroni I/O Expander (to drive colors to the LED button)
* Metal LED power button
* I'm considering adding the Sony AI camera (https://www.raspberrypi.com/products/ai-camera/) for onboard inference.
The actual LLM pipeline is working really well. When the device is ready for input, the LED button is green. When the user presses and holds the button, the LED button turns blue and the RPi records the user's voice. When the user releases the button, the lED button turns red, the RPi transcribes voice (via whisper), processes it (via an ollama model), and speaks the response (via espeak).
The persistently tricky part is power. I want this to be mobile, I want the RPI to use multiple cores, and I want the MAX9744 amplifier to play sound at a decent volume.
I measured the peak voltage of the RPi, during inference, as about 10 watts. The amplifier isn't running anywhere near 20 watts.
I tried using a 30W power bank (https://www.amazon.com/gp/product/B0C14 ... le_o00_s00). Unfortunately, it wasn't viable: the RPi repeatedly turned off at the start of the second inference attempt. I thought that it might be a memory issue, but when I swapped out the power bank and ran the RPi off of wall power with a 5V / 3.5-amp Argon charger, it was completely stable.
Next, I tried the Anker 737 140-watt power bank (https://www.anker.com/products/a1289?va ... 4285041814), without the amplifier or sound. The RPi ran stably for about 15 minutes, and the Anker reported delivering about 10 watts at peak usage. To my surprise, the RPi occasionally complained of undervoltage, and eventually it crashed. I don't know why a 140-watt power bank is incapable of supplying a clean 5V power at the peak of its battery charge, but that appears to be the case.
I'm now trying the Anker 250-watt power bank (https://www.anker.com/products/a1340-250w-power-bank), which is absolutely overkill for a 10-watt RPi and a 10-watt amplifier. But honestly, I'm just tired of dealing with the issue and figured I'd go big or go home. If that doesn't work and I have some patience left, I will look into power regulation devices.
The other observation is that after running inference for quite a while, vcgencmd reported temps of 85C and peak throttling. The device definitely needs some better ventilation, so I'm buying a couple of 5V Noctua fans and hope to run them off of the I/O expander. It's possible that the last shutdown, while using the Anker power bank, was due to CPU temperature rather than power draw - but we'll see.
If anyone has any ideas I haven't considered yet, I'd be grateful for your suggestions.
Statistics: Posted by sfsdfd — Sat Nov 30, 2024 11:10 pm