Readit News logoReadit News
haunter · 2 years ago
I started buying Lenovo mini PCs instead, 18cm x 18cm x 3cm so it's still really small.

And you can get them dirt cheap nowadays, has proper casing and cooling etc https://psref.lenovo.com/syspool/Sys/PDF/ThinkCentre/ThinkCe...

I have one right next to me, i5-8500T, 32GB RAM, 2x SSDs and currently 5W at idle with powertop auto-tune https://wiki.archlinux.org/title/powertop

atoav · 2 years ago
No GPIOs, no I2C, no SPI. If you are purely in search for a small light seever, this is a suitable choice, but Raspis are popular also for the more unusual (for consumer/office devices) IO.

Good you could add a serial comnection to a microcontroller etc, but then the solution won't be as elegant.

weugek · 2 years ago
Is it elegant is use GPIO on an rpi?
squarefoot · 2 years ago
Last year I've migrated my Kodi media player from a RasPi 4 to a N3350 mini PC and didn't look back, then recently also moved my NAS (N5105) and services (3215U) machine to mini PCs and unlocked Chromebooks I got used on either Ebay or flea markets. They're cheap and the computing power is on another planet compared to the PI while still maintaining a low power consumption.
firecall · 2 years ago
I do the same, but with Dell 7060s.

There are thousands of them coming off lease for sale here in Australia.

8500T or 8700T chips are fantastic. They can officially run Windows 11 and have have hardware transcoding built In, so are great for a Plex server!

I’ve upgraded one of mine with 2x 2GB SSD and 64GB RAM.

Never any issues and it runs 247.

mths · 2 years ago
> There are thousands of them coming off lease for sale here in Australia.

Where would I find one and what do they go for?

As a Plex server, do you know if they can handle hardware transcoding?

Thanks!

thedougd · 2 years ago
Same. Bought a Dell refurb and I’ve had so many fewer hassles. The SD card was perhaps the biggest source of issues and after factoring the cost of outfitting a RPi with something like an SSD, it just wasn’t cost effective anymore.
mbirth · 2 years ago
This I never understood. I have 3 different RPis running since they were available to buy. A 2, a 3 and a 4. All with the OS on microSD and no optimisation like writing log files to tmpfs or similar. It’s plain Ubuntu Server for ARM.

And in all these years, I had maybe one microSD going read-only on me. Cloned that to a fresh one, did an fsck and the Pi was up and running again. That’s it. No other issues despite various sudden power losses etc.

I don’t understand where these rumours about bad reliability of SD cards in Pis come from.

robotnikman · 2 years ago
The fact that you can run just about any linux distro is also nice when compared to ARM hardware.
heavyset_go · 2 years ago
Yup, this is the reason I refuse to buy anymore ARM SBCs.

You're always dependent on someone building a bespoke Linux image for your particular device into perpetuity, which rarely happens.

cschneid · 2 years ago
Similarly, I saw some youtube videos just the other day talking about intel's (newish?) N100 chip. You can pick up a small mini pc for under $150 new, runs windows or linux, and is on-par speed-wise with the 8500 iirc. Plenty fast for small server use cases, although you do give up the raw GPIO pins.

Also of note, it's x86, which has me interested as a field-portable ham radio computer. Most software runs on rpi (my current setup), but x86 is universally supported by stuff. Just need to figure out the monitor situation.... maybe vnc from an ipad? :)

bee_rider · 2 years ago
I did a portable NUC for a while, just powered off some no-name li-ion backup battery (a slightly bigger one because you need the 12-20v out).

There are plenty of decent portable monitors out there. I ended up with an MSI one, but I think any laptop brand can make a monitor (I also had some no-name one which had a nicer resolution but ended up breaking pretty quickly).

Anyway, I eventually concluded that I was wasting time re-inventing the laptop. It was nice to be able to put the unnecessary extra stuff, like the monitor and battery away, when using it in desktop mode (saving desk space). And the ability to pick a proper keyboard is nice.

But buying part-by-part has some downsides. You pay extra because this is all niche stuff, it doesn’t slip as nicely into a backpack (otoh if I only needed to bring the NUC somewhere, it could fit in my cargo shorts or bike under-saddle bag which was neat). And random integration things—you need to bring all the right cables, the battery probably doesn’t talk to your OS so you have to check it manually.

I also did SSH and VNC from an iPad, it works fine, you’ll probably want a keyboard and end up with something like a laptop, but extra latency and your SSH/VNC client will waste some pixels. I also did SSH and VNC from my iPhone, got a little stand and everything. That combo actually got interested questions in a coffee shop, so nerdy niche computing mission accomplished, haha!

So, I hate to ask the boring question because it was a really fun project and I got years of use out of it, but: Why not a laptop?

mortos · 2 years ago
I got an N100 fanless computer for about $100, with the right Aliexpress deals. With 4x i266-v (2.5GbE). Just needed RAM and M.2. The total power draw is also quite low, I've seen people measure 7w total from the wall. At about $.32/kWh, this is saving me a lot over an eBay computer.

The CPU is enough for my needs, currently running Proxmox with OPNsense. And x86 was a major draw for me over the RPi.

woleium · 2 years ago
you can get a usb powered hdmi portable 10 inch monitor on aliexpress for less than $100
Ao7bei3s · 2 years ago
Does anyone know of a low power, mini PC that supports 2 or even 4 SATA SSDs (both connector and physical space wise)?

Most, incl. the Lenovo above, seem to support at best 1x M.2 + 1x SATA.

The best choice I have found is using a N100DC-ITX mainboard with a generic ITX case, and those are huge.

I am looking to replace my Raspi / USB SSDs combination.

squarefoot · 2 years ago
I recently migrated my NAS from a Mini-ITX mainboard to a mini PC. Although there are m.2 cards with multiple SATA ports on board, they would require either to keep the PC open or mount it into a bigger case with multiple disks bay. I chose instead a more costly but potentially more stable solution with a IcyBox IB-3780-C31 enclosure (8xSATA to USB3.1) which is well supported by either Linux and FreeBSD, which is important as I use XigmaNAS. The mini PC I used was the only one with a USB3.1 port on board CPU is a N5105 which is more than enough for 4 ZFS mirrors. Feel free to ask if you need more information.
logicprog · 2 years ago
The Dell Optiplex 7050 I have can take two SATA SSDs or one SATA HDD and one SATA SSD (my current config). Its processing power is pretty great (esp compared to an RPI), plus it's reasonably sized, sturdy, and it also has a low TDP. I use it as a server for several services (A discord clone, Nextcloud, my blog, video streaming). Got it for $20 at a surplus store, plus $80 for the HDD and $60 for the SSD (I went for very high quality ones) lol.
pi-rat · 2 years ago
There are m2 adapters that break out to 4-6 sata ports.

Ex: https://a.aliexpress.com/_Ey8htlr

piaste · 2 years ago
What is your use case? You can fit a few terabytes in the internal slots; if the additional storage were in the form of USB drives, would that be a problem?
youngtaff · 2 years ago
Some of the Lenovo models e.g. m720q have a PCI slot

One option would be to use a riser and a PCI card that supports multiple M2 drives

NelsonMinar · 2 years ago
There's a wonderful world of Mini-PCs and homelabs out there. The new N100 based ones are amazing for under $200.
eek2121 · 2 years ago
The Ryzen ones are pretty amazing. I have one with a 8 core 16 thread “Cezanne” chip that performs all kinds of tasks. (game servers, jellyfin, etc.)
krick · 2 years ago
What's your definition of dirt cheap? These don't seems cheap by any means.
StimDeck · 2 years ago
Certainly not compared to the Pi.
spacecadet · 2 years ago
Great advice. I also use Lenovo Mini PCs in place of SBC's on some projects and they can be very capable.
naikrovek · 2 years ago
Thanks. That is super helpful info on a post about Raspberry Pis.

The number of posts on this site that people want to turn into “show & tell” is very high.

Just make a Show HN post about whatever you want to show off and let the voters decide the fate of the post.

droopyEyelids · 2 years ago
another option in this vein is a used google chrome box. they're rated for 24x7 operation in dusty high vibration environments. can change the boot loader and install any distro. low idle power, std sodimm and nvme drives. super cheap used. usb-c and ethernet
eisa01 · 2 years ago
Any mini PCs with ECC RAM yet?

Would love to make a small all-flash NAS, but the only one I know about does not have it: https://www.asustor.com/en/product?p_id=80

transpute · 2 years ago
$150 PC Engines APU2 (RIP) has 4GB ECC soldered RAM, SATA, mSATA and mPCIE.

$650 QNAP TS435XeU 1U NAS supports 4GB-32GB ECC SODIMM, 4xSATA, 2xNVME, based on Marvell/Armada CN9130 Arm SoC. Debian for Arm64 can be installed via serial console.

Some 4x4 Ryzen Embedded V1000 mini PCs support ECC SODIMM, e.g. https://www.sapphiretech.com/en/commercial/fs-fp5#Specificat..., possibly ASRock.

6th · 2 years ago
I have a 32GB DDR5 ECC in a max spec https://www.solid-run.com/industrial-computers/bedrock-v3000... - it's headless.

You can go to 96 GB.

With displays: https://www.solid-run.com/industrial-computers/bedrock-r7000...

Deleted Comment

Joel_Mckay · 2 years ago
Mini PC (or laptop motherboards) also have normal SSD storage, and are thus much more reliable than micro sdcard MLC flash cards.

There are ways to make Pi's more reliable, but it usually requires a lot of extra parts.

The ARM SoC do not have an IME though... ;)

transpute · 2 years ago
> The ARM SoC do not have an IME though...

RPi 4 boots from GPU/VPU running closed firmware (incl. Microsoft ThreadX RTOS), which retains full control over the OS application processor, https://www.fsf.org/resources/hw/single-board-computers

  Boards based on the Broadcom VideoCore 4 family, such as the Raspberry Pi, require non-free software to startup, although signature checks are not enforced. A free proof-of-concept replacement firmware has been developed, but it is not in a usable state, and development has halted.

  By default, the GPU requires a blob running in this same startup firmware. However, Broadcom also supplies an "experimental" free software stack, which could run without blobs, if the startup firmware were free.

folmar · 2 years ago
Here in Europe I don't see any of them for less than $100. And for a small server something like OrangePi 3B (with nvme etc.) would be be reasonable at half the price and half the power.

Deleted Comment

ed_balls · 2 years ago
Interesting, I was always in Mac mini camp.

I'd be cool to do a comparison new Lenovo mini vs used M1 Mac mini - should be the same price.

nonethewiser · 2 years ago
I imagine it comes down to if you want macos or not. Seems like a niche choice when comparing to something like the pi.
photonbeam · 2 years ago
Is ssd and ram replaceable on these?
mmsimanga · 2 years ago
Not OP but I am in the process of upgrading my M93P Lenovo Thinkcenter. Uses laptop ram and simple to replace hard drive and RAM. I am far from a hardware guy.
haunter · 2 years ago
Yes to both

2x DDR4 SO-DIMM RAM (notebook) slots. No ECC though (at least on this model)

And you can have 2 storage drives. One 2.5" and one M2.NVME

youngtaff · 2 years ago
Yes, almost everything is replaceable https://download.lenovo.com/pccbbs/thinkcentre_pdf/m910q_ug_...

Can even add HDMI, serial ports etc.

SJetKaran · 2 years ago
How do you monitor the power consumption of your pc?
blacklion · 2 years ago
First advice is to enable journaling mode on FS.

First advice must be to mount FS in read only mode, mount /var in memory and forward al logs to one nide, which may be not RPi but something with proper UPS and nut running. Power loss becomes absolutely bening if your FSes RO or temporary.

It is overkill if you have one RPi but author claims that he uses multiple RPis all around a house.

Also, good idea to have A/B system partitions and upgrade system with full partition rewrite and changing active one. Thus way your system will have one good system partition in any case, even if new version has fatal bugs, and recovery become trivial.

I'm using several small/single board PCs im different roles in such way for 20+ years with great success.

pama · 2 years ago
The author links to read only advice a couple of lines down.

https://www.dzombak.com/blog/2021/11/Reducing-SD-Card-Wear-o...

Avamander · 2 years ago
In addition to /var, tmpfs should also be used for /tmp and similar. That should lengthen the SD card's lifetime immensely.
diggan · 2 years ago
Isn't this (tmpfs for /tmp) the default setup for most OSes, and surely raspberry's homegrown OS too?
blacklion · 2 years ago
Anything transient writable must be in memfs/tmpfs/how it is mamre im your OS, of course.

Logs to log server, if system needs non-volatile writable storage — NFS or second storage device, depending on requirements.

Of course, it is too much hassle for single such system (but I started to use it from very beginning out of curiosity) but uf you have many single-task small devices it is very convenient.

joshuanapoli · 2 years ago
> have A/B system partitions and upgrade system with full partition rewrite and changing active one

What’s your upgrade process like? How do you make the new disk image? Do you log into each device to upgrade it, or do you have automation?

blacklion · 2 years ago
I have one «beefy» server/NAS (now it is EPYC2 on SuperMicro platform becuse they are cheap used on AliExpress, before it was Intel E3 12xx-badsed systems, two ir three generation old, always bought used). It is not my router, but NAS for all my data, NFS and build server.

I'm using FreeBSD, and here is script for preparing such installations named NanoBSD. To be honest, it is nothing special — build system with provided config (to strip it down, for example full FreeBSD installation includes toolchain and it is waste of space on «embedded» systems), mount file as loop device, create FS, install system to this FS by standard system means, add needed packages.

I'm building system once, make several images (as each device needs its own set of packages, of course), login to each device via ssh and run simple sh script which detects current active partition (by simply looking at output of mount command — from whic root is mounted) and then «dd if=/net/images/$hostname.img of=/dev/da0p$otherpart bs=128k», set this updated partition as «bootonce» in bootmanager and reboots. Last startup script in boot sequence check availability of network and liveleness of sshd, and if these simple checks are Ok, set this partition as alwaysboot (it is all UEFI features, mostly). If something goes wrong — one power cycle and device will boot from previous partition.

I don't have enough devices to automate «login and call script» part :-)

Eduard · 2 years ago
> First advice must be to mount FS in read only mode

How then do you update the system or install new software?

sokoloff · 2 years ago
Remount in read-write mode when you specifically want to make changes.

  sudo mount -o remount,rw .....

blacklion · 2 years ago
As you update any embedded device: preparing new system image («firmware») and «re-flash» it.

Of course, devices don't build their own systems.

LtWorf · 2 years ago
There is no need to mount read only if you forward the logs somewhere else.
heavyset_go · 2 years ago
Note that mounting /var in memory might exceed your device's memory if you're using something like Docker. You might have to move /var/lib/docker to secondary storage.
bdemirkir · 2 years ago
> have A/B system partitions and upgrade system with full partition rewrite and changing active one

Are there any solutions available for this?

nsteel · 2 years ago
There appears to be new support for A/B partitions in the bootloader that might help: https://www.raspberrypi.com/documentation/computers/raspberr... and https://www.raspberrypi.com/documentation/computers/config_t...
blacklion · 2 years ago
I'm using FreeBSD, not Linux for all my headless systems, and FreeBSD has NanoBSD script for such installations like forever.
akdev1l · 2 years ago
Abroot and OSTree
e12e · 2 years ago
Might be worth considering a different journaling fs, like nilfs2 for filesystems you need r/w.
hcfman · 2 years ago
Back in 2011 I made a commercial product that ran on the earliest plug computers from global scale technologies. I only sold 20 of 'em and every single one of them was being returned with SD card corruption problems. I had to quickly pivot to keeping the rootfs read-only. I've been a fan ever since.

Incidentally, that early commercial product was a home security product with a very small amount of home automation. I released this into open source with a new name in 2021 and now runs on the Jetson series SBCs (https://github.com/hcfman/sbts-install). Except then including high end YOLO models as triggers.

Because it was intended to be a standalone product it supported https with a GUI wrapper around all of the certificate operations. This still exists in my open source version, making it easy to use self signed certificates for intra-device rest calls.

But I've kept and expanded upon the multi-partition memory overlayFS approach and the installation of this system first asks you to install the sbts-base system, which installs the multi-partition memory overlayFS so that other's can use this as their own base systems.

ikari_pl · 2 years ago
I had a similar experience when I hacked a $5 Pentium 2 PC into a fanless (whether it liked it or not) and noiseless workstation. Replaced the HDD with a CF card. After a while, the system started stalling for 1-2s on disk writes, and that was a pain.
MPSimmons · 2 years ago
I am urging everyone who wants to do this to see if they can't first do what they need with a small board like an ESP32. Their energy usage is a small fraction, they cost ones of dollars, and they're sufficient for a whole lot. If you're of the Python persuasion, many boards support both MicroPython and CircuitPython.

It's worth looking into for the cost savings on initial purchase and ongoing power draw.

n8henrie · 2 years ago
I hear the nay-sayers regarding the time cost and complexity of embedded programming, but as a hobbyist I think this is a great recommendation to at least consider.

The projects that I've been able to accomplish on a microcontroller have been more reliable (over decades) than my Pi-based projects, and I don't have to worry as much about them being part of some botnet because I forgot to change the default ssh configuration (wasn't it `pi:raspberry`?).

Beyond micropython, the no_std rust support for ESP32C3 is getting better every month. For people doing little home automation projects because they are fun, the additional constraints can make things a lot of fun and very rewarding.

But yes, for those that are already handy with Linux, a Pi is generally going to be much easier, though IME will be at least 10x as expensive, and the additional setup to get it in the same ballpark for reliability (SSD booting vs network booting vs ro-rootfs, watchdog setup, etc) and the increased power usage (esp for a Pi5) should at least be small part of the decision making.

Macha · 2 years ago
Can I run a media server on an ESP32? No.

Can I run my password manager server on an ESP32? No

PiHole? No. Unifi controller? No

I think people making this comment are envisioning people use Pis as garage door controllers etc., but reflexively suggesting ESP32s as Pi replacements isn't helpful.

petee · 2 years ago
So the straightforward answer is no, an ESP doesnt work for your situation.

They only asked that people consider it; no need to get snooty because it doesn't work for you, despite being very obviously inappropriate for your use case

MPSimmons · 2 years ago
I don't disagree with what you're saying at all, but in my defense, the blog entry posted literally starts with the sentence:

>I use Raspberry Pis around my home as everything from low-power FM transmitters to UPS energy monitors.

master-lincoln · 2 years ago
Why would you buy an expensive raspi for these scenarios? Wasn't it meant to be a computer learning platform for people with no access to powerful computers? Your applications don't need GPIO pins, so you could just use any small PC.
cuu508 · 2 years ago
Even for the garage door opener use case, one nice thing about Pi is you can run Cloudflare tunnel on it – you can then access it from the internet without messing with port forwarding or TLS certificates.
zer00eyz · 2 years ago
How about a camera server?

Why yes a ESP32 can do that, and you get to play with a another cheap stack.

How about Blinking leds or a Display, or led strip... WLED and artwix have you covered!

A project like ESPresense is evolved from a PIzero project.

Let's not forget that some of the pi's appeal is that it has those GPIO headers!

milesvp · 2 years ago
> Can I run a media server on an ESP32? No.

You can run a web server though. Now I’m curious if I can push enough data to stream audio. I know the chip can handle 48k stereo samples, presumably I could stream that much over wifi. I may have to play with that this week. The bigger problem is attaching storage, many ESP32 SoCs don’t have USB peripherals, and many SDcard implementations can’t do much better than Audio throughput (SD has some insane lisencing costs for small dev shops), so I’m not sure the best solution for large amounts of data.

analog31 · 2 years ago
A useful deciding factor is whether your application benefits from an operating system and its utilities, or not.
arendtio · 2 years ago
How about PhotoPrism? ;-)

No, honestly I am impressed at the performance with which my RaspPi 4 is running a PhotoPrism instance with about 120.000 photos. This might also be thanks to the application software, but still quite a feat.

weugek · 2 years ago
Can I run a media server on an RPI? No. -- It's interface is slow and bus speed is shared. No redundancy either.

Can I run my password manager server on an RPI? No -- do you want to loose all your data because the SD card failed?

PiHole? Unifi controller? Maybe

graphe · 2 years ago
If you need an OS and a cheap device use your old netbook or computer you have laying around instead of turning it into ewaste. You can do it all with a rpi zero if you insist on buying another device.
sofixa · 2 years ago
ESP32 and related are pretty cool, but it's a whole different mindset; if something doesn't work, you can't just connect an HDMI/keyboard to debug live with all the regular utilities one might know and that come for free with any mainiline Linux.
MPSimmons · 2 years ago
There are definitely limitations, but if it's a thing that you're going to leave untouched for years, it's worth looking at a device that will use 1/10th (or less) the power.
vel0city · 2 years ago
If something goes wrong I just plug USB into my regular computer with all my normal tooling and can see the serial console, edit the filesystem, etc.

Or if I'm using something like Circuit python I'll just connect the web console.

forinti · 2 years ago
A Pi Zero will spend about a dollar a year in electricity.
NoboruWataya · 2 years ago
I've recently taken an interest in embedded programming and this is something I can't look past. Unless you're producing something at scale, I don't think there is much of a difference economically between writing code in a low level language and running it on a bare metal ESP32, vs running a Python script on a lightweight Linux on a Pi Zero. The cost difference is going to be a few dollars over the life of the product.
krzyk · 2 years ago
Electricity cost is not a problem, lack of ability to run of batteries (because if large amount of power consumption) is in some applications.
treprinum · 2 years ago
N100 costs about $1.50 a year in electricity yet is like 30x faster.
herbst · 2 years ago
For my last (a bit more complex setup) I am using a pi zero and several ESP8266/32 communicating with it, simply with http and wifi. My first time using micro python instead of Arduino as well.

Absolutely love it and to my surprise it's super stable. Something about WiFi power states often messed up my projects in the long run, but no issue with micro python architecture so far.

doctor_eval · 2 years ago
Just want to shout out to tinygo, for this old Go programmer, it makes working with ESP and friends loads of fun.

Admittedly, reverse engineering a single digit 7-segment LED display wasn’t the best use of my time, but by crikey it was fun.

https://github.com/doctor-eval/clocky

torblerone · 2 years ago
I had a look into it but unfortunately there seems no tinygo support for ESP Wifi/BT.
roeles · 2 years ago
Do these devices allow for effortless OTA updating?
LtWorf · 2 years ago
I run ssh, https server (with some dynamic pages written in python), there is a 3T disk attached to get files, I download torrent.

The other ones have touchscreens and speaker attached, I use them to listen to internet radio, watch cartoons and control relays.

Your cheaper solution doesn't allow for any of this to happen.

petee · 2 years ago
They simply asked people to consider it as an option, clearly your use wouldn't run on an esp very well
hn8305823 · 2 years ago
> This feels like a hack, but based on hours of reading online discussions, most people seem to settle on a script that periodically checks whether the WiFi connection is good, and restarts the WiFi interface or the whole Pi if it’s not.

It's not a hack, it's best practice! Just like important servers in a data center should have some kind of out of band connectivity (IPMI, remote controllable RPDU outlets, etc), Important servers in remote difficult to reach locations should have some kind of watchdog script. The script should of course be tuned to the specific use case, considering the impact of a reboot vs downtime until reboot. At the very least it could log adverse events for later investigation.

A simple bash watchdog script was the very first thing I did when I deployed a remote RPI. Not just for wi-fi issues but for any of the dozens of things that could break and be fixed with a reboot.

bravetraveler · 2 years ago
Nowadays this watchdog is init/PID1, on most distributions - systemd.

If init can't be relied on to manage services, what guarantees do you have for the system to provide them?

Sure, one could reinvent this in scripts, but we've moved past this. I mention systemd a lot, but that's not to cast favor - there are alternatives.

Most services don't make appropriate use of the environment they exist in. I assume they expect some site customization, ie: declaring your web server needs these mounts.

A commonly overlooked directive is 'PartOf='. You can tie restarts of one service/resource to another.

Heck, more simply, I think NetworkManager offers a way to customize the wifi/portal checking. You may not have to go completely heavy-handed

smashed · 2 years ago
Just a fyi if you're going down that path, the hardware watchdog is disabled by default in most distros.

Enable it using RuntimeWatchdogSec.

Second, if you are running an important service, see if it can support systemd notification socket and even better the software watchdog protocol with systemd. The service just has to send a heart-beat every X seconds otherwise systemd will restart it.

This ties in the hw watchdog to systemd and systemd watches your service, ensuring both the hardware and the software is running, otherwise the system will restart.

Lots of details here: http://0pointer.de/blog/projects/watchdog.html

None of this is raspberry pi specific, it works on every system with a supported hardware watchdog, which includes raspberry pis.

hn8305823 · 2 years ago
> Nowadays this watchdog is init/PID1, on most distributions - systemd. If init can't be relied on to manage services, what guarantees do you have for the system to provide them?

Does that monitor actual (not just apparent link up) network connectivity or the ability to access remote networks? Not all failure modes are local to OS processes and sometimes rebooting fixes things like this.

process-specific monitoring can be quite useful but so is a generic "I can't reach my server I just want it to reboot now" with minimal complexity/dependencies. I can't imagine all of the failure modes where a simple bash watchdog script would help - and that's the point!

Now, if you have OOB access to the server (like in a data center) then you might not need or even want a watchdog script. But for remote and/or difficult to access servers (I refer to these as "mars probes"), they can be a life saver.

myself248 · 2 years ago
Similarly, I use an ESP8266 to supervise my wifi router and cable modem. Any trouble, reboot!

For the router, it just tries to connect to the appropriate SSID, and then tries to ping the router, and if either of those fails, it swaps to the other router. I have two identical routers with identical configs, with their power connected to the NO and NC contacts of an SPDT relay. If one fails, it just toggles the relay state to switch to the other.

If the router is up, the watchdog tries to load the cable modem's status page, and tries to ping any of three different IPs I've identified within my ISP's network which seem to be either the CMTS or closely associated hardware, and should indicate aliveness of the HFC plant -- I don't want to bother rebooting if the failure isn't something that a reboot could solve. Sadly I haven't figured out how to have two cable modems with the same MAC such that I could swap between them too, and the ISP won't let me have two modems on the same account, so my only resort if the CM fails is to reboot it and hope for the best.

This, plus a rack of batteries that'll keep the router and modem running for 30-plus hours in a utility outage, has kept me online nearly-continuously since May of 2020 when I built it. The code is an absolute horror-show (I'm much better with a soldering iron than an arduino), but in practice it's been rock solid.

slicktux · 2 years ago
Totally agree! Watchdog timers are essential for microcontroller and even computers running software “forever”. Things do happen that even perfect code and design can’t prevent and a watchdog timer will break out of an infinite loop and reset. Things such as cosmic rays flipping a bit or even brownouts…in raspberry pi you also have to worry about SD card corruption…

Edit: raspberry pi’s have built-in hardware watchdog timers I believe. I know Arduino's do!

IshKebab · 2 years ago
It's best practice and a hack. It shouldn't be required, but bugs exist so it is.

Dead Comment

mvip · 2 years ago
We’ve been running thousands of Pis in production for about a decade now. We’re beginning to shift to x86. The price/performance isn’t what it once was for the Pi. I gave a talk about our experience recently at State of Open Con here (https://youtu.be/vX-qK9mxKZI).
dividuum · 2 years ago
Fellow Raspberry Pi digital signage CEO here :). Surprised you didn’t mention the secure boot support, available since the Pi4, in your talk. While our service doesn’t use it (yet?) it sounds quite solid on paper and allows you to protect the data on disk/SD.

We’re still pretty happy with the Pi and the move to more open source APIs (Mesa/DRM/KMS/FFmpeg) is, now that they are finally in a state they feel usable, really promising. As our main use case is still digital signage, the raw processes power isn’t really that relevant as the expensive part (video decoding) is obviously accelerated and the backwards compatibility that’s possible with the Pi is awesome. We still have customers running Pi1B+ devices continuously for almost 10 years with the latest OS release we provide.

anon-3988 · 2 years ago
Hello I am just curious what does different digital signage companies offer? From the outside, I can't think of any innovations or differentiation that is possible. Perhaps its all a matter of cutting down cost?
mvip · 2 years ago
Agree on the adoption of open standards. That’s a step in the right direction. But even with secure boot on the pi, it’s still missing a TPM for other cryptographic operations. We use a lot of Zero Trust stuff on x86 and you can’t do that on the Pi.

The Pi is fine for videos/images (less proper storage), but chokes on a lot of modern web assets.

spintin · 2 years ago
You need to factor in usage, if you are idling a lot ARM > X86.

And you need to look at longevity, there I suspect ARM will outlive X86 too.

For modularity ARM > X86 too, because they are cheaper to have many small of.

But for scalability (= business in the current economy) X86 > ARM.

Also all graphs should be per watt, that 2 -> 4 is more performant is not news, that it is more performant per watt is!

And if you did that you would see that Raspberry 5 is not getting as much per watt performance increase as it should.

We have peaked permanently for the eternity of mankind.

Let that sink in.

Last but not least, the ONLY hope for any progress (openness not performance) at this point is the JH7110, but they are lagging behind in 3D support.

AshamedCaptain · 2 years ago
> if you are idling a lot ARM > X86.

Don't take this for a given. RPis are notoriously bad at idle power consumption. The x86 replacement I bought for my home server ended up having about half the idle power consumption of the RPi it was replacing.

pi-rat · 2 years ago
I think maybe you’re overestimating the idle power consumption of modernish x86 (especially the last 5-6 gens of intel)? I’ve a i5 9600T system that’s drawing less than a Pi at idle. And cost me the same to buy.

https://docs.google.com/spreadsheets/u/1/d/1LHvT2fRp7I6Hf18L...

mvip · 2 years ago
Our customer don’t really care about power usage much as they are all connected to a TV which draws order of magnitude more power. For some use cases power usage is key, but not for our use case.
Always42 · 2 years ago
is this for running displays or is this being used for production assembly machines? (eg gpio usage)
mvip · 2 years ago
Currently for displays, but with Edge Apps you will be able to run workloads that interfaces with sensors etc over GPIO (and USB/Serial).
Horffupolde · 2 years ago
What do you mean shift to x86?
treprinum · 2 years ago
Maybe Alder Lake-N? Those cores are very fast (Skylake-level) and <10W full load. RPi5 is at 8.5W full load and much slower.
rcarmo · 2 years ago
I did exactly zero of any of those things and have had some Pis run for multiple years without any issues until being replaces by a newer model (my HomeKit/Zigbee gateway and data logger is now a Pi 4). I guess it all boils down to good SD cards and stable power supplies.
Towaway69 · 2 years ago
> I guess it all boils down to good SD cards and stable power supplies.

Agree, I've also been running a number of PIs and when they broke, it was because of failing SD cards.

I found pibenchmark to be a good source of info --> https://pibenchmarks.com/

Definitely compare SD cards before buying.

not_your_vase · 2 years ago

  > I guess it all boils down to good SD cards and stable power supplies.
I would say your sample size has a stronger effect on your experiences. With a large enough pool of devices everything will go wrong, what can go wrong. And there will be also new failure modes that you have never even dreamed of.

megous · 2 years ago
6 years of the same Orange Pi PCs/PC2s often on the same cheap Sandisk SD cards, some with postgresql databases on them logging environment data every 5 seconds (though batched to 15 minutes), frequent large system updates (Arch Linux ARM is a bit more churny). And not much issues, to be concerned about doing something annoying like other people suggest here with A/B updates and readonly stuff, that prevents comfy normal use of a general purpose Linux OS.

I don't run a completely storage oblivious setup, I use f2fs, which has much better write patterns than ext4, and I disable stats collection in postgresql, which causes constant churn, but logging, and other stuff has negligible effects and I don't care doing anything special about it, and I leave it default OS configuration.

I have probably 4-5 uSD card failures over 60 year total runtime of my 15 SBCs that I run 24/7. So that's 1 failure per 12 years. Nothing that can't be dealt with using backups. (I didn't even need them that much, yet, because all uSD cards so far failed by turning read only, which for f2fs means you'll get the last consistent snapshot on the card, that you can just copy to a new card and continue. 10-15 minutes of recovery time.

All that complexity and limitations that people talk about seem way overkill for a home setup.

And I think other reason why I don't get many uSD card failures is because I run most cards in HS25 mode. Not in SDR104 or whatnot. 3-4x the frequency really causes the cards to heat up a LOT during activity. Can't be great for the flash chips. 2 of those failures were in SDR104 enabled hosts. Copying data to uSD card using SDR104 capable USB3.0 adapter really makes the card scarry hot in just a few seconds.

goodburb · 2 years ago
Older Raspbian OS around year 2016 and below had access time enabled, with any file read there was a write, which is probably why there were many reports of corrupted cards ever since.
Tor3 · 2 years ago
Same here - I've been running a couple of Pi3 as Cups servers for years of uptime (the only time that uptime gets reset is when there's a power outage - and that's very rare indeed). Did nothing more than install Raspbian on a micro SD card, set up Cups, connect USB to printer (for one of them - the other manages a networked printer). And left them alone after that.
Havoc · 2 years ago
It’s very hit and miss. I’ve had endless problems with some and others seem fine for extended timeframes. No clear pattern in sight

I’ve switched to SSDs for most of them now. It’s too much of a dice roll otherwise

dgellow · 2 years ago
My Raspberry Pi 2 ran for a while, used as a pihole, a VLC-hacked media center, and a weather stations for simple sensors, until it didn’t :(

In ~2022 I started to get random errors, then found out the SD card was failing. I never took the time to fix it, it had lots of little things locally I don’t feel like doing again.

domh · 2 years ago
It may seem like overkill, but I learnt how to use ansible to manage the software on my pi. I've actually just upgraded my rpi4 to use an ssd so am choosing to start from scratch. I'm hoping my ansible playbooks are written well.
LtWorf · 2 years ago
Get a new SD card and replace it?
yjftsjthsd-h · 2 years ago
I'm shocked the SD card bit isn't first, and more surprised that the post doesn't suggest USB boot (I have one pi that's been on ~24/7 for years now, and I attribute its lack of problems to 1. using Alpine configured to barely touch disk, and 2. not having an SD card to corrupt - I don't know why USB would be more reliable, but anecdotally it is)
txdv · 2 years ago
My argon rasberry setup with an SSD is also stable.

Only reason why it fails is when the electricity goes out. A battery that could keep it running for 10 minutes would completely be enough.

rubatuga · 2 years ago
It's kind of a shame that laptops are the equivalent of a Server + UPS, but with none of the enterprise features.
severino · 2 years ago
> 1. using Alpine configured to barely touch disk

I'd like to learn some tips on how to do that... do you mean something like https://wiki.alpinelinux.org/wiki/Installation#Diskless_Mode ?

yjftsjthsd-h · 2 years ago
Yeah, exactly - as best as I can remember, and https://wiki.alpinelinux.org/wiki/Raspberry_Pi seems to agree, Alpine on the pi defaults to a mode where it mosly loads itself into a ram disk on boot and just stays there. Which means that you're constrained on "disk" space (your root filesystem is no larger than ram), but if your workload fits in that then you have the advantage that after boot the system pretty much doesn't touch disk.
Tor3 · 2 years ago
I've had a bunch of Pi cards, running on SD without problems. But a single one suddenly developed a super-hot SD card, this was a brand new Pi which I was just setting up. Got the card out, and that one and the next Pi got a USB SSD and are now using those. That was a bit scary. But as mentioned I've also been running Pi with micro SD as Cups servers for years, with no problems at all.
Aerbil313 · 2 years ago
Would there be any reliability problems with using USB flash drives instead of USB SSDs?
breton · 2 years ago
I agree with this completely. All my rpi failures were because of SD cards. I have 2 rpis, both boot and run from usb, both for several years now.
JoBrad · 2 years ago
Same. I use an M.2 HAT to boot from SSD, and it works great.