I run similar (gitea, scrypted+ffmpeg instead of frigate, plex instead of jellyfin) plus some Minecraft servers, *arr stack, notes, dns, and my VM for development.
It's an i7-4790k from 12 years ago, it barely breaks a sweat most hours of the day.
It's not really that impressive, or (not to be a jerk) you've overestimated how expensive these services are to run.
ZFS doesn't really need huge amounts of RAM. Most of the memory usage people see is the Adaptive Replacement Cache (ARC), which will happily use as much memory as you throw at it, but will also shrink very quickly under memory pressure. ZFS really works fine with very little RAM (even less than the recommended 2GB), just with a smaller cache and thus lower performance. The only exception is if you enable deduplication, which will try to keep the entire Deduplication Table (DDT) in memory. But for most workloads, it doesn't make sense to enable that feature anyways.
That + full-disk encryption is why I went with BTRFS inside LUKS for my NAS.
They recommend 1GB RAM per 1TB storage for ZFS. Maybe they mean redundant storage, so even 2x16TB should use 16GB RAM? But it's painful enough building a NAS server when HDD prices have gone up so much lately.
The total price tag already feels like you're about to build another gaming PC rather than just a place to back up your machines and serve some videos. -_-
That said, you sure need to be educated on BTRFS to use it in fail scenarios like degraded mode. If ZFS has a better UX around that, maybe it's a better choice for most people.
1GB RAM per 1TB storage is really only required if you enable deduplication, which rarely makes sense.
Otherwise, the only benefit more RAM gets you is better performance. But it's not like ZFS performs terribly with little RAM. It's just going to more closely reflect raw disk speed, similar to other filesystems that don't do much caching.
I've run ZFS on almost all my machines for years, some with only 512MiB of RAM. It's always been rock-solid. Is more RAM better? Sure. But it's absolutely not required. Don't choose a different file system just because you think it'll perform better with little RAM. It probably won't, except under very extreme circumstances.
Way way overspeced for what I listed, but I use it for lots of video processing, numerical simulations, and some local AI too.
I have a similar subset of this stuff running at my mom's house on a 16 GB ram Beelink minicomputer. With openvino frigate can still do fully local object detection on the security case, whish is sweet.
Not impressive at all. I run just about as many services, plus several game servers, on a Ryzen 5, and most of the time CPU usage is in the low single digits. Most stuff is idle most of the time. Something like a Home Assistant instance used by a single household is basically costless to run in terms of CPU.
Ultimately, basically. I have two servers in my homelab, one that is more beefy, which hosts a bunch of stuff (basically everything parent outlined + ), including a DHT crawler, download clients, indexers, databases and a lot more. It's sitting and using 16GB (out of available 126GB) right now. Then I have another which only runs the security system + Frigate + Home Assistant, it's using 2.3GB out of 32GB available.
No, you definitely can’t. Or at least, not 3B+. I wound up buying https://www.amazon.com/ACEMAGICIAN-M1-Computers-Computer-3-2... which was $50 less a month ago (!!) because so many things don’t fit well. Immich is amazing, but you wouldn’t get a lot of the coolness of it if you can’t run the ai bits, which are quite heavy.
> Impressive that all that can run on one machine. Mind sharing the specs?
Not GP but I have lots of fun running VMs and lots of containers on an old HP Z440 workstation from 2014 or so. This thing has 64 GB of ECC RAM and costs next to nothing (a bit more now with RAM that went up). Thing is: it doesn't need to be on 24/7. I only power it up when I first need it during the day. 14 cores Xeon for lots of fun.
Only thing I haven't moved to it yet is Plex, which still runs on a very old HP Elitedesk NUC. Dunno if Plex (and/or Jellyfin) would work fine on an old Xeon: but I'll be trying soon.
Before that I had my VMs and containers on a core i7-6700K from 2015 IIRC. But at some point I just wanted ECC RAM so I bought a used Xeon workstation.
As someone commented: most services simply do not need that beefy of a machine. Especially not when you're strangled by a 1 Gbit/s Internet connection to the outside world anyway.
For compilation and overall raw power, my daily workstation is a more powerful machine. But for a homelab: old hardware is totally fine (especially if it's not on 24/7 and I really don't need access to my stuff when I sleep).
Cheap to buy old hardware, but electricity to run those old rigs isn't really cheap in many areas now. My server is costing me about $100/month in electricity costs.
It does have 16 spinning disks in it, so I accept that I pay for the energy to keep them spinning 24/7, but I like the redundancy of RAID10, and I have two 8-disk arrays in the machine. And a Ryzen-7 5700G, 10gbit NIC, 16 port RAID card, and 96GB of RAM.
It depends on the type of hardware that you use for your server. If it's really server grade you're totally right. For example cheap memory+CPU+MB x99 off AliExpress are cheap but they're not very efficient.
In my case I fell in love with the tiny/mini/micros and have a refurbish Lenovo m710q running 24/7 and only using 5W when idling. I know it doesn't support ECC memory or more than 8 threads, but for my use case is more than enough
I’ve been watching some storage and homelab-themed videos and I heard there’s a lot of optimizations you can do to lower power usage - spinning the disks down, turning the machine on for a limited time, etc.
That doesn't work for me. The main server is constantly using the disks to record security cameras, run VMs 24/7, Plex, a web server, a VPN (so I can dial in to my local network remotely), and a lot more.
How have you measured the power usage/cost? That seems like a incredibly high price for electricity, similar to a 600W constant load in my part of the world.
All of my IT equipment in my office is running through a single UPS that measures power consumption.
I do have a bit more than just that server hooked up to it. There's also a Dell i5 running DDWRT as my main gateway/router, the fiber internet modem, a small Synology NAS, a couple of WIFI routers, etc. It all adds up.
That doesn't include my backup server out in the garage with another 8-disk RAID10 array and an LTO tape drive that is often backing up data, 5 more WIFI routers around the property, and 10 or so security cameras. So I'm probably well over $100/mo for all my tech stuff.
I run similar (gitea, scrypted+ffmpeg instead of frigate, plex instead of jellyfin) plus some Minecraft servers, *arr stack, notes, dns, and my VM for development.
It's an i7-4790k from 12 years ago, it barely breaks a sweat most hours of the day.
It's not really that impressive, or (not to be a jerk) you've overestimated how expensive these services are to run.
Video is usually offloaded too to the igpu on these. I have like 13 vms running on a AMD 3400g with 32gb
Fair enough. How much RAM though?
16GB would be plenty. I've got like a dozen services running on an 8GB i7-4970 and it's only using 5GB of RAM right now.
If you're running ZFS, it's advisable to use more RAM. ZFS is a RAM hog. I'm using 32GB on my home server.
ZFS doesn't really need huge amounts of RAM. Most of the memory usage people see is the Adaptive Replacement Cache (ARC), which will happily use as much memory as you throw at it, but will also shrink very quickly under memory pressure. ZFS really works fine with very little RAM (even less than the recommended 2GB), just with a smaller cache and thus lower performance. The only exception is if you enable deduplication, which will try to keep the entire Deduplication Table (DDT) in memory. But for most workloads, it doesn't make sense to enable that feature anyways.
That + full-disk encryption is why I went with BTRFS inside LUKS for my NAS.
They recommend 1GB RAM per 1TB storage for ZFS. Maybe they mean redundant storage, so even 2x16TB should use 16GB RAM? But it's painful enough building a NAS server when HDD prices have gone up so much lately.
The total price tag already feels like you're about to build another gaming PC rather than just a place to back up your machines and serve some videos. -_-
That said, you sure need to be educated on BTRFS to use it in fail scenarios like degraded mode. If ZFS has a better UX around that, maybe it's a better choice for most people.
1GB RAM per 1TB storage is really only required if you enable deduplication, which rarely makes sense.
Otherwise, the only benefit more RAM gets you is better performance. But it's not like ZFS performs terribly with little RAM. It's just going to more closely reflect raw disk speed, similar to other filesystems that don't do much caching.
I've run ZFS on almost all my machines for years, some with only 512MiB of RAM. It's always been rock-solid. Is more RAM better? Sure. But it's absolutely not required. Don't choose a different file system just because you think it'll perform better with little RAM. It probably won't, except under very extreme circumstances.
32gb for me because half of that is given to the development VM
Ryzen 5950x cpu, 64 gb ecc ram, dual 16 tb drives for zfs, Nvidia 5070 gpu.
Way way overspeced for what I listed, but I use it for lots of video processing, numerical simulations, and some local AI too.
I have a similar subset of this stuff running at my mom's house on a 16 GB ram Beelink minicomputer. With openvino frigate can still do fully local object detection on the security case, whish is sweet.
Not impressive at all. I run just about as many services, plus several game servers, on a Ryzen 5, and most of the time CPU usage is in the low single digits. Most stuff is idle most of the time. Something like a Home Assistant instance used by a single household is basically costless to run in terms of CPU.
Not costless in terms of RAM though, surely?
Ultimately, basically. I have two servers in my homelab, one that is more beefy, which hosts a bunch of stuff (basically everything parent outlined + ), including a DHT crawler, download clients, indexers, databases and a lot more. It's sitting and using 16GB (out of available 126GB) right now. Then I have another which only runs the security system + Frigate + Home Assistant, it's using 2.3GB out of 32GB available.
Web apps like Home Assistant are very light, things like game servers are heavier since they have to load maps etc.
You could easily run all of that on a rpi…
No, you definitely can’t. Or at least, not 3B+. I wound up buying https://www.amazon.com/ACEMAGICIAN-M1-Computers-Computer-3-2... which was $50 less a month ago (!!) because so many things don’t fit well. Immich is amazing, but you wouldn’t get a lot of the coolness of it if you can’t run the ai bits, which are quite heavy.
[dead]
> Impressive that all that can run on one machine. Mind sharing the specs?
Not GP but I have lots of fun running VMs and lots of containers on an old HP Z440 workstation from 2014 or so. This thing has 64 GB of ECC RAM and costs next to nothing (a bit more now with RAM that went up). Thing is: it doesn't need to be on 24/7. I only power it up when I first need it during the day. 14 cores Xeon for lots of fun.
Only thing I haven't moved to it yet is Plex, which still runs on a very old HP Elitedesk NUC. Dunno if Plex (and/or Jellyfin) would work fine on an old Xeon: but I'll be trying soon.
Before that I had my VMs and containers on a core i7-6700K from 2015 IIRC. But at some point I just wanted ECC RAM so I bought a used Xeon workstation.
As someone commented: most services simply do not need that beefy of a machine. Especially not when you're strangled by a 1 Gbit/s Internet connection to the outside world anyway.
For compilation and overall raw power, my daily workstation is a more powerful machine. But for a homelab: old hardware is totally fine (especially if it's not on 24/7 and I really don't need access to my stuff when I sleep).
Cheap to buy old hardware, but electricity to run those old rigs isn't really cheap in many areas now. My server is costing me about $100/month in electricity costs.
It does have 16 spinning disks in it, so I accept that I pay for the energy to keep them spinning 24/7, but I like the redundancy of RAID10, and I have two 8-disk arrays in the machine. And a Ryzen-7 5700G, 10gbit NIC, 16 port RAID card, and 96GB of RAM.
It depends on the type of hardware that you use for your server. If it's really server grade you're totally right. For example cheap memory+CPU+MB x99 off AliExpress are cheap but they're not very efficient.
In my case I fell in love with the tiny/mini/micros and have a refurbish Lenovo m710q running 24/7 and only using 5W when idling. I know it doesn't support ECC memory or more than 8 threads, but for my use case is more than enough
I’ve been watching some storage and homelab-themed videos and I heard there’s a lot of optimizations you can do to lower power usage - spinning the disks down, turning the machine on for a limited time, etc.
That doesn't work for me. The main server is constantly using the disks to record security cameras, run VMs 24/7, Plex, a web server, a VPN (so I can dial in to my local network remotely), and a lot more.
How have you measured the power usage/cost? That seems like a incredibly high price for electricity, similar to a 600W constant load in my part of the world.
All of my IT equipment in my office is running through a single UPS that measures power consumption.
I do have a bit more than just that server hooked up to it. There's also a Dell i5 running DDWRT as my main gateway/router, the fiber internet modem, a small Synology NAS, a couple of WIFI routers, etc. It all adds up.
That doesn't include my backup server out in the garage with another 8-disk RAID10 array and an LTO tape drive that is often backing up data, 5 more WIFI routers around the property, and 10 or so security cameras. So I'm probably well over $100/mo for all my tech stuff.