Full Lab Details - September 2021
Last year in September I posted a full update to my home lab/home network. Since it has been a year, I thought I would do it again. You can read last years post below. I encourage you to read it, as I go into detail about some things there that I may not here.
Like last time, let's start with Network and IoT
The general layout of everything has stayed the same, but I have added more patches to my patch panels, and added extra switches. Here are the full details from top to bottom:
The top section remains the same with 2 x Monoprice patch panels, and some cable management in the middle. I have added patches for more cameras, and more end devices. I am still using the Monoprice Slimrun cat6a cables without issue.
The PFSENSE firewall remains the same using a Pentium Gold G5500, Supermicro server board, 8GB ECC, SSD boot drive etc (More details here)
Below that is my Dell X1052P switch, I mentioned in the last post I wanted to replace it, and this is still the case. I am waiting for a good deal on a Cisco switch I have my eye on. This Dell switch handles all PoE devices and all general 1G ports, except for management ports (IPMI, UPS's, OOBM ports etc)
Because I added more devices to the network, that Dell switch was getting very full. I decided to get a Cisco SG300-28 switch for IoT devices and management ports. This switch is fanless and very low power. This is connected with a 1G Trunk port back to the Dell switch, but I also moved my Management VLAN onto a separate interface on the firewall and connected it directly to this Cisco switch. So all Management traffic flows through this switch first. This switch has decent CLI, and a really decent WebUI too.
Below that is my favorite new device, its a Cisco SX350X-24F switch, its a 24 Port 10G SFP+ switch, with 4 of them being 10G-BaseT combo ports. I have 2 connected to my ESXi server, 2 connected to my TrueNAS server, 1 to my Firewall, 1 to my Desktop and 1 to my garage to extend the network over there. Then there is 2 x SFP+ DAC's connecting back to my Dell X1052P for a total of 20G of bandwidth between them. This has let me have a MASSIVE improvement in performance. With my new NAS being able to easily saturate a 10G link, I had to get more devices onto 10G (Or 2 x 10G LACP). All Transceivers are either old Cisco 10G-SR or FS.com cheap ones. You can see I have a SMF run in there now, that goes to my garage
The IoT stuff remains the same. A Hive Hub which I want to remove, a Hubitat which I don't like and my trusty old Ripe Atlas
I am still using the Geist 15-P for temperature monitoring. Its been working great, and just keeps on going.
Onto the the middle of the rack which is Servers and Storage
This section has changed a lot from the last past
Up top is my ESXi server which hasn't changed too much. Its running 2 x E5-2680 V4's with 256GB of ECC DDR4, Dual 10G NIC's, Micron 9100 Pro 3.4TB HHHL NVMe Datastore etc. Since last time, I added an LSI 9207-8i which I passed through to a TrueNAS VM. I added 12 x 8TB NL-SAS drives I got for cheap, and put them into a large RAIDZ2 array for my media. This server is good, but boy does it use a lot of power and generate a lot of heat. I am looking for my next upgrade which should be able to be downsized a little, as my compute needs are actually quite low. I swapped the fans in the ESXi server for the FAN-0094L4 9G0812P1G09 models. These are SO MUCH QUIETER than the old Nidec fans.
ESXi runs VM's for pretty much everything I use. Docker hosts for PLEX and supporting services, Windows VM's for Veeam, Arq etc, all the stuff I mess with.
Below that is my new TrueNAS physical Server that replaced the Synology.
I loved the DS1817+, but the limitations of the performance and functionally of the Synology were getting to me, and it seems the newer models are not much better when you need to encrypt your data. I figured I could easily build a killer TrueNAS machine for less money than a rackmount Synology.
This server only has 6 x 4TB NL-SAS drives in three mirrored vdev's, and 2 x 800GB Intel DC S3610 SSD's for a metadata vdev (Also called Fusion pool). Here are the rest of the specs
This server doesn't need much space (Hence the array configuration) but gives me GREAT speed and reliability. It can easily max out the 10G link reading and writing, and the rebuild for a failed drive is super fast. All that is on this server is my actual IMPORTANT data, like photos, documents etc. All bulk media is on the TrueNAS VM in the ESXi server.
I keep a big stock of spare 8TB and 4TB disks for these 2 storage systems. And more info on how I am backing up and replicating data can be found here
Below that is my Blue Iris NVR. The system is largely the same as before, but now its in a 2u case for more expandability. It also has an LSI 9207-8i, and now contains 2 x 12TB SATA disks, and 2 x 8TB SATA disks for camera clip storage.
Since it has an i7 8700K and I have a lot of spare CPU power, I'm now running DeepStack AI on all cameras to form a virtual perimeter around my house which notifies me with push notifications if anyone breaches is. Here is the mailman doing just that
I also added a second PSU to this, as well as all the other servers just to protect against PSU failure. Here is a shot of the boring internals
Below this is my Power setup, which has gotten quite an upgrade too
I'm using the same PDU's, but they are now connected to the same UPS. There is a whole lot more detail in this post
The UPS is a 3000va double conversion UPS which now powers everything electronic in my whole house. So far this UPS has been AWESOME.
Now that I have a 27kw Natural Gas standby generator that powers my whole home, this UPS only has to run for a maximum of 15 seconds, as that's how long it takes the generator to sense utility outage, spin up the generator and switch power.
The generator is a Generac RG027, which contains a 2.4l Mitsubishi Inline-4 engine, and runs at 1800rpm. These are much more durable and reliable than the 999c china-made air cooled Generac's that run at 3600rpm. These liquid cooled units are also much quieter and use less natural gas.
Here is a wider shot that shows the transfer switch location also
Here is a demo of how fast it switches over. The loud click you can hear at the end is the transfer switch moving from Utility, to generator power.
I just want to add a side note on power usage here. You may think this all uses a ton of power, and then see my UPS/PDU load and see that yes, it does use a fair amount of power. Keep in mind that ALL of my homes electronics are powered from my rack, that's everything in the rack, both my entire desktop setup as well as my wife's, my entire living room HTPC/TV setup, and the HTPC/TV setup in my office. So it usually has a fair amount of load which everyone else has spread around the home.
On to my Garage Rack which has seen no changes since last time other than the SMF upgrade
You can read about that more here:
And that's all!