my home server needs to be reconstructed and i’m seeking ideas on how to future proof it. here’s some ascii art in a screenshot to help describe how it’s currently setup:
description (left to right):
- laptops, smartphones, tables, etc connect to an access point configured on a windows 10 virtual machine (vm). the windows10 vm uses pci passthrough on the wireless adapter and this is done to get gigabit wifi speeds since intel’s drivers won’t allow linux to do this in ap mode; but will allow it just fine if you’re using windows.
- requests from the wifi clients are passed via dns & ip masquerade to another virtual machine based on pfsense
- pfsense serves as the router, firewall, vpn, ad blocking & web hosting and it’s also configured to use pci passthrough on the primary network interface to gap internet traffic from the server
- the center of the drawing shows how i perform data backups using a 3 gig wired connection with a hardware switch and i setup the host ubuntu server to manage dhcp on the secondary network interface & the devices that are connect to the switch. the data is stored using rsync and harddrives are setup to use an extremely large lvm made of several different types of hard drives.
i’ve rebuilt this server multiple times each time i encountered a “gotcha” or a surprise that i had not anticipated and it made some needful component stop working; so i’m seeking advice from Lemmy on how to redesign this to mitigate future surprises.
some of the surprises i’ve encountered so far are:
- the pfsense logs overfilled the root volume of the bsd based vm because logrotate was configured for linux. the image is hardcoded with a single volume so i will need to find a way to borrow some space from the backup volume using nfs and configure the logs to write there instead of locally.
- i have no key for the windows10 vm; so i’m forced to clone it’s qcow image and manually configure the hotspot each time the 30 day free trial from microsoft expires. I intend to improve upon this creating an ansible job to rotate this virtual machine every 30 days automatically and include powershell based tasks to configure the hotspot in windows automatically
- intel limits the speed for linux native internet connection sharing to 100 megabits (already mentioned & fixed above)
- the local users home volume overfills when trying to take my google backups (already fixed)
- my cats & dogs LOVE the taste of cat6 capables and cat6 is required for 3 gigabit speeds (also fixed)
constraints:
- don’t spend anymore $$$
- gigabit wifi speed is A MUST
- 3 gigabit backups speeds are a must too
You keep cloning and configuring shit on a Win10 instance because you can’t find the key?
That’s silly and you should just stop doing that: https://github.com/massgravel/Microsoft-Activation-Scripts
There you go! One less problem to deal with.
This is why I asked. Thank you!!!
Commenting in case I need this someday.
Maybe you can use the spicy tape to prevent your pets from eating the cables (assuming that works on them)?
Orher than that, maybe you can setup some metrics (and alerting?) to keep an eye on the diskspace?
spicy tape
That’s fucking hilarious.
It’s real; I used to use it in my furniture to keep my dogs from chewing on it
i used this sour apple spray designed to keep pets away and treated the ethernet cables w it.
sour apple spray
Doesn’t it attract ants etc?
i don’t think it’s real apple and it has no smell, so no; or atleast not yet in the last five or so years.
Maybe a stupid question, but isn’t it just easier to get a secondhand AP on eBay or something than deal with this windows WiFi BS?
You ask about future proofing but Windows 10 is EoL in 8 months.
I use one as a backup when problems happen and I go this route for the additional features that routers usually don’t have like the spam blocking or vpn or Internet accessible storage like a cloud.
The Windows vm only faces internally so I’m okay w it not getting support and it’s only purpose is as an access point
Window only faces internally so I’m okay w it not getting support
There is room some future proofing.
All your mobile devices connect directly to that Windows. Consider them ‘unsafe’. Consider Windows ‘unsafe’ as well.
‘unsafe’ + ‘unsafe’ = incubator for all kinds of trouble IMHO.
i’d like to replace it with something else; but the wireless network adapter is intel and i can only get 100 megabits using the linux driver whereas i get 1 gigabit using the windows driver.
The attacking bots will surely be liking all your ‘buts’ ;-)
I hope you help me better appreciate your recommendation better; the windows machine only faces internally so if there’s bots they would be coming from my personal Linux laptop or work MacBook and those things never leave the house.
It feels like if pfsense is unable to help them out; then I stand little chance of doing myself so myself.
The standard (automated) attacker manipulates the ‘inside’ device first (for example, it executes a JavaScript) and makes it perform an attack on the WiFi router, to which the device is connected.
If the inside device is a windows pc and the WiFi router has it’s inside port open for administrative actions, this is an easy game. Millions of WiFi routers have been turned into bots this way.
In your case the WiFi router is windows. This is different from the usual plastic router, but still not really a safe situation.
The router is a pfsense virtual machine based on openbsd; Windows is only the wifi access point and no administration whatsoever is conducted from it.
However the delineation between router and Wi-Fi access point gets murky for me here since the an access point is a effectively router, but by this same loose definition, it’s also, effectively, a proxy.
Since this Windows virtual machine is headless like is host server, so the only possible entry vector would come from its clients entirely made up of Linux, android, and Mac machines. If those are compromised; then I don’t think there’s any way for me to stop it.
An AP is just a WiFi point, you can use pretty much any AP with your pfsense router.
That’s what most of us do, using this windows VM just for WiFi is only going to cause you a headache in the future.
it causes headache now since, but i don’t think i have another choice if i want faster than 100 megabit speeds.
Pretty much any wireless AC AP from the last 10 years can hit those speeds with no headache, no keys, and no Windows.
802.11ac will hit 600-800Mbps easily, and those APs are dirt cheap since it’s old tech.
I would not want to expose my home server to the internet. VMs have been breached, network stacks too, and any exposed services. I would dump the windows stuff, if nothing else it is not future proof. Consider an AP. Backup consider a hot mount sata enclosure. One can then do swapable high speed backups. I would want off line and off site backups. One issue with rsync is it may not store all file attributes. Just be aware and it may not keep historical snapshots. Some of this depends on how it is configured. Also rsync may not be that secure though it can be too depending how configured.
Edit: Cool concept though. Thanks for sharing.
Edit: Might want to consider some level of volume shadowing or Raid on your server and NAS. Maybe some snapshotting.
Edit: Future proof, have enough memory, cores, and storage.
not a concept; an actual, physical server that i’ve been using for almost five years now.
VMs have been breached
i have automatic updating enabled to incorporate cve fixes/updates asap on all of the instances and the host server.
now that you’ve made me aware, i intend to create automated jobs to destroy & create both vm’s from an immutable golden image that are also pre-staged to capture all updates before they replace their older live and possibly compromised predecessors.
the host server is also gaped from internet access via pci passthrough dedicated to the pfsense vm; so the only entry vector, afaik, is through the pfsense firewall.
i was also wondering if an immutable distro for the host server would help with security as well and now i think i’ll do that too.
I would dump the windows stuff, if nothing else it is not future proof. Consider an AP.
i’m limited to sub-100 megabit wifi speeds without the windows vm since intel will not allow the linux driver to have gigabit speeds in ap mode. i feel like this is the weakest part of the entire design and i was hoping someone had a better idea that didn’t require AP purchase. all of the AP’s i’ve purchased in the past eventually lost support from their manufacturers and they became compromise-able anyways so it’s less future proof imo; whereas i plan on keeping this server running for atleast another decade and support is virtually guaranteed to be never ending.
also: i haven’t yet encountered an AP that is capable of providing all of the features that i currently use. ie ad blocking; personal vpn; web hosting; and cloud-like internet accessible storage via ssh tunnel (in addition to others). purchasing a dedicated AP would effectively deny myself these capabilties and i would have pay $$$ for the privilege.
Backup consider a hot mount sata enclosure. One can then do swapable high speed backups. I would want off line and off site backups.
it feels silly to to me to purchase hardware to duplicate the same capability that i already have and that cloud like internet accessible storage is reason why offline backups don’t work for me, but i can see the wisdom of having gapped backup duplicates nonetheless; so i’ll figure out a way to incorporate it somehow.
these 3 very valid points are exactly why i asked this question and thanks for giving me this awareness.
i haven’t yet encountered an AP that is capable of providing all of the features that i currently use. ie ad blocking; personal vpn;
Pfsense does both of these. pfblocker NG in particular is a very powerful network adblocker with lots of lists. Pfsense can also run VPNs, it supports openvpn and wireguard in both client and server mode and you can set up multiple so one client, one server.
web hosting; and cloud-like internet accessible storage via ssh tunnel (in addition to others).
If you just need personal services it would be best to run something local, setup a wireguard tunnel on pfsense that gives access to your network and VPN in to access things remotely. If you need to share with others I suppose this can become a problem.
Yes, I use a pfsense based virtual machine as my firewall and I have availed myself to some of these capabilities like I’ve mentioned earlier.
I’ve grown accustomed to have this broad range of capabilities and the idea of getting a home router without this functionality feels foolish because I would literally be paying for the privilege of denying myself these utilities.
Regrading AP. Why can’t you just use the wifi functionlity and let your server do the rest? APs are really just glorified WiFi cards with a bridge.
Im stuck at sub-100-megabit Wi-Fi speeds if I use Intel Linux driver; but their Windows driver doesn’t have any such restriction, so I give the Windows virtual machine full control of the wireless adapter via PCI passthrough to workaround this annoying and pointless restriction.
Connect the AP to a Gigabit ethernet port. No way that should be limited to 100MBit.
My point is use ethernet not a WiFi connection. Also use an AP with a Gigabit port.
why ubuntu instead of debian?
Live patching; I don’t reboot
Interesting. How well has this feature been working out for you?
well so far; the only time this machine reboots is when it loses power and it’s usually once or twice per year.
What about on major version update?
they happen when the reboots occur.
Is there a reason you’re making your own access point instead of buying an off-the-shelf one? I know you said you don’t want to spend more, but a proper AP would let you simplify your server and remove the Windows VM entirely while still providing greater than Gigabit speeds (depending on the speed of your switch ports).
This way I get all of the features that I want that including things that off the shelf access points usually didn’t have (eg ad blocking, internet accessible storage, and vpn) and for free since it’s all 100% made out of old hardware cannibalized from only laptops and workstations. It’s also faster than the most expensive expensive alternatives.
It’s also a perpetual project that forces me to keep alive the knowledge of how to work with these technologies since I stopped doing it professionally once I became a software engineer.
Also: I’m not too happy w the idea of the software getting old and compromised and adding to the zombie hordes of botnets or letting the Chinese or American governments decide what’s best for my home network.
That’s fair. My UniFi gear has added that in recent updates, though that’s an investment. If your system works for you, that great; stick with it!
But, I would try to find an alternative to Windows 10. Paying for ESU’s would be better spent getting something else. What that might be, I’m not sure.
What’s an ESU?
I wish I could get rid of the windows VM but doing so would slow my Wi-Fi speeds below 100 megabits since Intel won’t allow the Linux driver to do the same in AP mode.
I like unifi; I would probably be using them if I didn’t hate the idea of throwing away perfectly good equipment.
I spent a little over ten years in IT and it always saddened me to witness and commit the staggering volume of wastefullness of all of it, so try not to now
Extended Security Updates.
I agree with you on throwing out perfectly good hardware. Either you hang on it until it’s useless, or you throw it on eBay and let someone else have it.
deleted by creator
