Have suggestions for a "roll your own file server"?

[jonhall80 at comcast.net]

OK:

s/RPi4/some-other-cheap-computer-with-USB-3.x>/g

Unless you build multiple Ethernet or WiFi or LTE modem connections your networking will still be the slowest thing.

You do not need huge amounts of CPU power, or huge amounts of RAM.

My basic point is that if you stick with simple RAID (like mirroring) but also set up a unit that is remote from your own home you could protect your own data from fire, flood and theft to a reasonable level and even protect your friend's data by backing up their data to your device.

Add snapshots as suggested by Tom Buskey,perhaps encryption of file systems and data-streams and you can have a rather simple, server where you learn a lot by planning it out and setting it up rather than buying an "off the shelf" solution or simply using a "web backup".

And good catch on the USB power supply.

md
> On 03/10/2021 6:53 PM Joshua Judson Rosen <rozzin at hackerposse.com> wrote:
> 
>  
> I'm not sure about the Raspberry Pi 4, but up thru the raspi 3+ there are... problems, e.g.:
> 
> Beware of USB on the raspi: there are some bugs in the silicon that pretty severely
> cripple performance when multiple `bulk' devices are used at simultaneously,
> sometimes to the point of making it unusable (e.g. if you want to use a better Wi-Fi
> adapter/antenna than the one built onto the board, and connect an LTE modem so that
> your raspi roam onto that if Wi-Fi becomes unavailable, throughput on whichever of those
> interfaces you're actually using can become abysmal). IIRC the issue is basically
> that the number of USB endpoints that can be assigned interrupts by the raspi controller
> is _incredibly small_; and it's common for high-throughput devices to have multiple endpoints per device--
> sometimes even one USB device will have more endpoints that the raspi USB controller can handle.
> 
> Also, `network fileserver with USB-attached hard drives' is kind of the `peak unfitness'
> for the raspberry pi. Specifically if you've got it attached to ethernet,
> the ethernet is attached through the same slow-ish USB bus as your HDDs.
> 
> (the onboard Wi-Fi BTW is SDIO; so if you avoid using the onboard Wi-Fi, I guess you might also
>  be able to make your µSD card faster...)
> 
> ALSO: you'll really want to use an externally-powered USB hub for USB devices
> that are not totally trivial, because the raspi's µUSB power supply is already
> strained... (and if you're trying to power your raspi from some random USB power supply,
> don't. Ideally you power it through the 5V pins on the expansion header...).
> 
> 
> Though there is a lot of neat stuff that can be done with a Raspberry Pi,
> it's really easy to overestimate it.
> 
> But on the other hand: YMMV, and there are scenarios where the issues don't matter,
> and might not even be noticeable. e.g., if you're dumping periodic backups to your
> raspi asynchronously instead of (say) NFS mounting it and trying to use it interactively,
> you might not even notice the weird bottlenecks because you're never looking at them.
> And if you have enough of them as spares running simultaneously, you may not care
> that every once in a while your fileystems get corrupted or your USB ports stop working
> or whatever.
> 
> 
> On 3/8/21 9:56 PM, jonhall80 at comcast.net wrote:
> > I will suggest something and let people rip it apart:
> > 
> > Get two RPis that have at least USB 2.0  Attach two large capacity disks to each one in a RAID-1 configuration (also known as "mirroring") to keep it simple.  If one disk fails the other will still keep working (but you should replace it as soon as possible).
> > 
> > Put all of your data on both systems.
> > 
> > Take one of your systems to a friends or relatives house who you trust that has relatively good WiFi.  Make sure the friend is relatively close, but is not in the same flood plain or fire area you are.
> > 
> > Do an rsync every night to keep them in sync.
> > 
> > Help your friend/relative do the same thing, keeping a copy of their data in your house.   If your disks are big enough you could share systems and disks.
> > 
> > Use encryption as you wish.
> > 
> > Disk failure?   Replace the disk and the data will be replicated.
> > Fire, theft, earthquake?   Take the replaced system over to your friends/relatives and copy the data at high speed, then take the copied system back to your house and start using it again.
> > 
> > You would need three disks to fail at relatively the same time to lose your data.   Or an asteroid crashing that wipes out all life on the planet.  Unlikely.
> > 
> > Realize that nothing is forever.
> > 
> > md
> >> On 03/08/2021 7:33 PM Bruce Labitt <bdlabitt at gmail.com> wrote:
> >>  
> >>  
> >> For the second time in 3 months I have had a computer failure.  Oddly, it was a PS on the motherboard both times.  (Two different MB's.)  Fortunately the disks were ok.  I'm living on borrowed time.  Next time, I may not be that lucky.  
> >>  
> >> Need a file server system with some sort of RAID redundancy.  I want to backup 2 main computers, plus photos.  Maybe this RPI4 too, since that's what I'm running on, due to the second failure.  If this SSD goes, I'm gonna be a sad puppy.  This is for home use, so we are not talking Exabytes.  I'm thinking about 2-4TB of RAID.  Unless of course, RAID is obsolete these days.  Honestly, I find some of the levels of RAID confusing.  I want something that will survive a disk
> >> failure (or two) out of the array.  Have any ideas, or can you point me to some place that discusses this somewhat intelligently?
> >>  
> >> Are there reasonable systems that one can put together oneself these days?  Can I repurpose an older PC for this purpose?  Or an RPI4?  What are the gotchas of going this way?
> >>  
> >> I want to be able to set up a daily rsync or equivalent so we will lose as little as possible.  At the moment, I'm not thinking about surviving fire or disaster.  Maybe I should, but I suspect the costs balloon considerably.  I do not want to backup to the cloud because, plain and simple, I don't trust it to be fully secure.
> 
> -- 
> Connect with me on the GNU social network! <https://status.hackerposse.com/rozzin>
> Not on the network? Ask me for more info!
> _______________________________________________
> gnhlug-discuss mailing list
> gnhlug-discuss at mail.gnhlug.org
> http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/