Have suggestions for a "roll your own file server"?

Bruce Labitt bdlabitt at gmail.com
Wed Mar 10 20:49:00 EST 2021


I'll take a look at that.  Thanks for the link.

On Wed, Mar 10, 2021 at 8:15 PM Marc Nozell (marc at nozell.com) <
nozell at gmail.com> wrote:

> Just to put a plug in for a colleague's work:
> https://perfectmediaserver.com/    It covers everything from disk
> purchasing strategies, burn-in, filesystems (ZFS, SnapRAID, etc).
>
> He also hosts a podcast that folks here may find interesting:
> https://selfhosted.show/
>
> -marc
>
> On Wed, Mar 10, 2021 at 8:08 PM <jonhall80 at comcast.net> wrote:
>
>> OK:
>>
>> s/RPi4/some-other-cheap-computer-with-USB-3.x>/g
>>
>> Unless you build multiple Ethernet or WiFi or LTE modem connections your
>> networking will still be the slowest thing.
>>
>> You do not need huge amounts of CPU power, or huge amounts of RAM.
>>
>> My basic point is that if you stick with simple RAID (like mirroring) but
>> also set up a unit that is remote from your own home you could protect your
>> own data from fire, flood and theft to a reasonable level and even protect
>> your friend's data by backing up their data to your device.
>>
>> Add snapshots as suggested by Tom Buskey,perhaps encryption of file
>> systems and data-streams and you can have a rather simple, server where you
>> learn a lot by planning it out and setting it up rather than buying an "off
>> the shelf" solution or simply using a "web backup".
>>
>> And good catch on the USB power supply.
>>
>> md
>> > On 03/10/2021 6:53 PM Joshua Judson Rosen <rozzin at hackerposse.com>
>> wrote:
>> >
>> >
>> > I'm not sure about the Raspberry Pi 4, but up thru the raspi 3+ there
>> are... problems, e.g.:
>> >
>> > Beware of USB on the raspi: there are some bugs in the silicon that
>> pretty severely
>> > cripple performance when multiple `bulk' devices are used at
>> simultaneously,
>> > sometimes to the point of making it unusable (e.g. if you want to use a
>> better Wi-Fi
>> > adapter/antenna than the one built onto the board, and connect an LTE
>> modem so that
>> > your raspi roam onto that if Wi-Fi becomes unavailable, throughput on
>> whichever of those
>> > interfaces you're actually using can become abysmal). IIRC the issue is
>> basically
>> > that the number of USB endpoints that can be assigned interrupts by the
>> raspi controller
>> > is _incredibly small_; and it's common for high-throughput devices to
>> have multiple endpoints per device--
>> > sometimes even one USB device will have more endpoints that the raspi
>> USB controller can handle.
>> >
>> > Also, `network fileserver with USB-attached hard drives' is kind of the
>> `peak unfitness'
>> > for the raspberry pi. Specifically if you've got it attached to
>> ethernet,
>> > the ethernet is attached through the same slow-ish USB bus as your HDDs.
>> >
>> > (the onboard Wi-Fi BTW is SDIO; so if you avoid using the onboard
>> Wi-Fi, I guess you might also
>> >  be able to make your µSD card faster...)
>> >
>> > ALSO: you'll really want to use an externally-powered USB hub for USB
>> devices
>> > that are not totally trivial, because the raspi's µUSB power supply is
>> already
>> > strained... (and if you're trying to power your raspi from some random
>> USB power supply,
>> > don't. Ideally you power it through the 5V pins on the expansion
>> header...).
>> >
>> >
>> > Though there is a lot of neat stuff that can be done with a Raspberry
>> Pi,
>> > it's really easy to overestimate it.
>> >
>> > But on the other hand: YMMV, and there are scenarios where the issues
>> don't matter,
>> > and might not even be noticeable. e.g., if you're dumping periodic
>> backups to your
>> > raspi asynchronously instead of (say) NFS mounting it and trying to use
>> it interactively,
>> > you might not even notice the weird bottlenecks because you're never
>> looking at them.
>> > And if you have enough of them as spares running simultaneously, you
>> may not care
>> > that every once in a while your fileystems get corrupted or your USB
>> ports stop working
>> > or whatever.
>> >
>> >
>> > On 3/8/21 9:56 PM, jonhall80 at comcast.net wrote:
>> > > I will suggest something and let people rip it apart:
>> > >
>> > > Get two RPis that have at least USB 2.0  Attach two large capacity
>> disks to each one in a RAID-1 configuration (also known as "mirroring") to
>> keep it simple.  If one disk fails the other will still keep working (but
>> you should replace it as soon as possible).
>> > >
>> > > Put all of your data on both systems.
>> > >
>> > > Take one of your systems to a friends or relatives house who you
>> trust that has relatively good WiFi.  Make sure the friend is relatively
>> close, but is not in the same flood plain or fire area you are.
>> > >
>> > > Do an rsync every night to keep them in sync.
>> > >
>> > > Help your friend/relative do the same thing, keeping a copy of their
>> data in your house.   If your disks are big enough you could share systems
>> and disks.
>> > >
>> > > Use encryption as you wish.
>> > >
>> > > Disk failure?   Replace the disk and the data will be replicated.
>> > > Fire, theft, earthquake?   Take the replaced system over to your
>> friends/relatives and copy the data at high speed, then take the copied
>> system back to your house and start using it again.
>> > >
>> > > You would need three disks to fail at relatively the same time to
>> lose your data.   Or an asteroid crashing that wipes out all life on the
>> planet.  Unlikely.
>> > >
>> > > Realize that nothing is forever.
>> > >
>> > > md
>> > >> On 03/08/2021 7:33 PM Bruce Labitt <bdlabitt at gmail.com> wrote:
>> > >>
>> > >>
>> > >> For the second time in 3 months I have had a computer failure.
>> Oddly, it was a PS on the motherboard both times.  (Two different MB's.)
>> Fortunately the disks were ok.  I'm living on borrowed time.  Next time, I
>> may not be that lucky.
>> > >>
>> > >> Need a file server system with some sort of RAID redundancy.  I want
>> to backup 2 main computers, plus photos.  Maybe this RPI4 too, since that's
>> what I'm running on, due to the second failure.  If this SSD goes, I'm
>> gonna be a sad puppy.  This is for home use, so we are not talking
>> Exabytes.  I'm thinking about 2-4TB of RAID.  Unless of course, RAID is
>> obsolete these days.  Honestly, I find some of the levels of RAID
>> confusing.  I want something that will survive a disk
>> > >> failure (or two) out of the array.  Have any ideas, or can you point
>> me to some place that discusses this somewhat intelligently?
>> > >>
>> > >> Are there reasonable systems that one can put together oneself these
>> days?  Can I repurpose an older PC for this purpose?  Or an RPI4?  What are
>> the gotchas of going this way?
>> > >>
>> > >> I want to be able to set up a daily rsync or equivalent so we will
>> lose as little as possible.  At the moment, I'm not thinking about
>> surviving fire or disaster.  Maybe I should, but I suspect the costs
>> balloon considerably.  I do not want to backup to the cloud because, plain
>> and simple, I don't trust it to be fully secure.
>> >
>> > --
>> > Connect with me on the GNU social network! <
>> https://status.hackerposse.com/rozzin>
>> > Not on the network? Ask me for more info!
>> > _______________________________________________
>> > gnhlug-discuss mailing list
>> > gnhlug-discuss at mail.gnhlug.org
>> > http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
>>
>> _______________________________________________
>> gnhlug-discuss mailing list
>> gnhlug-discuss at mail.gnhlug.org
>> http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
>>
>
>
> --
> Marc Nozell (marc at nozell.com) http://www.nozell.com/blog
> _______________________________________________
> gnhlug-discuss mailing list
> gnhlug-discuss at mail.gnhlug.org
> http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.gnhlug.org/pipermail/gnhlug-discuss/attachments/20210310/3a862136/attachment.html 


More information about the gnhlug-discuss mailing list