Have suggestions for a "roll your own file server"?

Wed Mar 10 14:04:29 EST 2021

And make sure you get alerts when something fails.

There are a number of commercial NAS like Synology out there if you don't
want to build it yourself.

I'd also add some kind of offsite storage.  If you have decent internet
speed, cloud storage is a good choice.

Something with error correction is also good.

What I do:
I have the OS on its own disk.  I've done RAID in the past, but I'm ok with
rebuilding the OS.  A failed drive == forced upgrade.  For home, I'm ok
with this for heat/electricity costs.

For data, I've been using ZFS on Solaris -> OpenSolaris -> Ubuntu,  I think
15+ years.  I'm very happy with it.
I've used as low as 1GB, but more is better.  Dual core with 4GB is enough
for a home system IMO
If you've ever used a NetApp, it is similar.
It has snapshots that can persist forever.  My system takes one every hour,
day, week, month and keeps them for some multiple of that time.
You can divide the pool into "partitions" with changing size limits
(quota).
RAID is part of the FS.  mirrors, stripes and a raid5 like raidz.  Also
variants that can survive the failure of 2 drives ( raidz2 ~ raid6 and
triple mirrors)
Error detection.  If there is an error, the FS stops writing to prevent
more.  It's a bad idea to run ZFS on single drives.
If you have RAID, the FS has another copy to reference and can self heal
the error.
It has compression built in with negligible performance hit.
It has dedup.  Don't use it IMO.  Maybe if you have infinite RAM.

You can't expand the pool by adding disks.  So a 3 disk RAIDZ can't be
expanded by adding a 4th.
You can replace/rebuild disks with larger disks to increase the pool size.
It works well.

I create mirrors.  When I need more space, I add another mirror with newer
drives.
Fewer spinning drives means less electricity used if that's a big issue.

The family uses samba, NFS or the web server to get to data.  SFTP also
works.  They don't have to care about the disks underneath unless they fill
up the share.  Then it's a quick quota change and ask them to pare down how
many copies of that CD they really need.

In the past I used Crashplan for offsite.  They no longer offer the
unlimited systems for personal use, but I think a single system with
unlimited space is available.

Snapshots cover: Oops!  I deleted it by accident, can I get it back?  Well,
except for deleting a "partition" or pool.
Off site covers:  Oops! The house burned down, where are the family photos.

On Mon, Mar 8, 2021 at 10:00 PM <jonhall80 at comcast.net> wrote:

> I forgot one thing:
>
> Set up a shell script to do a simple diagnostic on both systems to detect
> a failed or failing system.  Run two or three times a day.
>
> md
>
> On 03/08/2021 9:56 PM jonhall80 at comcast.net wrote:
>
>
> I will suggest something and let people rip it apart:
>
> Get two RPis that have at least USB 2.0  Attach two large capacity disks
> to each one in a RAID-1 configuration (also known as "mirroring") to keep
> it simple.  If one disk fails the other will still keep working (but you
> should replace it as soon as possible).
>
> Put all of your data on both systems.
>
> Take one of your systems to a friends or relatives house who you trust
> that has relatively good WiFi.  Make sure the friend is relatively close,
> but is not in the same flood plain or fire area you are.
>
> Do an rsync every night to keep them in sync.
>
> Help your friend/relative do the same thing, keeping a copy of their data
> in your house.   If your disks are big enough you could share systems and
> disks.
>
> Use encryption as you wish.
>
> Disk failure?   Replace the disk and the data will be replicated.
> Fire, theft, earthquake?   Take the replaced system over to your
> friends/relatives and copy the data at high speed, then take the copied
> system back to your house and start using it again.
>
> You would need three disks to fail at relatively the same time to lose
> your data.   Or an asteroid crashing that wipes out all life on the
> planet.  Unlikely.
>
> Realize that nothing is forever.
>
> md
>
> On 03/08/2021 7:33 PM Bruce Labitt <bdlabitt at gmail.com> wrote:
>
>
> For the second time in 3 months I have had a computer failure.  Oddly, it
> was a PS on the motherboard both times.  (Two different MB's.)  Fortunately
> the disks were ok.  I'm living on borrowed time.  Next time, I may not be
> that lucky.
>
> Need a file server system with some sort of RAID redundancy.  I want to
> backup 2 main computers, plus photos.  Maybe this RPI4 too, since that's
> what I'm running on, due to the second failure.  If this SSD goes, I'm
> gonna be a sad puppy.  This is for home use, so we are not talking
> Exabytes.  I'm thinking about 2-4TB of RAID.  Unless of course, RAID is
> obsolete these days.  Honestly, I find some of the levels of RAID
> confusing.  I want something that will survive a disk failure (or two) out
> of the array.  Have any ideas, or can you point me to some place that
> discusses this somewhat intelligently?
>
> Are there reasonable systems that one can put together oneself these
> days?  Can I repurpose an older PC for this purpose?  Or an RPI4?  What are
> the gotchas of going this way?
>
> I want to be able to set up a daily rsync or equivalent so we will lose as
> little as possible.  At the moment, I'm not thinking about surviving fire
> or disaster.  Maybe I should, but I suspect the costs balloon
> considerably.  I do not want to backup to the cloud because, plain and
> simple, I don't trust it to be fully secure.
>
> Thanks for any and all suggestions.
> _______________________________________________
> gnhlug-discuss mailing list
> gnhlug-discuss at mail.gnhlug.org
> http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
>
> _______________________________________________
> gnhlug-discuss mailing list
> gnhlug-discuss at mail.gnhlug.org
> http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
>
> _______________________________________________
> gnhlug-discuss mailing list
> gnhlug-discuss at mail.gnhlug.org
> http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.gnhlug.org/pipermail/gnhlug-discuss/attachments/20210310/56c1924e/attachment-0001.html 