Opened 8 years ago

Closed 19 months ago

#7521 closed Question/How do I...? (fixed)

need advice on LAN file server setup

Reported by: HistoriCUSS Owned by: Ross
Priority: Medium Component: Tech
Keywords: LAN FileServe samba Cc: historiCUSS
Sensitive: no

Description (last modified by HistoriCUSS)

Backing up a little...

Over the years I have accumulated TBs of files, text documents, audios, videos, photos. Currently it is 'decentralized', on numerous computers (primarily windows OS), external hard drives, CDs/DVDs, flash drives, even floppies, often duplicated perhaps multiple instances! It is unwieldy to say the least.

Aside from being a resource to me personally, I put a small portion of my library online via torrents, the demand for which maxes out my 'meager' 2mb/s UL bandwidth, resulting in about 300 GB of files served per month. This is limited by the nature of most bittorrent clients, which can't handle seeding thousands of torrents. Once my files are organized, I envision making custom torrents for collections, so that within 1 torrent there may be a thousand topically related files. With that strategy, seeding 200 such torrents could make over 100,000 files available.

Another concern is media creation, primarily hidef video. Seems Apple's Final Cut Pro is the dominant ap for that. Pre-Intel PowerMacs are cost effective for this. There are also multiple windows based video/audio editing programs too. And maybe one day there will be comparable FLOSS aps for linux. Until then, having a linux file server to handle the media for such projects seems to be a good start, with samba being the inclusionary element to integrate windows and OSx workstations along with those running linux.

My personal fileserve project will serve as a testbed of sorts. I am working with an activist-artists collective whose focus is on performing and producing hiphop media (audio and video). We are discussing how to implement a creative media center to facilitate their projects. I work with miniDV tape cameras and also the newer compressed formats, but prefer capturing/recording in uncompressed or raw modes, which than later be edited and encoded with less conversion loss. That obviously will require LOTS of disk space.

So the basic outline is to build a LAN domain controller/file server that can accommodate the NAS needs of linux, windows and OSx workstations. Samba 4.0 looks good to me. WAN access to files, either through torrents or FTP might also be implemented on this server or another that is able to share the files on it.

That said...

Having finally procured the needed materials, I am ready to configure my file server.

My plan is to initially use 2 4TB drives (non-raid) for storing the media/data files. Once my current files are consolidated and de-duplicated, I am hoping 8TB will suffice to provide a little space for future media creation projects.

I have installed and configured debian several times for testing and learning but recognize I do not yet comprehend all of the dos and don'ts. I am in the process of playing with the basic configuration, using 2 80GB drives. The encryption process is time consuming. I can't imagine how many days it will take to initialize 2 4TB drives.

Every attempt I have made using the debian DVD installs disks has failed. Seems I can install from the CD but not from the DVDs. I haven't tried the v7.1 DVDs yet but trying earlier versions on different HW platforms never succeeded for me.

Given that the install process using huge drives with encryption is so time consuming, I'd like to make sure I am creating the right/best partition scheme, hence my verbose background/purpose details above.

What would be the best way to configure the partitions, assuming there would be a small boot partition then the rest encrypted ala mayfirst style? Where does swap fit in the mix?

I have reviewed the Mayfirst server configuration guide. That focuses on most command line configuration, with very little explanation of how to use debian's graphical install options. I have worked with some command line but like the GUI. I am more than willing to make screenshots along the way to help create a GUI tutorial that documents the configuration process.

Eventually I'd like to use the ZFS file system as it appears to be the most advanced, most reliable. While debian kFreeBSD supports it, ZoL (ZFS on Linux) is still a work in progress, not yet incorporated into the official distro package.

Where to begin?

What partitions are needed, desired? Which partitions need to be primary? What partitions need to be mirrored? (boot and OS?) If encryption may be desired, can it be done at a later time? Or does that process clear all data and prep all the space for reloading the files?

My thought is not to RAID the data partitions, at least initially. I am not sure how many TB I will need for consolidation. (Whatever data is on these 2 drives will be backed up to either external drives or another (RAID6?) backup server.)

Change History (11)

comment:1 Changed 8 years ago by HistoriCUSS

Description: modified (diff)

comment:2 Changed 8 years ago by Dana

Keywords: samba added
Owner: set to Ross
Status: newassigned

Hi historicuss,

Here's a document Jamie created that could be helpful:

I also did a quick search & found these links:

Ross, do you have any advice on this?


comment:3 Changed 8 years ago by Ross

I think those are pretty good examples. historicuss is Glen from Detroit, we discussed this on irc last night a little bit. I think he's decided to go with the 4TB drives configured with lvm, which is what we generally recommend.


Last edited 8 years ago by Ross (previous) (diff)

comment:4 Changed 8 years ago by HistoriCUSS

Description: modified (diff)

comment:5 Changed 7 years ago by HistoriCUSS

Cc: historiCUSS added

comment:6 Changed 7 years ago by HistoriCUSS

This project got pushed to the back burner but I'd like to get it going soon.

Here's what I am working with for use as a media center/torrent file server: MB: ASUS E35-M1M Pro (low power, 5 int SATA + 1 eSATA 6Gb/s ports, USB 3.0) 8 GB RAM add-on: PCI-e x1 card for 2 more internal SATA ports and 2 eSATA Total available SATA ports: 7 internal, 3 external

Right now, I have debian 7.4 with KDE installed on an SSD on 1 port 6 x 2TB drives on the remaining internal ports, ready to be aligned, partitioned, and formatted to RAIDS DVD is currently cabled to an external eSATA port.

Once configured, I think SAMBA, NFS (if needed for linux clients), and transmission, deluge or Tixati for torrent seeding will be the 24/7 applications.

After lots of reading and some advice, I have abandoned the idea of using ZFS for now (seems a MB with ECC RAM is VERY recommended) and will probably use ext2 or ext4 for the data partition.

My thoughts right now are to setup partitions very similar to what I think mayfirst uses as described at

Is that information still current practice?

As an aside, my drives are mix and match, bought at different times and althoug all are labeled as Sata 6Gb/s, my preliminary benchmark testing indicates a rather large variance in read and write performance. (Toshiba was tops, over WD, Hitachi and Seagate)

I tried to partition with disk utility but always got that dreaded "This partition is not aligned" warning. Same with the gparted GUI. So looks like taking control with the command line it the route to use to get the drive alignment right.

Before that though, I need to decide what to do with the OS. Right now, as I said, it is installed on an SSD. I am noting that the mayfirst servers use a highly redundant RAID 1 across 4 drives, so the system could boot from any drive as I read it correct? So, should I copy the debian install to the drives or keep it as is on the SSD?

My thinking is to make 3 partitions on all 6 drives (sda,b,c,d,f,g) as follows, based on the standard install doc above, perhaps making the OS partition larger?): parted /dev/sdx -- mklabel gpt parted /dev/sdx -- unit s mkpart biosboot 8192 16383 parted /dev/sdx -- set 1 bios_grub on parted /dev/sdx -- unit s mkpart boot 16384 1015807 (make bigger?) parted /dev/sdx -- set 2 raid on parted /dev/sdx -- unit s mkpart pv 1015808 -1 parted /dev/sdx -- set 3 raid on

At the next point, I am a bit confused on the example of making RAIDs on sittingbull

I see sdx2 partitions formed as md0 and sdx3 partitions as md1. What happened to the sdx1 partitions used for grub? Aren't they also combined as a RAID1?

I was thinking to mirror the little grub bios partition across all 6 drives: mdadm --create --raid-devices=1 --level=1 --metadata=1.0 --verbose /dev/md0 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sdf1 /dev/sdg1

then, make 3 pairs of striped raids for the OS mdadm --create --raid-devices=2 --level=0 --metadata=1.0 --verbose /dev/md1 /dev/sda2 /dev/sdb2 mdadm --create --raid-devices=2 --level=0 --metadata=1.0 --verbose /dev/md2 /dev/sdc2 /dev/sdd2 mdadm --create --raid-devices=2 --level=0 --metadata=1.0 --verbose /dev/md3 /dev/sdf2 /dev/sdg2

mirror these 3 raid 0 pairs: mdadm --create --raid-devices=3 --level=1 --metadata=1.0 --verbose /dev/md4 /dev/md1 /dev/md2 /dev/md3

Provide some redundancy for the data partition, either RAID 5, 6 or 10, resulting in 10TB, 8TB or 6TB available respectively: mdadm --create --raid-devices=6 --level=6 --metadata=1.0 --verbose /dev/md5 /dev/sda3 /dev/sdb3 /dev/sdc3 /dev/sdd3 /dev/sdf3 /dev/sdg3

Again, my quandaries: 1] Should I keep the OS on the SSD or install it on the HDDs?

If the latter: 2] Is the 1st partition for grub to be mirrored across all 6 drives? (confused as that was not mentioned on the server set-up guide) 3] is it better to have the 2nd partition for the OS also mirrored across all 6 drives or might there be some advantage to mirroring 3 striped pairs? 4] I am leaning towards RAID 5 for the data partition, obviously for maximum space, gaining 2-4TB over RAID 10 and RAID 6 for this topolopgy. Your thoughts?

comment:7 Changed 7 years ago by HistoriCUSS

As to my questions 1 and 3, my further experiments revealed that a RAID 1 made of partitions on all 6 HDD is slightly faster read time (253 MB/s) compared to the SATA2 SSD (243 MB/s). (For reference, my tests on a SATA3 SSD showed it about 10% faster than this RAID1, at 277 MB/s). I'll opt for installing the OS on the HDD RAID instead of the SSD.

As to question 2, I'm still confused as to whether or not the bios_grub partition is also a RAID 1 across the 6 drives. Disk Utility can't be right benchmarking this 4MB partition. It is reporting a peak read time of over 300 GB/s, with an average of 37 GB/s.

Question 4 is still an open book.

Last edited 7 years ago by HistoriCUSS (previous) (diff)

comment:8 Changed 21 months ago by updater

Sensitive: set

Changed to sensitive as part of leadership decision to make all tickets sensitive.

comment:9 Changed 20 months ago by HistoriCUSS

Sensitive: unset

comment:10 in reply to:  7 Changed 20 months ago by JaimeV

Resolution: fixed
Status: assignedfeedback

This is really old so you may not even be using this setup anymore but...

As to question 2, I'm still confused as to whether or not the bios_grub partition is also a RAID 1 across the 6 drives.

You want the bios_grub partition on all drives but you don't have to compose a raid array with them

Question 4 is still an open book.

You can find lots of debates online about RAID5 or 6. Personally I would just stick with RAID1 and jump up to RAID10 if you need better RW speed. ZFS is really cool but is something new to get your brain around, lots of advances in ZFS for linux but in theory one of the *bsd is still the optimal system to use it on. See projects like FreeNAS for good support.

comment:11 Changed 19 months ago by automatic

Status: feedbackclosed

No news is good news (we hope)! Given the lack of feedback, we think this ticket can be closed.

Please login to add comments to this ticket.

Note: See TracTickets for help on using tickets.