HP Proliant G7 N54L Freenas 9.3 NAS Build Log

NAS Server build LOG ( It is still improving, many errors)

For my own reference

Start time: Jan 04, 2015

Project completion time: Jan 28, 2015

0.  Introduction

  • Planning:

Pure central storage server

  • Requirements:

Minimum Cost, Reliable Hardware, Data Redundancy, Fast & stable local file sharing, anywhere data accessing; the more storage space, the better.

1. Hardware selections:

1.1 Server:

  • Model: HP Proliant MicroServer G7 744900-001 4-HDD Bay
  • CPU: AMD Turion II Neo N54L CPU, 25W TDP

Similar performance CPU for comparsion -> (Intel Core i5 520M)

  • Build-in RAM: SKhynix 4GB, 1333MHz, PC3-10600E-9, DDR3, dual-rank x8, 1.50V, unbuffered with ECC dual in-line memory module (UDIMM)

Model: HMT351U7EFR8C-H9 T0 AF

Part Number: 500210-171

  • NIC: Broadcom NC107i Gbps
  • Reason: $199, HP Warranty, Hard drive bays, Sever grade hardware & design

Dual core CPU satisfies home NAS requirements

2.2Ghz is good enough for single thread CIFS share

Two slot ECC RAM, expandable to 16G, important for ZFS file system

Direct USB port on the motherboard, convenient for OS USB drive.

1.2. Additional Hardware:

RAM:

  • Crucial CT102472BA160B 8GB 240-Pin Unbuffered DIMM Memory Module PC3-12800 1.5V CAS 11 $90

Reason:

Total 12Gb ECC RAM, In the future could replace the system build-in 4Gb RAM to 8GB as well.

Capacity and ECC is more important than speed, 12TB ZFS RAID-Z1 will have 9TB storage space, according to ZFS, 1TB roughly requires 1G RAM

HDD:

  • WD 3TB Network HDD Retail Kit (WD30EFRX, Red Drive) x 4

WDBMMA0030HNC-NRSN   $109 each

Reason:

7×24 NAS drive, reliable

USB Drive:

  • 2x 16Gb USB Drives for OS

Since OS is installed on USB drive, two of them will make a mirror, if any of them fails, system won’t have trouble. Freenas 9.3 supports multiple boot environments and mirror boot device, 16Gb is preferred over 8Gb.

1.3 Sections summary

Energy efficient, All server-grade hardware, more than enough storage space for now.

Later on test shows SMB shares have 110Mb/s reads and 90-95Mb/s writes w/ 20 meters CAT5E, 2 level Gb routers, windows 10 client, SMB Ver. 3.0 @Server side

The speed is very consistent for files size up to 50Gb ISO image, which is good enough for home environment. CPU temp ranging from 40C to 65C w/ room temp 60-75F; HDD temp around 30-35C

2. Installation:

2.1. Planning:

  • OS in mind:

Freenas, Nas4free, VMware ESXi+Freenas, Windows Server+HyperV+Freenas

Decision: FreeNAS-9.3-STABLE-201501212031

  • Reason:

The hardware is not powerful enough to be a decent Virtual Machine server, since the server is purely used as NAS anyway, just choose Freenas directly on top of the hardware, which is also the suggested way by Freeenas project.

2.2 RAID planning:

Since ZFS is enterprise level solution, many things have be to carefully learned and planned ahead.

Terminology MUST be learned:

ZFS, pool, zdev, zvol, raid-z1, z2, z3, mirror, ARC, L2ARC, ZIL, snapshot

The most helpful tutorial: (will take around 1-2 hrs to read for noobs)

https://forums.freenas.org/index.php?threads/slideshow-explaining-vdev-zpool-zil-and-l2arc-for-noobs.7775/

http://doc.freenas.org/9.3/freenas.html

 

2.3. Decision:

RAID-Z1 (Similar to RAID-5), since it will provide some level of redundancy while providing Maximum storage space. Meanwhile, it is unlikely 2 drives fails at the same time in Home environment.

Advantages:

  • Zpool can be expanded in the future by swapping to high capacity hard drives in the future w/ ‘zpool expansion’ function.
  • Zpool provides several useful functions such as: scrub, snapshots

Attentions:

  • zpool can ADD more zdev, but CAN’T Remove any zdev. Any failure of zdev will resulting in completely loss of data.
  • RAID must be planned ahead of time, once it is set, the number of hard drives Can’t be changed. If add an additional HDD to the zpool, it will not be added to the raid array, rather it is a strip, which could be a single failure point in the future. (If this single HDD fails, entire zpool will lost).
  • Home server most likely won’t need SSD as ARC, L2ARC, ZIL cache.

2.4. Installation

2.4.1. Install HWs:

  1. Open the door of the server, unscrew the screw at the back(on the top) to remove the top panel
  2. Remove the door. loose two Blue screws on the motherboard, disconnect all the wires and pull the motherboard tray out
  • Install the RAM on the empty slot
  1. Use the tool and screws on the Door to install hard drives
  2. The door can be disassembled to add a thin dust filter

A good tutorial video: https://www.youtube.com/watch?v=EgPriN_oKos

2.4.2. HP N54L mod:

A comprehensive guide to HP Gen 7 mod:

http://homeservershow.com/forums/index.php?/topic/5190-microserver-hardware-links/

2.4.2.1. BIOS mod

Original BIOS lacks of some functions, flash a modded BIOS can add the following functions: enable the 5th data port on the motherboard to data 3Gb/s, enable hot swap (might still have problem, I added HDD to 4th cage while system on, 3rd HDD disconnected)

Tutorial:

Follow this post:

http://terfmop.co.uk/blog/2013/07/31/hp-proliant-n54l-bios-modification-guide-allow-hot-plug-sata-and-5th-sata-port/

My Procedures:

http://youtil.tistory.com/attachment/cfile1.uf@2752B63F53B427202EC0BB.zip

  • Replace the rom file in the flash drive with the mod rom flash
  • Boot into N54, the screen should display ‘updating blocks’ when flashing
  • After that, boot into BIOS

From the main screen go to ‘Chipset > Southbridge Configuration > SB SATA Configuration’ and make sure your settings are the same as below:

  • OnChip SATA Channel = Enabled
  • OnChip IDE Type = IDE
  • SATA IDE Combined Mode = Disabled
  • SATA EPS on all PORT = Enabled
  • SATA Power on all PORT = Enabled
  • Return to the main screen then go to ‘Advanced > IDE Configuration’ and again, make sure your settings are the same as below.
  • Embedded SATA Link Rate = 3.0Gbps MAX

2.4.2.2 Fan mod:

The server’s noise mainly came from the PSU. One guide to mod that is here:

http://jeffgraves.me/2013/10/29/replace-noisy-power-supply-fan-in-hp-proliant-gen7-microserver/

I think if it is really noisy, it’s better to replace the entire PSU, it won’t cost too much.

 

3. Server Configuration

Main Sources:

http://doc.freenas.org/9.3/freenas.html

https://forums.freenas.org/index.php?forums/how-to-guides.13/

https://www.youtube.com/playlist?list=PLgKBSCAcGA57juSDNE09bc7Ub491DIFVj

Useful to know: Basic BSD commands, understand ‘Cron’ ‘Jail’, Zpool Commands, Tuntable

3.1 Wizard

  • Freenas 9.3 come with a setup wizard, follow the wizard to create Directory Service, NFS shares, AFP shares, AFP Time Machine Shares. Will config the CIFS shares manually.
  • Make sure to give the server some name, which will be useful for later access and identification.

 

3.2 Detail configurations

 

1). Setup up root account email:

System -> Email

Here’s settings for Gmail account:

  • Outgoing server: smtp.googlemail.com
  • Port: 465
  • Select SSL
  • Use SMTP
  • Take a look at BSD system’s ‘sendmail’ command.

https://www.freebsd.org/doc/handbook/sendmail.html

2). User accounts:

Freenas added a feature to support Microsoft Account 12/30/2014:

Windows 8,8.1,10 use Microsoft account email address as Login username, creating problems to access NAS. Solution is here:

https://bugs.freenas.org/issues/7340

When creating the user account, add the E-Mail address match user’s Microsoft account mail address and enter the same password, mark ‘Microsoft Account’, So Freenas will create a mapping from that email address to user’s username internally.

* If the user’s Auxiliary groups has ‘wheel’, it will obtain some super privileges. (Entering any user’s folder and make change; When FTP sets ‘always chroot’, wheel group can bypass that restriction)

See: http://en.wikipedia.org/wiki/Wheel_(Unix_term)

3). SSH service

Enable it in the ‘Services’

On Mac or other Unix machines:

ssh Username@ServerIP -p Portnumber

4). WebUI Secure http (https) access:

  • Config Secure connection access to the WebUI:

System -> CAs-> Create Internal CA (This will create an internal Certificate Authority)

  • After that:

System -> Certificates -> Create Internal Certificate

* Attention:

  • when create certificates, using legit names and email address, otherwise when using browsers such as Chrome, Chrome will show Certificate internal error, and unable to proceed.
  • With these self-signed certificate, browsers will alert Your connection is not private‘’, but will allow user to proceed

Will create 2 certificate here, one for WebUI, one for FTP TLS.

  • Https WebUI:

System -> General

Protocol: HTTP+HTTPS

Certificate: Select the one just created

  • Then set port forward accordingly in the Router

5). Work flow to setup a Share:

‘storage’ icon at the top-> ‘create dataset’ button at the bottom tools box section -> ‘change permission’ button next to it to change permissions -> left section ‘sharing’ menu create shares -> ‘services’ icon to configure corresponding service-> enable services

Regarding permissions:

http://en.wikipedia.org/wiki/File_system_permissions

6). CIFS shares (SMB)

Notes:

Very useful tutorial:

https://forums.freenas.org/index.php?threads/how-to-private-group-folder-private-user-folder-common-folder.11557/

Mine:

  • Zpool top level dataset permission; 777
  • User’s dataset permission: 700 (only that user can access)
  • User’s dataset owner: user’s name, nogroup
  • For Windows CIFS shares, even though dataset has the option to select ‘permission type’ to ‘windows’, I change it to Unix. Otherwise the tutorial in the above link won’t work.
  • In the CIFS settings: I enter File mask to 700, otherwise ‘.exe’ files won’t be executed in the Windows environment. (700: only user read, write, execute)

Commands to check files detailed permissions:

> getacl [filename]

To change permissions of all the *.exe files under a folder to allow execution: (if for some reason forgot to set ‘File mask’)

> find . -name *.exe -exec chmod 700 {} ;

Some additional notes:

* Choose ‘Server Maximum protocol’ to SMB 3.0 (Windows 8 start to support it) will increase writing speed a lot: in my case, from 80Mb/s to 90-95 Mb/s

* In each folder’s sharing settings, add the following to Auxillary Parameters can prevent Mac Machines create annoying .DS_store files into the sharing CIFS folders:

veto files = /Temporary Items/.DS_Store/.AppleDB/.TemporaryItems/.AppleDouble/.bin/.AppleDesktop/Network Trash Folder/.Spotlight/.Trashes/.fseventd/

delete veto files = yes

Reference link:

http://blog.graceabundant.com/2011/10/02/freenas-simultaneous-afpcifs-shares-done-neatly/comment-page-1/

7).Config FTP:

Each user must have a valid home directory in user settings to use FTP. (Except Anonymous Login)

  • Services -> FTP settings:
  • Port: 21
  • Clients: 5
  • Connection: 10
  • Login Attempts: 20
  • Timeout: 600
  • Allow Root Login: no
  • Allow Anonymous Login: no
  • Path: (If allow anonymous login, specify the path here)
  • Display Login: ******Welcome*******
  • File permission: 660
  • Directory Permission: 770
  • Always Choot: Yes (User can only view this home directory, except users in wheel group, which is Me in this case)
  • Masquerade address: mydomain.duckdns.org (Since the server is behind the server, if login the ftp remotely in passive mode, server will respond will an internal IP, which is wrong, so tell the client its Public IP here; Alternative way is to set the remote client software using active mode, this is slow to list directory during my tests)
  • Min Passive port: 10000
  • Max passive port: 11000 (These are for ftp transmissions, set port forward on the router for these ports, otherwise unable to transfer file through ftp remotely)
  • Enable TLS: Yes (This will allow FTP secure connection, in Filezilla, using ftpes://server address to establish connection)
  • TLS policy: off (Since some ftp client software don’t support ftpes, so it’s better to not enforce the policy, however if client support TLS, it will establish a secure connection)
  • Certificate: FTPTLS (created earlier in the certificate section)

Notes:

  • After these, set port forward accordingly in the router
  • iOS ftp client that allows superuser (in ‘wheel’ group) to change path beyond their home directory: iTransfer

8). Wake on Lan (WOL)

  1. Enable WOL in the BIOS
  2. In the command line, check if system has enabled WOL:

>ifconfig -m bge0

if the interface has capability ‘WOL_MAGIC’, then it is enabled.

To enable it, follow this post:

https://forums.freenas.org/index.php?threads/wake-on-lan.13508/page-4

Check post #65, basically it extract if_bge.ko from NAS4Free project ISO file and put it into Freenas’ ‘/boot/kernel’

Then add a new tunable in Freenas WebGUI:

  • Variable: if_bge_load
  • Value: YES
  • Type: Loader
  • Comment: Wake On Lan
  • Enabled: yes

9).UPS service

UPS Hardware: Cyberpower CP550HG

Settings:

  • Identifier: cpsups (in command line, to identify the UPS)
  • Driver: Cyberpower AE550 USB(usbhid-ups) (Can’t find my model, this one works)
  • Port: /dev/ugenx.x (This can be found by using: sudo usbconfig or sudo dmesg)
  • Description: CyberPower CP550HG standby
  • Shutdown mode: UPS reaches low battery
    • (Test shows the server will shutdown around 20mins with this model)
  • Monitor User: upsmon
  • Monitor Password: cpsmonitor
  • Extra users:
    • [admin]
    • password = cpsmm
    • instcmds = all
  • Send Email status updates: yes
  • To email: myemail@provider.com
  • Email subject: My Server UPS report

Some useful commands:

UPS status: > upsc cpsups@localhost

UPS command:

list all commands: >upscmd -l cpsups

quicktest: >upscmd cpsups test.battery.start.quick

username: admin

password: cpsmm

After UPS shuts down the server, the server can be woke up in the router by send magic packets to its MAC address.

Reference:

http://doc.freenas.org/9.3/freenas_services.html#ups

http://www.networkupstools.org/docs/man/upscmd.html

http://www.networkupstools.org/docs/man/upsd.users.html

10) Dynamic DNS service

Since the server can be access anywhere, it’s better to have a web address.

Freenas’ internal DynDNS utilizes ‘inadyn’, document:

http://www.inatech.eu/inadyn/readme.html

Attention: To configure Dyndns sucessfully, each time you have to re-enter the password!

A good free dynamic DNS service came to my mind -> DuckDNS:

To configure DuckDNS:

Add a Cron Jobs:

  • User: root
  • Command:

curl -sS https://duckdns.org/update/SUBDOMAINNAME/TOKEN-NUMBER

  • Select ‘Redirect stdout’, so you won’t receive email every time DNS updates

11). Automatic Configuration File Backup

  • Create a shell script: /mnt/POOLNAME/DATASETNAME/system_bak/bkconfig.sh, put the following scripts:

cp /data/freenas-v1.db /mnt/POOLNAME/DATASETNAME/system_bak/config/`date +%Y%m%d`.db

(‘date’ followed by a whitespace)

  • > chmod 700 bkconfig.sh   So the script will be executable.
  • Create a Cron job at 01:05AM everyday:
    • User: root
    • Command: sh /mnt/POOLNAME/DATASETNAME/system_bak/bkconfig.sh
    • Description: System Configuration Nightly Backup

Reference:

https://forums.freenas.org/index.php?threads/backup-config-file-every-night-automatically.8237/

12). some additional commands:

CPU Temp:

> sysctl -a | egrep -E “cpu.[0-9]+.temp”

HDD SMART:

> smartctl -i -H -A -l error /dev/ada0          <- (ada1-ada3)

13). Zpool operations, replace HDD

WebGUI can do a lot, but sometimes command lines are still necessary.

Common zpool commands:

>zpool list

>zpool status  OR  >zpool status poolname

>scrub

>zpool offline/online

14). Windows Backup App:

After a carefully selection, ‘Cobian’ is indeed excellent to use.

Reference: http://www.cobiansoft.com/cobianbackup.htm

 

To be Continue…

License: CC BY-NC-SA 4.0

Advertisements

One thought on “HP Proliant G7 N54L Freenas 9.3 NAS Build Log”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s