A Software-only Version of Isilon: IsilonSD Edge

For more than a decade, Isilon has been delivered as an appliance, based on standard x64 servers using internal disks on which the scale out filesystem resides. Beside the scale-out character of the system and the ease of use, I strongly believe that the appliance idea is one of the main reason for the success of Isilon (and even other NAS appliances in the market). You get everything pre-installed, pre-tested, pre-optimized and easy to support system. However, there are also use cases, for which customers have asked for a software only version of Isilon. And because it’s based on FreeBSD and runs on standard x64 servers it’s just a matter of configuration, support and testing. The nodes in the Isilon appliance that EMC delivers are nothing else than x64 servers with different number of CPUs, memory, SSDs and HDDs, depending on the not type you choose. There is basically only one piece of hardware which has been specifically developed by the Isilon team. This is a NVRAM card that is used for the filesystem journal and caching. However, with OneFS 8.0, the code has been adopted so that now SSDs can be used as well for this purpose.

The Architecture

Considering the potential use cases, the decision has been made to provide the first version of IsilonSD Edge (SD stands for Software Defined) in a virtualized environment, based on VMWare ESXi 5.5 or 6.  Since Isilon has a scale-out architecture and does erasure coding across nodes, it’s required to run a OneFS cluster on at least 3 ESX servers to maintain the availability the Isilon users are used to have. The OneFS cluster nodes run as virtual machines (VMs) and IsilonSD Edge supports only direct-attached disks and VMFS-5 datastores (data disks) that are created from the direct-attached disks. The maximum number of nodes in an IsilonSD Edge cluster currently is 6.
The components that are included in the download package [4] are:

• The IsilonSD Edge Management Server  (runs in a VM)
• IsilonSD Management Plug-in  for VMWare vCenter
• the OneFS virtual machine files.

 

Figure 1: Architecture of IsilonSD Edge

Current Configuration Options

 

The Software-only version of Isilon allows currently to be deployed with the following configurations:

• Three to six nodes with one node per ESXi server
• Data Disks:
• Either 6 or 12 defined data disks
• Minimum size of each data disk—64 GB
• Maximum size of each data disk—2 TB
• Minimum of eight of disks per node 8 (6 data disks, 1 journal disk, and 1 boot disk)
• Maximum of 14 disks per node14 (12 data disks, 1 SSD for journal disk and 1 boot disk)
• Minimum cluster capacity—1152 GB (Calculated as shown: Minimum disk size * # of drives per node * 3 (minimum number of nodes))
• Maximum cluster capacity—Varies depending on your licenses and the resources available on your system.
• Journal Disks
• One SSD for the journal journal per node with at least 1 GB of free space
• Boot Disk
• One SSD or HDD  for the OS per node with at least 20 GB of free space
• Memory: minimum of 6 GB of free memory per node
  

 

Supported Servers

 

EMC IsilonSD Edge is supported on all of the VMware Virtual SAN compatible systems that meet the minimum deployment requirements. You can identify the compatible systems on the VMWare Compatibility Guide . Please note: although we use the vSAN compatible HCL to identify the supported systems, IsilonSD Edge itself does not support vSAN at this point in time. That might sound a bit strange but think about how OneFS protects data: with erasure coding across nodes using native disks. Although it will most probably work, using vSAN would add a redundant level of data protection and would most probably be contra-productive for performance. A typical good system for an IsilonSD Edge deployment would be the Dell PowerEdge R630 servers. If you spend a bit more by using a Dell PowerEdge FX2, you would get a 4 node IsilonSD cluster in a single 2U chassis.

A Free Version of IsilonSD Edge

There are two versions of IsilonSD Edge available. One is the regular paid version and one free community version. A lot of features are enabled in both versions except SyncIQ, SmartLock, CloudPools which are only enabled on the community version. You can start installing the free version and acquire license keys later on which can then be entered via the UI. The following table lists the differences of enabled features in both versions.

Feature	Function	Free license	Paid license
SmartPools	Groups nodes and files into pools	yes	yes
CloudPools	Transparent File Archiving to the Cloud	no	yes
Protocols NAS	NFS, SMB, HTTP,FTP,HDFS	yes	yes
Object Protocols	Swift	yes	yes
InsightIQ	Nice Monitoring for performance, capacity and forecasting	yes	yes
SyncIQ	Policy based and parallel synchronization between clusters	no	yes
SmartLock	WORM functionality for directories which require compliance protection	no	yes
SmartConnect Advanced	Load Balancing: round robin or based on CPU-utilization, connection count or throughput	yes	yes
SmartDedupe	Post process deduplication	yes	yes
SmartQuota	Enforcement and monitoring for quotas	yes	yes
SnapshotIQ	Filesystem or directory based Snapshots	yes	yes
NDMP Backup	IsilonSD Edge only supports 3way backups (no 2way due to lack of Fibre Channel connection)	yes	yes

Table 1: Supported features for the free and paid version of IsilonSD Edge

Use Cases

The most obvious use case for the software-only version are remote offices where no data center is available or locations where everything runs in a virtual environment. Using SyncIQ you can pull data that’s stored in the remote location into a central datacenter for backup purposes for example. Or you can even push content from a central location towards the edges (remote offices). You can even combine this with CloudPools [5] which enables you to keep the actual content local, while files that have not been used for some time are transparently pushed out the cloud. This can be very powerful because you get local performance with a small footprint but logically your NAS could be huge! What people also like is the fact that the Isilon appliance that resides in a data center is being managed in the same way as the virtual instances in remote locations (except the additional management VM that’s being used for the deployment).

The OneFS code is the very same code used in the appliance and SD version of Isilon.  Therefore, IsilonSD Edge might be a good vehicle for functional testing. Be aware that performance tests are very much depending on the underlying hardware so they might not make sense if you want to know performance characteristics for an appliance version.

 

Webcast and more Information

I am running a short 30-40 minutes WebCast explaining IsilonSD Edge:
Title: Extend your Data Center to your Remote Office with Software Defined Storage
When:  14 September 2016 – 14:00 UK time / 15:00 Berlin time
Register here:  This link works if you want to see the recording.
 
Here are some more links with useful information. Especially [1] contains almost everything you need.

[1] Isilon Info Hub: https://community.emc.com/docs/DOC-49267
[2] Technical Demo: EMC IsilonSD Edge Installation: https://www.youtube.com/watch?v=BgNzHRZMmo4
[3] EMC Community Network: Isilon: https://community.emc.com/community/products/isilon
[4] IsilonSD Edge Download packages: https://www.emc.com/products-solutions/trial-software-download/isilonsd-edge.htm
[5] Transparent Storage Tiering to the Cloud using Isilon Cloudpools : http://stefanradtke.blogspot.de/2016/02/transparent-storage-tiering-to-cloud.html

A good alternative: the Isilon x210 Fast Start Bundle

While I am writing this post, EMC hast just announced an entry level starter kit, containing three or more x210 nodes for a very attractive price. If you want to start with a small Isilon cluster, this might be a fantastic option to consider as well. As every appliance, it come pre-installed, pre-tested, ready to go. The new fast start bundle kit contains:

• Three to eight X210 12TB Nodes
• TWO 8-Port Mellanox Infiniband Switches (for the interal network) – no need to manage.
• Enterprise Bundle:
  • SmartConnect
  • SnapshotIQ
    SmartQuotas
  • Cables & Accessories
  • Optional Support

This bundle is aggressively priced and the promotion runs until December 31st 2016. It can be acquired through  an EMC partner. The maximum cluster size you can get is 8x 12TB = 96TB (raw). However, this limit is only a promo limit, not a technical one. It means that you can extend your cluster with any other available node type, including additional x210. In that case please note two things: 1.) You need to purchase bigger Infiniband Switsches if you want to build bigger clusters.  2.) For the configuration options you choose beyond the special offer, your regular company discount applies.

 

Summary

IsilonSD Edge is the first software-only version of Isilon and is a good starting point for a virtualized environment (based on VMWare ESXi  - other hypervisors might be supported in the future).  It’s a good way to connect remote locations with your central Data Lake built on an EMC Isilon Scale out Cluster. The functionality is equal to the appliance version of Isilon ( the free version has some restrictions). My personal preferred alternative would be a small Isilon cluster based on the appliance with x210 nodes, but this attractive promotion only runs until end of 2016.