FSx

• Allows us to launch 3rd party high-performance file systems on AWS

• Useful when we don’t want to use an AWS managed file system like S3

• Can be accessed from your on-premise infrastructure

Types of FSx

• FSx for Windows File Server

• FSx for Lustre

• FSx for NetApp ONTAP

• FSx for OpenZFS

FSx for Windows

• FSx for Windows is a Shared File System for Windows (like EFS but for Windows)

• Supports SMB protocol, Windows NTFS, Microsoft Active Directory integration, ACLs, user quotas

• Can be mounted on Linux EC2 instances

• Built on SSD, scale up to 10s of GB/s, millions of IOPS, 100s PB of data

• Supports Multi-AZ (high availability)

• Data is backed-up daily to S3

• Can be accessed from your on-premises infrastructure (VPN or Direct Connect)

• Does not integrate with S3 (cannot store cold data)

• Storage Options:

  • SSD – latency sensitive workloads (databases, media processing, data analytics, …)

  • HDD – broad spectrum of workloads (home directory, CMS, …)

FSx for Lustre

• Parallel distributed file system for HPC (like EFS but for HPC)

• Scales up to 100s GB/s, millions of IOPS, sub-ms latencies

• Only works with Linux

• Seamless integration with S3

  • Can read S3 buckets as a file system (through FSx)

  • Can write the output back to S3 (through FSx)

• Ability to both process the hot data in a parallel and distributed fashion as well as easily store the cold data on Amazon S3

• Storage Options:

• SSD – low-latency, IOPS intensive workloads, small & random file operations

• HDD – throughput-intensive workloads, large & sequential file operations

FSx Lustre - File System Deployment Options

• Scratch File System

  • Temporary storage (cheaper)

  • Data is not replicated (data lost if the file server fails)

  • High burst (6x faster than persistent file system)

  • Usage: short-term processing

• Persistent File System

  • Long-term storage (expensive)

  • Data is replicated within same AZ

  • Failed files are replaced within minutes

  • Usage: long-term processing, sensitive data

FSx for NetApp ONTAP

• Managed NetApp ONTAP on AWS

• File System compatible with NFS, SMB, iSCSI protocol

• Move workloads running on ONTAP or NAS to AWS

• Works with:

  • Linux

  • Windows

  • MacOS

  • VMware Cloud on AWS

  • Amazon Workspaces & AppStream 2.0

  • Amazon EC2, ECS and EKS

• Storage shrinks or grows automatically

• Snapshots, replication, low-cost, compression and data de-duplication

• Point-in-time instantaneous cloning (helpful for testing new workloads)

FSx for OpenZFS

• Managed OpenZFS file system on AWS

• File System compatible with NFS (v3, v4, v4.1, v4.2)

• Move workloads running on ZFS to AWS

• Works with:

  • Linux

  • Windows

  • MacOS

  • VMware Cloud on AWS

  • Amazon Workspaces & AppStream 2.0

  • Amazon EC2, ECS and EKS

• Up to 1,000,000 IOPS with < 0.5ms latency

• Snapshots, compression and low-cost

• Point-in-time instantaneous cloning (helpful for testing new workloads)

Transfer Family

• AWS managed service to transfer files in and out of Amazon S3 or Amazon EFS using FTP protocol (instead of using proprietary methods)

• Supported Protocols

  • FTP (File Transfer Protocol) - unencrypted in flight

  • FTPS (File Transfer Protocol over SSL) - encrypted in flight

  • SFTP (Secure File Transfer Protocol) - encrypted in flight

• Supports Multi AZ

• Pay per provisioned endpoint per hour + data transfers in GB

• Clients can either connect directly to the FTP endpoint or optionally through Route 53

• Transfer Family will need permission to read or put data into S3 or EFS

DataSync

• Move large amount of data to and from

  • On-premises / other cloud to AWS (NFS, SMB, HDFS, S3 API… protocols) – needs agent

  • AWS to AWS (different storage services) – no agent needed

• Can synchronize to:

  • S3 (all storage classes)

  • EFS

  • FSx for Windows

• Scheduled Replication (hourly, daily, weekly but not continuous)

• File permissions and metadata are preserved (NFS POSIX, SMB…)

• Need to install AWS DataSync Agent on premises

  

• Can also be used to transfer between two EFS in different regions

  

• Perfect to move large amounts of historical data from on-premises to S3 Glacier Deep Archive (directly).