Product SiteDocumentation Site

Gluster File System 3.2.5

Administration Guide

Edition 1

Red Hat Engineering Content Services

Legal Notice

Copyright © 2011 Red Hat Inc.
The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shadowman logo, JBoss, MetaMatrix, Fedora, the Infinity Logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
Java® is a registered trademark of Oracle and/or its affiliates.
XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
All other trademarks are the property of their respective owners.

1801 Varsity Drive
Phone: +1 919 754 3700
Phone: 888 733 4281
Fax: +1 919 754 3701

This guide describes Gluster File System (GlusterFS), how to configure, operate, and manage GlusterFS.

1. Audience
2. License
3. Document Conventions
3.1. Typographic Conventions
3.2. Pull-quote Conventions
3.3. Notes and Warnings
4. Feedback
1. Introducing Gluster File System
2. Starting and Stopping the glusterd Daemon
2.1. Starting and Stopping glusterd Manually
2.2. Starting glusterd Automatically
2.2.1. Red Hat-based Systems
2.2.2. Debian-based Systems
2.2.3. Systems Other than Red Hat and Debain
3. Gluster Console Manager – Command Line Utility for Configuration and Management
4. Trusted Storage Pools – Preparing GlusterFS for Management
4.1. Adding Servers to Trusted Storage Pool
4.2. Removing Server from the Trusted Storage Pool
5. Setting Up GlusterFS Server Volumes
5.1. Creating Distributed Volumes
5.2. Creating Replicated Volumes
5.3. Creating Striped Volumes
5.4. Creating Distributed Striped Volumes
5.5. Creating Distributed Replicated Volumes
5.6. Starting Volumes
5.7. Displaying Volume Information
6. Accessing Data - Setting Up GlusterFS Client
6.1. Gluster Native Client
6.1.1. Installing the Gluster Native Client
6.1.2. Mounting Volumes
6.2. NFS
6.2.1. Using NFS to Mount Volumes
6.3. CIFS
6.3.1. Using CIFS to Mount Volumes
7. Managing GlusterFS Volumes
7.1. Tuning Volume Options
7.2. Expanding Volumes
7.3. Shrinking Volumes
7.4. Migrating Volumes
7.5. Rebalancing Volumes
7.5.1. Rebalancing Volume to Fix Layout Changes
7.5.2. Rebalancing Volume to Migrate Existing Data
7.5.3. Rebalancing Volume to Fix Layout and Migrate Existing Data
7.6. Stopping Volumes
7.7. Deleting Volumes
7.8. Triggering Self-Heal on Replicate
8. Managing GlusterFS Geo-replication
8.1. Replicated Volumes vs Geo-replication
8.2. Preparing to Deploy GlusterFS Geo-replication
8.2.1. Exploring Geo-replication Deployment Scenarios
8.2.2. GlusterFS Geo-replication Deployment Overview
8.2.3. Checking Geo-replication Minimum Requirements
8.2.4. Setting Up the Environment for Geo-replication
8.3. Starting GlusterFS Geo-replication
8.3.1. Starting Geo-replication
8.3.2. Verifying Successful Deployment
8.3.3. Displaying Geo-replication Status Information
8.3.4. Configuring Geo-replication
8.3.5. Stopping Geo-replication
8.4. Restoring Data from the Slave
8.5. Best Practices
9. Monitoring your GlusterFS Workload
9.1. Running GlusterFS Volume Profile Command
9.1.1. Start Profiling
9.1.2. Displaying the I/0 Information
9.1.3. Stop Profiling
9.2. Running GlusterFS Volume TOP Command
9.2.1. Viewing Open fd Count and Maximum fd Count
9.2.2. Viewing Highest File Read Calls
9.2.3. Viewing Highest File Write Calls
9.2.4. Viewing Highest Open Calls on Directories
9.2.5. Viewing Highest Read Calls on Directory
9.2.6. Viewing List of Read Performance on each Brick
9.2.7. Viewing List of Write Performance on each Brick
10. Managing Directory Quota
10.1. Enabling Quota
10.2. Disabling Quota
10.3. Setting or Replacing Disk Limit
10.4. Displaying Disk Limit Information
10.5. Updating Memory Cache Size
10.6. Removing Disk Limit
11. POSIX Access Control Lists
11.1. Activating POSIX ACLs Support
11.1.1. Activating POSIX ACLs Support on Sever
11.1.2. Activating POSIX ACLs Support on Client
11.2. Setting POSIX ACLs
11.2.1. Setting Access ACLs
11.2.2. Setting Default ACLs
11.3. Retrieving POSIX ACLs
11.4. Removing POSIX ACLs
11.5. Samba and ACLs
11.6. NFS and ACLs
12. Troubleshooting GlusterFS
12.1. Managing GlusterFS Logs
12.1.1. Setting the Log Directory
12.1.2. Rotating Logs
12.2. Troubleshooting Geo-replication
12.2.1. Locating Log Files
12.2.2. Synchronization is not complete
12.2.3. Issues in Data Synchronization
12.2.4. Geo-replication status displays Faulty very often
12.2.5. Intermediate Master goes to Faulty State
12.3. Troubleshooting POSIX ACLs
12.3.1. setfacl command fails with “setfacl: <file or directory name>: Operation not supported” error
12.4. Troubleshooting NFS
12.4.1. mount command on NFS client fails with “RPC Error: Program not registered”
12.4.2. NFS server start-up fails with “Port is already in use” error in the log file."
12.4.3. mount command fails with “rpc.statd” related error message
12.4.4. mount command takes too long finish.
12.4.5. NFS server glusterfsd starts but initialization fails with “nfsrpc- service: portmap registration of program failed” error message in the log.
12.4.6. mount command fails with NFS server failed error.
12.4.7. showmount fails with clnt_create: RPC: Unable to receive
12.4.8. Application fails with "Invalid argument" or "Value too large for defined data type" error.
13. Command Reference
13.1. gluster Command
13.2. glusterd Daemon
14. Glossary