{"id":6192,"date":"2020-06-18T01:23:09","date_gmt":"2020-06-17T22:23:09","guid":{"rendered":"https:\/\/kifarunix.com\/?p=6192"},"modified":"2024-03-14T21:16:45","modified_gmt":"2024-03-14T18:16:45","slug":"setup-glusterfs-distributed-replicated-volume-on-centos-8","status":"publish","type":"post","link":"https:\/\/kifarunix.com\/setup-glusterfs-distributed-replicated-volume-on-centos-8\/","title":{"rendered":"Setup GlusterFS Distributed Replicated Volume on CentOS 8"},"content":{"rendered":"\n<p>In this tutorial, we are going to learn how to setup glusterfs distributed replicated volume on CentOS 8. <a rel=\"noreferrer noopener\" href=\"https:\/\/www.gluster.org\/\" target=\"_blank\">Gluster<\/a> is a free and open source scalable network filesystem which enables you to create large, distributed storage solutions for media streaming, data analysis, and other data- and bandwidth-intensive tasks.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"960\" height=\"439\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/06\/distributed-replicated-glusterfs-arc.png\" alt=\"Setup GlusterFS Distributed Replicated Volume on CentOS 8\" class=\"wp-image-6203\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/06\/distributed-replicated-glusterfs-arc.png 960w, https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/06\/distributed-replicated-glusterfs-arc-768x351.png 768w\" sizes=\"(max-width: 960px) 100vw, 960px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Setting up GlusterFS Distributed Replicated Volume<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Prerequisites<\/h3>\n\n\n\n<p>Before you can proceed, ensure that the following are met;<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Have 6 nodes for your GlusterFS cluster. <strong>An even number of bricks must be used in this type of volume<\/strong>.<\/li>\n\n\n\n<li>Attach an extract disk (different from the \/ partition) for use in providing Gluster storage unit (<code>brick<\/code>)<\/li>\n\n\n\n<li>Partition the disks using LVM and format the disk\/brick with an XFS filesystem.<\/li>\n\n\n\n<li>Ensure time is synchronized among your cluster notes<\/li>\n\n\n\n<li>Open the required Gluster Ports\/Services on Firewall on all your cluster nodes. <\/li>\n<\/ol>\n\n\n\n<p>In our previous tutorial, we covered how to install and setup GlusterFS storage cluster on CentOS 8 and all the above requirements. Follow the link below to check it.<\/p>\n\n\n\n<p><a rel=\"noreferrer noopener\" href=\"https:\/\/kifarunix.com\/install-and-setup-glusterfs-storage-cluster-on-centos-8\" target=\"_blank\">Install and Setup GlusterFS Storage Cluster on CentOS 8<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Types of Gluster Volumes<\/h3>\n\n\n\n<p>GlusterFS supports different types of volumes that offers various features. These include;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong><code>Distributed<\/code><\/strong>: Files are distributed across the bricks in the volume.<\/li>\n\n\n\n<li><strong><code>Replicated<\/code><\/strong>: Files are replicated across the bricks in the volume. It ensures high storage availability and reliability.<\/li>\n\n\n\n<li><strong><code>Distributed Replicated<\/code><\/strong>: Files are distributed across the replicated bricks in the volume. Ensures high-reliability, scalability and improved read performance.<\/li>\n\n\n\n<li><strong><code>Arbitrated Replicated<\/code><\/strong>: Files are replicated across two bricks in a replica set and only the metadata is replicated to the third brick. Ensures data consistency.<\/li>\n\n\n\n<li><strong><code>Dispersed<\/code><\/strong>: Files are dispersed across the bricks in the volume.<\/li>\n\n\n\n<li><strong><code>Distributed Dispersed<\/code><\/strong>: Data is distributed across the dispersed sub-volume.<\/li>\n<\/ul>\n\n\n\n<p>In our previous, we covered how to set-up replicated glusterfs storage volume.<\/p>\n\n\n\n<p><a rel=\"noreferrer noopener\" href=\"https:\/\/kifarunix.com\/install-and-setup-glusterfs-storage-cluster-on-centos-8\/#replicatedglustervolume\" target=\"_blank\">How to Setup Replicated Gluster Volume on CentOS 8<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"distributedreplicatedglustervolume\"><a href=\"#distributedreplicatedglustervolume\">Setup GlusterFS Distributed Replicated Volume<\/a><\/h3>\n\n\n\n<p>In a glusterfs distributed replicated setup, the number of bricks must be a multiple of the replica count. Also, the order in which bricks are specified is crucial in the sense that, &nbsp;adjacent bricks become replicas of each other.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Cluster Nodes<\/h4>\n\n\n\n<p>Below are the details of our distributed replicated Gluster volume nodes<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table><tbody><tr><td>#<\/td><td><strong>Hostname<\/strong><\/td><td><strong>IP Address<\/strong><\/td><\/tr><tr><td>1<\/td><td>gfs01.kifarunix-demo.com<\/td><td>192.168.56.111<\/td><\/tr><tr><td>2<\/td><td>gfs02.kifarunix-demo.com<\/td><td>192.168.56.112<\/td><\/tr><tr><td>3<\/td><td>gfs03.kifarunix-demo.com<\/td><td>192.168.57.114<\/td><\/tr><tr><td>4<\/td><td>gfs04.kifarunix-demo.com<\/td><td>192.168.57.113<\/td><\/tr><tr><td>5<\/td><td>gfs05.kifarunix-demo.com<\/td><td>192.168.57.117<\/td><\/tr><tr><td>6<\/td><td>gfs06.kifarunix-demo.com<\/td><td>192.168.57.118<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<h4 class=\"wp-block-heading\">Install GlusterFS Server on CentOS 8<\/h4>\n\n\n\n<p>Follow the link below to install GlusterFS server package on CentOS 8 nodes;<\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/install-and-setup-glusterfs-storage-cluster-on-centos-8\/#installglusterfscentos8\" target=\"_blank\" rel=\"noreferrer noopener\">How to Install GlusterFS Server Package on CentOS 8<\/a><\/p>\n\n\n\n<p>Checking the status of the GlusterFS server;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>systemctl status glusterd<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\n\u25cf glusterd.service - GlusterFS, a clustered file-system server\n   Loaded: loaded (\/usr\/lib\/systemd\/system\/glusterd.service; enabled; vendor preset: enabled)\n   Active: active (running) since Mon 2020-06-15 21:56:23 EAT; 11s ago\n     Docs: man:glusterd(8)\n Main PID: 2368 (glusterd)\n    Tasks: 9 (limit: 5027)\n   Memory: 3.9M\n   CGroup: \/system.slice\/glusterd.service\n           \u2514\u25002368 \/usr\/sbin\/glusterd -p \/var\/run\/glusterd.pid --log-level INFO\n\nJun 15 21:56:22 gfs01.kifarunix-demo.com systemd[1]: Starting GlusterFS, a clustered file-system server...\nJun 15 21:56:23 gfs01.kifarunix-demo.com systemd[1]: Started GlusterFS, a clustered file-system server.\n<\/code><\/pre>\n\n\n\n<p>You can do the same on other nodes.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Open\/Allow GlusterFS Service\/Ports on Firewall<\/h4>\n\n\n\n<p>Open GlusterFS ports or services on a firewall to enable the nodes communicate.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>The&nbsp;<code><strong>24007-24008\/TCP<\/strong><\/code>&nbsp;are used for the communication between nodes;<\/li>\n\n\n\n<li><code><strong>24009-24108\/TCP<\/strong><\/code>&nbsp;are required for client communication.<\/li>\n<\/ul>\n\n\n\n<p>You can simply use the service,&nbsp;<strong><code>glusterfs<\/code><\/strong> instead of ports;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>firewall-cmd --add-service=glusterfs --permanent;firewall-cmd --reload<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">Verifying the GlusterFS Storage cluster disks<\/h4>\n\n\n\n<p>We are using LVM disks each of 4GBs across the nodes.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>lvs<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\n  LV   VG    Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert\n  root cl    -wi-ao----  &lt;6.20g                                                    \n  swap cl    -wi-ao---- 820.00m                                                    \n<strong>  gfs  drgfs -wi-ao----  &lt;4.00g<\/strong>\n<\/code><\/pre>\n\n\n\n<p>The disks have <strong><code>XFS<\/code><\/strong> filesystems and mounted under, <code><strong>\/data\/glusterfs\/<\/strong><\/code>.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>df -hT<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\nFilesystem            Type      Size  Used Avail Use% Mounted on\n...\n\/dev\/mapper\/cl-root   xfs       6.2G  1.7G  4.6G  27% \/\n\/dev\/sda1             ext4      976M  260M  650M  29% \/boot\ntmpfs                 tmpfs      82M     0   82M   0% \/run\/user\/0\n<strong>\/dev\/mapper\/drgfs-gfs xfs       4.0G   61M  4.0G   2% \/data\/glusterfs\/<\/strong>\n<\/code><\/pre>\n\n\n\n<p>To automount the drive on system boot, enter the following entry on the <code>\/etc\/fstab<\/code> configuration file;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>echo \"<strong>\/dev\/mapper\/drgfs-gfs<\/strong> \/data\/glusterfs xfs defaults 1 2\" &gt;&gt; \/etc\/fstab<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">Create Gluster TSP<\/h4>\n\n\n\n<p>Create a gluster <strong><code>trusted storage pool (TSP)<\/code><\/strong>&nbsp;using a&nbsp;<code><strong>gluster peer probe<\/strong><\/code>&nbsp;command. It is enough to probe all the other cluster nodes from the one of the nodes;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>for i in gfs{02..06}; do <strong>gluster peer probe<\/strong> $i; done<\/code><\/pre>\n\n\n\n<p>You should get a success for each node, in the order with which they are probed;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>\npeer probe: success.\npeer probe: success.\npeer probe: success.\npeer probe: success.\npeer probe: success.\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">Check the status of Gluster peers<\/h4>\n\n\n\n<p>From any node, you can run gluster peer status command to display the status of peers;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>gluster peer status<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\nNumber of Peers: 5\n\nHostname: gfs02\nUuid: 148dcf14-76c9-412c-9911-aac17cc5801f\nState: Peer in Cluster (Connected)\n\nHostname: gfs03\nUuid: 22d2a6ea-e3a4-49fc-8df6-bd70a9545b30\nState: Peer in Cluster (Connected)\n\nHostname: gfs04\nUuid: 89ddf393-8144-4529-81f6-98128a5f1b71\nState: Peer in Cluster (Connected)\n\nHostname: gfs05\nUuid: 53b7b05a-28ac-4dfc-8598-651dee9d2431\nState: Peer in Cluster (Connected)\n\nHostname: gfs06\nUuid: 29d2d128-5e59-4123-b265-d27ef08f024b\nState: Peer in Cluster (Connected)\n<\/code><\/pre>\n\n\n\n<p>You can verify the peering status from other nodes.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Configure Distributed Replicated Storage Volume<\/h4>\n\n\n\n<p><code><strong>gluster volume create<\/strong><\/code> command can be used to create a Gluster distributed-replicated volume. The syntax of the complete command is;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>gluster volume create NEW-VOLNAME [replica COUNT] [transport [tcp | rdma | tcp,rdma]] NEW-BRICK...<\/code><\/pre>\n\n\n\n<p><strong>NOTE<\/strong>:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>an even number of bricks must be used this when creating a distributed replicated volume (We have 6 of them). That means, the number of bricks must be a multiple of the replica count.<\/li>\n\n\n\n<li>The order in which bricks are specified determines how they are replicated with each other. For example, if you specify a replica of two, then it means that the first two adjacent bricks specified becomes a replicate(mirror) of each other and the next two bricks in the sequence replicate each other.<\/li>\n\n\n\n<li>Two-way distributed replicated volumes is NOT RECOMMENDED due to <a rel=\"noreferrer noopener\" href=\"https:\/\/docs.gluster.org\/en\/latest\/Administrator%20Guide\/Split%20brain%20and%20ways%20to%20deal%20with%20it\/\" target=\"_blank\">split brain<\/a> issues (<em><strong>inconsistency in either data or metadata (permissions, uid\/gid, extended attributes etc<\/strong><\/em>)) Hence, the use the three-way distributed replicated volume.<\/li>\n\n\n\n<li>If you have multiple bricks on your cluster nodes, ensure that you list the first brick on every server, then the second brick on every server in the same order etc so that replica-set members are not placed on the same node.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Creating a Three-way Distributed Replicated Volume<\/h3>\n\n\n\n<p>To create a three-way distributed replicated volume, we use six nodes in our demo with a replica of 3. This means that, the first 3 adjacent bricks with form a replica, same thing to the next three. <\/p>\n\n\n\n<p>In our setup, we named the brick data directory as <code>gfsbrick<\/code>. This data directory will be created if it doesn&#8217;t exist. <strong>Replace the names accordingly<\/strong> in the following command.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>gluster volume create dist-repl-gfs replica 3 transport tcp \\\ngfs01:\/data\/glusterfs\/brick01 gfs02:\/data\/glusterfs\/brick02 \\\ngfs03:\/data\/glusterfs\/brick03 gfs04:\/data\/glusterfs\/brick04 \\\ngfs05:\/data\/glusterfs\/brick05 gfs06:\/data\/glusterfs\/brick06<\/code><\/pre>\n\n\n\n<p>If all is well, you should a message about the volume creation being succesful.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>volume create: dist-repl-gfs: success: please start the volume to access data<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">Starting Distributed Replicated GlusterFS volume<\/h4>\n\n\n\n<p>You can start your volume with gluster volume start command. Replace&nbsp;<code><strong>dist-repl-gfs<\/strong><\/code>&nbsp;with the name of your volume.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>gluster volume start <strong>dist-repl-gfs<\/strong><\/code><\/pre>\n\n\n\n<p>And there you go, <strong><code>volume start: dist-repl-gfs: success<\/code><\/strong>. Your volume is up.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Verify GlusterFS Volumes<\/h4>\n\n\n\n<p>You can verify GlusterFS volumes with <code>gluster volume info<\/code> command.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>gluster volume info all<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\nVolume Name: dist-repl-gfs\nType: Distributed-Replicate\nVolume ID: 45f4416d-9842-4352-8802-b280a243036b\nStatus: Started\nSnapshot Count: 0\nNumber of Bricks: 2 x 3 = 6\nTransport-type: tcp\nBricks:\nBrick1: gfs01:\/data\/glusterfs\/brick01\nBrick2: gfs02:\/data\/glusterfs\/brick02\nBrick3: gfs03:\/data\/glusterfs\/brick03\nBrick4: gfs04:\/data\/glusterfs\/brick04\nBrick5: gfs05:\/data\/glusterfs\/brick05\nBrick6: gfs06:\/data\/glusterfs\/brick06\nOptions Reconfigured:\ntransport.address-family: inet\nstorage.fips-mode-rchecksum: on\nnfs.disable: on\nperformance.client-io-threads: off\n<\/code><\/pre>\n\n\n\n<p>To get the status of the volume;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>gluster volume status dist-repl-gfs<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\nStatus of volume: dist-repl-gfs\nGluster process                             TCP Port  RDMA Port  Online  Pid\n------------------------------------------------------------------------------\nBrick gfs01:\/data\/glusterfs\/brick01         49152     0          Y       4510 \nBrick gfs02:\/data\/glusterfs\/brick02         49152     0          Y       4194 \nBrick gfs03:\/data\/glusterfs\/brick03         49152     0          Y       4078 \nBrick gfs04:\/data\/glusterfs\/brick04         49152     0          Y       4080 \nBrick gfs05:\/data\/glusterfs\/brick05         49152     0          Y       5330 \nBrick gfs06:\/data\/glusterfs\/brick06         49152     0          Y       4077 \nSelf-heal Daemon on localhost               N\/A       N\/A        Y       4532 \nSelf-heal Daemon on gfs03                   N\/A       N\/A        Y       4099 \nSelf-heal Daemon on gfs02                   N\/A       N\/A        Y       4215 \nSelf-heal Daemon on gfs04                   N\/A       N\/A        Y       4101 \nSelf-heal Daemon on gfs05                   N\/A       N\/A        Y       5359 \nSelf-heal Daemon on gfs06                   N\/A       N\/A        Y       4098 \n \nTask Status of Volume dist-repl-gfs\n------------------------------------------------------------------------------\nThere are no active volume tasks\n<\/code><\/pre>\n\n\n\n<p>To list additional information about the bricks;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>gluster volume status dist-repl-gfs detail<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\nStatus of volume: dist-repl-gfs\n------------------------------------------------------------------------------\nBrick                : Brick gfs01:\/data\/glusterfs\/brick01\nTCP Port             : 49152               \nRDMA Port            : 0                   \nOnline               : Y                   \nPid                  : 4510                \nFile System          : xfs                 \nDevice               : \/dev\/mapper\/drgfs-gfs\nMount Options        : rw,seclabel,relatime,attr2,inode64,noquota\nInode Size           : 512                 \nDisk Space Free      : 3.9GB               \nTotal Disk Space     : 4.0GB               \nInode Count          : 2095104             \nFree Inodes          : 2095086             \n------------------------------------------------------------------------------\nBrick                : Brick gfs02:\/data\/glusterfs\/brick02\nTCP Port             : 49152\n...\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Mounting GlusterFS Storage Volumes on Clients<\/h3>\n\n\n\n<p>Once the distributed replicated volumes are setup, you can then mount them on clients and start writing data into them.<\/p>\n\n\n\n<p>For the purposes of demoing how to mount glusterfs volumes, we will be using a CentOS 8 client.<\/p>\n\n\n\n<p>There are different methods in which Gluster Storage volumes can be accessed. These include the use of;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Native GlusterFS Client <\/li>\n\n\n\n<li>Network File System (NFS) v3<\/li>\n\n\n\n<li>Server Message Block (SMB)<\/li>\n<\/ul>\n\n\n\n<p>We will be using Native GlusterFS client method in this case.<\/p>\n\n\n\n<p>Install GlusterFS native client on CentOS 8<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>dnf install glusterfs glusterfs-fuse<\/code><\/pre>\n\n\n\n<p>Once the installation is done, create a Gluster Storage volume mount point. We use, <code>\/mnt\/glusterfs<\/code> as the mount point.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>mkdir \/mnt\/glusterfs<\/code><\/pre>\n\n\n\n<p>Before you can proceed to mount the Gluster volumes, ensure that all nodes are reachable from the client.<\/p>\n\n\n\n<p>The mounting can be done using the <code>mount<\/code> command and specifying the file-system type as <code>glusterfs<\/code> and the node and name of the volume.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>mount -t glusterfs gfs01:\/dist-repl-gfs \/mnt\/glusterfs<\/code><\/pre>\n\n\n\n<p>For automatic mounting during boot, enter the line below on the \/etc\/fstab, replacing the particular glusterfs storage volume and the mount point.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>gfs01:\/dist-repl-gfs \/mnt\/glusterfs\/ glusterfs defaults,_netdev 0 0<\/code><\/pre>\n\n\n\n<p><strong>NOTE<\/strong> that you can configure the backup volfile servers in clients by using the <code><strong>backup-volfile-servers<\/strong><\/code> mount option.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>backup-volfile-servers=&lt;volfile_server2&gt;:&lt;volfile_server3&gt;:...:&lt;volfile_serverN&gt;<\/code><\/li>\n\n\n\n<li>If this option is specified while mounting the fuse client as show above, when the first volfile server fails, the servers specified in&nbsp;<code>backup-volfile-servers<\/code>&nbsp;option are used as volfile servers to mount the client until the mount is successful.<\/li>\n<\/ul>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>mount -t glusterfs -o backup-volfile-servers=server2:server3:.... ..:serverN server1:\/VOLUME-NAME MOUNT-POINT<\/code><\/pre>\n\n\n\n<p>This might look like;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>mount -t glusterfs -o backup-volfile-servers=gfs02:gfs03 gfs01:\/dist-repl-gfs \/mnt\/glusterfs<\/code><\/pre>\n\n\n\n<p>Confirm the mounting;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>df -hT -P \/mnt\/glusterfs\/<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">Testing Mounted Volumes<\/h4>\n\n\n\n<p>To test the distribution and replication of data on our distributed replicated gluster storage volume that is mounted on the client, we will create some bogus files;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>cd \/mnt\/glusterfs\/<\/code><\/pre>\n\n\n\n<p>Create files;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>for i in {1..10};do echo hello &gt; \"File${i}.txt\"; done<\/code><\/pre>\n\n\n\n<p>Verify the files that get stored on each node&#8217;s brick.<\/p>\n\n\n\n<p><strong>Same data (replication) on the first three bricks;<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>[root@gfs01 ~]# ls \/data\/glusterfs\/brick01\/\n<strong>File1.txt File3.txt File5.txt File6.txt File8.txt<\/strong><\/code><\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>[root@gfs02 ~]# ls \/data\/glusterfs\/brick02\/\n<strong>File1.txt File3.txt File5.txt File6.txt File8.txt<\/strong><\/code><\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>[root@gfs03 ~]# ls \/data\/glusterfs\/brick03\/\n<strong>File1.txt File3.txt File5.txt File6.txt File8.txt<\/strong><\/code><\/pre>\n\n\n\n<p><strong>Same data (replication) on the next three bricks;<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>[root@gfs04 ~]# ls \/data\/glusterfs\/brick04\/\n<strong>File10.txt File2.txt File4.txt File7.txt File9.txt<\/strong><\/code><\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>[root@gfs05 ~]# ls \/data\/glusterfs\/brick05\/\n<strong>File10.txt File2.txt File4.txt File7.txt File9.txt<\/strong><\/code><\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>[root@gfs06 ~]# ls \/data\/glusterfs\/brick06\/\n<strong>File10.txt File2.txt File4.txt File7.txt File9.txt<\/strong><\/code><\/pre>\n\n\n\n<p>And that is how data is distributed and replicated on the glusterfs distributed replicated storage volume.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Reference<\/h3>\n\n\n\n<p><a href=\"https:\/\/docs.gluster.org\/en\/latest\/Administrator%20Guide\/Setting%20Up%20Volumes\/\" target=\"_blank\" rel=\"noreferrer noopener\">Setting up GlusterFS Volumes<\/a><\/p>\n\n\n\n<p><a rel=\"noreferrer noopener\" href=\"https:\/\/access.redhat.com\/documentation\/en-us\/red_hat_gluster_storage\/3.5\/html\/administration_guide\/sect-creating_distributed_replicated_volumes\" target=\"_blank\">Creating Distributed Replicated Volumes Red Hat Gluster Storage<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Related Tutorials<\/h3>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/install-and-setup-glusterfs-storage-cluster-on-centos-8\/\" target=\"_blank\" rel=\"noreferrer noopener\">Install and Setup GlusterFS Storage Cluster on CentOS 8<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/install-and-setup-glusterfs-on-ubuntu-18-04\/\" target=\"_blank\" rel=\"noreferrer noopener\">Install and Setup GlusterFS on Ubuntu 18.04<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/install-and-configure-ceph-block-device-on-ubuntu-18-04\/\" target=\"_blank\" rel=\"noreferrer noopener\">Install and Configure Ceph Block Device on Ubuntu 18.04<\/a><\/p>\n\n\n\n<p><a rel=\"noreferrer noopener\" href=\"https:\/\/kifarunix.com\/how-to-install-and-configure-iscsi-storage-server-on-ubuntu-18-04\/\" target=\"_blank\">How to install and Configure iSCSI Storage Server on Ubuntu 18.04<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this tutorial, we are going to learn how to setup glusterfs distributed replicated volume on CentOS 8. Gluster is a free and open source<\/p>\n","protected":false},"author":1,"featured_media":13046,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"rank_math_lock_modified_date":false,"footnotes":""},"categories":[39,345,121],"tags":[1142,1702,1700,1703,346,1701],"class_list":["post-6192","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-storage","category-glusterfs","category-howtos","tag-centos-8","tag-configure-distributed-replicated-glusterfs-volume","tag-distributed-replicated-glusterfs-volume","tag-gluster-fs-volumes","tag-glusterfs","tag-setup-glusterfs-distributed-replicated-storage-volume","generate-columns","tablet-grid-50","mobile-grid-100","grid-parent","grid-50","resize-featured-image"],"_links":{"self":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/6192"}],"collection":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/comments?post=6192"}],"version-history":[{"count":8,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/6192\/revisions"}],"predecessor-version":[{"id":21422,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/6192\/revisions\/21422"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/media\/13046"}],"wp:attachment":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/media?parent=6192"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/categories?post=6192"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/tags?post=6192"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}