{"id":13048,"date":"2022-06-02T20:21:33","date_gmt":"2022-06-02T17:21:33","guid":{"rendered":"https:\/\/kifarunix.com\/?p=13048"},"modified":"2024-03-09T15:45:51","modified_gmt":"2024-03-09T12:45:51","slug":"install-and-setup-glusterfs-on-ubuntu","status":"publish","type":"post","link":"https:\/\/kifarunix.com\/install-and-setup-glusterfs-on-ubuntu\/","title":{"rendered":"Install and setup GlusterFS on Ubuntu 22.04\/Ubuntu 20.04"},"content":{"rendered":"\n<p>Welcome to our guide on how to install and setup GlusterFS on Ubuntu 22.04\/Ubuntu 20.04. Well, so what is GlusterFS? GlusterFS is an opensource distributed and scalable network file system that clusters various disk storage resources into a single global namespace. GlusterFS is suitable for data-intensive tasks such as cloud storage and media streaming.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Install and Setup GlusterFS on Ubuntu 22.04\/Ubuntu 20.04<\/h2>\n\n\n\n<p>While setting up GlusterFS, there are <a href=\"https:\/\/docs.gluster.org\/en\/main\/Quick-Start-Guide\/Architecture\/\" target=\"_blank\" rel=\"noreferrer noopener\">different types of Volume architectures<\/a> that you may want to consider. These include;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"#distributed\">Distributed GlusterFS Volume<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/kifarunix.com\/setup-replicated-glusterfs-volume-on-ubuntu\/\">Replicated GlusterFS Volume<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/kifarunix.com\/setup-glusterfs-distributed-replicated-volume-on-centos-8\/\" target=\"_blank\" rel=\"noreferrer noopener\">Distributed Replicated GlusterFS Volume<\/a><\/li>\n\n\n\n<li>Striped GlusterFS Volume<\/li>\n\n\n\n<li>Distributed Striped GlusterFS Volume<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"distributed\">How to Setup Distributed GlusterFS Volume on Ubuntu 22.04\/Ubuntu 20.04<\/h3>\n\n\n\n<p>In this guide, we are going to learn how to setup <code>distributed GlusterFS<\/code>. With the distributed volume, files are distributed across various bricks in the volume such that file A is stored on one of the volumes and file B on the other. The purpose of this architecture is to cheaply scale the volume size. However, it doesn&#8217;t provide redundancy and a failure of the volume will lead to a complete loss of the data stored in that volume.<\/p>\n\n\n\n<p>As a result, our environment consists of two storage nodes and a single client. Their details are as shown below.<\/p>\n\n\n\n<p>Note, do not use the root partition for GlusterFS. Add another storage partition, preferably on a different drive.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Storage Node 1:\n<ul class=\"wp-block-list\">\n<li>Hostname: gfs01.kifarunix-demo.com<\/li>\n\n\n\n<li>IP address: 192.168.57.6<\/li>\n\n\n\n<li>Gluster Storage Disk: \/dev\/sdb1<\/li>\n\n\n\n<li>Size: 4GB<\/li>\n\n\n\n<li>Mount Point: \/gfsvolume<\/li>\n\n\n\n<li>OS: Ubuntu 22.04\/Ubuntu 20.04 <\/li>\n<\/ul>\n<\/li>\n\n\n\n<li>Storage Node 2:\n<ul class=\"wp-block-list\">\n<li>Hostname: gfs02.kifarunix-demo.com<\/li>\n\n\n\n<li>IP address: 192.168.56.124<\/li>\n\n\n\n<li>Gluster Storage Disk: \/dev\/sdb1<\/li>\n\n\n\n<li>Size: 4GB<\/li>\n\n\n\n<li>Mount Point: \/gfsvolume<\/li>\n\n\n\n<li>OS: Ubuntu 22.04\/Ubuntu 20.04 <\/li>\n<\/ul>\n<\/li>\n\n\n\n<li>GlusterFS Client:\n<ul class=\"wp-block-list\">\n<li>Hostname: gfsclient.kifarunix-demo.com<\/li>\n\n\n\n<li>IP address: 192.168.43.197<\/li>\n\n\n\n<li>OS: Ubuntu 22.04\/Ubuntu 20.04 <\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Ensure that the hostsnames are resolvable. If you dont have a DNS server, then populate the hosts file of each Server accordingly such that the three servers are reachable via the hostnames.<\/strong><\/li>\n\n\n\n<li><strong>Another thing to consider is the NTP server. Ensure that the time is synchronized for the three servers.<\/strong><\/li>\n<\/ul>\n\n\n\n<p>Update and upgrade your system packages.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>apt update<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"install-glusterfs\"><a href=\"#install-glusterfs\">Install GlusterFS Server on Ubuntu 22.04\/Ubuntu 20.04 Nodes<\/a><\/h3>\n\n\n\n<p>GlusterFS packages is available on the default Ubuntu 22.04\/Ubuntu 20.04. Run the command below to install GlusterFS server.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>apt install glusterfs-server<\/code><\/pre>\n\n\n\n<p>Start and enable GlusterFS server (<code>glusterd<\/code>) to run on system boot;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>systemctl enable --now glusterd<\/code><\/pre>\n\n\n\n<p>Check the status;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>systemctl status glusterd<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\u25cf glusterd.service - GlusterFS, a clustered file-system server\n     Loaded: loaded (\/lib\/systemd\/system\/glusterd.service; enabled; vendor preset: enabled)\n     Active: active (running) since Thu 2022-06-02 17:59:13 UTC; 11s ago\n       Docs: man:glusterd(8)\n    Process: 3346 ExecStart=\/usr\/sbin\/glusterd -p \/var\/run\/glusterd.pid --log-level $LOG_LEVEL $GLUSTERD_OPTIONS (code=exited, status=0\/SUCCESS)\n   Main PID: 3347 (glusterd)\n      Tasks: 9 (limit: 2241)\n     Memory: 7.1M\n        CPU: 1.394s\n     CGroup: \/system.slice\/glusterd.service\n             \u2514\u25003347 \/usr\/sbin\/glusterd -p \/var\/run\/glusterd.pid --log-level INFO\n\nJun 02 17:59:11 gfs02.kifarunix-demo.com systemd[1]: Starting GlusterFS, a clustered file-system server...\nJun 02 17:59:13 gfs02.kifarunix-demo.com systemd[1]: Started GlusterFS, a clustered file-system server.\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Setup Distributed GlusterFS Volume on Ubuntu 22.04\/Ubuntu 20.04<\/h3>\n\n\n\n<ul class=\"wp-block-list\" id=\"glusterfs-firewall\">\n<li><a href=\"#glusterfs-firewall\">Open GlusterFS ports on Firewall<\/a><\/li>\n<\/ul>\n\n\n\n<p>If Firewall is running, run the command below to allow the Gluster storage nodes to communicate with each other via the Gluster daemon service port, 24007\/TCP<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ufw allow from <strong>&lt;other-node-IP&gt;<\/strong> to any port 24007 proto tcp comment \"GlusterFS Management\"<\/code><\/pre>\n\n\n\n<p>If you are using iptables;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>iptables -A INPUT -s <strong>&lt;other-node-IP&gt;<\/strong> -p tcp --dport 24007 -j ACCEPT -m comment --comment \"GlusterFS Management\"<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>cp \/etc\/iptables\/rules.v4{,.old}\niptables-save &gt; \/etc\/iptables\/rules.v4<\/code><\/pre>\n\n\n\n<p>Also allow GlusterFS clients to connect to GlusterFS daemon;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ufw allow from <strong>&lt;Client-IP&gt;<\/strong> to any port 24007 proto tcp comment \"GlusterFS Client Access\"<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Configure GlusterFS Trusted Pool<\/li>\n<\/ul>\n\n\n\n<p>To create a trusted storage pool between the GlusterFS nodes, run the probe from GlusterFS Node01 as shown below;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>gluster peer probe gfs02.kifarunix-demo.com<\/code><\/pre>\n\n\n\n<p>Sample output;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>peer probe: success.<\/code><\/pre>\n\n\n\n<p>To check the status of the trusted pool just created above, execute the command below;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>gluster peer status<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>Number of Peers: 1\n\nHostname: gfs02.kifarunix-demo.com\nUuid: b81803a8-893a-499e-9a87-6bac00a62822\nState: Accepted peer request (Connected)<\/code><\/pre>\n\n\n\n<p>If you get <code>State: Peer Rejected (Connected)<\/code>, see the resolution <a href=\"https:\/\/staged-gluster-docs.readthedocs.io\/en\/release3.7.0beta1\/Administrator%20Guide\/Resolving%20Peer%20Rejected\/\" target=\"_blank\" rel=\"noopener\">here<\/a>.<\/p>\n\n\n\n<p>Output from the second node;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>Number of Peers: 1\n\nHostname: gfs01.kifarunix-demo.com\nUuid: 26fe538a-91c2-42a1-b34a-67c2c94c7492\nState: Peer in Cluster (Connected)<\/code><\/pre>\n\n\n\n<p>To list the storage pools;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>gluster pool list<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>UUID\t\t\t\t\tHostname                \tState\nb81803a8-893a-499e-9a87-6bac00a62822\tgfs02.kifarunix-demo.com\tConnected \n26fe538a-91c2-42a1-b34a-67c2c94c7492\tlocalhost               \tConnected<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Create Distributed GlisterFS Volume<\/li>\n<\/ul>\n\n\n\n<p>Create a brick directory for GlusterFS volumes on the GlusterFS storage device mount point on <strong>both<\/strong> storage nodes.<\/p>\n\n\n\n<p><em>A Brick is&nbsp;<strong>the basic unit of storage in GlusterFS, represented by an export directory on a server in the trusted storage pool<\/strong>. A brick is expressed by combining a server with an export directory in the following format: `SERVER:EXPORT` For example: `myhostname:\/exports\/myexportdir\/`<\/em>.<\/p>\n\n\n\n<p>Thus, on <strong>BOTH<\/strong> Nodes, create brick directory.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>mkdir \/gfsvolume\/gv0<\/code><\/pre>\n\n\n\n<p>Please note that our GlusterFS disk, \/dev\/sdb1 is mounted on \/gfsvolume directory.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>df -hT -P \/gfsvolume<\/code><\/pre>\n\n\n\n<p>Sample output;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>Filesystem     Type  Size  Used Avail Use% Mounted on\n\/dev\/sdb1      ext4  3.9G   24K  3.7G   1% \/gfsvolume<\/code><\/pre>\n\n\n\n<p>Next, create a distributed volume called <code>distributed_vol<\/code> on the nodes. The name can be anything!<\/p>\n\n\n\n<p>Run this command on either of the nodes, once.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>gluster volume create <span style=\"font-size: inherit; color: initial;\"><strong>distributed_vol<\/strong><\/span> transport tcp gfs01:\/gfsvolume\/gv0 gfs02:\/gfsvolume\/gv0<\/code><\/pre>\n\n\n\n<p>Sample output;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code> volume create: distributed_vol: success: please start the volume to access data<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Start the created volume.<\/li>\n<\/ul>\n\n\n\n<p>You can now start the created volume brick.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>gluster volume start distributed_vol<\/code><\/pre>\n\n\n\n<p>Sample output;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>volume start: distributed_vol: success<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Show information about the created volume.<\/li>\n<\/ul>\n\n\n\n<p>You can show information about created brick volume using the command below;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>gluster volume info<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code> \nVolume Name: distributed_vol\nType: Distribute\nVolume ID: 98519652-97a2-4fb8-bd1a-9b6a83d8936e\nStatus: Started\nSnapshot Count: 0\nNumber of Bricks: 2\nTransport-type: tcp\nBricks:\nBrick1: gfs01:\/gfsvolume\/gv0\nBrick2: gfs02:\/gfsvolume\/gv0\nOptions Reconfigured:\ntransport.address-family: inet\nstorage.fips-mode-rchecksum: on\nnfs.disable: on\n<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Get GlusterFS volume status;<\/li>\n<\/ul>\n\n\n\n<pre class=\"wp-block-code\"><code>gluster volume status<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>Status of volume: distributed_vol\nGluster process                             TCP Port  RDMA Port  Online  Pid\n------------------------------------------------------------------------------\nBrick gfs01:\/gfsvolume\/gv0                  49152     0          Y       6110 \nBrick gfs02:\/gfsvolume\/gv0                  60116     0          Y       4501 \n \nTask Status of Volume distributed_vol\n------------------------------------------------------------------------------\nThere are no active volume tasks\n<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Open GlusterFS Volumes Ports on Firewall<\/li>\n<\/ul>\n\n\n\n<p>In order for the clients to connect to the volumes created, you need to open the respective node volume port on firewall. The ports are shown above.<\/p>\n\n\n\n<p>Also ensure that the nodes can talk to each other on these ports.<\/p>\n\n\n\n<p>For example, on GlusterFS node 01, open port 49152 to allow clients to mount the volume.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ufw allow from <strong>&lt;Client-IP-or-Network&gt;<\/strong> to any port 49152 proto tcp comment \"GlusterFS Client Access\"<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>ufw allow from <strong>&lt;Node02 IP&gt;<\/strong> to any port 49152 proto tcp comment \"GlusterFS Node02\"<\/code><\/pre>\n\n\n\n<p>On Node 02;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ufw allow from <strong><strong>&lt;Client-IP-or-Network&gt;<\/strong><\/strong> to any port 60116 proto tcp comment \"GlusterFS Client Access\"<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>ufw allow from <strong><strong>&lt;Node01 IP&gt;<\/strong><\/strong> to any port 60116 proto tcp comment \"GlusterFS Node01\"<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"install-glusterfs-client\"><a href=\"#install-glusterfs-client\">Install GlusterFS Client on Ubuntu 22.04\/Ubuntu 20.04 Client<\/a><\/h3>\n\n\n\n<p>On Ubuntu 22.04\/Ubuntu 20.04, run the command to install GlusterFS client on Ubuntu 22.04\/Ubuntu 20.04;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>apt update<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>apt install glusterfs-client<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Mount the GlusterFS Volume on GlusterFS client<\/li>\n<\/ul>\n\n\n\n<p>We are going to use the native GlusterFS client to mount the GlusterFS nodes.<\/p>\n\n\n\n<p>Create the mount point<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>mkdir \/mnt\/gfsvol<\/code><\/pre>\n\n\n\n<p>Mount the distributed volume. If using domain names, ensure they are resolvable.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>mount -t glusterfs gfs01:\/distributed_vol \/mnt\/gfsvol\/<\/code><\/pre>\n\n\n\n<p>Run the df command to check the mounted filesystems.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>df -hTP \/mnt\/gfsvol\/<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>Filesystem             Type            Size  Used Avail Use% Mounted on\ngfs01:\/distributed_vol fuse.glusterfs  7.8G   97M  7.3G   2% \/mnt\/gfsvol<\/code><\/pre>\n\n\n\n<p>From other clients, you can mount the volume on the other node;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>mount -t glusterfs gfs02:\/distributed_vol \/mnt\/gfsvol\/<\/code><\/pre>\n\n\n\n<p>To auto-mount the volume on system boot, you need to add the line below to <code>\/etc\/fstab<\/code>.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>gfs01:\/distributed_vol \/mnt\/gfsvol glusterfs defaults,_netdev 0 0<\/code><\/pre>\n\n\n\n<p>To test the data distribution, create two test files on the client. One of the file will be stored one of the volumes and the other file on the other volume. see example below;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>mkdir \/mnt\/gfsvol\/Test-dir\ntouch \/mnt\/gfsvol\/Test-dir\/{test-file,test-file-two}<\/code><\/pre>\n\n\n\n<p>If you can check on node01,<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>ls \/gfsvolume\/gv0\/Test-dir\/\ntest-file-two<\/code><\/pre>\n\n\n\n<p>On node02,<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>ls \/gfsvolume\/gv0\/Test-dir\/\ntest-file<\/code><\/pre>\n\n\n\n<p>That concludes our guide on setting up GlusterFS on Ubuntu 22.04\/Ubuntu 20.04, specifically how to setup distributed GlusterFS volumes. In our next tutorial, we will learn how to setup replicated GlusterFS volumes.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Other Tutorials<\/h2>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/install-and-configure-nfs-server-on-rocky-linux-8\/\" target=\"_blank\" rel=\"noreferrer noopener\">Install and Configure NFS Server on Rocky Linux 8<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/easily-install-and-configure-samba-file-server-on-ubuntu-22-04\/\" target=\"_blank\" rel=\"noreferrer noopener\">Easily Install and Configure Samba File Server on Ubuntu 22.04<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/setup-replicated-glusterfs-volume-on-ubuntu\/\" target=\"_blank\" rel=\"noreferrer noopener\">Setup Replicated GlusterFS Volume on Ubuntu<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Welcome to our guide on how to install and setup GlusterFS on Ubuntu 22.04\/Ubuntu 20.04. Well, so what is GlusterFS? GlusterFS is an opensource distributed<\/p>\n","protected":false},"author":1,"featured_media":13046,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"rank_math_lock_modified_date":false,"footnotes":""},"categories":[39,345,121],"tags":[5343,5344,5341,5342,90,1200,4412],"class_list":["post-13048","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-storage","category-glusterfs","category-howtos","tag-distributed-glusterfs-ubuntu","tag-glusterfs-ports-24007","tag-install-glusterfs-ubuntu-22-04","tag-setup-glusterfs-ubuntu-20-04","tag-storage","tag-ubuntu-20-04","tag-ubuntu-22-04","generate-columns","tablet-grid-50","mobile-grid-100","grid-parent","grid-50","resize-featured-image"],"_links":{"self":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/13048"}],"collection":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/comments?post=13048"}],"version-history":[{"count":14,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/13048\/revisions"}],"predecessor-version":[{"id":20577,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/13048\/revisions\/20577"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/media\/13046"}],"wp:attachment":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/media?parent=13048"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/categories?post=13048"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/tags?post=13048"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}