{"id":7706,"date":"2021-01-19T22:21:45","date_gmt":"2021-01-19T19:21:45","guid":{"rendered":"https:\/\/kifarunix.com\/?p=7706"},"modified":"2024-03-19T20:52:44","modified_gmt":"2024-03-19T17:52:44","slug":"install-spiderfoot-on-ubuntu","status":"publish","type":"post","link":"https:\/\/kifarunix.com\/install-spiderfoot-on-ubuntu\/","title":{"rendered":"Install SpiderFoot on Ubuntu 20.04"},"content":{"rendered":"\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"941\" height=\"304\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot.png\" alt=\"spiderfoot ubuntu 20.04\" class=\"wp-image-7709\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot.png?v=1611074753 941w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-768x248.png?v=1611074753 768w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-150x48.png?v=1611074753 150w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-300x97.png?v=1611074753 300w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-696x225.png?v=1611074753 696w\" sizes=\"(max-width: 941px) 100vw, 941px\" \/><\/figure>\n\n\n\n<p>In this tutorial, you will learn how to install SpiderFoot on Ubuntu 20.04. <a aria-label=\"SpiderFoot (opens in a new tab)\" href=\"https:\/\/www.spiderfoot.net\/documentation\/\" target=\"_blank\" rel=\"noreferrer noopener\" class=\"rank-math-link\">SpiderFoot<\/a> <em>is a reconnaissance tool that automatically queries over 100 public data sources (OSINT) to gather intelligence on IP addresses, domain names, e-mail addresses, names and more. You simply specify the target you want to investigate, pick which modules to enable and then SpiderFoot will collect data to build up an understanding of all the entities and how they relate to each other<\/em>.<\/p>\n\n\n\n<p>Some of the common use cases of SpiderFoot include;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a aria-label=\" (opens in a new tab)\" href=\"https:\/\/www.spiderfoot.net\/osint-for-cyber-threat-intelligence\/\" target=\"_blank\" rel=\"noreferrer noopener\" class=\"rank-math-link\">Cyber Threat Intelligence<\/a><\/li>\n\n\n\n<li><a aria-label=\" (opens in a new tab)\" href=\"https:\/\/www.spiderfoot.net\/osint-for-security-assessments\/\" target=\"_blank\" rel=\"noreferrer noopener\" class=\"rank-math-link\">Security Assessments<\/a><\/li>\n\n\n\n<li><a aria-label=\" (opens in a new tab)\" href=\"https:\/\/www.spiderfoot.net\/osint-for-it-asset-discovery\/\" target=\"_blank\" rel=\"noreferrer noopener\" class=\"rank-math-link\">Asset Discovery<\/a><\/li>\n\n\n\n<li><a aria-label=\" (opens in a new tab)\" href=\"https:\/\/www.spiderfoot.net\/osint-for-attack-surface-monitoring\/\" target=\"_blank\" rel=\"noreferrer noopener\" class=\"rank-math-link\">Attack Surface Monitoring<\/a><\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Installing SpiderFoot on Ubuntu 20.04<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Prerequisites<\/h3>\n\n\n\n<p>In order to install SpiderFoot, you need to install a few package dependencies.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Run System Update<\/h4>\n\n\n\n<p>Update and upgrade your system packages;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>apt update<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>apt upgrade<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">Install Python 3 on Ubuntu 20.04<\/h4>\n\n\n\n<p>SpiderFoot is a Python based application and hence, it requires Python 3.6+ in order to run. Ubuntu 20.04 ships with Python 3 pre-installed. You can check installed version of Python3 by running the command below;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>apt list -a python3<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>python3\/focal,now 3.8.2-0ubuntu2 amd64 &#91;installed]<\/code><\/pre>\n\n\n\n<p>As you can see, we have Python 3.8.2 which is fine.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Install Python 3 PIP<\/h4>\n\n\n\n<p>Run the command below to install Python 3 PIP;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>apt install python3-pip<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Install SpiderFoot<\/h3>\n\n\n\n<p>Download the stable latest release version of SipderFoot source code from <a aria-label=\"Github repository (opens in a new tab)\" href=\"https:\/\/github.com\/smicallef\/spiderfoot\/releases\" target=\"_blank\" rel=\"noreferrer noopener\" class=\"rank-math-link\">Github repository<\/a>, which is v3.2.1 as of this writing;<\/p>\n\n\n\n<p>You can simply pull it and extract at the same using the command below;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>curl -Ls https:\/\/github.com\/smicallef\/spiderfoot\/archive\/v3.2.1.tar.gz | tar xz<\/code><\/pre>\n\n\n\n<p>If you check the current directory, you should see such a folder;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>ls<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>spiderfoot-3.2.1<\/code><\/pre>\n\n\n\n<p>To install SpiderFoot including other package requirements, navigate to the directory above and run the installation as shown below;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>cd spiderfoot-3.2.1<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>pip3 install -r requirements.txt<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Running SpiderFoot<\/h3>\n\n\n\n<p>Once the installation is done, SpiderFoot can be run in two modes;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Scan mode<\/strong>: Running SpiderFoot scans on command line with no web ui being started.<\/li>\n\n\n\n<li><strong>Web UI mode<\/strong>: Starts SpiderFoot with its built in web server to enable the management of scans and configs from the browser.<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Running SpiderFoot on Scan mode<\/h4>\n\n\n\n<p>To run SpiderFoot in scan mode, use the command;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>cd spiderfoot-3.2.1<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>python3 sf.py OPTION<\/code><\/pre>\n\n\n\n<p>To can list the available options by running the command;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>python3 sf.py --help<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>usage: sf.py &#91;-h] &#91;-d] &#91;-l IP:port] &#91;-m mod1,mod2,...] &#91;-M] &#91;-s TARGET] &#91;-t type1,type2,...] &#91;-T] &#91;-o tab|csv|json] &#91;-H] &#91;-n] &#91;-r] &#91;-S LENGTH] &#91;-D DELIMITER] &#91;-f]\n             &#91;-F type1,type2,...] &#91;-x] &#91;-q]\n\nSpiderFoot 3.2.1: Open Source Intelligence Automation.\n\noptional arguments:\n  -h, --help          show this help message and exit\n  -d, --debug         Enable debug output.\n  -l IP:port          IP and port to listen on.\n  -m mod1,mod2,...    Modules to enable.\n  -M, --modules       List available modules.\n  -s TARGET           Target for the scan.\n  -t type1,type2,...  Event types to collect (modules selected automatically).\n  -T, --types         List available event types.\n  -o tab|csv|json     Output format. Tab is default. If using json, -q is enforced.\n  -H                  Don't print field headers, just data.\n  -n                  Strip newlines from data.\n  -r                  Include the source data field in tab\/csv output.\n  -S LENGTH           Maximum data length to display. By default, all data is shown.\n  -D DELIMITER        Delimiter to use for CSV output. Default is ,.\n  -f                  Filter out other event types that weren't requested with -t.\n  -F type1,type2,...  Show only a set of event types, comma-separated.\n  -x                  STRICT MODE. Will only enable modules that can directly consume your target, and if -t was specified only those events will be consumed by modules.\n                      This overrides -t and -m options.\n  -q                  Disable logging. This will also hide errors!<\/code><\/pre>\n\n\n\n<p>The options are self explanatory.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Running SpiderFoot on Web UI mode<\/h4>\n\n\n\n<p>To be able to access and run SpiderFoot scans from the web interface, you can start SpiderFoot in web ui mode. For this, you need to specify the interface IP and port to access SpiderFoot with. Note, you can use any port as long no other application is using the same port.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>cd spiderfoot-3.2.1<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>python3 sf.py -l 192.168.57.3:8080<\/code><\/pre>\n\n\n\n<p>The above command enables SpiderFoot to be accessible from browser using the address, <code>http:\/\/192.168.57.3:8080<\/code>.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>Starting web server at 192.168.57.3:8080 ...\n\n********************************************************************\nWarning: passwd file contains no passwords. Authentication disabled.\n********************************************************************\n\n\n*************************************************************\n Use SpiderFoot by starting your web browser of choice and \n browse to http:\/\/192.168.57.3:8080\n*************************************************************\n\n\n&#91;19\/Jan\/2021:18:43:55] ENGINE Listening for SIGTERM.\n&#91;19\/Jan\/2021:18:43:55] ENGINE Listening for SIGHUP.\n&#91;19\/Jan\/2021:18:43:55] ENGINE Listening for SIGUSR1.\n&#91;19\/Jan\/2021:18:43:55] ENGINE Bus STARTING\n&#91;19\/Jan\/2021:18:43:55] ENGINE Serving on http:\/\/192.168.57.3:8080\n&#91;19\/Jan\/2021:18:43:55] ENGINE Bus STARTED<\/code><\/pre>\n\n\n\n<p>If firewall is running, open the port to allow external access.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>ufw allow 8080\/tcp<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1913\" height=\"297\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-scans.png\" alt=\"SpiderFoot scans\" class=\"wp-image-7711\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-scans.png?v=1611083587 1913w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-scans-768x119.png?v=1611083587 768w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-scans-1536x238.png?v=1611083587 1536w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-scans-150x23.png?v=1611083587 150w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-scans-300x47.png?v=1611083587 300w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-scans-696x108.png?v=1611083587 696w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-scans-1068x166.png?v=1611083587 1068w\" sizes=\"(max-width: 1913px) 100vw, 1913px\" \/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">Enable SpiderFoot Basic Authentication<\/h3>\n\n\n\n<p>By default, SpiderFoot doesn&#8217;t implement any authentication to the Web UI and any can access the dashboard. To enable basic auth, enter a username and <strong>passwd<\/strong> to a file in the SpiderFoot directory in the format (still in plain text though);<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code><code>username:password<\/code><\/code><\/pre>\n\n\n\n<p>You can simply echo the username and password to file;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>echo \"admin:password\" &gt; passwd<\/code><\/pre>\n\n\n\n<p>Restart SpiderFoot and verify the basic authentication;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>python3 sf.py -l 192.168.57.3:8080<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>python3 sf.py -l 192.168.57.3:8080\nStarting web server at 192.168.57.3:8080 ...\n<strong>Enabling authentication based on supplied passwd file.<\/strong>\n\n\n*************************************************************\n Use SpiderFoot by starting your web browser of choice and \n browse to http:\/\/192.168.57.3:8080\n*************************************************************\n\n\n[19\/Jan\/2021:18:54:38] ENGINE Listening for SIGTERM.\n[19\/Jan\/2021:18:54:38] ENGINE Listening for SIGHUP.\n[19\/Jan\/2021:18:54:38] ENGINE Listening for SIGUSR1.\n[19\/Jan\/2021:18:54:38] ENGINE Bus STARTING\n[19\/Jan\/2021:18:54:38] ENGINE Serving on http:\/\/192.168.57.3:8080\n[19\/Jan\/2021:18:54:38] ENGINE Bus STARTED<\/code><\/pre>\n\n\n\n<p>Navigate through the SpiderFoot web UI to perform your desired scans.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1666\" height=\"942\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-new-scan.png\" alt=\"new SpiderFoot scan\" class=\"wp-image-7710\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-new-scan.png?v=1611083562 1666w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-new-scan-768x434.png?v=1611083562 768w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-new-scan-1536x868.png?v=1611083562 1536w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-new-scan-150x85.png?v=1611083562 150w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-new-scan-300x170.png?v=1611083562 300w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-new-scan-696x394.png?v=1611083562 696w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-new-scan-1068x604.png?v=1611083562 1068w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-new-scan-743x420.png?v=1611083562 743w\" sizes=\"(max-width: 1666px) 100vw, 1666px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1639\" height=\"934\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-settings.png\" alt=\"SpiderFoot scan settings\" class=\"wp-image-7712\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-settings.png?v=1611083662 1639w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-settings-768x438.png?v=1611083662 768w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-settings-1536x875.png?v=1611083662 1536w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-settings-150x85.png?v=1611083662 150w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-settings-300x171.png?v=1611083662 300w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-settings-696x397.png?v=1611083662 696w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-settings-1068x609.png?v=1611083662 1068w, https:\/\/kifarunix.com\/wp-content\/uploads\/2021\/01\/spiderfoot-settings-737x420.png?v=1611083662 737w\" sizes=\"(max-width: 1639px) 100vw, 1639px\" \/><\/figure>\n\n\n\n<p>That is it on how to install SpiderFoot. Feel free to explore this tool and utilize it as you wish<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Reference and Further Reading<\/h3>\n\n\n\n<p>In order to further configure and setup SpiderFoot, please refer to its documentation page.<\/p>\n\n\n\n<p><a aria-label=\"SpiderFoot Documentation (opens in a new tab)\" href=\"https:\/\/www.spiderfoot.net\/documentation\/\" target=\"_blank\" rel=\"noreferrer noopener\" class=\"rank-math-link\">SpiderFoot Documentation<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Other tutorials<\/h3>\n\n\n\n<p><a aria-label=\" (opens in a new tab)\" href=\"https:\/\/kifarunix.com\/how-to-install-and-use-clamav-antivirus-on-ubuntu-18-04\/\" target=\"_blank\" rel=\"noreferrer noopener\" class=\"rank-math-link\">How to Install and Use ClamAV Antivirus on Ubuntu 18.04<\/a><\/p>\n\n\n\n<p><a aria-label=\" (opens in a new tab)\" href=\"https:\/\/kifarunix.com\/install-and-setup-nessus-scanner-on-ubuntu-20-04\/\" target=\"_blank\" rel=\"noreferrer noopener\" class=\"rank-math-link\">Install and Setup Nessus Scanner on Ubuntu 20.04<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/install-rengine-on-ubuntu\/\" target=\"_blank\" aria-label=\" (opens in a new tab)\" rel=\"noreferrer noopener\" class=\"rank-math-link\">Install reNgine on Ubuntu 20.04|Ubuntu 18.04<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this tutorial, you will learn how to install SpiderFoot on Ubuntu 20.04. SpiderFoot is a reconnaissance tool that automatically queries over 100 public data<\/p>\n","protected":false},"author":1,"featured_media":7709,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"rank_math_lock_modified_date":false,"footnotes":""},"categories":[34,121],"tags":[3030,3032,3034,3033,1200,3031],"class_list":["post-7706","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-security","category-howtos","tag-install-spiderfoot-ubuntu","tag-spiderfoot-install","tag-spiderfoot-install-ubuntu-20-04","tag-ubuntu","tag-ubuntu-20-04","tag-ubuntu-spiderfoot","generate-columns","tablet-grid-50","mobile-grid-100","grid-parent","grid-50","resize-featured-image"],"_links":{"self":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/7706"}],"collection":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/comments?post=7706"}],"version-history":[{"count":4,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/7706\/revisions"}],"predecessor-version":[{"id":21965,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/7706\/revisions\/21965"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/media\/7709"}],"wp:attachment":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/media?parent=7706"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/categories?post=7706"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/tags?post=7706"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}