Ftp-proxy was developed to help ftp clients transverse an OpenBSD PF firewall. The ftp protocol is notoriously difficult to use if a firewall is involved due to active and passive negotiation methods and its non-standard control and data connections. In 2005 a new proxy called "pftpx" was developed for the purpose of helping ftp clients contact ftp servers on either side of a firewall. The OpenBSD group eventually accepted pftpx as its standard ftp proxy and renamed it to "ftp-proxy". ftp-proxy is included by default on OpenBSD 4.0 and later.
ftp-proxy is a ftp proxy with many advanced features such as:
Think of a single firewall protecting an internal LAN from the Internet. If your machines are on the inside (LAN) and you connect outward to the Internet then this is considered a "forward" direction. Connections made from the Internet to a server located inside the firewall are considered to be in the "reverse" direction.
Forward Proxy is a ftp proxy allowing clients on the inside of the firewall to connect to ftp servers on the Internet.
lan client --> firewall --> internet ftp server
Reverse Proxy is a ftp proxy used to allow public internet clients to access ftp servers on the inside of the firewall.
internet client --> firewall --> internal lan ftp server
Ftp-proxy is a proxy for the Internet File Transfer Protocol. FTP control connections are redirected into the proxy using the pf(4) rdr command, after which the proxy connects to the server on behalf of the client.
The proxy allows data connections to pass, rewriting and redirecting them so the correct addresses are used. All connections from the client to the server have their source address rewritten so they appear to come from the proxy. Consequently, all connections from the server to the proxy have their destination address rewritten, so they are redirected to the client. The proxy uses the pf(4) anchor facility for this. (man ftp-proxy)
In effect, the client is connecting to the proxy and the proxy is connecting to the ftp server. When a client initiates a connection the redirection rules (rdr) inside of PF intercept the packets. The client is routed to the ftp proxy and the the proxy translates any IP addresses from the internal network to the external interface (IP). The proxy then connects to the ftp server as a liaison for the client. When requests are returned from the server to the client the proxy forwards the responses.
Though the setup is different for a forward or reverse proxy the methodology is the same.
The use of a proxy helps secure the client as well as the server. Since the ftp proxy is acting as the middle man all commands must pass though the proxy. If an illegal command is sent either from the client or the server the proxy will not accept it.
A "forward" ftp proxy is used to allow internal LAN clients to connect to Internet ftp servers when connecting through a PF firewall. All control and data connection rules are made by the ftp-proxy daemon and inserted into PF using anchors.
Step 1: You have two choices in setting up a forward ftp proxy. The most common setup is using the default external ip address as the source of ftp connections from the firewall. This will be option 1. You can instead use option 2 to allow the ftp proxy to use an alternate aliased ip address on the external interface. This is useful if you have a DMZ and want to allow those machine to connect to external ftp servers using a dedicated ip address.
Option 1 - FTP proxy using the default ip on the external interface: Start the ftp-proxy daemon. The daemon is what will translate the clients requests and also make the pf rules to allow the data connections through.
The following command will start the daemon using the options:
You can start the daemon manually by using the following command:
/usr/sbin/ftp-proxy -D7 -v
To make the ftp proxy start on boot put the following line into your /etc/rc.conf.local :
ftpproxy_flags="-D7 -v"
Option 2 - FTP proxy using an alias on the external interface: Start the ftp-proxy daemon. This option uses the argument "-a" to tell the ftp proxy to source all of its connections from a specific ip address on the external interface. This option is only used if you have multiple ips on the external interface and want to bind ftp traffic to one of those ips. The daemon is what will translate the clients requests and also make the pf rules to allow the data connections through.
The following command will start the daemon using the options:
You can start the daemon manually by using the following command:
/usr/sbin/ftp-proxy -a 10.20.30.40 -D 7 -v
To make the ftp proxy start on boot put the following line into your /etc/rc.conf.local :
ftpproxy_flags="-a 10.20.30.40 -D 7 -v"
Step 2: Edit the PF rules to add the anchors necessary for ftp-proxy. An anchor is used to insert and delete rules from PF dynamically as needed. Ftp-proxy will use these anchors to insert the rules necessary to let internal ftp clients connect to external ftp servers.
The ftp-proxy daemon listens on localhost (127.0.0.1) port 8021 by default. We need to have a redirection rule (divert-to) to redirect all requests from the internal network ($IntIf:network) to any external ftp server (to any port ftp) and send those requests to the ftp-proxy daemon (lo0 port 8021).
If you use strict ordering in PF then here are the lines separated into sections. If you need an example of these anchors in a working PF rule set check out our moneyslow.com PF config page.
Note: This syntax is for OpenBSD v5.x and later.
################ Filtering ################################# # Ftp ( secure forward ftp proxy ) anchor "ftp-proxy/*" pass in log on $IntIf inet proto tcp from $IntIf:network to !$IntIf port ftp flags S/SAFR modulate state divert-to lo0 port 8021
Step 3: Apply the new Pf rules using the following line:
pfctl -f /etc/pf.conf
Step 4: Test the connection using a client on the inside of the firewall. Do not use a client on the firewall itself for testing as the proxy can not be used transparently for local clients. Since the ftp daemon is running and the PF rules have been applied you should be able to use any client to connect to any external ftp site. For example, the following line will use ftp and connect to the primary OpenBSD ftp server anonymously.
ftp -p ftp.openbsd.org
A "reverse" ftp proxy is used to allow internet clients to connect to internal lan ftp servers when connecting through a PF firewall. All control and data connection rules are made by the ftp-proxy daemon and inserted into PF using anchors.
In this example we will setup a reverse proxy to accept connections from any ip address to port 21 on the firewall and forward that to the proxy. The proxy in turn will connect to the ftp server on the internal lan. Our setup will look like so:
(any ip) --> (firewall ip port 21) --> (rdr to ftp-proxy port 8021) --> (internal ftp server port 21)
Step 1: Start the ftp-proxy daemon in the reverse proxy configuration. This means we need to tell the proxy what port to listen on as well as what ip and port to forward the connection to. Since the proxy normally listens on localhost port 8021 we will keep this setting. We must also tell the proxy to connect to our internal ftp server on ip 10.10.10.50 port 21. This is the line we will use:
/usr/sbin/ftp-proxy -p 8021 -R 10.10.10.50 -P 21 -D7 -v
Step 2: Add the PF rules to allow external connections to be redirected to the proxy. This involves adding a redirection rule (divert-to) and filter rule like so to pf:
Note: This syntax is for OpenBSD v5.x and later.
################ Filtering ################################# # Ftp ( secure reverse ftp proxy ) anchor "ftp-proxy/*" pass in log on $ExtIf inet proto tcp from any to ($ExtIf) port ftp flags S/SAFR modulate state divert-to lo0 port 8021
Step 3: Apply the new Pf rules using the following line:
pfctl -f /etc/pf.conf
Step 4: Test the connection using a client on the outside of the firewall. Since the ftp daemon is running and the PF rules have been applied you should be able to use any client to connect to the external interface of the firewall and be forwarded to the internal ftp server. For example here we are using ftp to connect to the external name of the firewall on port 21.
ftp -p external_ftp_server.com
Note how a ftp-proxy works: When a LAN client want to ftp to a ftp server on the internet,the ftp-proxy is the middle man. The LAN client will send the ftp request to the internet server. The Pf "divert-to" line will intercept this request and send the ftp traffic to the ftp-proxy daemon. The ftp-proxy daemon will then make the connection to the internet ftp server on behalf of the LAN client. Return traffic from the internet ftp server will return to the firewall, go through the ftp-proxy daemon and back to the lan client.
We have to be careful when testing ftp. Different ftp binaries for different OS's use different default options. For example, the ftp binary on OpenBSD v5.1 uses passive ftp by default, so the the commands "ftp" and "ftp -p" are exactly the same. Some older Solaris machines use active only and ftp on Ubuntu 10.10 uses active by default.
Passive should work from your firewall, but active (PORT) probably will not. Active will only work if you accept a connection from any ip source port 20 to any upper port on the firewall. Not very common and not very secure. Remember, ftp connections initiated on the firewall do not use the ftp-proxy daemon; only ftp requests that go through the firewall.
The LAN machine behind the firewall should be able to do active and passive because the ftp-proxy, if setup correctly, will anchor the proper rules to allow both connection types. using the ftp-proxy is a very secure setup.
For testing you can setup the ftp-proxy daemon to log its connections to /var/log/daemon using, "/usr/sbin/ftp-proxy -D7 -v". You may also want to add the "log" variable to your Pf rules so you can watch the logs with "tcpdump -n -e -ttt -v -i pflog0".
Then make sure you are using the correct ftp arguments for your ftp binary to make a passive and a active connection from your LAN machine. Check the man page on the machine behind the firewall.
How do I list out the ftp-proxy anchors and rules ?
Every new ftp connection makes a new ftp-proxy/* anchor. The anchor only lives for as long as the ftp connection from the LAN host is active. So, to look at a ftp-proxy sub anchor you must print out the anchors and find the numerical id of the connection you want to list out. If you have many ftp connections active at once the job becomes harder, but you can script the following lines to make it easier for you.
## On firewall we see the parent anchor of the ## ftp-proxy daemon. There are no sub anchors ## because there are no current ftp session active. root@firewall# pfctl -vv -sA ftp-proxy ## on a machine behind the firewall on the LAN we ## initiate a passive ftp connection to ftp.openbsd.org user@lanhost$ ftp -p ftp.openbsd.org ## back on the firewall we list out all the anchors, ## notice a new anchor for our lan machine connection ## has been made (25552.5) root@firewall# pfctl -vv -sA ftp-proxy ftp-proxy/25552.5 ## listing out that anchor shows the divert-to and ## pass rules and statistics for the connection root@firewall# pfctl -vv -a ftp-proxy/25552.5 -sr @0 pass in log (all) quick inet proto tcp from 10.10.10.10 to 129.128.5.191 port = 44411 flags S/SA keep state (max 1) rtable 0 divert-to 129.128.5.191 port 33322 [ Evaluations: 25 Packets: 0 Bytes: 0 States: 0 ] [ Inserted: uid 82 pid 10347 State Creations: 0 ] @1 pass out log (all) quick inet proto tcp from 10.10.10.10 to 129.128.5.191 port = 33322 flags S/SA keep state (max 1) rtable 0 nat-to 192.168.100.200 [ Evaluations: 22 Packets: 0 Bytes: 0 States: 0 ] [ Inserted: uid 82 pid 10347 State Creations: 0 ]
Can I use CARP with a reverse ftp proxy? Yes, you can. Bind the reverse ftp proxy to the carp address not the external interface, for example carp0, and pass it to the internal ftp server. Remember carp0 is the virtual interface which is bound to the carp0 ip address.
################ Filtering ################################# # Ftp ( secure reverse ftp proxy ) pass in log on $ExtIf inet proto tcp from any to carp0 port ftp flags S/SAFR modulate state divert-to lo0 port 8021
A problem you might eventually see is when the CARP firewalls fail over. Current ftp connections to the ftp server will die when the backup firewall takes over because it does not have the ftp-proxy anchors from the first firewall. The anchors are not pfsync states and thus are not transferred to the backup firewall through pfsync. But, if the users issue a re-connect to your ftp server after the firewall fail over they will connect without issue.
Is there a limit to how many CARP bound ftp-proxy daemons I can run? The limit really depends on your hardware and how much bandwidth you have. We have setup CARP firewalls with as many as 12 separate CARP ip addresses bound to 12 different ftp-proxy daemons to 12 back end ftp servers. No problems at all. Just remember to run each ftp-proxy daemon on a different port (8021 for the first and 8022 for the second, and so on). Then just use Pf and the redirection rule (rdr) to direct the right client to the right ftp-proxy.
How do I watch the pf logs in real time?
tcpdump -n -e -ttt -i pflog0
How do I cat the pf log file?
tcpdump -n -e -ttt -r pflog0
How do I tell pf to re-read the pf.conf file after I make a change?
pfctl -f /etc/pf.conf
Can multiple ftp-proxy daemons be run on the same machine? Yes, as long as they are listening on different ports. The daemons will use the same ftp-proxy table to keep track of client to server sessions, but each proxy will use a sub-anchor so there is no contention between the daemons. Sub anchors are named as follows: "ftp-proxy.$pid.$session", where $pid is the process id of ftp-proxy, and $session the ftp-proxy session sequence number. So multiple ftp-proxy's will never collide.
The ftp-proxy daemon dies unexpectedly sometimes. What can I do?In some of the older versions of ftp-proxy (used to be called pftpx) the daemon used to die unexpectedly under load. A simple solution is to employ a ftp-proxy watcher though cron. The following cron job will look for the ftp-proxy daemon executed with "ftp-proxy -D7 -v". If the pid does not exist then the cron job will execute it.
#minute (0-59) #| hour (0-23) #| | day of the month (1-31) #| | | month of the year (1-12 or Jan-Dec) #| | | | day of the week (0-6 with 0=Sun or Sun-Sat) #| | | | | commands #| | | | | | #### ftp-proxy watcher */15 * * * * /usr/bin/pgrep -f "ftp-proxy -D7 -v" 2>&1 >/dev/null || /usr/sbin/ftp-proxy -D7 -v
Can you explain the different connection types of FTP and why it has problems with firewalls?
FTP runs exclusively over TCP. FTP servers by default listen on port 21 for incoming connections from FTP clients. A connection to this port from the FTP Client forms the control stream on which commands are passed to the FTP server from the FTP client and on occasion from the FTP server to the FTP client. FTP uses out-of-band control, which means it uses a separate connection for control and data. Thus, for the actual file transfer to take place, a different connection is required which is called the data stream. Depending on the transfer mode, the process of setting up the data stream is different.
In active mode, the FTP client opens a random port (> 1023), sends the FTP server the random port number on which it is listening over the control stream and waits for a connection from the FTP server. When the FTP server initiates the data connection to the FTP client it binds the source port to port 20 on the FTP server.
In order to use active mode, the client sends a PORT command, with the IP and port as argument. The format for the IP and port is "h1,h2,h3,h4,p1,p2". Each field is a decimal representation of 8 bits of the host IP, followed by the chosen data port. For example, a client with an IP of 192.168.0.1, listening on port 1025 for the data connection will send the command "PORT 192,168,0,1,4,1". The port fields should be interpreted as p1 x 256 + p2 = port, or, in this example, 4 x 256 + 1 = 1025.
In passive mode, the FTP server opens a random port (> 1023), sends the FTP client the server's IP address to connect to and the port on which it is listening (a 16 bit value broken into a high and low byte, like explained before) over the control stream and waits for a connection from the FTP client. In this case the FTP client binds the source port of the connection to a random port greater than 1023.
To use passive mode, the client sends the PASV command to which the server would reply with something similar to "227 Entering Passive Mode (127,0,0,1,78,52)". The syntax of the IP address and port are the same as for the argument to the PORT command.
In extended passive mode, the FTP server operates exactly the same as passive mode, however it only transmits the port number (not broken into high and low bytes) and the client is to assume that it connects to the same IP address that was originally connected to. Extended passive mode was added by RFC 2428 in September 1998.
While data is being transferred via the data stream, the control stream sits idle. This can cause problems with large data transfers through firewalls which time out sessions after lengthy periods of idleness. While the file may well be successfully transferred, the control session can be disconnected by the firewall, causing an error to be generated.
The FTP protocol supports resuming of interrupted downloads using the REST command. The client passes the number of bytes it has already received as argument to the REST command and restarts the transfer. In some command line clients for example, there is an often-ignored but valuable command, "reget" (meaning "get again") that will cause an interrupted "get" command to be continued, hopefully to completion, after a communications interruption.
Resuming uploads is not as easy. Although the FTP protocol supports the APPE command to append data to a file on the server, the client does not know the exact position at which a transfer got interrupted. It has to obtain the size of the file some other way, for example over a directory listing or using the SIZE command. Wikipedia, FTP