File Transfer with cURL and the FTP Protocol

cURL is a tool for transferring data using URLs and it supports FTP as well as many other protocols such as HTTP, FTP, LDAP, POP3, SMTP and more. For a full list of supported protocols, see this official list of Protocols cURL Supports. The cURL command line tool is pre-installed on macOS X and higher. It can be installed on Unix and Linux systems. If you need to download cURL, visit the official cURL Download Page. This post shows how file transfer with cURL can be used to download and upload files from a remote FTP server. If you are behind a network proxy, have a look at the below post. It describes how to set proxy environment variables for a terminal session.

Confirm cURL Installation on macOS

For the below examples, I used the bash shell on macOS Sierra. To check whether cURL is installed, run the below command to see the version information. Here, the cURL tool and the libcurl library, version 7.54.0, are both installed on my macOS Sierra system.

bash-3.2$ curl -V
curl 7.54.0 (x86_64-apple-darwin16.0) libcurl/7.54.0 SecureTransport zlib/1.2.8
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz UnixSockets

Using cURL to List Directory Contents on a FTP Server

To simply connect to an anonymous FTP server (one that does not require authentication) and list the contents of the root directory, run the curl command, followed by its FTP URL.

curl ftp://speedtest.tele2.net

This results in the below response. The tele2 anonymous FTP server can be used for testing upload and download speeds.

bash-3.2$ curl ftp://speedtest.tele2.net
-rw-r--r--  1 0  0  1073741824000 Feb 19 2016 1000GB.zip
-rw-r--r--  1 0  0   107374182400 Feb 19 2016  100GB.zip
-rw-r--r--  1 0  0         102400 Feb 19 2016 100KB.zip
-rw-r--r--  1 0  0      104857600 Feb 19 2016 100MB.zip
-rw-r--r--  1 0  0    10737418240 Feb 19 2016 10GB.zip
-rw-r--r--  1 0  0       10485760 Feb 19 2016 10MB.zip
-rw-r--r--  1 0  0     1073741824 Feb 19 2016 1GB.zip
-rw-r--r--  1 0  0           1024 Feb 19 2016 1KB.zip
-rw-r--r--  1 0  0        1048576 Feb 19 2016 1MB.zip
-rw-r--r--  1 0  0      209715200 Feb 19 2016 200MB.zip
-rw-r--r--  1 0  0       20971520 Feb 19 2016 20MB.zip
-rw-r--r--  1 0  0        2097152 Feb 19 2016 2MB.zip
-rw-r--r--  1 0  0        3145728 Feb 19 2016 3MB.zip
-rw-r--r--  1 0  0      524288000 Feb 19 2016 500MB.zip
-rw-r--r--  1 0  0       52428800 Feb 19 2016 50MB.zip
-rw-r--r--  1 0  0         524288 Feb 19 2016 512KB.zip
-rw-r--r--  1 0  0        5242880 Feb 19 2016 5MB.zip
drwxr-xr-x  2 105  108      16384 Apr 25 11:41 upload

You can list directory contents, using singlecwd for the --ftp-method option. See cURL Directory Traversing for more information on this.

curl --ftp-method singlecwd ftp://speedtest.tele2.net/upload/

Alternatively, use the --list-only option as shown below.

curl --list-only ftp://speedtest.tele2.net/upload/

cURL FTP Connection with Authentication

If the FTP server you are connecting to requires authentication with a username and password, you can use the --user option.

curl --user <username>:<password> ftp://speedtest.tele2.net

Alternatively, you can also embed the username and password in the URL as shown below.

curl ftp://<username>:<password>@ftp://speedtest.tele2.net

File Transfer with cURL – Download Files

You can download files with cURL as weell. You need to tell it to navigate to the target directory and then download the file. Here, the file named 5MB.zip in the root directory is downloaded and saved locally using the name specified by the -o option. By default, cURL will try to use a passive FTP connection. Refer to this thread on stackoverflow for more information on active vs. passive FTP.

curl -o 5MB.zip ftp://speedtest.tele2.net/5MB.zip

cURL will start the download and show progress information and download speed.

% Total     % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 5120k  100 5120k    0     0   251k      0  0:00:20  0:00:20 –:–:–  314k

File Transfer with cURL – Upload Files

Files can be uploaded to a FTP server with the --upload-file or -T option. In the below example, one file named 5mb-test-file.zip is uploaded.

bash-3.2$ curl --upload-file 5mb-test-file.zip ftp://speedtest.tele2.net/upload/

This command produces output showing the progress of the file upload.

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 5120k    0     0  100 5120k      0   457k  0:00:11  0:00:11 --:--:--  544k
bash-3.2$

Multiple files can be specified for upload using globbing.

bash-3.2$ curl --upload-file '{file1,file2,file2}.zip' ftp://speedtest.tele2.net/upload/

For more information on using cURL for FTP transfers, see the cURL and FTP Documentation.

One thought on “File Transfer with cURL and the FTP Protocol

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.