linux,

How to download files with curl

Dec 01, 2021 · 2 mins read · Post a comment
How to download files with curl

curl and wget are the most popular command-line tools (with curl being a library) for downloading remote content from the CLI. On one side, wget is simple to use as it gets, downloading web content using the command: curl <some_remote_file_url>.
On the other side, curl is more than that. curl supports multiple protocols including HTTP, HTTPS, FTP, SFTP, SMTP, SMB, SCP and others. You could find the major differences here. Nonetheless the topic for today is to learn how easily we could download remote content with curl.

Prerequisites

  • Shell environment

Installation

### Debian-based distros
$ apt install -y curl
### RHEL-based distros
$ yum install -y curl
### CentOS8
$ dnf install -y curl
### MacOS
~ brew install curl 

Verify installation:

$ curl --version

Example output:

curl 7.77.0 (x86_64-apple-darwin21.0) libcurl/7.77.0 (SecureTransport) LibreSSL/2.8.3 zlib/1.2.11 nghttp2/1.42.0
Release-Date: 2021-05-26
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp 
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL UnixSockets

Basic usage

Step 1. Let’s take for example the devcoops site robots.txt file and download it:

$ curl -O https://devcoops.com/robots.txt

-O: saves the file locally with the same name as the remote one. If you want to save the remote file on the local machine using custom name, use -o instead. For instance:

$ curl -o devcoops_robots.txt https://devcoops.com/robots.txt

Step 2. Verify the downloaded robots.txt file:

$ cat robots.txt

Expected output:

User-agent: *
Sitemap: https://devcoops.com/sitemap.xml

Now, if you have noticed, most of the third-party software that comes with curl installations, includes a remote bash script called install.sh and use redirection (pipe) to run the script. Let’s take the Node Version Manager nvm curl command installation as an example:

$ curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash

Note: Since -o specifies output file, the second dash - after the -o, points to the standard output, which means that the install.sh file will be downloaded and redirected as a standard input to the bash invocation.

Conclusion

As always, hope this was convenient enough. Feel free to leave a comment below and if you find this tutorial useful, follow our official channel on telegram.