wget are the most popular command-line tools (with curl being a library) for downloading remote content from the CLI. On one side, wget is simple to use as it gets, downloading web content using the command:
On the other side, curl is more than that. curl supports multiple protocols including HTTP, HTTPS, FTP, SFTP, SMTP, SMB, SCP and others. You could find the major differences here. Nonetheless, the topic for today is to learn how easily we could download remote content with curl.
- Shell environment
### Debian-based distros apt install -y curl ### RHEL-based distros yum install -y curl ### CentOS8 dnf install -y curl ### macOS brew install curl
curl 7.77.0 (x86_64-apple-darwin21.0) libcurl/7.77.0 (SecureTransport) LibreSSL/2.8.3 zlib/1.2.11 nghttp2/1.42.0 Release-Date: 2021-05-26 Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL UnixSockets
Step 1. Let’s take for example the DevCoops site robots.txt file and download it.
curl -O https://devcoops.com/robots.txt
-O: Saves the file locally with the same name as the remote one. If you want to save the remote file on the local machine using custom name, use
-o instead. For instance:
curl -o devcoops_robots.txt https://devcoops.com/robots.txt
Step 2. Verify the downloaded robots.txt file.
User-agent: * Sitemap: https://devcoops.com/sitemap.xml
Now, if you have noticed, most of the third-party software that comes with curl installations, includes a remote bash script called install.sh and use redirection (pipe) to run the script. Let’s take the Node Version Manager
nvm curl command installation as an example:
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
-o specifies output file, the second dash
- after the
-o, points to the standard output, which means that the install.sh file will be downloaded and redirected as a standard input to the bash invocation.
As always, hope this was convenient enough. Feel free to leave a comment below and if you find this tutorial useful, follow our official channel on Telegram.