curl is a powerful command-line tool for transferring data with URL syntax.
It supports various protocols including HTTP, HTTPS, FTP, and many others.
curl [options] [URL]
Basic HTTP GET Request:
curl http://example.com
Save Output to a File:
curl -o output.html http://example.com
The -o option saves the content to a file named output.html.
Follow Redirects:
curl -L http://example.com
The -L option tells curl to follow HTTP redirects.
Include Headers in the Output:
$ curl -i http://example.com
HTTP/1.1 200 OK
Accept-Ranges: bytes
Age: 425101
Cache-Control: max-age=604800
Content-Type: text/html; charset=UTF-8
Date: Tue, 28 May 2024 16:46:42 GMT
Etag: "3147526947+gzip"
Expires: Tue, 04 Jun 2024 16:46:42 GMT
Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT
Server: ECAcc (nyd/D191)
Vary: Accept-Encoding
X-Cache: HIT
Content-Length: 1256
The -i option includes the HTTP response headers in the output.
Send a POST Request:
curl -X POST -d "param1=value1¶m2=value2" http://example.com
The -X POST option specifies the request method as POST, and -d specifies the data to send.
Send a JSON POST Request:
curl -X POST -H "Content-Type: application/json" -d '{"key1":"value1", "key2":"value2"}' http://example.com
The -H option adds a header to specify the content type as JSON.
Download a File:
curl -O http://example.com/file.zip
The -O option saves the file with its original name.
Send Custom Headers:
curl -H "Authorization: Bearer YOUR_TOKEN" http://example.com
The -H option sends custom headers, such as an authorization token.
Get Only HTTP Headers:
curl -I http://example.com
The -I option fetches only the HTTP headers.
Use a Proxy:
curl -x http://proxy.example.com:8080 http://example.com
The -x option specifies a proxy to use for the request.
Example
curl -s http://www.gutenberg.org/cache/epub/2701/pg2701.txt | gunzip | sed -r 's/\s+/\n/g' | grep -i whale | wc -l
Explanation:
curl -s http://www.gutenberg.org/files/2701/2701-0.txt:
Fetches the content of "Moby Dick" from Project Gutenberg silently (-s for silent mode).
sed 's/[^a-zA-Z]/\n/g':
Uses sed to replace any non-alphabet character with a newline, effectively placing each word on a new line.
grep -i whale:
Searches for the word "whale" in a case-insensitive manner (-i for case-insensitive).
wc -l:
Counts the number of lines, which corresponds to the number of occurrences of the word "whale".