lh | 9ed821d | 2023-04-07 01:36:19 -0700 | [diff] [blame] | 1 | LATEST VERSION |
| 2 | |
| 3 | You always find news about what's going on as well as the latest versions |
| 4 | from the curl web pages, located at: |
| 5 | |
| 6 | https://curl.haxx.se |
| 7 | |
| 8 | SIMPLE USAGE |
| 9 | |
| 10 | Get the main page from Netscape's web-server: |
| 11 | |
| 12 | curl http://www.netscape.com/ |
| 13 | |
| 14 | Get the README file the user's home directory at funet's ftp-server: |
| 15 | |
| 16 | curl ftp://ftp.funet.fi/README |
| 17 | |
| 18 | Get a web page from a server using port 8000: |
| 19 | |
| 20 | curl http://www.weirdserver.com:8000/ |
| 21 | |
| 22 | Get a directory listing of an FTP site: |
| 23 | |
| 24 | curl ftp://cool.haxx.se/ |
| 25 | |
| 26 | Get the definition of curl from a dictionary: |
| 27 | |
| 28 | curl dict://dict.org/m:curl |
| 29 | |
| 30 | Fetch two documents at once: |
| 31 | |
| 32 | curl ftp://cool.haxx.se/ http://www.weirdserver.com:8000/ |
| 33 | |
| 34 | Get a file off an FTPS server: |
| 35 | |
| 36 | curl ftps://files.are.secure.com/secrets.txt |
| 37 | |
| 38 | or use the more appropriate FTPS way to get the same file: |
| 39 | |
| 40 | curl --ftp-ssl ftp://files.are.secure.com/secrets.txt |
| 41 | |
| 42 | Get a file from an SSH server using SFTP: |
| 43 | |
| 44 | curl -u username sftp://example.com/etc/issue |
| 45 | |
| 46 | Get a file from an SSH server using SCP using a private key |
| 47 | (not password-protected) to authenticate: |
| 48 | |
| 49 | curl -u username: --key ~/.ssh/id_rsa \ |
| 50 | scp://example.com/~/file.txt |
| 51 | |
| 52 | Get a file from an SSH server using SCP using a private key |
| 53 | (password-protected) to authenticate: |
| 54 | |
| 55 | curl -u username: --key ~/.ssh/id_rsa --pass private_key_password \ |
| 56 | scp://example.com/~/file.txt |
| 57 | |
| 58 | Get the main page from an IPv6 web server: |
| 59 | |
| 60 | curl "http://[2001:1890:1112:1::20]/" |
| 61 | |
| 62 | Get a file from an SMB server: |
| 63 | |
| 64 | curl -u "domain\username:passwd" smb://server.example.com/share/file.txt |
| 65 | |
| 66 | DOWNLOAD TO A FILE |
| 67 | |
| 68 | Get a web page and store in a local file with a specific name: |
| 69 | |
| 70 | curl -o thatpage.html http://www.netscape.com/ |
| 71 | |
| 72 | Get a web page and store in a local file, make the local file get the name |
| 73 | of the remote document (if no file name part is specified in the URL, this |
| 74 | will fail): |
| 75 | |
| 76 | curl -O http://www.netscape.com/index.html |
| 77 | |
| 78 | Fetch two files and store them with their remote names: |
| 79 | |
| 80 | curl -O www.haxx.se/index.html -O curl.haxx.se/download.html |
| 81 | |
| 82 | USING PASSWORDS |
| 83 | |
| 84 | FTP |
| 85 | |
| 86 | To ftp files using name+passwd, include them in the URL like: |
| 87 | |
| 88 | curl ftp://name:passwd@machine.domain:port/full/path/to/file |
| 89 | |
| 90 | or specify them with the -u flag like |
| 91 | |
| 92 | curl -u name:passwd ftp://machine.domain:port/full/path/to/file |
| 93 | |
| 94 | FTPS |
| 95 | |
| 96 | It is just like for FTP, but you may also want to specify and use |
| 97 | SSL-specific options for certificates etc. |
| 98 | |
| 99 | Note that using FTPS:// as prefix is the "implicit" way as described in the |
| 100 | standards while the recommended "explicit" way is done by using FTP:// and |
| 101 | the --ftp-ssl option. |
| 102 | |
| 103 | SFTP / SCP |
| 104 | |
| 105 | This is similar to FTP, but you can use the --key option to specify a |
| 106 | private key to use instead of a password. Note that the private key may |
| 107 | itself be protected by a password that is unrelated to the login password |
| 108 | of the remote system; this password is specified using the --pass option. |
| 109 | Typically, curl will automatically extract the public key from the private |
| 110 | key file, but in cases where curl does not have the proper library support, |
| 111 | a matching public key file must be specified using the --pubkey option. |
| 112 | |
| 113 | HTTP |
| 114 | |
| 115 | Curl also supports user and password in HTTP URLs, thus you can pick a file |
| 116 | like: |
| 117 | |
| 118 | curl http://name:passwd@machine.domain/full/path/to/file |
| 119 | |
| 120 | or specify user and password separately like in |
| 121 | |
| 122 | curl -u name:passwd http://machine.domain/full/path/to/file |
| 123 | |
| 124 | HTTP offers many different methods of authentication and curl supports |
| 125 | several: Basic, Digest, NTLM and Negotiate (SPNEGO). Without telling which |
| 126 | method to use, curl defaults to Basic. You can also ask curl to pick the |
| 127 | most secure ones out of the ones that the server accepts for the given URL, |
| 128 | by using --anyauth. |
| 129 | |
| 130 | NOTE! According to the URL specification, HTTP URLs can not contain a user |
| 131 | and password, so that style will not work when using curl via a proxy, even |
| 132 | though curl allows it at other times. When using a proxy, you _must_ use |
| 133 | the -u style for user and password. |
| 134 | |
| 135 | HTTPS |
| 136 | |
| 137 | Probably most commonly used with private certificates, as explained below. |
| 138 | |
| 139 | PROXY |
| 140 | |
| 141 | curl supports both HTTP and SOCKS proxy servers, with optional authentication. |
| 142 | It does not have special support for FTP proxy servers since there are no |
| 143 | standards for those, but it can still be made to work with many of them. You |
| 144 | can also use both HTTP and SOCKS proxies to transfer files to and from FTP |
| 145 | servers. |
| 146 | |
| 147 | Get an ftp file using an HTTP proxy named my-proxy that uses port 888: |
| 148 | |
| 149 | curl -x my-proxy:888 ftp://ftp.leachsite.com/README |
| 150 | |
| 151 | Get a file from an HTTP server that requires user and password, using the |
| 152 | same proxy as above: |
| 153 | |
| 154 | curl -u user:passwd -x my-proxy:888 http://www.get.this/ |
| 155 | |
| 156 | Some proxies require special authentication. Specify by using -U as above: |
| 157 | |
| 158 | curl -U user:passwd -x my-proxy:888 http://www.get.this/ |
| 159 | |
| 160 | A comma-separated list of hosts and domains which do not use the proxy can |
| 161 | be specified as: |
| 162 | |
| 163 | curl --noproxy localhost,get.this -x my-proxy:888 http://www.get.this/ |
| 164 | |
| 165 | If the proxy is specified with --proxy1.0 instead of --proxy or -x, then |
| 166 | curl will use HTTP/1.0 instead of HTTP/1.1 for any CONNECT attempts. |
| 167 | |
| 168 | curl also supports SOCKS4 and SOCKS5 proxies with --socks4 and --socks5. |
| 169 | |
| 170 | See also the environment variables Curl supports that offer further proxy |
| 171 | control. |
| 172 | |
| 173 | Most FTP proxy servers are set up to appear as a normal FTP server from the |
| 174 | client's perspective, with special commands to select the remote FTP server. |
| 175 | curl supports the -u, -Q and --ftp-account options that can be used to |
| 176 | set up transfers through many FTP proxies. For example, a file can be |
| 177 | uploaded to a remote FTP server using a Blue Coat FTP proxy with the |
| 178 | options: |
| 179 | |
| 180 | curl -u "Remote-FTP-Username@remote.ftp.server Proxy-Username:Remote-Pass" \ |
| 181 | --ftp-account Proxy-Password --upload-file local-file \ |
| 182 | ftp://my-ftp.proxy.server:21/remote/upload/path/ |
| 183 | |
| 184 | See the manual for your FTP proxy to determine the form it expects to set up |
| 185 | transfers, and curl's -v option to see exactly what curl is sending. |
| 186 | |
| 187 | RANGES |
| 188 | |
| 189 | HTTP 1.1 introduced byte-ranges. Using this, a client can request |
| 190 | to get only one or more subparts of a specified document. Curl supports |
| 191 | this with the -r flag. |
| 192 | |
| 193 | Get the first 100 bytes of a document: |
| 194 | |
| 195 | curl -r 0-99 http://www.get.this/ |
| 196 | |
| 197 | Get the last 500 bytes of a document: |
| 198 | |
| 199 | curl -r -500 http://www.get.this/ |
| 200 | |
| 201 | Curl also supports simple ranges for FTP files as well. Then you can only |
| 202 | specify start and stop position. |
| 203 | |
| 204 | Get the first 100 bytes of a document using FTP: |
| 205 | |
| 206 | curl -r 0-99 ftp://www.get.this/README |
| 207 | |
| 208 | UPLOADING |
| 209 | |
| 210 | FTP / FTPS / SFTP / SCP |
| 211 | |
| 212 | Upload all data on stdin to a specified server: |
| 213 | |
| 214 | curl -T - ftp://ftp.upload.com/myfile |
| 215 | |
| 216 | Upload data from a specified file, login with user and password: |
| 217 | |
| 218 | curl -T uploadfile -u user:passwd ftp://ftp.upload.com/myfile |
| 219 | |
| 220 | Upload a local file to the remote site, and use the local file name at the remote |
| 221 | site too: |
| 222 | |
| 223 | curl -T uploadfile -u user:passwd ftp://ftp.upload.com/ |
| 224 | |
| 225 | Upload a local file to get appended to the remote file: |
| 226 | |
| 227 | curl -T localfile -a ftp://ftp.upload.com/remotefile |
| 228 | |
| 229 | Curl also supports ftp upload through a proxy, but only if the proxy is |
| 230 | configured to allow that kind of tunneling. If it does, you can run curl in |
| 231 | a fashion similar to: |
| 232 | |
| 233 | curl --proxytunnel -x proxy:port -T localfile ftp.upload.com |
| 234 | |
| 235 | SMB / SMBS |
| 236 | |
| 237 | curl -T file.txt -u "domain\username:passwd" |
| 238 | smb://server.example.com/share/ |
| 239 | |
| 240 | HTTP |
| 241 | |
| 242 | Upload all data on stdin to a specified HTTP site: |
| 243 | |
| 244 | curl -T - http://www.upload.com/myfile |
| 245 | |
| 246 | Note that the HTTP server must have been configured to accept PUT before |
| 247 | this can be done successfully. |
| 248 | |
| 249 | For other ways to do HTTP data upload, see the POST section below. |
| 250 | |
| 251 | VERBOSE / DEBUG |
| 252 | |
| 253 | If curl fails where it isn't supposed to, if the servers don't let you in, |
| 254 | if you can't understand the responses: use the -v flag to get verbose |
| 255 | fetching. Curl will output lots of info and what it sends and receives in |
| 256 | order to let the user see all client-server interaction (but it won't show |
| 257 | you the actual data). |
| 258 | |
| 259 | curl -v ftp://ftp.upload.com/ |
| 260 | |
| 261 | To get even more details and information on what curl does, try using the |
| 262 | --trace or --trace-ascii options with a given file name to log to, like |
| 263 | this: |
| 264 | |
| 265 | curl --trace trace.txt www.haxx.se |
| 266 | |
| 267 | |
| 268 | DETAILED INFORMATION |
| 269 | |
| 270 | Different protocols provide different ways of getting detailed information |
| 271 | about specific files/documents. To get curl to show detailed information |
| 272 | about a single file, you should use -I/--head option. It displays all |
| 273 | available info on a single file for HTTP and FTP. The HTTP information is a |
| 274 | lot more extensive. |
| 275 | |
| 276 | For HTTP, you can get the header information (the same as -I would show) |
| 277 | shown before the data by using -i/--include. Curl understands the |
| 278 | -D/--dump-header option when getting files from both FTP and HTTP, and it |
| 279 | will then store the headers in the specified file. |
| 280 | |
| 281 | Store the HTTP headers in a separate file (headers.txt in the example): |
| 282 | |
| 283 | curl --dump-header headers.txt curl.haxx.se |
| 284 | |
| 285 | Note that headers stored in a separate file can be very useful at a later |
| 286 | time if you want curl to use cookies sent by the server. More about that in |
| 287 | the cookies section. |
| 288 | |
| 289 | POST (HTTP) |
| 290 | |
| 291 | It's easy to post data using curl. This is done using the -d <data> |
| 292 | option. The post data must be urlencoded. |
| 293 | |
| 294 | Post a simple "name" and "phone" guestbook. |
| 295 | |
| 296 | curl -d "name=Rafael%20Sagula&phone=3320780" \ |
| 297 | http://www.where.com/guest.cgi |
| 298 | |
| 299 | How to post a form with curl, lesson #1: |
| 300 | |
| 301 | Dig out all the <input> tags in the form that you want to fill in. (There's |
| 302 | a perl program called formfind.pl on the curl site that helps with this). |
| 303 | |
| 304 | If there's a "normal" post, you use -d to post. -d takes a full "post |
| 305 | string", which is in the format |
| 306 | |
| 307 | <variable1>=<data1>&<variable2>=<data2>&... |
| 308 | |
| 309 | The 'variable' names are the names set with "name=" in the <input> tags, and |
| 310 | the data is the contents you want to fill in for the inputs. The data *must* |
| 311 | be properly URL encoded. That means you replace space with + and that you |
| 312 | replace weird letters with %XX where XX is the hexadecimal representation of |
| 313 | the letter's ASCII code. |
| 314 | |
| 315 | Example: |
| 316 | |
| 317 | (page located at http://www.formpost.com/getthis/ |
| 318 | |
| 319 | <form action="post.cgi" method="post"> |
| 320 | <input name=user size=10> |
| 321 | <input name=pass type=password size=10> |
| 322 | <input name=id type=hidden value="blablabla"> |
| 323 | <input name=ding value="submit"> |
| 324 | </form> |
| 325 | |
| 326 | We want to enter user 'foobar' with password '12345'. |
| 327 | |
| 328 | To post to this, you enter a curl command line like: |
| 329 | |
| 330 | curl -d "user=foobar&pass=12345&id=blablabla&ding=submit" (continues) |
| 331 | http://www.formpost.com/getthis/post.cgi |
| 332 | |
| 333 | |
| 334 | While -d uses the application/x-www-form-urlencoded mime-type, generally |
| 335 | understood by CGI's and similar, curl also supports the more capable |
| 336 | multipart/form-data type. This latter type supports things like file upload. |
| 337 | |
| 338 | -F accepts parameters like -F "name=contents". If you want the contents to |
| 339 | be read from a file, use <@filename> as contents. When specifying a file, |
| 340 | you can also specify the file content type by appending ';type=<mime type>' |
| 341 | to the file name. You can also post the contents of several files in one |
| 342 | field. For example, the field name 'coolfiles' is used to send three files, |
| 343 | with different content types using the following syntax: |
| 344 | |
| 345 | curl -F "coolfiles=@fil1.gif;type=image/gif,fil2.txt,fil3.html" \ |
| 346 | http://www.post.com/postit.cgi |
| 347 | |
| 348 | If the content-type is not specified, curl will try to guess from the file |
| 349 | extension (it only knows a few), or use the previously specified type (from |
| 350 | an earlier file if several files are specified in a list) or else it will |
| 351 | use the default type 'application/octet-stream'. |
| 352 | |
| 353 | Emulate a fill-in form with -F. Let's say you fill in three fields in a |
| 354 | form. One field is a file name which to post, one field is your name and one |
| 355 | field is a file description. We want to post the file we have written named |
| 356 | "cooltext.txt". To let curl do the posting of this data instead of your |
| 357 | favourite browser, you have to read the HTML source of the form page and |
| 358 | find the names of the input fields. In our example, the input field names |
| 359 | are 'file', 'yourname' and 'filedescription'. |
| 360 | |
| 361 | curl -F "file=@cooltext.txt" -F "yourname=Daniel" \ |
| 362 | -F "filedescription=Cool text file with cool text inside" \ |
| 363 | http://www.post.com/postit.cgi |
| 364 | |
| 365 | To send two files in one post you can do it in two ways: |
| 366 | |
| 367 | 1. Send multiple files in a single "field" with a single field name: |
| 368 | |
| 369 | curl -F "pictures=@dog.gif,cat.gif" |
| 370 | |
| 371 | 2. Send two fields with two field names: |
| 372 | |
| 373 | curl -F "docpicture=@dog.gif" -F "catpicture=@cat.gif" |
| 374 | |
| 375 | To send a field value literally without interpreting a leading '@' |
| 376 | or '<', or an embedded ';type=', use --form-string instead of |
| 377 | -F. This is recommended when the value is obtained from a user or |
| 378 | some other unpredictable source. Under these circumstances, using |
| 379 | -F instead of --form-string would allow a user to trick curl into |
| 380 | uploading a file. |
| 381 | |
| 382 | REFERRER |
| 383 | |
| 384 | An HTTP request has the option to include information about which address |
| 385 | referred it to the actual page. Curl allows you to specify the |
| 386 | referrer to be used on the command line. It is especially useful to |
| 387 | fool or trick stupid servers or CGI scripts that rely on that information |
| 388 | being available or contain certain data. |
| 389 | |
| 390 | curl -e www.coolsite.com http://www.showme.com/ |
| 391 | |
| 392 | NOTE: The Referer: [sic] field is defined in the HTTP spec to be a full URL. |
| 393 | |
| 394 | USER AGENT |
| 395 | |
| 396 | An HTTP request has the option to include information about the browser |
| 397 | that generated the request. Curl allows it to be specified on the command |
| 398 | line. It is especially useful to fool or trick stupid servers or CGI |
| 399 | scripts that only accept certain browsers. |
| 400 | |
| 401 | Example: |
| 402 | |
| 403 | curl -A 'Mozilla/3.0 (Win95; I)' http://www.nationsbank.com/ |
| 404 | |
| 405 | Other common strings: |
| 406 | 'Mozilla/3.0 (Win95; I)' Netscape Version 3 for Windows 95 |
| 407 | 'Mozilla/3.04 (Win95; U)' Netscape Version 3 for Windows 95 |
| 408 | 'Mozilla/2.02 (OS/2; U)' Netscape Version 2 for OS/2 |
| 409 | 'Mozilla/4.04 [en] (X11; U; AIX 4.2; Nav)' NS for AIX |
| 410 | 'Mozilla/4.05 [en] (X11; U; Linux 2.0.32 i586)' NS for Linux |
| 411 | |
| 412 | Note that Internet Explorer tries hard to be compatible in every way: |
| 413 | 'Mozilla/4.0 (compatible; MSIE 4.01; Windows 95)' MSIE for W95 |
| 414 | |
| 415 | Mozilla is not the only possible User-Agent name: |
| 416 | 'Konqueror/1.0' KDE File Manager desktop client |
| 417 | 'Lynx/2.7.1 libwww-FM/2.14' Lynx command line browser |
| 418 | |
| 419 | COOKIES |
| 420 | |
| 421 | Cookies are generally used by web servers to keep state information at the |
| 422 | client's side. The server sets cookies by sending a response line in the |
| 423 | headers that looks like 'Set-Cookie: <data>' where the data part then |
| 424 | typically contains a set of NAME=VALUE pairs (separated by semicolons ';' |
| 425 | like "NAME1=VALUE1; NAME2=VALUE2;"). The server can also specify for what |
| 426 | path the "cookie" should be used for (by specifying "path=value"), when the |
| 427 | cookie should expire ("expire=DATE"), for what domain to use it |
| 428 | ("domain=NAME") and if it should be used on secure connections only |
| 429 | ("secure"). |
| 430 | |
| 431 | If you've received a page from a server that contains a header like: |
| 432 | Set-Cookie: sessionid=boo123; path="/foo"; |
| 433 | |
| 434 | it means the server wants that first pair passed on when we get anything in |
| 435 | a path beginning with "/foo". |
| 436 | |
| 437 | Example, get a page that wants my name passed in a cookie: |
| 438 | |
| 439 | curl -b "name=Daniel" www.sillypage.com |
| 440 | |
| 441 | Curl also has the ability to use previously received cookies in following |
| 442 | sessions. If you get cookies from a server and store them in a file in a |
| 443 | manner similar to: |
| 444 | |
| 445 | curl --dump-header headers www.example.com |
| 446 | |
| 447 | ... you can then in a second connect to that (or another) site, use the |
| 448 | cookies from the 'headers' file like: |
| 449 | |
| 450 | curl -b headers www.example.com |
| 451 | |
| 452 | While saving headers to a file is a working way to store cookies, it is |
| 453 | however error-prone and not the preferred way to do this. Instead, make curl |
| 454 | save the incoming cookies using the well-known netscape cookie format like |
| 455 | this: |
| 456 | |
| 457 | curl -c cookies.txt www.example.com |
| 458 | |
| 459 | Note that by specifying -b you enable the "cookie awareness" and with -L |
| 460 | you can make curl follow a location: (which often is used in combination |
| 461 | with cookies). So that if a site sends cookies and a location, you can |
| 462 | use a non-existing file to trigger the cookie awareness like: |
| 463 | |
| 464 | curl -L -b empty.txt www.example.com |
| 465 | |
| 466 | The file to read cookies from must be formatted using plain HTTP headers OR |
| 467 | as netscape's cookie file. Curl will determine what kind it is based on the |
| 468 | file contents. In the above command, curl will parse the header and store |
| 469 | the cookies received from www.example.com. curl will send to the server the |
| 470 | stored cookies which match the request as it follows the location. The |
| 471 | file "empty.txt" may be a nonexistent file. |
| 472 | |
| 473 | To read and write cookies from a netscape cookie file, you can set both -b |
| 474 | and -c to use the same file: |
| 475 | |
| 476 | curl -b cookies.txt -c cookies.txt www.example.com |
| 477 | |
| 478 | PROGRESS METER |
| 479 | |
| 480 | The progress meter exists to show a user that something actually is |
| 481 | happening. The different fields in the output have the following meaning: |
| 482 | |
| 483 | % Total % Received % Xferd Average Speed Time Curr. |
| 484 | Dload Upload Total Current Left Speed |
| 485 | 0 151M 0 38608 0 0 9406 0 4:41:43 0:00:04 4:41:39 9287 |
| 486 | |
| 487 | From left-to-right: |
| 488 | % - percentage completed of the whole transfer |
| 489 | Total - total size of the whole expected transfer |
| 490 | % - percentage completed of the download |
| 491 | Received - currently downloaded amount of bytes |
| 492 | % - percentage completed of the upload |
| 493 | Xferd - currently uploaded amount of bytes |
| 494 | Average Speed |
| 495 | Dload - the average transfer speed of the download |
| 496 | Average Speed |
| 497 | Upload - the average transfer speed of the upload |
| 498 | Time Total - expected time to complete the operation |
| 499 | Time Current - time passed since the invoke |
| 500 | Time Left - expected time left to completion |
| 501 | Curr.Speed - the average transfer speed the last 5 seconds (the first |
| 502 | 5 seconds of a transfer is based on less time of course.) |
| 503 | |
| 504 | The -# option will display a totally different progress bar that doesn't |
| 505 | need much explanation! |
| 506 | |
| 507 | SPEED LIMIT |
| 508 | |
| 509 | Curl allows the user to set the transfer speed conditions that must be met |
| 510 | to let the transfer keep going. By using the switch -y and -Y you |
| 511 | can make curl abort transfers if the transfer speed is below the specified |
| 512 | lowest limit for a specified time. |
| 513 | |
| 514 | To have curl abort the download if the speed is slower than 3000 bytes per |
| 515 | second for 1 minute, run: |
| 516 | |
| 517 | curl -Y 3000 -y 60 www.far-away-site.com |
| 518 | |
| 519 | This can very well be used in combination with the overall time limit, so |
| 520 | that the above operation must be completed in whole within 30 minutes: |
| 521 | |
| 522 | curl -m 1800 -Y 3000 -y 60 www.far-away-site.com |
| 523 | |
| 524 | Forcing curl not to transfer data faster than a given rate is also possible, |
| 525 | which might be useful if you're using a limited bandwidth connection and you |
| 526 | don't want your transfer to use all of it (sometimes referred to as |
| 527 | "bandwidth throttle"). |
| 528 | |
| 529 | Make curl transfer data no faster than 10 kilobytes per second: |
| 530 | |
| 531 | curl --limit-rate 10K www.far-away-site.com |
| 532 | |
| 533 | or |
| 534 | |
| 535 | curl --limit-rate 10240 www.far-away-site.com |
| 536 | |
| 537 | Or prevent curl from uploading data faster than 1 megabyte per second: |
| 538 | |
| 539 | curl -T upload --limit-rate 1M ftp://uploadshereplease.com |
| 540 | |
| 541 | When using the --limit-rate option, the transfer rate is regulated on a |
| 542 | per-second basis, which will cause the total transfer speed to become lower |
| 543 | than the given number. Sometimes of course substantially lower, if your |
| 544 | transfer stalls during periods. |
| 545 | |
| 546 | CONFIG FILE |
| 547 | |
| 548 | Curl automatically tries to read the .curlrc file (or _curlrc file on win32 |
| 549 | systems) from the user's home dir on startup. |
| 550 | |
| 551 | The config file could be made up with normal command line switches, but you |
| 552 | can also specify the long options without the dashes to make it more |
| 553 | readable. You can separate the options and the parameter with spaces, or |
| 554 | with = or :. Comments can be used within the file. If the first letter on a |
| 555 | line is a '#'-symbol the rest of the line is treated as a comment. |
| 556 | |
| 557 | If you want the parameter to contain spaces, you must enclose the entire |
| 558 | parameter within double quotes ("). Within those quotes, you specify a |
| 559 | quote as \". |
| 560 | |
| 561 | NOTE: You must specify options and their arguments on the same line. |
| 562 | |
| 563 | Example, set default time out and proxy in a config file: |
| 564 | |
| 565 | # We want a 30 minute timeout: |
| 566 | -m 1800 |
| 567 | # ... and we use a proxy for all accesses: |
| 568 | proxy = proxy.our.domain.com:8080 |
| 569 | |
| 570 | White spaces ARE significant at the end of lines, but all white spaces |
| 571 | leading up to the first characters of each line are ignored. |
| 572 | |
| 573 | Prevent curl from reading the default file by using -q as the first command |
| 574 | line parameter, like: |
| 575 | |
| 576 | curl -q www.thatsite.com |
| 577 | |
| 578 | Force curl to get and display a local help page in case it is invoked |
| 579 | without URL by making a config file similar to: |
| 580 | |
| 581 | # default url to get |
| 582 | url = "http://help.with.curl.com/curlhelp.html" |
| 583 | |
| 584 | You can specify another config file to be read by using the -K/--config |
| 585 | flag. If you set config file name to "-" it'll read the config from stdin, |
| 586 | which can be handy if you want to hide options from being visible in process |
| 587 | tables etc: |
| 588 | |
| 589 | echo "user = user:passwd" | curl -K - http://that.secret.site.com |
| 590 | |
| 591 | EXTRA HEADERS |
| 592 | |
| 593 | When using curl in your own very special programs, you may end up needing |
| 594 | to pass on your own custom headers when getting a web page. You can do |
| 595 | this by using the -H flag. |
| 596 | |
| 597 | Example, send the header "X-you-and-me: yes" to the server when getting a |
| 598 | page: |
| 599 | |
| 600 | curl -H "X-you-and-me: yes" www.love.com |
| 601 | |
| 602 | This can also be useful in case you want curl to send a different text in a |
| 603 | header than it normally does. The -H header you specify then replaces the |
| 604 | header curl would normally send. If you replace an internal header with an |
| 605 | empty one, you prevent that header from being sent. To prevent the Host: |
| 606 | header from being used: |
| 607 | |
| 608 | curl -H "Host:" www.server.com |
| 609 | |
| 610 | FTP and PATH NAMES |
| 611 | |
| 612 | Do note that when getting files with the ftp:// URL, the given path is |
| 613 | relative the directory you enter. To get the file 'README' from your home |
| 614 | directory at your ftp site, do: |
| 615 | |
| 616 | curl ftp://user:passwd@my.site.com/README |
| 617 | |
| 618 | But if you want the README file from the root directory of that very same |
| 619 | site, you need to specify the absolute file name: |
| 620 | |
| 621 | curl ftp://user:passwd@my.site.com//README |
| 622 | |
| 623 | (I.e with an extra slash in front of the file name.) |
| 624 | |
| 625 | SFTP and SCP and PATH NAMES |
| 626 | |
| 627 | With sftp: and scp: URLs, the path name given is the absolute name on the |
| 628 | server. To access a file relative to the remote user's home directory, |
| 629 | prefix the file with /~/ , such as: |
| 630 | |
| 631 | curl -u $USER sftp://home.example.com/~/.bashrc |
| 632 | |
| 633 | FTP and firewalls |
| 634 | |
| 635 | The FTP protocol requires one of the involved parties to open a second |
| 636 | connection as soon as data is about to get transferred. There are two ways to |
| 637 | do this. |
| 638 | |
| 639 | The default way for curl is to issue the PASV command which causes the |
| 640 | server to open another port and await another connection performed by the |
| 641 | client. This is good if the client is behind a firewall that doesn't allow |
| 642 | incoming connections. |
| 643 | |
| 644 | curl ftp.download.com |
| 645 | |
| 646 | If the server, for example, is behind a firewall that doesn't allow connections |
| 647 | on ports other than 21 (or if it just doesn't support the PASV command), the |
| 648 | other way to do it is to use the PORT command and instruct the server to |
| 649 | connect to the client on the given IP number and port (as parameters to the |
| 650 | PORT command). |
| 651 | |
| 652 | The -P flag to curl supports a few different options. Your machine may have |
| 653 | several IP-addresses and/or network interfaces and curl allows you to select |
| 654 | which of them to use. Default address can also be used: |
| 655 | |
| 656 | curl -P - ftp.download.com |
| 657 | |
| 658 | Download with PORT but use the IP address of our 'le0' interface (this does |
| 659 | not work on windows): |
| 660 | |
| 661 | curl -P le0 ftp.download.com |
| 662 | |
| 663 | Download with PORT but use 192.168.0.10 as our IP address to use: |
| 664 | |
| 665 | curl -P 192.168.0.10 ftp.download.com |
| 666 | |
| 667 | NETWORK INTERFACE |
| 668 | |
| 669 | Get a web page from a server using a specified port for the interface: |
| 670 | |
| 671 | curl --interface eth0:1 http://www.netscape.com/ |
| 672 | |
| 673 | or |
| 674 | |
| 675 | curl --interface 192.168.1.10 http://www.netscape.com/ |
| 676 | |
| 677 | HTTPS |
| 678 | |
| 679 | Secure HTTP requires SSL libraries to be installed and used when curl is |
| 680 | built. If that is done, curl is capable of retrieving and posting documents |
| 681 | using the HTTPS protocol. |
| 682 | |
| 683 | Example: |
| 684 | |
| 685 | curl https://www.secure-site.com |
| 686 | |
| 687 | Curl is also capable of using your personal certificates to get/post files |
| 688 | from sites that require valid certificates. The only drawback is that the |
| 689 | certificate needs to be in PEM-format. PEM is a standard and open format to |
| 690 | store certificates with, but it is not used by the most commonly used |
| 691 | browsers (Netscape and MSIE both use the so called PKCS#12 format). If you |
| 692 | want curl to use the certificates you use with your (favourite) browser, you |
| 693 | may need to download/compile a converter that can convert your browser's |
| 694 | formatted certificates to PEM formatted ones. This kind of converter is |
| 695 | included in recent versions of OpenSSL, and for older versions Dr Stephen |
| 696 | N. Henson has written a patch for SSLeay that adds this functionality. You |
| 697 | can get his patch (that requires an SSLeay installation) from his site at: |
| 698 | http://www.drh-consultancy.demon.co.uk/ |
| 699 | |
| 700 | Example on how to automatically retrieve a document using a certificate with |
| 701 | a personal password: |
| 702 | |
| 703 | curl -E /path/to/cert.pem:password https://secure.site.com/ |
| 704 | |
| 705 | If you neglect to specify the password on the command line, you will be |
| 706 | prompted for the correct password before any data can be received. |
| 707 | |
| 708 | Many older SSL-servers have problems with SSLv3 or TLS, which newer versions |
| 709 | of OpenSSL etc use, therefore it is sometimes useful to specify what |
| 710 | SSL-version curl should use. Use -3, -2 or -1 to specify that exact SSL |
| 711 | version to use (for SSLv3, SSLv2 or TLSv1 respectively): |
| 712 | |
| 713 | curl -2 https://secure.site.com/ |
| 714 | |
| 715 | Otherwise, curl will first attempt to use v3 and then v2. |
| 716 | |
| 717 | To use OpenSSL to convert your favourite browser's certificate into a PEM |
| 718 | formatted one that curl can use, do something like this: |
| 719 | |
| 720 | In Netscape, you start with hitting the 'Security' menu button. |
| 721 | |
| 722 | Select 'certificates->yours' and then pick a certificate in the list |
| 723 | |
| 724 | Press the 'Export' button |
| 725 | |
| 726 | enter your PIN code for the certs |
| 727 | |
| 728 | select a proper place to save it |
| 729 | |
| 730 | Run the 'openssl' application to convert the certificate. If you cd to the |
| 731 | openssl installation, you can do it like: |
| 732 | |
| 733 | # ./apps/openssl pkcs12 -in [file you saved] -clcerts -out [PEMfile] |
| 734 | |
| 735 | In Firefox, select Options, then Advanced, then the Encryption tab, |
| 736 | View Certificates. This opens the Certificate Manager, where you can |
| 737 | Export. Be sure to select PEM for the Save as type. |
| 738 | |
| 739 | In Internet Explorer, select Internet Options, then the Content tab, then |
| 740 | Certificates. Then you can Export, and depending on the format you may |
| 741 | need to convert to PEM. |
| 742 | |
| 743 | In Chrome, select Settings, then Show Advanced Settings. Under HTTPS/SSL |
| 744 | select Manage Certificates. |
| 745 | |
| 746 | RESUMING FILE TRANSFERS |
| 747 | |
| 748 | To continue a file transfer where it was previously aborted, curl supports |
| 749 | resume on HTTP(S) downloads as well as FTP uploads and downloads. |
| 750 | |
| 751 | Continue downloading a document: |
| 752 | |
| 753 | curl -C - -o file ftp://ftp.server.com/path/file |
| 754 | |
| 755 | Continue uploading a document(*1): |
| 756 | |
| 757 | curl -C - -T file ftp://ftp.server.com/path/file |
| 758 | |
| 759 | Continue downloading a document from a web server(*2): |
| 760 | |
| 761 | curl -C - -o file http://www.server.com/ |
| 762 | |
| 763 | (*1) = This requires that the FTP server supports the non-standard command |
| 764 | SIZE. If it doesn't, curl will say so. |
| 765 | |
| 766 | (*2) = This requires that the web server supports at least HTTP/1.1. If it |
| 767 | doesn't, curl will say so. |
| 768 | |
| 769 | TIME CONDITIONS |
| 770 | |
| 771 | HTTP allows a client to specify a time condition for the document it |
| 772 | requests. It is If-Modified-Since or If-Unmodified-Since. Curl allows you to |
| 773 | specify them with the -z/--time-cond flag. |
| 774 | |
| 775 | For example, you can easily make a download that only gets performed if the |
| 776 | remote file is newer than a local copy. It would be made like: |
| 777 | |
| 778 | curl -z local.html http://remote.server.com/remote.html |
| 779 | |
| 780 | Or you can download a file only if the local file is newer than the remote |
| 781 | one. Do this by prepending the date string with a '-', as in: |
| 782 | |
| 783 | curl -z -local.html http://remote.server.com/remote.html |
| 784 | |
| 785 | You can specify a "free text" date as condition. Tell curl to only download |
| 786 | the file if it was updated since January 12, 2012: |
| 787 | |
| 788 | curl -z "Jan 12 2012" http://remote.server.com/remote.html |
| 789 | |
| 790 | Curl will then accept a wide range of date formats. You always make the date |
| 791 | check the other way around by prepending it with a dash '-'. |
| 792 | |
| 793 | DICT |
| 794 | |
| 795 | For fun try |
| 796 | |
| 797 | curl dict://dict.org/m:curl |
| 798 | curl dict://dict.org/d:heisenbug:jargon |
| 799 | curl dict://dict.org/d:daniel:web1913 |
| 800 | |
| 801 | Aliases for 'm' are 'match' and 'find', and aliases for 'd' are 'define' |
| 802 | and 'lookup'. For example, |
| 803 | |
| 804 | curl dict://dict.org/find:curl |
| 805 | |
| 806 | Commands that break the URL description of the RFC (but not the DICT |
| 807 | protocol) are |
| 808 | |
| 809 | curl dict://dict.org/show:db |
| 810 | curl dict://dict.org/show:strat |
| 811 | |
| 812 | Authentication is still missing (but this is not required by the RFC) |
| 813 | |
| 814 | LDAP |
| 815 | |
| 816 | If you have installed the OpenLDAP library, curl can take advantage of it |
| 817 | and offer ldap:// support. |
| 818 | On Windows, curl will use WinLDAP from Platform SDK by default. |
| 819 | |
| 820 | Default protocol version used by curl is LDAPv3. LDAPv2 will be used as |
| 821 | fallback mechanism in case if LDAPv3 will fail to connect. |
| 822 | |
| 823 | LDAP is a complex thing and writing an LDAP query is not an easy task. I do |
| 824 | advise you to dig up the syntax description for that elsewhere. One such |
| 825 | place might be: |
| 826 | |
| 827 | RFC 2255, "The LDAP URL Format" https://curl.haxx.se/rfc/rfc2255.txt |
| 828 | |
| 829 | To show you an example, this is how I can get all people from my local LDAP |
| 830 | server that has a certain sub-domain in their email address: |
| 831 | |
| 832 | curl -B "ldap://ldap.frontec.se/o=frontec??sub?mail=*sth.frontec.se" |
| 833 | |
| 834 | If I want the same info in HTML format, I can get it by not using the -B |
| 835 | (enforce ASCII) flag. |
| 836 | |
| 837 | You also can use authentication when accessing LDAP catalog: |
| 838 | |
| 839 | curl -u user:passwd "ldap://ldap.frontec.se/o=frontec??sub?mail=*" |
| 840 | curl "ldap://user:passwd@ldap.frontec.se/o=frontec??sub?mail=*" |
| 841 | |
| 842 | By default, if user and password provided, OpenLDAP/WinLDAP will use basic |
| 843 | authentication. On Windows you can control this behavior by providing |
| 844 | one of --basic, --ntlm or --digest option in curl command line |
| 845 | |
| 846 | curl --ntlm "ldap://user:passwd@ldap.frontec.se/o=frontec??sub?mail=*" |
| 847 | |
| 848 | On Windows, if no user/password specified, auto-negotiation mechanism will |
| 849 | be used with current logon credentials (SSPI/SPNEGO). |
| 850 | |
| 851 | ENVIRONMENT VARIABLES |
| 852 | |
| 853 | Curl reads and understands the following environment variables: |
| 854 | |
| 855 | http_proxy, HTTPS_PROXY, FTP_PROXY |
| 856 | |
| 857 | They should be set for protocol-specific proxies. General proxy should be |
| 858 | set with |
| 859 | |
| 860 | ALL_PROXY |
| 861 | |
| 862 | A comma-separated list of host names that shouldn't go through any proxy is |
| 863 | set in (only an asterisk, '*' matches all hosts) |
| 864 | |
| 865 | NO_PROXY |
| 866 | |
| 867 | If the host name matches one of these strings, or the host is within the |
| 868 | domain of one of these strings, transactions with that node will not be |
| 869 | proxied. When a domain is used, it needs to start with a period. A user can |
| 870 | specify that both www.example.com and foo.example.com should not uses a |
| 871 | proxy by setting NO_PROXY to ".example.com". By including the full name you |
| 872 | can exclude specific host names, so to make www.example.com not use a proxy |
| 873 | but still have foo.example.com do it, set NO_PROXY to "www.example.com" |
| 874 | |
| 875 | The usage of the -x/--proxy flag overrides the environment variables. |
| 876 | |
| 877 | NETRC |
| 878 | |
| 879 | Unix introduced the .netrc concept a long time ago. It is a way for a user |
| 880 | to specify name and password for commonly visited FTP sites in a file so |
| 881 | that you don't have to type them in each time you visit those sites. You |
| 882 | realize this is a big security risk if someone else gets hold of your |
| 883 | passwords, so therefore most unix programs won't read this file unless it is |
| 884 | only readable by yourself (curl doesn't care though). |
| 885 | |
| 886 | Curl supports .netrc files if told to (using the -n/--netrc and |
| 887 | --netrc-optional options). This is not restricted to just FTP, |
| 888 | so curl can use it for all protocols where authentication is used. |
| 889 | |
| 890 | A very simple .netrc file could look something like: |
| 891 | |
| 892 | machine curl.haxx.se login iamdaniel password mysecret |
| 893 | |
| 894 | CUSTOM OUTPUT |
| 895 | |
| 896 | To better allow script programmers to get to know about the progress of |
| 897 | curl, the -w/--write-out option was introduced. Using this, you can specify |
| 898 | what information from the previous transfer you want to extract. |
| 899 | |
| 900 | To display the amount of bytes downloaded together with some text and an |
| 901 | ending newline: |
| 902 | |
| 903 | curl -w 'We downloaded %{size_download} bytes\n' www.download.com |
| 904 | |
| 905 | KERBEROS FTP TRANSFER |
| 906 | |
| 907 | Curl supports kerberos4 and kerberos5/GSSAPI for FTP transfers. You need |
| 908 | the kerberos package installed and used at curl build time for it to be |
| 909 | available. |
| 910 | |
| 911 | First, get the krb-ticket the normal way, like with the kinit/kauth tool. |
| 912 | Then use curl in way similar to: |
| 913 | |
| 914 | curl --krb private ftp://krb4site.com -u username:fakepwd |
| 915 | |
| 916 | There's no use for a password on the -u switch, but a blank one will make |
| 917 | curl ask for one and you already entered the real password to kinit/kauth. |
| 918 | |
| 919 | TELNET |
| 920 | |
| 921 | The curl telnet support is basic and very easy to use. Curl passes all data |
| 922 | passed to it on stdin to the remote server. Connect to a remote telnet |
| 923 | server using a command line similar to: |
| 924 | |
| 925 | curl telnet://remote.server.com |
| 926 | |
| 927 | And enter the data to pass to the server on stdin. The result will be sent |
| 928 | to stdout or to the file you specify with -o. |
| 929 | |
| 930 | You might want the -N/--no-buffer option to switch off the buffered output |
| 931 | for slow connections or similar. |
| 932 | |
| 933 | Pass options to the telnet protocol negotiation, by using the -t option. To |
| 934 | tell the server we use a vt100 terminal, try something like: |
| 935 | |
| 936 | curl -tTTYPE=vt100 telnet://remote.server.com |
| 937 | |
| 938 | Other interesting options for it -t include: |
| 939 | |
| 940 | - XDISPLOC=<X display> Sets the X display location. |
| 941 | |
| 942 | - NEW_ENV=<var,val> Sets an environment variable. |
| 943 | |
| 944 | NOTE: The telnet protocol does not specify any way to login with a specified |
| 945 | user and password so curl can't do that automatically. To do that, you need |
| 946 | to track when the login prompt is received and send the username and |
| 947 | password accordingly. |
| 948 | |
| 949 | PERSISTENT CONNECTIONS |
| 950 | |
| 951 | Specifying multiple files on a single command line will make curl transfer |
| 952 | all of them, one after the other in the specified order. |
| 953 | |
| 954 | libcurl will attempt to use persistent connections for the transfers so that |
| 955 | the second transfer to the same host can use the same connection that was |
| 956 | already initiated and was left open in the previous transfer. This greatly |
| 957 | decreases connection time for all but the first transfer and it makes a far |
| 958 | better use of the network. |
| 959 | |
| 960 | Note that curl cannot use persistent connections for transfers that are used |
| 961 | in subsequence curl invokes. Try to stuff as many URLs as possible on the |
| 962 | same command line if they are using the same host, as that'll make the |
| 963 | transfers faster. If you use an HTTP proxy for file transfers, practically |
| 964 | all transfers will be persistent. |
| 965 | |
| 966 | MULTIPLE TRANSFERS WITH A SINGLE COMMAND LINE |
| 967 | |
| 968 | As is mentioned above, you can download multiple files with one command line |
| 969 | by simply adding more URLs. If you want those to get saved to a local file |
| 970 | instead of just printed to stdout, you need to add one save option for each |
| 971 | URL you specify. Note that this also goes for the -O option (but not |
| 972 | --remote-name-all). |
| 973 | |
| 974 | For example: get two files and use -O for the first and a custom file |
| 975 | name for the second: |
| 976 | |
| 977 | curl -O http://url.com/file.txt ftp://ftp.com/moo.exe -o moo.jpg |
| 978 | |
| 979 | You can also upload multiple files in a similar fashion: |
| 980 | |
| 981 | curl -T local1 ftp://ftp.com/moo.exe -T local2 ftp://ftp.com/moo2.txt |
| 982 | |
| 983 | IPv6 |
| 984 | |
| 985 | curl will connect to a server with IPv6 when a host lookup returns an IPv6 |
| 986 | address and fall back to IPv4 if the connection fails. The --ipv4 and --ipv6 |
| 987 | options can specify which address to use when both are available. IPv6 |
| 988 | addresses can also be specified directly in URLs using the syntax: |
| 989 | |
| 990 | http://[2001:1890:1112:1::20]/overview.html |
| 991 | |
| 992 | When this style is used, the -g option must be given to stop curl from |
| 993 | interpreting the square brackets as special globbing characters. Link local |
| 994 | and site local addresses including a scope identifier, such as fe80::1234%1, |
| 995 | may also be used, but the scope portion must be numeric or match an existing |
| 996 | network interface on Linux and the percent character must be URL escaped. The |
| 997 | previous example in an SFTP URL might look like: |
| 998 | |
| 999 | sftp://[fe80::1234%251]/ |
| 1000 | |
| 1001 | IPv6 addresses provided other than in URLs (e.g. to the --proxy, --interface |
| 1002 | or --ftp-port options) should not be URL encoded. |
| 1003 | |
| 1004 | METALINK |
| 1005 | |
| 1006 | Curl supports Metalink (both version 3 and 4 (RFC 5854) are supported), a way |
| 1007 | to list multiple URIs and hashes for a file. Curl will make use of the mirrors |
| 1008 | listed within for failover if there are errors (such as the file or server not |
| 1009 | being available). It will also verify the hash of the file after the download |
| 1010 | completes. The Metalink file itself is downloaded and processed in memory and |
| 1011 | not stored in the local file system. |
| 1012 | |
| 1013 | Example to use a remote Metalink file: |
| 1014 | |
| 1015 | curl --metalink http://www.example.com/example.metalink |
| 1016 | |
| 1017 | To use a Metalink file in the local file system, use FILE protocol (file://): |
| 1018 | |
| 1019 | curl --metalink file://example.metalink |
| 1020 | |
| 1021 | Please note that if FILE protocol is disabled, there is no way to use a local |
| 1022 | Metalink file at the time of this writing. Also note that if --metalink and |
| 1023 | --include are used together, --include will be ignored. This is because including |
| 1024 | headers in the response will break Metalink parser and if the headers are included |
| 1025 | in the file described in Metalink file, hash check will fail. |
| 1026 | |
| 1027 | MAILING LISTS |
| 1028 | |
| 1029 | For your convenience, we have several open mailing lists to discuss curl, |
| 1030 | its development and things relevant to this. Get all info at |
| 1031 | https://curl.haxx.se/mail/. Some of the lists available are: |
| 1032 | |
| 1033 | curl-users |
| 1034 | |
| 1035 | Users of the command line tool. How to use it, what doesn't work, new |
| 1036 | features, related tools, questions, news, installations, compilations, |
| 1037 | running, porting etc. |
| 1038 | |
| 1039 | curl-library |
| 1040 | |
| 1041 | Developers using or developing libcurl. Bugs, extensions, improvements. |
| 1042 | |
| 1043 | curl-announce |
| 1044 | |
| 1045 | Low-traffic. Only receives announcements of new public versions. At worst, |
| 1046 | that makes something like one or two mails per month, but usually only one |
| 1047 | mail every second month. |
| 1048 | |
| 1049 | curl-and-php |
| 1050 | |
| 1051 | Using the curl functions in PHP. Everything curl with a PHP angle. Or PHP |
| 1052 | with a curl angle. |
| 1053 | |
| 1054 | curl-and-python |
| 1055 | |
| 1056 | Python hackers using curl with or without the python binding pycurl. |
| 1057 | |
| 1058 | Please direct curl questions, feature requests and trouble reports to one of |
| 1059 | these mailing lists instead of mailing any individual. |