curl 检测代理

curl -m 10 -o out.html --socks4


curl   shell 抓取网页命令

-m 10    连接超时时间,最大为10秒

-o out.html    抓取网页输出到 out.html

--socks4   代理协议采用 --socks4


此外,curl 7.21.7版本以上,还支持代理协议格式如下:

curl -m 10 -o out.html -x socks4://

curl -m 10 -o out.html --proxy socks4://


更多请详见米扑代理的代码示例   (Shell curl wget)



shell 完整脚本代码

# sudo apt-get install autossh
# at 2015-02-09

# remote host and port

# proxy ip addr

# proxy port

# proxy spider url


function __help(){
    echo 'usage:
    proxy [run | show | check | log]
    proxy run       # run proxy
    proxy show      # show proxy information
    proxy check     # check proxy status
    proxy log       # show proxy log'

function __show(){
    OUTPUT=`ps -ef | grep -v "grep" | grep "autossh"`
    if [[ -n $OUTPUT ]]; then
        echo $OUTPUT
        echo 'Proxy not working'

# options for autossh:
# -M      specifies the base monitoring port to use.
# -f      run in background (autossh handles this, and does not pass it to ssh.)
# options for ssh:
# -q      Quiet mode.
# -T      Disable pseudo-tty allocation.
# -f      Requests ssh to go to background just before command execution.
# -n      Redirects stdin from /dev/null (actually, prevents reading from stdin).  This must be used when ssh is run in the background.
# -N      Do not execute a remote command.  This is useful for just forwarding ports (protocol version 2 only).
# -g      Allows remote hosts to connect to local forwarded ports.
# -D port Specifies a local dynamic application-level port forwarding.

function __start(){
    echo 'try to connect remote host...'

function __check(){
    cmd_4="curl $CHECK_URL -m 10 --socks4 $IP:$PORT"
    cmd_5="curl $CHECK_URL -m 10 --socks5 $IP:$PORT"
    echo "cmd_4 : $cmd_4"
    echo "try to connect proxy $IP:$PORT"

    dtime=$(date +'%Y-%m-%d__%H:%M:%S')
    echo $dtime
    if [[ -n $RESPONSE ]]; then
        echo $RESPONSE
        echo 'Proxy is working...'
        echo "$dtime -- Proxy is working" >> $LOG
        echo 'Proxy not working...'
        echo "$dtime -- Proxy not working" >> $LOG

if [[ $# -eq 0 ]]; then
    case $action in
            tail -n 10 $log


curl 相关参数

homer@ubuntu:~/script$ curl --help
Usage: curl [options...] <url>
Options: (H) means HTTP/HTTPS only, (F) means FTP only
 -G, --get           Send the -d data with a HTTP GET (H)
 -H, --header LINE   Custom header to pass to server (H)
 -I, --head          Show document info only
 -h, --help          This help text
 -0, --http1.0       Use HTTP 1.0 (H)
     --http1.1       Use HTTP 1.1 (H)
     --http2.0       Use HTTP 2.0 (H)
 -4, --ipv4          Resolve name to IPv4 address
 -6, --ipv6          Resolve name to IPv6 address
 -l, --list-only     List only mode (F/POP3)
     --local-port RANGE  Force use of these local port numbers
 -L, --location      Follow redirects (H)
 -M, --manual        Display the full manual
     --mail-from FROM  Mail from this address (SMTP)
     --mail-rcpt TO  Mail to this/these addresses (SMTP)
     --mail-auth AUTH  Originator address of the original email (SMTP)
 -m, --max-time SECONDS  Maximum time allowed for the transfer
 -o, --output FILE   Write output to <file> instead of stdout
     --pass PASS     Pass phrase for the private key (SSL/SSH)
 -#, --progress-bar  Display transfer progress as a progress bar
     --proto PROTOCOLS  Enable/disable specified protocols
     --proto-redir PROTOCOLS  Enable/disable specified protocols on redirect
 -x, --proxy [PROTOCOL://]HOST[:PORT] Use proxy on given port
     --proxy-anyauth Pick "any" proxy authentication method (H)
     --proxy-basic   Use Basic authentication on the proxy (H)
     --proxy-digest  Use Digest authentication on the proxy (H)
     --proxy-negotiate Use Negotiate authentication on the proxy (H)
     --proxy-ntlm    Use NTLM authentication on the proxy (H)
 -U, --proxy-user USER[:PASSWORD]  Proxy user and password
     --proxy1.0 HOST[:PORT]  Use HTTP/1.0 proxy on given port
 -p, --proxytunnel   Operate through a HTTP proxy tunnel (using CONNECT)
     --pubkey KEY    Public key file name (SSH)
 -e, --referer       Referer URL (H)
 -J, --remote-header-name Use the header-provided filename (H)
 -O, --remote-name   Write output to a file named as the remote file
     --remote-name-all Use the remote file name for all URLs
 -R, --remote-time   Set the remote file's time on the local output
 -X, --request COMMAND  Specify request command to use
     --resolve HOST:PORT:ADDRESS  Force resolve of HOST:PORT to ADDRESS
     --retry NUM   Retry request NUM times if transient problems occur
     --retry-delay SECONDS When retrying, wait this many seconds between each
     --retry-max-time SECONDS  Retry only within this period
 -s, --silent        Silent mode. Don't output anything
     --socks4 HOST[:PORT]  SOCKS4 proxy on given host + port
     --socks4a HOST[:PORT]  SOCKS4a proxy on given host + port
     --socks5 HOST[:PORT]  SOCKS5 proxy on given host + port
     --socks5-hostname HOST[:PORT] SOCKS5 proxy, pass host name to proxy
     --socks5-gssapi-service NAME  SOCKS5 proxy service name for gssapi
     --socks5-gssapi-nec  Compatibility with NEC SOCKS5 server
 -Y, --speed-limit RATE  Stop transfers below speed-limit for 'speed-time' secs
 -y, --speed-time SECONDS  Time for trig speed-limit abort. Defaults to 30
     --ssl           Try SSL/TLS (FTP, IMAP, POP3, SMTP)
     --ssl-reqd      Require SSL/TLS (FTP, IMAP, POP3, SMTP)
 -t, --telnet-option OPT=VAL  Set telnet option
 -z, --time-cond TIME  Transfer based on a time condition
 -u, --user USER[:PASSWORD][;OPTIONS]  Server user, password and login options
     --tlsuser USER  TLS username
     --tlspassword STRING TLS password
     --tlsauthtype STRING  TLS authentication type (default SRP)
 -A, --user-agent STRING  User-Agent to send to server (H)
 -V, --version       Show version number and quit
 -w, --write-out FORMAT  What to output after completion
     --xattr        Store metadata in extended file attributes


shell 参数表示

$1 表示第一个参数,$2 表示第二个参数 …




以”参数1 参数2 … ” 形式保存所有参数

以”参数1″ “参数2″ … 形式保存所有参数




curl POST登陆(未测试)

# at 2015-02-09
# example:



# login
# -m connect timeout
# -d your login information 
# -c save cookie to a temp file
# -s silence mode

curl $LOGIN_URL -m 10 --data-urlencode -d $LOGIN_URL"sid=QuvD3j&frames=yes&admin_password=$PASSWORD&admin_questionid=0&admin_answer=" -c $COOKIES_FILE -o curl_login.html

# update
# -b use the cookie file
RESPONSE=`curl $UPDATE_URL -b $COOKIES_FILE -o curl_update.html | grep $USERNAME`
dtime=`date +'%Y-%m-%d %H:%M:%S'`

# write the response to log file
echo "$dtime | $RESPONSE" >> $LOG

# delete cookie temp file




Linux curl 命令模拟 POST/GET 请求

PHP 模拟POST提交2种方法

PHP 异步后台处理