Search This Blog

2013-05-14

从下载文件看HTTP协议

今天去下载qt-linux-opensource-5.0.2-x86_64-offline.run,在Firefox浏览器,用的是DownThemAll,下载的时候闪了一下metalink,我看下载链接是个很普通的文件链接,这事比较奇怪,稍微跟踪了一下。

1 文件元数据

我先确认一下这个下载链接应该返回的内容。
>  curl --head http://download.qt-project.org/official_releases/qt/5.0/5.0.2/qt-linux-opensource-5.0.2-x86_64-offline.run
HTTP/1.1 200 OK
Date: Mon, 13 May 2013 03:25:33 GMT
Server: Apache/2.2.15 (CentOS)
X-Prefix: 122.240.0.0/13
X-AS: 4134
Last-Modified: Tue, 09 Apr 2013 11:53:01 GMT
ETag: "120117-189c7765-4d9ec31fcc140"
Accept-Ranges: bytes
Content-Length: 412907365
Connection: close
Content-Type: text/plain; charset=UTF-8
很明显是个文件,将近400MB,符合网页上显示的大小。那metalink怎么可能出来呢?还是抓包看看吧,用的是Wireshark。

2 请求头、响应头、响应内容

Pre[-]
GET /official_releases/qt/5.0/5.0.2/qt-linux-opensource-5.0.2-x86_64-offline.run HTTP/1.1
Host: download.qt-project.org
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:20.0) Gecko/20100101 Firefox/20.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.8,zh-cn;q=0.5,zh;q=0.3
Accept-Encoding: gzip, deflate
DNT: 1
Referer: http://qt-project.org/downloads
Connection: keep-alive

HTTP/1.1 302 Found
Date: Mon, 13 May 2013 04:09:22 GMT
Server: Apache/2.2.15 (CentOS)
X-Prefix: 122.240.0.0/13
X-AS: 4134
X-MirrorBrain-Mirror: mirrors.ustc.edu.cn
X-MirrorBrain-Realm: country
Link: <http://download.qt-project.org/official_releases/qt/5.0/5.0.2/qt-linux-opensource-5.0.2-x86_64-offline.run.meta4>; rel=describedby; type="application/metalink4+xml"
Link: <http://download.qt-project.org/official_releases/qt/5.0/5.0.2/qt-linux-opensource-5.0.2-x86_64-offline.run.torrent>; rel=describedby; type="application/x-bittorrent"
Link: <http://mirrors.ustc.edu.cn/qtproject/official_releases/qt/5.0/5.0.2/qt-linux-opensource-5.0.2-x86_64-offline.run>; rel=duplicate; pri=1; geo=cn
Link: <http://mirrors.neusoft.edu.cn/qt/official_releases/qt/5.0/5.0.2/qt-linux-opensource-5.0.2-x86_64-offline.run>; rel=duplicate; pri=2; geo=cn
Link: <http://ftp.jaist.ac.jp/pub/qtproject/official_releases/qt/5.0/5.0.2/qt-linux-opensource-5.0.2-x86_64-offline.run>; rel=duplicate; pri=3; geo=jp
Link: <http://master.qt-project.org/official_releases/qt/5.0/5.0.2/qt-linux-opensource-5.0.2-x86_64-offline.run>; rel=duplicate; pri=4; geo=fi
Link: <http://www.nic.funet.fi/pub/mirrors/download.qt-project.org/official_releases/qt/5.0/5.0.2/qt-linux-opensource-5.0.2-x86_64-offline.run>; rel=duplicate; pri=5; geo=fi
Digest: MD5=bbJjkMgQIwimVWhOyVetyQ==
Digest: SHA=HJA5DwAdhRTN1/VuDF9tfwY2Bwg=
Digest: SHA-256=eSPqSjbYzSA3nd9Pt5JkbzSZMjhBcX30DwJANdV9GsM=
Location: http://mirrors.ustc.edu.cn/qtproject/official_releases/qt/5.0/5.0.2/qt-linux-opensource-5.0.2-x86_64-offline.run
Content-Length: 385
Connection: close
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a href="http://mirrors.ustc.edu.cn/qtproject/official_releases/qt/5.0/5.0.2/qt-linux-opensource-5.0.2-x86_64-offline.run">here</a>.</p>
<hr>
<address>Apache/2.2.15 (CentOS) Server at download.qt-project.org Port 80</address>
</body></html>

这是个重定向响应(状态码302),重定向的位置在Location,地址是一个国内的下载链接,这是通过请求对应的ip地址计算出来的,X-Prefix响应头还返回了ip所在区域。

有很多Link响应头,包括metalink、bittorrent,以及一些带优先级跟所在国家的http下载地址。

还有几个Digest响应头,对应的值应该是二进制数据的Base64表示形式。我一开始没查到这个响应头,后来在Google搜索http response "Digest: SHA" -auth查到的。

有一个DNT请求头,代表do not track。

3 深入阅读

http://www.w3.org/Protocols/rfc2616/rfc2616-sec6.html
https://en.wikipedia.org/wiki/List_of_HTTP_header_fields
RFC3230 Instance Digests in HTTP - tools.ietf.org
https://en.wikipedia.org/wiki/HTTP_ETag

=文章版本=

20130513

No comments:

Post a Comment