Python 对比抓取两个网页耗时
最近,把自己的博客域名,从切换成了,并把博客老域名映射到了新域名,于是想看下访问两个域名的耗时。
下面自己用Python写了一段小脚本,用于记录每次访问耗时以及平均耗时,主要用到了Python的装饰器,命令行传参,urllib2抓取网页等小知识。
示例代码: mimvp_costtime.py
#!/usr/bin/env python
# -*- coding:utf-8 -*-
#
# Copyright 2015 mimvp.com
'''
对比访问N次我的博客,比较域名切换后的访问时间
新博客: https://blog.mimvp.com
老博客: http://blog.ithomer.net
'''
import time
import urllib2
import functools
import sys
reload(sys)
sys.setdefaultencoding('utf8')
DOWN_NUM = 10 # 默认比较次数,通过命令行可以传参
mimvp_NEW = 'https://blog.mimvp.com'
mimvp_OLD = 'http://blog.ithomer.net'
def timefunc(func):
@functools.wraps(func)
def __do__(*args, **kwargs):
start = time.time()
func(*args, **kwargs)
costtime = time.time() - start
# print('%s \t %s \t costtime: %ss' % (str(args[1]), func.__name__, costtime))
return costtime
return __do__
@timefunc
def down(url, mimvp_type='mimvp_new'):
try:
req = urllib2.Request(url)
content = urllib2.urlopen(req, timeout=30).read()
# print("%s \t url: %s \t content len: %d" % (mimvp_type, url, len(content)))
except Exception, ex:
print("main() -- error_msg: %s" % (str(ex),))
def main(down_num = DOWN_NUM):
mimvp_old_costtime = 0.0
mimvp_new_costtime = 0.0
for i in xrange(down_num):
result_old = down(mimvp_OLD, 'mimvp_old')
result_new = down(mimvp_NEW, 'mimvp_new')
print('down_%d \t\t url: %s \t costtime: %f' % (i, mimvp_OLD, result_old))
print('down_%d \t\t url: %s \t costtime: %f' % (i, mimvp_NEW, result_new))
mimvp_old_costtime += result_old
mimvp_new_costtime += result_new
mimvp_old_avg = mimvp_old_costtime / down_num
mimvp_new_avg = mimvp_new_costtime / down_num
print('down_num: %d \t\t url: %s \t costtime_avg: %f' % (down_num, mimvp_OLD, mimvp_old_avg))
print('down_num: %d \t\t url: %s \t costtime_avg: %f' % (down_num, mimvp_NEW, mimvp_new_avg))
if __name__ == "__main__":
# 获取比较次数参数,例如:python mimvp_costtime.py 5
down_num = int(sys.argv[1]) if len(sys.argv) > 1 else DOWN_NUM
print 'down_num: ', down_num
main(down_num)
print("end.")
运行结果:
homer@ubuntu:~/$ python mimvp_costtime.py 5
down_num: 5
down_0 url: http://blog.ithomer.net costtime: 0.444852
down_0 url: https://blog.mimvp.com costtime: 0.783525
down_1 url: http://blog.ithomer.net costtime: 1.197499
down_1 url: https://blog.mimvp.com costtime: 0.877678
down_2 url: http://blog.ithomer.net costtime: 0.635510
down_2 url: https://blog.mimvp.com costtime: 0.470353
down_3 url: http://blog.ithomer.net costtime: 1.234132
down_3 url: https://blog.mimvp.com costtime: 0.911674
down_4 url: http://blog.ithomer.net costtime: 3.554443
down_4 url: https://blog.mimvp.com costtime: 1.306112
down_num: 5 url: http://blog.ithomer.net costtime_avg: 1.413287
down_num: 5 url: https://blog.mimvp.com costtime_avg: 0.869868
版权所有: 本文系米扑博客原创、转载、摘录,或修订后发表,最后更新于 2015-02-20 19:12:31
侵权处理: 本个人博客,不盈利,若侵犯了您的作品权,请联系博主删除,莫恶意,索钱财,感谢!
转载注明: Python 对比抓取两个网页耗时 (米扑博客)