Python 对比抓取两个网页耗时
最近,把自己的博客域名,从切换成了,并把博客老域名映射到了新域名,于是想看下访问两个域名的耗时。
下面自己用Python写了一段小脚本,用于记录每次访问耗时以及平均耗时,主要用到了Python的装饰器,命令行传参,urllib2抓取网页等小知识。
示例代码: mimvp_costtime.py
#!/usr/bin/env python # -*- coding:utf-8 -*- # # Copyright 2015 mimvp.com ''' 对比访问N次我的博客,比较域名切换后的访问时间 新博客: https://blog.mimvp.com 老博客: http://blog.ithomer.net ''' import time import urllib2 import functools import sys reload(sys) sys.setdefaultencoding('utf8') DOWN_NUM = 10 # 默认比较次数,通过命令行可以传参 mimvp_NEW = 'https://blog.mimvp.com' mimvp_OLD = 'http://blog.ithomer.net' def timefunc(func): @functools.wraps(func) def __do__(*args, **kwargs): start = time.time() func(*args, **kwargs) costtime = time.time() - start # print('%s \t %s \t costtime: %ss' % (str(args[1]), func.__name__, costtime)) return costtime return __do__ @timefunc def down(url, mimvp_type='mimvp_new'): try: req = urllib2.Request(url) content = urllib2.urlopen(req, timeout=30).read() # print("%s \t url: %s \t content len: %d" % (mimvp_type, url, len(content))) except Exception, ex: print("main() -- error_msg: %s" % (str(ex),)) def main(down_num = DOWN_NUM): mimvp_old_costtime = 0.0 mimvp_new_costtime = 0.0 for i in xrange(down_num): result_old = down(mimvp_OLD, 'mimvp_old') result_new = down(mimvp_NEW, 'mimvp_new') print('down_%d \t\t url: %s \t costtime: %f' % (i, mimvp_OLD, result_old)) print('down_%d \t\t url: %s \t costtime: %f' % (i, mimvp_NEW, result_new)) mimvp_old_costtime += result_old mimvp_new_costtime += result_new mimvp_old_avg = mimvp_old_costtime / down_num mimvp_new_avg = mimvp_new_costtime / down_num print('down_num: %d \t\t url: %s \t costtime_avg: %f' % (down_num, mimvp_OLD, mimvp_old_avg)) print('down_num: %d \t\t url: %s \t costtime_avg: %f' % (down_num, mimvp_NEW, mimvp_new_avg)) if __name__ == "__main__": # 获取比较次数参数,例如:python mimvp_costtime.py 5 down_num = int(sys.argv[1]) if len(sys.argv) > 1 else DOWN_NUM print 'down_num: ', down_num main(down_num) print("end.")
运行结果:
homer@ubuntu:~/$ python mimvp_costtime.py 5
down_num: 5
down_0 url: http://blog.ithomer.net costtime: 0.444852
down_0 url: https://blog.mimvp.com costtime: 0.783525
down_1 url: http://blog.ithomer.net costtime: 1.197499
down_1 url: https://blog.mimvp.com costtime: 0.877678
down_2 url: http://blog.ithomer.net costtime: 0.635510
down_2 url: https://blog.mimvp.com costtime: 0.470353
down_3 url: http://blog.ithomer.net costtime: 1.234132
down_3 url: https://blog.mimvp.com costtime: 0.911674
down_4 url: http://blog.ithomer.net costtime: 3.554443
down_4 url: https://blog.mimvp.com costtime: 1.306112
down_num: 5 url: http://blog.ithomer.net costtime_avg: 1.413287
down_num: 5 url: https://blog.mimvp.com costtime_avg: 0.869868
版权所有: 本文系米扑博客原创、转载、摘录,或修订后发表,最后更新于 2015-02-20 19:12:31
侵权处理: 本个人博客,不盈利,若侵犯了您的作品权,请联系博主删除,莫恶意,索钱财,感谢!
转载注明: Python 对比抓取两个网页耗时 (米扑博客)