Python + Selenium2 + Chrome 爬取网页
CentOS 安装 chrome
CentOS 6.5 安装chrome浏览器
vim /etc/yum.repos.d/CentOS-Base.repo
yum update // 更新源,使其生效
根据系统版本,在最末尾增加一个节点,如下:
32-bit
[google] name=Google - i386 baseurl=http://dl.google.com/linux/rpm/stable/i386 enabled=1 gpgcheck=1 gpgkey=https://dl-ssl.google.com/linux/linux_signing_key.pub
64-bit
[google64] name=Google - x86_64 baseurl=http://dl.google.com/linux/rpm/stable/x86_64 enabled=1 gpgcheck=1 gpgkey=https://dl-ssl.google.com/linux/linux_signing_key.pub
安装稳定版本: yum install google-chrome-stable
安装测试版本: yum install google-chrome-beta (推荐,安装成功)
参考推荐: Ubuntu常用软件安装
chromedriver 下载配置
chromedriver 官方下载:
http://chromedriver.storage.googleapis.com/index.html
chromedriver 配置:
1. 解压 chromedriver_linux32.zip
unzip chromedriver_linux32.zip
2. 拷贝到 /usr/bin/ 目录下
cp chromedriver /usr/bin/
Python + Selenium2 + Chrome 爬取网页
For Linux
-
Check you have installed latest version of chrome brwoser->
chromium-browser -version
-
If not, install latest version of chrome
sudo apt-get install chromium-browser
- get appropriate version of chrome driver from here
- Unzip the chromedriver.zip
-
Move the file to
/usr/bin
directorysudo mv chromedriver /usr/bin
-
Goto
/usr/bin
directory and you would need to run something likechmod a+x chromedriver
to mark it executable. -
finally you can execute the code.
import os from selenium import webdriver from pyvirtualdisplay import Display display = Display(visible=0, size=(800, 600)) display.start() driver = webdriver.Chrome() driver.get("http://www.google.com") print driver.page_source.encode('utf-8') driver.quit() display.stop()
参考推荐:
Running webdriver chrome with Selenium
版权所有: 本文系米扑博客原创、转载、摘录,或修订后发表,最后更新于 2016-06-18 20:56:49
侵权处理: 本个人博客,不盈利,若侵犯了您的作品权,请联系博主删除,莫恶意,索钱财,感谢!