我想凑这个网站:Webscraping - 找不到元素
https://www.novanthealth.org/home/patients--visitors/locations/clinics.aspx?behavioral-health=yes
我想要得到的诊所名称和地址,这是我使用
的Python代码from selenium import webdriver
import pd
import time
#driver = webdriver.Chrome()
specialty = ["behavioral-health","dermatology","colon","ear-nose-and- throat","endocrine","express","family-practice","foot-and-ankle",
"gastroenterology","heart-%26-vascular","hepatobiliary-and-pancreas","infectious-disease","inpatient","internal-medicine",
"neurology","nutrition","ob%2Fgyn","occupational-medicine","oncology","orthopedics","osteoporosis","pain-management",
"pediatrics","plastic-surgery","pulmonary","rehabilitation","rheumatology","sleep","spine","sports-medicine","surgical","urgent-care",
"urology","weight-loss","wound-care","pharmacy"]
name = []
address = []
for q in specialty:
driver = webdriver.Chrome()
driver.get("https://www.novanthealth.org/home/patients-- visitors/locations/clinics.aspx?"+q+"=yes")
x = driver.find_element_by_class_name("loc-link-right")
num_page = str(x.text).split(" ")
x.click()
for i in num_page:
btn = driver.find_element_by_xpath('//*[@id="searchResults"]/div[2]/div[2]/button['+i+']')
btn.click()
time.sleep(8) #instaed of this use waituntil #
temp = driver.find_element_by_class_name("gray-background").text
temp0 = temp.replace("Get directions Website View providers\n","")
x_temp = temp0.split("\n\n\n")
for j in range(0,len(x_temp)-1):
temp1 = x_temp[j].split("Phone:")
name.append(temp1[0].split("\n")[1])
temp3 = temp1[1].split("Office hours:")
temp4 = temp3[0].split("\n")
temp5 = temp4[1:len(temp4)]
address.append(" ".join(temp5))
driver.close()
此代码工作正常,如果我用它只有一个特殊的时间,但是当我通过特色的循环如上,代码与错误的第二次迭代失败:
Traceback (most recent call last):
File "<stdin>", line 10, in <module>
File "C:\Anaconda2\lib\site- packages\selenium\webdriver\remote\webelement.py", line 77, in click self._execute(Command.CLICK_ELEMENT)
File C:\Anaconda2\lib\sitepackages\selenium\webdriver\remote\webelement.py", line 493, in _execute return self._parent.execute(command, params)
File "C:\Anaconda2\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 249, in execute self.error_handler.check_response(response)
File "C:\Anaconda2\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 193, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotVisibleException: Message: element not visible
(Session info: chrome=46.0.2490.80)
(Driver info: chromedriver=2.19.346078 (6f1f0cde889532d48ce8242342d0b84f94b114a1),platform=Windows NT 6.1 SP1 x86_64
我不使用Python太多的经验,任何帮助将不胜感激
您必须让您的网络驱动程序等待几秒钟,直到相应的elem出现在页面上。看看webdriver_wait函数.. –
我已经在阅读这个文档,但正面临一些实现它的问题,你能给它一个示例代码吗?谢谢 ! – Vaibhav
这里是http://stackoverflow.com/a/41832157/3297613 –