Python: Selenium, Nosuchelementexception On Generic Xpath

I have code which allows me to return all searched parts from a specific website, given a keyword. When the search term 'HL4RPV-50' is used, I can get back all returned values as e

Solution 1:

This is the working code I disabled the images because My Internet connection was slow and the website was taking time to load the page. I used css selector instead xPath for price and its fully working>

import time
#Need Selenium for interacting with web elementsfrom selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
#Need numpy/pandas to interact with large datasetsimport numpy as np
import pandas as pd

chrome_path = r".\web_driver\chromedriver.exe"
chrome_options = webdriver.ChromeOptions()
prefs = {"profile.managed_default_content_settings.images": 2}
chrome_options.add_experimental_option("prefs", prefs)
driver = webdriver.Chrome(chrome_path, chrome_options=chrome_options)
driver.maximize_window()
driver.get("https://www.tessco.com/login")

userName = "FirstName.SurName321123@gmail.com"
password = "PasswordForThis123"#Set a wait, for elements to load into the DOM
wait10 = WebDriverWait(driver, 10)
wait20 = WebDriverWait(driver, 20)
wait30 = WebDriverWait(driver, 30)

elem = wait10.until(EC.element_to_be_clickable((By.ID, "userID")))
elem.send_keys(userName)

elem = wait10.until(EC.element_to_be_clickable((By.ID, "password")))
elem.send_keys(password)

#Press the login button
driver.find_element_by_xpath("/html/body/account-login/div/div[1]/form/div[6]/div/button").click()

#Expand the search bar# searchIcon = wait10.until(EC.element_to_be_clickable((By.XPATH, "")))# searchIcon.click()

searchBar = wait10.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#searchBar input")))

#Enter information into the search bar
searchBar.send_keys("FSJ4-50B")
driver.find_element_by_css_selector('a.inputButton').click()
time.sleep(5)

# wait for the products information to be loaded
products = driver.find_elements_by_xpath( "//div[@class='CoveoResult']")
# create a dictionary to store product and price
productInfo = {}
# iterate through all products in the search result and add details to dictionaryfor product in products:
    # get product name
    productName = product.find_element_by_xpath("//a[@class='productName CoveoResultLink hidden-xs']").text
    # get price
    price = product.find_element_by_css_selector("div.price").text.split('\n')[1]
    # add details to dictionary
    productInfo[productName] = price
# print products informationprint(productInfo)
#time.sleep(5)
driver.close()

Output:

{"8' Jumper-FSJ4-50B NM/NM": '$147.55'}

Edited:

How to choose selector

As you can see in the above screenshot, I hover over the searchBar and come to know this have an ID, and we know ID is always unique element on the webpage so we can also use:

driver.find_element_by_id("searchBar")

but to reach to the input field I prefer css_selector and then send keys.

For finding a.inputButton css selector:

For a.button css selector see select the searchButton you will see the following html in the dom:

<aclass="CoveoSearchButton inputButton button"><spanclass="coveo-icon">Search</span><iclass="fa fa-search"aria-hidden="true"></i></a>

and we know <a> is the anchor tag, and from above html, we can deduce that one of the css_selector can be:

a.inputButton

NOTE

But this is unique here in this case sometimes the same class name can be used multiple time in different elements on the same page,so then you have to use the upper level of nodes to reach the child CSS element node. e.g, the a.inputButton can also be traversed as:

another css_selector for searchButton

div.divCoveoSearchbox > a.inputButton

as div is the parent element to our inputButton's anchor Tag.

I hope I've clear your point?

Solution 2:

Some products have different class name price sale that why you are getting NoSuchElementException. Update the price related line in the loop as below.

oPrice = product.find_element_by_xpath(".//div[@class='price' or @class='price sale']").text.split('\n')[-1]
price = oPrice[oPrice.find('$'):]