Why is the HTML from a website different from the HTML that Python's request library gives? -


i'm trying familiarize myself requests , beautifulsoup gave myself mini-project. i'm trying make program displays shoes on footlocker's release calendar (https://www.footlocker.com/release-dates/) so:

  • shoename#1 date#1
  • shoename#2 date#2
  • shoename#3 date#3
  • shoename#4 date#4

so far have this:

import requests req bs4 import beautifulsoup  def main():     url = "https://www.footlocker.com/release-dates/"     resp = req.get(url)      soup = beautifulsoup(resp.content, "html.parser")     print(soup)

however when load html parse beautifulsoup html code information containing dates , names of shoes not appear when use inspect element directly on website. assume because html shoe's information generated javascript. if how can load requests?

thank you.

if have selenium installed in machine, that's okay otherwise, install it. here how go.

from selenium import webdriver bs4 import beautifulsoup  driver = webdriver.chrome() driver.get("https://www.footlocker.com/release-dates/") soup = beautifulsoup(driver.page_source, "lxml") driver.quit() item in soup.select(".day"):     shoe = item.select_one(".productname").get_text()     date = item.select_one(".date").get_text()     print(shoe,date)

partial results:

jordan retro 1 hi og 1aug kids' jordan retro 1 hi og 1aug jordan retro 1 hi og 1aug kids' jordan retro 1 hi og 1aug nike kobe a.d. nxt 1aug nike dualtone racer 1aug

Comments

Post a Comment

Popular posts from this blog

python - Alternative to referencing variable before assignment -

import time # define variables username = "username" password = "password" customer_info_file_path = "c:\\users\megam\pycharmprojects\general stuff\the square pie co\customer " \ "information\{}.txt ".format(name) username_entry = input("enter username\n>>") password_entry = input("enter password\n>>") if username_entry != username or password_entry != password: print("access denied\nyou have entered username or password incorrectly. please check password , " "account name , try again.\n") else: print("access granted") choice_menu = input("\na) view list of customers\nb) add customer\nc) delete customer\nd) log out\n>>") if choice_menu == "a": customer_info = open(customer_info_file_path, 'a', ) file = open(customer_info_file_path, 'r') output = file.read() ...
Read more

vb.net - How to ignore if a cell is empty nothing -

how use roscolor() ignore value (like empty cell) ? color row if value upper 5 when there nothing in cell, want ignore roscolor() apply, how? public sub roscolor() integer = 0 quotedatagridview1.rows.count() - 1 step +1 dim val integer val = quotedatagridview1.rows(i).cells(3).value if val = vbempty quotedatagridview1.rows(i).defaultcellstyle.backcolor = color.white elseif val < 5 quotedatagridview1.rows(i).defaultcellstyle.backcolor = color.red elseif val > 5 , val < 10 quotedatagridview1.rows(i).defaultcellstyle.backcolor = color.lightyellow end if next end sub you can check empty value following public sub roscolor() integer = 0 quotedatagridview1.rows.count() - 1 step +1 dim val = quotedatagridview1.rows(i).cells(3).value if isdbnull(val) or val nothing quotedatagridview1.rows(i).defaultcellstyle.backcolor = color.white ...
Read more

Sort a complex associative array in PHP -

this question has answer here: php sort array subarray value 5 answers i have associative array in php like: $newarray1 = array ( [producta] => array ( [link] => http://sanjosespartan.com/blog/products/agile-java-development/ [visitcount] => 4 ) [productd] => array ( [link] => http://sanjosespartan.com/blog/products/intro-to-go/ [visitcount] => 1 ) [productg] => array ( [link] => http://sanjosespartan.com/blog/products/node-js-for-developers/ [visitcount] => 1 ) [productb] => array ( [link] => http://sanjosespartan.com/blog/products/beginning-mysql/ [visitcount] => 1 ) [productc] => array ( [link] => http://sanjosespartan.com/blog/products/computer-networks/ [visitcount] => 1 ) ...
Read more