python 2.7 - BeautifulSoup output is not transferred to CSV file -


i'm trying export output webscraper csv file. code works , right output when run in terminal, not transfer csv file.

question

when remove first loop works fine, can't figure out error in part?

code

import csv ; import requests bs4 import beautifulsoup  outfile = open('implementtest8.csv','w') writer = csv.writer(outfile) writer.writerow(["job_link", "job_desc"])  res = requests.get("http://implementconsultinggroup.com/career/#/6257").text soup = beautifulsoup(res,"lxml") links = soup.find_all("a")  li in soup.find('ul', class_='list-articles list').find_all('li'):     level = li.find_all('dd', {'class': 'author'})[1].get_text()     if "graduate" in level:         links = li.find_all("href")         link in links:             if "career" in link.get("href") , 'copenhagen' in link.text:                 item_link = link.get("href").strip()                 item_text = link.text.replace("view position","").encode('utf-8').strip()                 writer.writerow([item_link, item_text])                 print(item_link, item_text) 

edited code

import csv ; import requests bs4 import beautifulsoup  outfile = open('implementtest8.csv','w') writer = csv.writer(outfile) writer.writerow(["job_link", "job_desc"])  res = requests.get("http://implementconsultinggroup.com/career/#/6257").text soup = beautifulsoup(res,"lxml") links = soup.find_all("a")  li in soup.find('ul', class_='list-articles list').find_all('li'):     level = li.find_all('dd', {'class': 'author'})[1].get_text()     if "graduate" in level:         links = li.find_all(href=true)         link in links:             if "career" in link.get("href") , 'copenhagen' in link.text:                 item_link = link.get("href").strip()                 item_text = link.text.replace("view position","").encode('utf-8').strip()                 writer.writerow([item_link, item_text])                 print(item_link, item_text) 

href tag attribute not tag name. if want ensure links have href attribute can use keyward argument, else use tag name.

links = li.find_all(href=true) 

Comments

Popular posts from this blog

resizing Telegram inline keyboard -

command line - How can a Python program background itself? -

php - "cURL error 28: Resolving timed out" on Wordpress on Azure App Service on Linux -