Python Scraping Go To Next Page Using Beautifulsoup

This is my scraping code: import requests from bs4 import BeautifulSoup as soup def get_emails(_links:list): for i in range(len(_links)): new_d = soup(requests.get(_links[i]).text

Solution 1:

You put the 'soup' in a variable called 'd'.

So replace the following line:

next_page=soup.find('div', {'class': 'paging'}, 'weiter')

With this:

next_page = d.find('div', {'class': 'paging'}, 'weiter')