Quantcast
Channel: Using Python and BeautifulSoup (saved webpage source codes into a local file) - Stack Overflow
Viewing all articles
Browse latest Browse all 4

Using Python and BeautifulSoup (saved webpage source codes into a local file)

$
0
0

I am using Python 2.7 + BeautifulSoup 4.3.2.

I am trying to use Python and BeautifulSoup to pick up information on a webpage. Because the webpage is in the company website and requires login and redirection, I copied the target page's source code page into a file and saved it as “example.html” in C:\ for the convenience of practicing.

This is a part of the original code:

<tr class="ghj"><td><span class="city-sh"><sh src="./citys/1.jpg" alt="boy" title="boy" /></span><a href="./membercity.php?mode=view&amp;u=12563">port_new_cape</a></td><td class="position"><a href="./search.php?id=12563&amp;sr=positions" title="Search positions">452</a></td><td class="details"><div>South</div></td><td>May 09, 1997</td><td>Jan 23, 2009 12:05 pm&nbsp;</td></tr>

The code I worked out so far is:

from bs4 import BeautifulSoupimport reimport urllib2url = "C:\example.html"page = urllib2.urlopen(url)soup = BeautifulSoup(page.read())cities = soup.find_all('span', {'class' : 'city-sh'})for city in cities:print city

This is just the first stage of testing, so it's somewhat incomplete.

However, when I run it, it gives an error message. Seems it’s improper to use urllib2.urlopen to open a local file.

 Traceback (most recent call last):   File "C:\Python27\Testing.py", line 8, in <module>     page = urllib2.urlopen(url)   File "C:\Python27\lib\urllib2.py", line 127, in urlopen     return _opener.open(url, data, timeout)   File "C:\Python27\lib\urllib2.py", line 404, in open     response = self._open(req, data)   File "C:\Python27\lib\urllib2.py", line 427, in _open'unknown_open', req)   File "C:\Python27\lib\urllib2.py", line 382, in _call_chain     result = func(*args)   File "C:\Python27\lib\urllib2.py", line 1247, in unknown_open     raise URLError('unknown url type: %s' % type) URLError: <urlopen error unknown url type: c>

How can I practice using a local file?


Viewing all articles
Browse latest Browse all 4

Latest Images

Trending Articles





Latest Images