Member Avatar for Diaphanosoma

Hello,

I would like to scrape http://www.cpso.on.ca/publicregister/default.aspx?id=2048. by city. Specifically I have list of city names I would enter on the "form", hit submit and then collect/ parse all the results (across all pages), and then repeat for each city. I have never had to do a "post" before and this one is unlike any thing I've ever seen. Not even sure where to begin. Not sure how mechanize would work (or could work) in such a situation.

Thanks for any help.

B

  • 2 Contributors
  • 2 Replies
  • 121 Views
  • 2 Days Discussion Span
  • Latest Post Latest Post by Diaphanosoma
Member Avatar for Schol-R-LEA

I would recommend starting with BeautifulSoup, which is made for just such a purpose.

Member Avatar for Diaphanosoma

Sorry for the delay in response I was expecting an email notification...
My problem is not so much with the parsing of the data (beautiful soup) as the actual gathering of the data. I am unable to send the "post" back to the server to get the pages for parsing.

Thanks,

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.