Web Page Interaction Question

[sheff21]

My friend and I are working on a school project and our hope was to have a Java program take a zip code from the user and then connect to weather.com (or any other web site) and look up the local weather. I assume this is possible but I cannot quite figure it out. I am able to get the source code from a site by using the instream, however, it is difficult to find the information I need in the source code. Is there any way to access the page and retrieve the actual text from the site. Thanks!

[BestJewSinceJC]

Have you managed to send the site a request to look up the weather (based on the zip you send it) yet? If not, plain text isn't going to do you any good. So let us know which part you are stuck at.

[sheff21]

Well I don't know how to send a request. I just hard coded it by copying the URL from the site after performing a search and then replaced all the places they used the zip in their URL with a variable that contained it. So, I guess I'm stuck at both parts, sending a request as well as retrieving the info.

[stephen84s]

Sheff21, you need to learn first about HTTP. And how data is sent to a web page via GET or POST.

You send a request to a Web page via the GET or POST method, if you have done some basic web programming or written some forms in HTML you would have encountered them, or you could read about them in the page I have linked to HTTP.

Also once you have cleared your HTTP concepts, Here is an example on how to perform POST in Core Java.
Performing a GET is as simple as appending the the URL with a "?" and then appending the data with the format <param1>=<value1>&<param2>=<value2>& ... &<paramN>=<valueN> and invoking it.

[verruckt24]

Calling a URL and parsing it for plain text values is not what you would want. What you first require is an HTTP API from the site owners which will provide you with proper values. But this certainly requires an API defined at their end and would require you to spare them some usage fees if at all the API exists.
Then a typical interaction between your program and their site would be something like this: You call their URL using either HTTP GET or POST (as mentioned by stephen84s) as specified in their API specification providing them the zip code and they return you the current whether for that zip code as a response.

[~s.o.s~]

> But this certainly requires an API defined at their end

Not certainly, screen scraping is always an option and which I guess is what the OP is looking for in this case given that it's a school project.

> it is difficult to find the information I need in the source code

As long as you don't abuse the service, XQuery and some other options can make your screen scraping a bit easier.

This involves a two step process:

• Construct a valid HTTP request along with the necessary parameters to query the kind of data you require. Take a look at the HttpClient API offered by Apache.
• Scrape the data from the response returned using various screen scraping techniques.