URL.openConnection() doesn't wait for page to fully load

[claudiu_is]

I'm trying to get live feed for my server from this page http://dkm.inx-gaming.co.uk/stats/server.php. As you can see, when it first loads, it says "Querying Server x, Please wait.." for a fraction of a second.. And that's what the openConnection function returns, too. Is there any way I can tell it to wait for the page status to be done or use another function, maybe?

[masijade]

Have you enabled redirects on the URL, as it is probably getting redirected, which URLConnection does not automatically follow.

[jwenting]

the openConnection method doesn't read the content of the stream, it just opens the stream. So of course it doesn't wait for "the page to be loaded".

[claudiu_is]

Ty for your replies :)
@jwenting: the url doesn't return the content, it returns an urlConnection and THAT returns an input stream with the content.
@masijade: what do u mean I should redirect? to what? it's the same php link :( And I also think there's some AJAX involved..
Any more ideas are more than welcome :D

[masijade]

Like I said, it is probably getting redirected (and just because the address shown in the address bar doesn't change, doesn't mean it hasn't been redirected). And, if there is "AJAX involved", then the only thing you can do (without it getting much more complicated) is to print out the contents of the Script tag.

[claudiu_is]

Sry, I don't follow.. If it IS redirected, what can I do? How do I check if redirection is enabled or how do I enable it? As for the script tag, there are several and neither is helpful, even if you'd save the page after it has fully loaded. I just don't understand, help me, I'm noob at this :D

[masijade]

As far as the script tag goes, sorry, URLConnection (and HttpURLConnection) is not a browser. Script directives (such as onLoad) are (AFAIK) not carried out. It is not (again AFAIK) meant for that. If anyone else knows more about that, please feel free to speak up. As far as redirections, check the API docs for HttpURLConnection (and/or URLConnection) there is a method for that (followRedirects() or something like that).

[claudiu_is]

Then I guess (Http)UrlConnection is not the way to go.. I tried enabling HttpURLConnection redirects (which is true by default anyway) and it acts the same and even if it does follow redirect, I can't find the new Location I'm supposed to load again. So I might give up on these 2 classes, but I need another solution :D Anyone?

[masijade]

Well, take a look at the site, yourself, and figure out what it is doing in AJAX, and do that.

[~s.o.s~]

> but I need another solution Anyone?

As already mentioned, any library which just fetches the response for a given URI won't cut in for your needs (unless you plan to parse that response and make the necessary calls manually). I can think of two ways to tackle this problem:

• Analyze the way the page works; which Ajax calls are made after the page loads (pretty easy with the Firefox add-on Firebug). You would most probably end up with a different URI (from the same site most probably) which accepts the data in a particular format as used by the site Javascript. After that, all you'd need to do is use *that* URL with URLConnection by passing in the data as per the contact between the loaded page and the data service
• Use a library which emulates a browser. HtmlUnit and Lobo come to mind. By using the programmatic API of these libraries (would require a fair amount of research of course), you would definitely be able to achieve the actual *after page load* effect. You can then sanitize the response and parse it using something like Jericho.

HTH :)

[masijade]

HTH :)

Hold Their Hand? ;-)