Retrieving changing remote content?

[Edel_]

I'm not really well versed in web development techniques so I'm not sure if I'm describing it properly.

What I'd like to do is to be able to remotely access the content of a page that don't immediately appear when the page loads and don't show up on the page's source. Say, a tiny panel inside an article of a blog that shows the titles/links of the latest blog posts (for example). I don't know what it uses to achieve this (AJAX?) but using fopen() on my side only shows the static(?) content, not the bits that seem to load from somewhere else.

I wan't to retrieve from a remote page everything of a remote page that an user will be able to see with a browser, after everything is properly loaded. Is this possible? Thanks.

[Josh Connerty]

Yep make another file called page.php then use JavaScript and AJAX to retrieve it once your document is ready.

In this page.php you use file_get_contents() to retrieve the files contents.

[ShawnCplus]

Yep make another file called page.php then use JavaScript and AJAX to retrieve it once your document is ready.

In this page.php you use file_get_contents() to retrieve the files contents.

Note that you cannot use AJAX to fetch content from a domain other than the one the script runs on. The browser will not let you execute cross-domain AJAX requests.

[Josh Connerty]

Note that you cannot use AJAX to fetch content from a domain other than the one the script runs on. The browser will not let you execute cross-domain AJAX requests.

Yarp thats why you must make the page.php to retrieve it for you.

[Edel_]

Yep make another file called page.php then use JavaScript and AJAX to retrieve it once your document is ready.

In this page.php you use file_get_contents() to retrieve the files contents.

Thanks.

I am currently using a script on my server to retrieve an html file with file_get_contents()... the document is in another server.. Sorry, I really don't understand what I'm supposed to be doing. :( Do I use javascript and AJAX in this script to retrieve the document then get the contents when it's fully loaded (ready?)?

[Josh Connerty]

Basicly you use the page.php with file_get_contents() to retreive the remote page.

This will only happen once your page has loaded and when javascript uses AJAX to take whatever the page.php returns.

If you don't know much javascript and ajax then http://jquery.com/ will simplify the process a little.

I use jquery as it also saves allot of javascript coding.

[diafol]

f-g-c will work on a different server.

You don't really need the overhead of ajax if you're just hiding/showing the one source. If your hide/show data is dynamic, e.g. you've got 10 links with the info being shown in the same box on the page, then ajax may be useful. I love ajax, but it can be overused.

Have you thought about an iframe with js to change the src?

[Edel_]

I read the thread carefully again and I think I really couldn't explain myself properly. I'm not really trying to have a nested page of sorts i.e. get contents of a remote page to show in a box of my page (if I understand what you've told me to do..). What I'm really trying to do is that sometimes certain pages take a while to fully load and show the ultimate content (delays, counters etc.?) For these pages, I want to be able to run a script on my end and save an exact replica of the page as you'd see on a browser (after all the desired content have loaded remotely). I.e. fetch the file as it would appear on any browser.

I hope I'm not being super stupid :( Using file_get_contents() only seems to get the html content of the main page, not what appears as a result of these elements that seem to generate content on their own. Thanks.

[diafol]

So, if I get you right, you've got some javascripts that alter the data AFTER the page has loaded from the server, and these changes do not show in your f-g-c.

Is that it?

[Edel_]

So, if I get you right, you've got some javascripts that alter the data AFTER the page has loaded from the server, and these changes do not show in your f-g-c.

Is that it?

Yes, that seems to be it :) Except that I don't have the javascripts, it's on other peoples' pages.. The changes do not show, yes.

[diafol]

That'll be your problem. You'll need to check the legalities of using the data from other people's sites before proceeding with the following:

Search the source code of the other site, find the links to the js files or script snippits in the HEAD section. Copy script snips. Go to the url of the js files - depending on your browser settings, you can display or download the file(s).

Personally, I'd use cURL to get the page data. The data is put into a output string - you can cut this down to size (e.g. cut off head data and body tags to leave the bare content).

$myurl = 'http://www.example.com';

$curl_handle = curl_init();
curl_setopt ($curl_handle, CURLOPT_URL, $myurl);
curl_setopt ($curl_handle, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($curl_handle, CURLOPT_CONNECTTIMEOUT, 1);
$buffer = curl_exec($curl_handle);
curl_close($curl_handle);

echo $buffer;

This should output the whole page including head data etc.

A bit of parsing with various string functions should allow you to chop the page into bits and just use the relevant parts.
I use str_replace();strpos();substr();substr_replace(); strip_tags() and a few others to extract data. If you want to keep the original styling (I assume via CSS files), your best bet will be to supply an external link to it.

WARNING: if the external site isn't yours don't do the above unless you have permission.

Another way to do this, may be to use an iframe.

[Edel_]

Hey thanks! That makes it a lot more clearer now. I saw an example that does something similar and it seems to do what you just suggested!

I originally thought of using an iFrame but I don't really know how to use it properly (how it relates to redirection from server side :( ). I'll look more into it. Thanks everyone.