Making queries with external link

[hell hansen]

Hi guys, recently im faceng one problem.

i need to nake a queries from external link.
The good example is:

The website

www.example1.com

sell some products, and this website have reseller guy called

www.example2.com

so reseller he want all weeks get updated about prices from suply using queries like

www.example1.com?id=1234, example1.com?id=12345, example1.com?id=12346

for now im getting this value using parse external page and im getting the result wiht

 <?php
            $url = 'https://example1.com?id=1234';
            echo file_get_contents($url);
            ?>

So what i want is make this querie automatilly with ajax, and the result of

?id=1234

must be replced with one variable, making a loop, and that loop the ids are coming from mysql.

something like today is friday the loop start.
the ajax go to database get ids and make queries to example1.com and save the results somewhere. can be csv or sql.

Please guys hi hope get some help from you.

[cereal]

Hi,

the best solution would be for example1.com to serve a page where all price changes are published, so that example2.com needs to query only one time and not X number of times... if you have 10000 articles it means you are going to open that amount of connections against example1.com.

The serving method could go from a simple REST API returning XML or JSON, or an RSS feed.

[hell hansen]

Hi Cereal, thanks for replying.
But the big deal here is the example1.com dont serve full page for no one, the only way he serve is a simple form with text input and submit to enter the ID and get result from his website.
Like he only serve one by one, there no how to get full page with all ids, because he have many ressellers and each rellers have different products and different prices and different ID.

So the only way i've is getting price to my ids is one by one

[diafol]

That is an insane way of offering up data. If you're selling products through this guy, then surely it's in his interest to offer a simple, standard method of getting hold of the data. Will he consider creating an API or provide XML or JSON?

[jkon]

To be honest I didn't understood how you know product ID's if the reseller end don't communicate through some other ways with the seller end? If there is some other way of communication behind the scenes you could provide that , and it might be really better solution using that channel. If you know the ids you could create a scrapping job ( e.g. cron ) that runs every x time (e.g. every night) and get the product details in your db. Of course the best (as cereal and diafol wrote) would be to use an API of the seller end.

[hell hansen]

Thanks guys for replying! and really thanks all.
What im really looking is creating cron job, which can run all night or a day which i'll provide.

so like Diafol said, using Json or Xml, this the idea i've.
so how can i write the algorithm in php, to go through to the url, get the page, and do some rotine with all my id in Array.

and in this case my array is comming from Mysql.

or i can try deal how to make a loop to the some rotine run, but i've no idea how do i tell php like go to that url and back.

The way im using now Manual i type the url, in browser with the d like www.example1.com?id=1234 and then i get the update, and i do again with other ID's.

[hell hansen]

The way how i know the product ID's when i get invoice from him, he give me the ID's to track prices Update in his website.

But he don't have in his System like each reseller with own products. he provid some products with different people and different prices, and sometime depending on location or quantity you sell.

[diafol]

I'm a bit confused. You have products in your DB already that you're going to check on his site for changes in price?
You're also getting the IDs (new? old?) from invoices sent to you.

This doesn't sound as though he's offering up any sort of public connection, so it looks as though you're left with a product by product scrape.

This can be done of course, but it needs either to be done by CLI or cron job, otherwise if you try it through a browser, you'll probably timeout.

We're to assume that you're on a Linux server? If not, then you're not looking at cron jobs / CLI but Windows Task Manager / AT command.

If you need to automate this to happen in the background, say every Friday night, then do a cron. On the other hand, if you need to do an ad hoc update, CLI may be better.

You can extract IDs from a text file (or whatever you use) from the invoices too - that's pretty trivial and ask the cron script to do something with that.

If you could supply any code that you have already, it may shine a light on your needs, as I'm guessing a bit here.

[hell hansen]

Hi Diafol!
look when i get the products, from my suplier, i create those products in my DB, and i use some ID's i get from him. like i don't change the the IDs.

And really when i use background i get timeout with 2min.
so i'm on linux server and i get cli, so i'm writing the cron script, when im done i'll upload for you guys after i've tested and have something to discuss.

I'll be in touch in some hours. i dont know how far will be.