Member Avatar for feodal

Hey,

I have a scrapper that was written in php that gathers info from a website and outputs it in csv format. The problem that I have encountered is that there are short connection breaks which are enough for my script to stop executing and I am not sure whether it is anti bot program on the server side or it is because internet break for brief amount of time.
I use it once a week to gather lots of data so it can stop at any time and I have to run it manually from same place again.
So what I was trying to figure out is how to allow the script to run until it gets some result; in other words to let it try access specific link until it gets there.
Maybe I should try set_time_limit (0) or maybe something else.
Hope it's understandable :)

Thanks for help guys.

  • 1 Contributor
  • 1 Reply
  • 74 Views
  • 5 Days Discussion Span
  • Latest Post Latest Post by feodal
Member Avatar for feodal

The correct answer is

set_time_limit[0]
Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.