why curl function in php doesnt work for some sites like dir.yahoo.com?
function getThePage($page_url)
{
$options = array(
CURLOPT_RETURNTRANSFER => true, // return web page
CURLOPT_HEADER => false, // don't return headers
CURLOPT_FOLLOWLOCATION => true, // follow redirects
CURLOPT_ENCODING => "", // handle all encodings
CURLOPT_USERAGENT => "spider", // who am i
CURLOPT_AUTOREFERER => true, // set referer on redirect
CURLOPT_CONNECTTIMEOUT => 120, // timeout on connect
CURLOPT_TIMEOUT => 120, // timeout on response
CURLOPT_MAXREDIRS => 10, // stop after 10 redirects
);
$ch=curl_init($page_url)or die("Cannot initialize");
curl_setopt_array($ch,$options)or die("Cannot set options");
$content=curl_exec($ch)or die("Cannot execute");
$err=curl_errno($ch);
$errmsg=curl_error($ch);
$header=curl_getinfo($ch);
curl_close($ch);
$header['errno']=$err;
$header['errmsg']=$errmsg;
$header['content']=$content;
return($header);
}i used this code but i cant open some sites like http://dir.yahoo.com
www.cemunnar.org etc
