Retrieving information from another website

Question

Flufferman 0 Newbie Poster

14 Years Ago

Hi,

I'm looking for a method to get some information from another website (http://contests.covers.com/sportscontests/picksByDate.aspx?date=11/2/2007&ur=265839&contestID=15253&sportID=9) in an automated way. I need to get the total value of a NBA game between boston and another team (variable team) and if Boston got WIN or LOSS in its row.

I've tried this but I have a little problem:

<?php
$url = 'http://contests.covers.com/sportscontests/picksByDate.aspx?date=11/2/2007&ur=265839&contestID=15253&sportID=9&t=0';
 
$array = file($url);
 
$key = array_search('<table cellspacing="0" class="thepicks">
', $array);
 
$flop = $key + 12;
 
$result = $array[$flop];
 
print($result);
?>

This is a test script and it works. The way I want to retrieve my information is by searching the Boston team in the array (note that it is a link). The problem i'm facing is that I doesn't find: <td><a href="http://www.covers.com/pageLoader/pageLoader.aspx?page=/data/nba/teams/team404117.html">New Jersey</a></td>
in my array. So the following code doesn't give me the right line. It takes array[0] +15 which is wrong.

<?php
$url = 'http://contests.covers.com/sportscontests/picksByDate.aspx?date=11/2/2007&ur=265839&contestID=15253&sportID=9&t=0';
 
$array = file($url);
 
$key = array_search('<td><a href="http://www.covers.com/pageLoader/pageLoader.aspx?page=/data/nba/teams/team404117.html">New Jersey</a></td>
', $array);
 
$flop = $key + 15;
 
$result = $array[$flop];
 
print($result);
?>

The variable in the array (<td><a href="http://www.covers.com/pageLoader/pageLoader.aspx?page=/data/nba/teams/team404117.html">New Jersey</a></td>) can be shown using the first code fragment. So it is surely displayable.

So does anyone know why he can't find this link (<td><a href="http://www.covers.com/pageLoader/pageLoader.aspx?page=/data/nba/teams/team404117.html">New Jersey</a></td>) in my array?

Thanks is higly appreciated!

php

Edited 14 Years Ago by Flufferman because: n/a

4 Contributors
10 Replies
183 Views
1 Day Discussion Span
Latest Post 14 Years Ago Latest Post by me655321

me655321 1 Junior Poster in Training

14 Years Ago

When you use file() it returns the array of lines including the line breaks at the end of the line. Theoretically you can use the FILE_IGNORE_NEW_LINES flag to retrieve lines without them...

$array = file($url,FILE_IGNORE_NEW_LINES);

But there seems to be an issue with that so you're better off either searching for a line with the breaks...

$key = array_search("<table cellspacing=\"0\" class=\"thepicks\">\r\n", $array);

Or using file_get_contents() instead of file() to return a string instead of a line array.

me655321 1 Junior Poster in Training

14 Years Ago

Did you read my post earlier about the line breaks? This is still that same issue. If you replace lines 21 and 22 with $key = array_search(" <td><a href=\"http://www.covers.com/pageLoader/pageLoader.aspx?page=/data/nba/teams/team404169.html\">Boston</a></td>\r\n",$array); you get the correct key. Notice the line breaks at the end...\r\n. I also changed to double quotes because those escape codes don't work with single quotes. (So the quotes within the string had to be escaped as well)

me655321 1 Junior Poster in Training

14 Years Ago

I would guess it's taking too long to retrieve the document. I don't think there's any way to do a timeout on file() so maybe something else like curl or fsocketopen() would work better for getting the file. Either that or change the timeout time of your script with set_time_limit.

Reply to this topic

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.

Kligham 0 Light Poster · Answer 1 · 2009-12-16T05:16:15+00:00

I have just gotten a little closer with the problem. I have this now:

<?php
require_once ('C:\wamp\www\mysql.php');

$connection = db_connect();

$maand = array("11","11","11");
$dag = array("2","4","7");
$jaar = array("2007","2007","2007");

$expert = 265839;

for ($i = 0; $i <= 2; $i++){

	$url = 'http://contests.covers.com/sportscontests/picksByDate.aspx?date=' . $maand[$i] .'/' . $dag[$i] . '/' . $jaar[$i] .'&ur=' . $expert . '&contestID=15253&sportID=9&t=0';
	
	$array = file($url);
	
	$key = array_search('		<td><a href="http://www.covers.com/pageLoader/pageLoader.aspx?page=/data/nba/teams/team404169.html">Boston</a></td>
	',$array);
	
	$arraynba = $key - 1;
	
	$arraytest = $array[$arraynba];
	
	if($arraytest == "		<td>NBA</td>
	"){
		$sidepick = $key + 4;
		$totalpick2 = $key + 13;
	
		$result1 = $array[$sidepick];
		$result2 = $array[$totalpick2];
	
	
	} else {
	    $sidepick = $key + 4;
	    $totalpick2 = $key - 5;
	
	    $result1 = $array[$sidepick];
	    $result2 = $array[$totalpick2];
	}
	
	$datum = $maand[$i] . '/' . $dag[$i] . '/' .$jaar[$i];
	
	//De gegevens wegschrijven in de database
	$query = $connection->query("INSERT INTO nba (datum, boston, tegenstander) VALUES ('$datum','$result1', '$result2')");
	

	
}

But when I check my for loop with this:

require_once ('C:\wamp\www\mysql.php');

$connection = db_connect();

$maand = array("11","11","11");
$dag = array("2","4","7");
$jaar = array("2007","2007","2007");

$expert = 265839;

for ($i = 0; $i <= 2; $i++){

	$url = 'http://contests.covers.com/sportscontests/picksByDate.aspx?date=' . $maand[$i] .'/' . $dag[$i] . '/' . $jaar[$i] .'&ur=' . $expert . '&contestID=15253&sportID=9&t=0';
	
	$array = file($url);
print ($array);

I only get one line instead of 3 which are expected. Can anyone see a mistake in my for loop cause I can't.

Greetz

chrishea 182 Nearly a Posting Virtuoso · Answer 2 · 2009-12-16T21:21:21+00:00

I tried your loop and I got three hits. The code I used was:

<?PHP

$maand = array("11","11","11");
$dag = array("2","4","7");
$jaar = array("2007","2007","2007");

$expert = 265839;

for ($i = 0; $i <= 2; $i++){

	$url = 'http://contests.covers.com/sportscontests/picksByDate.aspx?date=' . $maand[$i] .'/' . $dag[$i] . '/' . $jaar[$i] .'&ur=' . $expert . '&contestID=15253&sportID=9&t=0';

	echo "<br><b>Reading $url </b>";
	$array = file($url);    
}

?>

Kligham 0 Light Poster · Answer 3 · 2009-12-17T01:15:43+00:00

Hi,

I see I made a mistake in my previous post. I also get the url when I ask for it, but I don't get the 2 messages with LOSS, WIN, PUSH.

This is the code with the loop:

<?php


$maand = array("11","11","11");
$dag = array("2","4","7");
$jaar = array("2007","2007","2007");

$expert = 265839;

for ($a=0;$a<=2;$a++){

}

for ($i = 0; $i <= 2; $i++){

			$url = 'http://contests.covers.com/sportscontests/picksByDate.aspx?date=' . $maand[$i] .'/' . $dag[$i] . '/' . $jaar[$i] .'&ur=' . $expert . '&contestID=15253&sportID=9&t=0';

			$array = file($url);

			print($array);

			$key = array_search('		<td><a href="http://www.covers.com/pageLoader/pageLoader.aspx?page=/data/nba/teams/team404169.html">Boston</a></td>
			',$array);

			$arraynba = $key - 1;

			$arraytest = $array[$arraynba];

			if($arraytest == "		<td>NBA</td>
			"){
				$sidepick = $key + 4;
				$totalpick2 = $key + 13;

				$result1 = $array[$sidepick];
				$result2 = $array[$totalpick2];


			} else {
				$sidepick = $key + 4;
				$totalpick2 = $key - 5;

				$result1 = $array[$sidepick];
				$result2 = $array[$totalpick2];
			}

			$datum = $maand[$i] . '/' . $dag[$i] . '/' .$jaar[$i];

			//De gegevens wegschrijven in de database
			print($result1);
			print($result2);
			print($datum);


}

The LOSS, WIN or PUSH are those from the game with BOSTON. As in the example in attachments. I don't understand why I don't get them, I think there is a problem with my $key, because when you ask for $key you don't get any result.

I hope anyone can solve this.

Greets.

Kligham 0 Light Poster · Answer 4 · 2009-12-17T04:41:45+00:00

Did you read my post earlier about the line breaks? This is still that same issue. If you replace lines 21 and 22 with $key = array_search(" <td><a href=\"http://www.covers.com/pageLoader/pageLoader.aspx?page=/data/nba/teams/team404169.html\">Boston</a></td>\r\n",$array); you get the correct key. Notice the line breaks at the end...\r\n. I also changed to double quotes because those escape codes don't work with single quotes. (So the quotes within the string had to be escaped as well)

Thanks, the script works now:

<?php



$dag = array("2", "4", "7", "9", "10", "13", "14", "16", "18", "21", "23", "24", "27", "29", "30", "2", "5", "7", "8", "12", "14", "16", "19", "21", "23", "26", "27", "29", "30", "2", "4", "5", "9", "11", "12", "14", "16", "18", "21", "23", "25", "27", "29", "31", "5", "6", "8", "10", "12", "13", "19", "20", "22", "24", "25", "27", "29", "2", "5", "7", "8", "10", "12", "14", "15", "17", "18", "20", "22", "24", "26", "28", "30", "1", "2", "5", "8", "9", "11", "12", "14", "16", "20", "23", "26", "28", "30", "2", "4", "6", "8", "10", "12", "14", "16", "18", "20", "22", "24", "26", "28", "30", "5", "8", "10", "12", "15", "17");
$maand = array("11", "11", "11", "11", "11", "11", "11", "11", "11", "11", "11", "11", "11", "11", "11", "12", "12", "12", "12", "12", "12", "12", "12", "12", "12", "12", "12", "12", "12", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "2", "2", "2", "2", "2", "2", "2", "2", "2", "2", "2", "2", "2", "3", "3", "3", "3", "3", "3", "3", "3", "3", "3", "3", "3", "3", "3", "3", "3", "4", "4", "4", "4", "4", "4", "4", "4", "4", "4", "4", "4", "4", "4", "5", "5", "5", "5", "5", "5", "5", "5", "5", "5", "5", "5", "5", "5", "5", "6", "6", "6", "6", "6", "6");
$jaar = array("2007", "2007", "2007", "2007", "2007", "2007", "2007", "2007", "2007", "2007", "2007", "2007", "2007", "2007", "2007", "2007", "2007", "2007", "2007", "2007", "2007", "2007", "2007", "2007", "2007", "2007", "2007", "2007", "2007", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008", "2008");

$expert = 265839;

//Connectie maken met database


for ($i = 0; $i <= count($dag); $i++){

			$url = 'http://contests.covers.com/sportscontests/picksByDate.aspx?date=' . $maand[$i] .'/' . $dag[$i] . '/' . $jaar[$i] .'&ur=' . $expert . '&contestID=15253&sportID=9&t=0';

			$array = file($url);

			$key = array_search("		<td><a href=\"http://www.covers.com/pageLoader/pageLoader.aspx?page=/data/nba/teams/team404169.html\">Boston</a></td>\r\n",$array);
			
			if($key === FALSE){
				$result1 = "none";
				$result2 = "none";
			} else {
			
				$arraynba = $key - 1;
	
				$arraytest = $array[$arraynba];
	
				if($arraytest == "		<td>NBA</td>\r\n"){
					$sidepick = $key + 4;
					$totalpick2 = $key + 13;
	
					$result1 = $array[$sidepick];
					$result2 = $array[$totalpick2];
	
	
				} else {
					$sidepick = $key + 4;
					$totalpick2 = $key - 5;
	
					$result1 = $array[$sidepick];
					$result2 = $array[$totalpick2];
				}
			}

			$datum = $maand[$i] . '-' . $dag[$i] . '-' .$jaar[$i];

			//De gegevens wegschrijven in de database
	print($result1);
	print($result2);
	print($datum);
	print("\r\n");

}
?>

However I stumbled upon another problem. I get the following fatal error:

Fatal error: Maximum execution time of 30 seconds exceeded in C:\Apache\htdocs\test.php on line 18

and line 18 is:

$array = file($url);

Anyone has an idea how to get past this?

Grtz

Kligham 0 Light Poster · Answer 5 · 2009-12-17T06:22:43+00:00

Thanks mate, I use the time out limit and it works. You rock m8! When this task is over and don't have any problems any more you definitely get my thanks!

Kligham 0 Light Poster · Answer 6 · 2009-12-17T17:16:06+00:00

Have one little question again:

$array = arrary("WIN", "WIN", "WIN","LOSS","WIN","LOSS","LOSS");

Is there a function that can count all the "LOSS" in the array, so that the answer would be 3?

Greetz

me655321 1 Junior Poster in Training · Answer 7 · 2009-12-17T17:23:58+00:00

Only way I know of is array_count_values Notice it returns an array of all the values and their count, not just the one.