Optimizing a lot of foreach loops. Please help. - Page 3

Question

langsor

16 Years Ago

I don't mind sharing...actually I'm getting out of the web design biz (but I love the work), so I'm trying to help people and share what I've got.

My little project isn't there yet, but I will post it up here when it's done. There's a couple interesting PHP and JavaScript classes and examples in the total mix of it.

Is your loop and everything working? It looks like it should!

Would you rather print to a file or to the browser window?

Still awake for a little while

Reply to this topic

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.

langsor · Answer 1 · 2008-08-23T11:27:26+00:00

Hey,

For better or worse I've spent the entire day playing with databases, dom, and templates...so I should be warmed up, but also a little brain-fried. :-)

The two things I notice off the top are in your final DOM loop you're calling the wrong object to fill its childNode

foreach ( $divs as $div )
{
if ( $div->getAttribute('id') == 'location_name' )
{
$tag->nodeValue = 'Location Name...';
}
}

Should be a $div

foreach ( $divs as $div )
{
	if ( $div->getAttribute('id') == 'location_name' )
	{
		$div->nodeValue = 'Location Name...';
	}
}

I also want to make sure that you're instantiating your dom-object before trying to use the methods in your database query loop?

$doc = new DOMDocument('1.0');
$doc->loadHTMLFile( 'test.html' );

foreach($rpthdr as $hdr)
{
	$customer_id = $hdr['customer_id'];
	$location_id = $hdr['location_id'];

	$morning = $db->Query("SELECT... WHERE... t1.customer_id = '$customer_id' AND t1.location_id = '$location_id' AND t3.report_type = 'morning'");
	foreach($morning as $rpt2)
	{
		$td = $doc->createElement('td');
		$td->nodeValue = $rpt2['report'];
		$rpt1->appendChild( $td );
	}

Here have you you already called the $doc = new DOMDocument('1.0'); and loadHTML methods?

:-)

langsor · Answer 2 · 2008-08-23T12:24:16+00:00

langsor

16 Years Ago

Right-O,

I'm currently developing an inventory management interface (similar to PHPMyAdmin but more user friendly -- I'm notorious for reinventing the wheel) and doesn't do any data-type validation, so I'm using VARCHAR for all the non-DATE fields.

Anyway, it's coming along well and a nice mix-up of PHP, regular HTML forms, JavaScript, CSS, and Ajax methods from the little Ajax class I built and revamped recently.

Anyway, I've attached an excerpt of the main page ... it's procedural programming structure but shows my DOM methods from my database query results.

I'm hoping a live example will make more sense than fragmented code blocks here.

...

example.php (3.4 KB)

<?php

// DATABASE CONNECT AND QUERIES
$host = 'localhost';
$user = 'test';
$pass = 'TopSecret!';
$dbase = 'test';
$table = 'some_table';

$conn = mysql_connect( $host, $user, $pass ) or die( mysql_error() );
mysql_query( "USE $dbase" );

//*** DELETED FOR EXAMPLE BREVITY ***//
// FORM REQUEST DATA AND VALIDATION
// AJAX FORM UPDATING REQUEST
// AJAX FORM DELETE REQUEST
// AJAX FORM CREATE REQUEST
// PAGING VALUES FROM TOP MENU

$result = mysql_query( "SHOW COLUMNS FROM `$table`" );
$cols = mysql_num_rows( $result );
while ( $record = mysql_fetch_object( $result ) ) {
  $labels[] = preg_replace( "/_/", ' ' , ucwords( $record->Field ) );
  $fields[] = $record->Field;
}
mysql_free_result( $result );

$result = mysql_query( "SELECT SQL_CALC_FOUND_ROWS * FROM `$table` $order_str $desc_str LIMIT $set,$num" );
$found = mysql_query( "SELECT FOUND_ROWS()" );
if ( $result ) {
  $rows = mysql_result( $found, 0, 0 );
  while ( $record = mysql_fetch_assoc( $result ) ) {
    $data[] = $record;
  }
  mysql_free_result( $result );
} else {
  $data = array();
}

// DOM PARSE AND PUBLISHING
$doc = new DOMDocument('1.0');
@$doc->loadHTMLFile( 'template.html' ) or 
  die( "Could not load html template" );
$nodes = $doc->getElementsByTagName('*');
foreach ( $nodes as $node ) {
  switch ( $node->getAttribute('id') ) {
    case 'tdata': $tdata = $node; break;
    case 'form_nav': $form_nav = $node; break;
    case 'link_nav': $link_nav = $node; break;
    case 'num': $page_num = $node; break;
    case 'idx': $page_idx = $node; break;
  }
}
$thead = $tdata->getElementsByTagName('thead')->item(0);
$tbody = $tdata->getElementsByTagName('tbody')->item(0);

//*** DELETED FOR EXAMPLE BREVITY ***//
// GENERATE PAGING MENUS
// total records in database
// field to order results by
// boolean, descending order
// number of records to display per page
// number of pages to generate nav for
// select option navigation menu
// link navigtion menu

// GENERATE TABLE HEADERS
foreach ( $labels as $i => $label ) {
  $th = $doc->createElement('th');
  $th->setAttribute('field',$fields[$i]);
  if ( $fields[$i] == $order ) {
    $th->setAttribute('class',( $desc ) ? 'order desc' : 'order asc');
    $th->nodeValue = $label;
  } else {
    $th->setAttribute('class',( $fields[$i] == 'key' ) ? 'num' : 'order');
    $th->nodeValue = ( $fields[$i] == 'key' ) ? '#' : $label;
  }
  $thead->appendChild($th);
}

// GENERATE TABLE DATA
foreach ( $data as $id => $row ) {
  $tr = $doc->createElement('tr');
  $tr->setAttribute('id',$row['key']);
  foreach ( $fields as $field ) {
    if ( $field == 'key' ) {
      $td = $doc->createElement('td');
      $td->setAttribute('class','key');
      $td->nodeValue = $set + $id + 1;
      $tr->appendChild( $td );
    } else {
      $div = $doc->createElement('div');
      $class = ( $field == 'created' || $field == 'updated' ) ? 'date' : 
        ( ( preg_match( '/image/', $row[$field] ) ) ? 'image' : 'data' );
      $more = ( $field == $order ) ? 'ord' : 
        ( ( $id % 2 ) ? 'evn' : 'odd' );
      $div->setAttribute('class',"$class $more");
      $div->setAttribute('field',$field);
      $div->nodeValue = ( $row[$field] ) ?
        $row[$field] : '&nbsp;';
      $td = $doc->createElement('td');
      $td->appendChild( $div );
      $tr->appendChild( $td );
    }
  }
  $tbody->appendChild($tr);
}

print $doc->saveHTML();

?>

tomjohnson 0 Newbie Poster · Answer 3 · 2008-08-23T12:56:10+00:00

Hey,
For better or worse I've spent the entire day playing with databases, dom, and templates...so I should be warmed up, but also a little brain-fried. :-)

I know how that can be. It can be equated to eating too much cheesecake sometimes. I feel like that quite often and have yet to learn that taking small breaks will help.

The two things I notice off the top are in your final DOM loop you're calling the wrong object to fill its childNode

Ok, now on this, That was left over from after I had converted all the tags to "td". It really isn't referencing anything at runtime. Good catch though. Thanks.

I also want to make sure that you're instantiating your dom-object before trying to use the methods in your database query loop?

I did not do that but I will try it now.

tomjohnson 0 Newbie Poster · Answer 4 · 2008-08-23T13:29:48+00:00

Langsor:

here is the dom

$doc = new DOMDocument('1.0');
$doc->loadHTMLFile( 'report.txt' );

$tags = $doc->getElementsByTagName('*');
foreach ( $tags as $i => $tag )
{
	if ( $tag->getAttribute('id') == 'location' )
	{
		$tag->nodeValue = '1';
	}

	if ( $tag->getAttribute('id') == 'addr' )
	{
		$tag->nodeValue = '2';
	}

	if ( $tag->getAttribute('id') == 'city' )
	{
		$tag->nodeValue = '3';
	}

	if ( $tag->getAttribute('id') == 'state' )
	{
		$tag->nodeValue = '4';
	}

	if ( $tag->getAttribute('id') == 'margin' )
	{
		$tag->nodeValue = '5';
	}

	if ( $tag->getAttribute('id') == 'employee' )
	{
		$tag->nodeValue = '6';
	}
}
print $doc->saveHTML();

This works great. What I see that it id doing is loading the html file. Where does my loop go?

langsor · Answer 5 · 2008-08-23T13:40:18+00:00

I'm sorry, what do you mean "where does your loop go?" I'm not clear what you are asking.
Is your code saving to the browser window with the DOM modifications?

By the way, just playing around -- you could optimize the above to this :-)

<?php
$doc = new DOMDocument('1.0');
$doc->loadHTMLFile( 'report.txt' );

$tags = $doc->getElementsByTagName('*');
foreach ( $tags as $tag )
  switch( $tag->getAttribute('id') ) {
    case 'location': $value = 1; break;
    case 'addr': $value = 2; break;
    case 'city': $value = 3; break;
    case 'state': $value = 4; break;
    case 'margin': $value = 5; break;
    case 'employee': $value = 6; break;
  }
  $tag->nodeValue = $value;
}

print $doc->saveHTML();
?>

tomjohnson 0 Newbie Poster · Answer 6 · 2008-08-23T13:45:45+00:00

Right-O,
I'm currently developing an inventory management interface (similar to PHPMyAdmin but more user friendly -- I'm notorious for reinventing the wheel) and doesn't do any data-type validation, so I'm using VARCHAR for all the non-DATE fields.

Congrats on that. I bet it will be nice. You do some nice coding also. A lot of c syntax I see. I am not that advanced yet but I can can get the idea of what is happening.

I must have missed this post because I think we possibly posted at the same time. Thanks for the example. I will look it over and see what can do.

Anyway, it's coming along well and a nice mix-up of PHP, regular HTML forms, JavaScript, CSS, and Ajax methods from the little Ajax class I built and revamped recently.

I would like to see that when you are done. Is this something that you are using for yourself only or will share?

tomjohnson 0 Newbie Poster · Answer 7 · 2008-08-23T13:51:50+00:00

Smartie.. :) I didn't even thing about using a switch. Thanks. Looks better too. :)

This is the loop that needs to loop the database records.

require_once("db.php");
$db = new Database('test');
$rpthdr = $db->Query("SELECT...");

$date = date("m/d/Y");

foreach($rpthdr as $hdr)
{
	$customer_id = $hdr['customer_id'];
	$location_id = $hdr['location_id'];

	$morning = $db->Query("SELECT... WHERE... t1.customer_id = '$customer_id' AND t1.location_id = '$location_id' AND t3.report_type = 'morning'");
	foreach($morning as $rpt2)
	{
		$td = $doc->createElement('td');
		$td->nodeValue = $rpt2['report'];
		$rpt1->appendChild( $td );
	}

	$evening = $db->Query("SELECT... WHERE... t1.customer_id = '$customer_id' AND t1.location_id = '$location_id' AND t3.report_type = 'evening'");
	foreach($evening as $rpt3)
	{
		$td = $doc->createElement('td');
		$td->nodeValue = $rpt3['report'];
		$rpt1->appendChild( $td );
	}
}

The code you have cleaned up by adding the switch statement.... The loop needs to loop all of the reports that are listed as "morning" and "evening" in the dom.

I'm sorry Langsor.. I am dense sometimes. Besides, I have never had to work with any xml type of code manipulation.

langsor · Answer 8 · 2008-08-23T13:57:51+00:00

I'm sorry Langsor.. I am dense sometimes. Besides, I have never had to work with any xml type of code manipulation.

Hah, I'm laughing -- you're having trouble wrapping your brain around the xml stuff and I'm having trouble visualizing the sql stuff...

Anyway, I'm dense most of the time, but stubborn.

Be right back after a look

tomjohnson 0 Newbie Poster · Answer 9 · 2008-08-23T13:59:21+00:00

I'm not sure if we are on the same page. Please let me explain again what I am trying to do, using your dom.

I have a plain old txt file with html in it. I am only interested in changing what is between the <td> tags. These fields will be:

location
address
city
state
margin
employee

Ok... the above are database fields and are also the ids in the report.txt file. These fileds will NOT loop. There is only going to be one report per location. Below are the fields that ALSO included in this report, but NEED to loop however many rows of data are in the database

morning < header and database field
evening

Now, under morning and evening, there could be one row of data or a bunch of rows. These are the rows that I am concerned with looping.

That being said, when the outer foreach (that gets the customer_location) finishes, that same outer loop needs to go to the NEXT customer location and grab all of the data for that customer. Does that make sense? :)

tomjohnson 0 Newbie Poster · Answer 10 · 2008-08-23T14:04:04+00:00

Hah, I'm laughing -- you're having trouble wrapping your brain around the xml stuff and I'm having trouble visualizing the sql stuff...
Anyway, I'm dense most of the time, but stubborn.
Be right back after a look

LOL.. I'm also laughing... I also am very dense and stubborn. :) I am going to throw together some actual sample data for you to look at. Maybe visualizing it will help. brb

langsor · Answer 11 · 2008-08-23T14:12:08+00:00

So you want to change

<td>location</td>
<td>address</td>
<td>city</td>
<td>state</td>
<td>margin</td>
<td>employee</td>

...to...

<td>1</td>
<td>2</td>
<td>3</td>
<td>4</td>
<td>5</td>
<td>6</td>

...or does your template file look like this...

<td id="location"></td>
<td id="address"></td>
<td id="city"></td>
<td id="state"></td>
<td id="margin"></td>
<td id="employee"></td>

...and you want to insert the numbers?

Also,
$td->nodeValue = $rpt3;
$rpt1->appendChild( $td );
is $prt1 a node that you've already grabbed ... getAttribute('id') == 'rpt1'

and is $rpt3 from a sql result, because it looks like $rpt1 and I just want to make sure it's not a node...

I'll wait for your sample data -- hopefully the output you're trying to create :-)

langsor · Answer 12 · 2008-08-23T14:22:32+00:00

I know I said I would wait, but it looks like maybe you're trying to do something like this.

foreach($morning as $rpt2)
  {
    $td = $doc->createElement('td');
    switch( $rpt2['report'] ) ) {
      case 'location': $value = 1; break;
      case 'addr': $value = 2; break;
      case 'city': $value = 3; break;
      case 'state': $value = 4; break;
      case 'margin': $value = 5; break;
      case 'employee': $value = 6; break;
    }
    $td->nodeValue = $value;
    $rpt1->appendChild( $td );
  }

langsor · Answer 13 · 2008-08-23T14:29:06+00:00

Or maybe this ...

$tags = $doc->getElementsByTagName('*');
foreach ( $tags as $tag )
  switch( $tag->getAttribute('id') ) {
    case 'location': $location = $tag; break;
    case 'addr': $addr = $tag; break;
    case 'city': $city = $tag; break;
    case 'state': $state = $tag; break;
    case 'margin': $margin = $tag; break;
    case 'employee': $employee = $tag; break;
  }
}

foreach($rpthdr as $hdr)
{
  $customer_id = $hdr['customer_id'];
  $location_id = $hdr['location_id'];

  $morning = $db->Query("SELECT... WHERE... t1.customer_id = '$customer_id' AND t1.location_id = '$location_id' AND t3.report_type = 'morning'");
  foreach($morning as $rpt2)
  {
    $td = $doc->createElement('td');
    switch( $rpt2['report'] ) ) {
      case 'location': $location.nodeValue = 1; break;
      case 'addr': $addr.nodeValue = 2; break;
      case 'city': $city.nodeValue = 3; break;
      case 'state': $state.nodeValue = 4; break;
      case 'margin': $margin.nodeValue = 5; break;
      case 'employee': $employee.nodeValue = 6; break;
    }
//    $rpt1->appendChild( $td ); // not sure when you want to append this...after you know it's done being filled up
  }

tomjohnson 0 Newbie Poster · Answer 14 · 2008-08-23T14:50:07+00:00

...or does your template file look like this...

<td id="location"></td>
<td id="address"></td>
<td id="city"></td>
<td id="state"></td>
<td id="margin"></td>
<td id="employee"></td>

Yes, this is exactly what it looks like. The tags are empty just like you have them and I reference the ids in the dom.

I drew you a cute little paint photo but I see that I can't upload files here on this forum. I have the attachment icon with an X. Let me try and write it out for you.
location	address			city		margin	employee
4 east	1422 main street	los angeles	6	78
1 south	4877 main street	los angeles	2	18
4 West	4744 main street	key west	8	12

Now, each one of these locations will have many reports associated with them denoted by morning and/or evening. All of these reports will be made available on one piece of paper (as provided by our report.txt template)

Here some sample data for the reports:

employee	time		actions		notes		report
12		8:00 AM		Started calls	some note	morning
133		9:00 AM		Started calls	some note	morning
13		10:00 AM	Started calls	some note	morning
32		7:00 PM		Started calls	some note	evening
54		9:00 PM		Started calls	some note	evening

Now, for each "location" as I have above, there will only one record with these "reports". These reports will be sorted by the database by the report column. Customer "X" can have multiple reports on his report.

Also,
$td->nodeValue = $rpt3;
$rpt1->appendChild( $td );
is $prt1 a node that you've already grabbed ... getAttribute('id') == 'rpt1'

well, when you say "grabbed". This is exactly where my foreach loop should come in. It needs to loop the database results to get the data in my last sample above. This is where I am having the issue.

from a sql result, because it looks like $rpt1 and I just want to make sure it's not a node...

The nodeValue() is exactly what I need to be looping. Actually, hang on... When I say that I need the nodeValue() to loop, what I ultimately need is the final report.txt to loop however many rows are in the db and show the following:

<td id="employee"></td>
<td id="time"></td>
<td id="actions"></td>
<td id="notes"></td>
<td id="report"></td>

:)

tomjohnson 0 Newbie Poster · Answer 15 · 2008-08-23T14:58:57+00:00

Or maybe this ...

$tags = $doc->getElementsByTagName('*');
foreach ( $tags as $tag )
  switch( $tag->getAttribute('id') ) {
    case 'location': $location = $tag; break;
    case 'addr': $addr = $tag; break;
    case 'city': $city = $tag; break;
    case 'state': $state = $tag; break;
    case 'margin': $margin = $tag; break;
    case 'employee': $employee = $tag; break;
  }
}

foreach($rpthdr as $hdr)
{
  $customer_id = $hdr['customer_id'];
  $location_id = $hdr['location_id'];

  $morning = $db->Query("SELECT... WHERE... t1.customer_id = '$customer_id' AND t1.location_id = '$location_id' AND t3.report_type = 'morning'");
  foreach($morning as $rpt2)
  {
    $td = $doc->createElement('td');
    switch( $rpt2['report'] ) ) {
      case 'location': $location.nodeValue = 1; break;
      case 'addr': $addr.nodeValue = 2; break;
      case 'city': $city.nodeValue = 3; break;
      case 'state': $state.nodeValue = 4; break;
      case 'margin': $margin.nodeValue = 5; break;
      case 'employee': $employee.nodeValue = 6; break;
    }
//    $rpt1->appendChild( $td ); // not sure when you want to append this...after you know it's done being filled up
  }

Yes Langsor, this is what I am trying to do. Hooray!. However, it doesn't work. :)

langsor · Answer 16 · 2008-08-23T15:08:31+00:00

I think we're getting close -- which is good because my vision is getting fuzzy....

do you have an html element wrapping these tags, if not can you make one with an id=""

Or we can just generate them fresh in the PHP code ...

Do they need the id="" values, or are they just hooks for the PHP code?

langsor · Answer 17 · 2008-08-23T15:10:17+00:00

I think we either need to create the <td></td> tags inside the PHP, or use cloneNode to copy and insert data into the <td> tags

It looks like this
source.html

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Untitled Document</title>
</head>
<body>
<div id="source">some content</div>
<div id="destin"></div>
</body>
</html>

dom.php

<?php
$doc = new DOMDocument('1.0');
$doc->loadHTMLFile( 'source.html' );
$tags = $doc->getElementsByTagName('*');

foreach ( $tags as $tag ) {
  switch( $tag->getAttribute('id') ) {
    case 'source': $source = $tag; break;
    case 'destin': $destin = $tag; break;
  }
}

$clone = $source->cloneNode(TRUE);
$destin->appendChild($clone);
print $doc->saveHTML();
?>

output source

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>Untitled Document</title>
</head>
<body>
<div id="source">some content</div>
<div id="destin"><div id="source">some content</div></div>
</body>
</html>

Okay, still working on it ...

The reasom is that any time you assign or move a 'grabbed' node it removes it from the old location or overwrites its values

langsor · Answer 18 · 2008-08-23T15:14:16+00:00

From the above, without cloning

dom.php

<?php
$doc = new DOMDocument('1.0');
$doc->loadHTMLFile( 'source.html' );
$tags = $doc->getElementsByTagName('*');

foreach ( $tags as $tag ) {
  switch( $tag->getAttribute('id') ) {
    case 'source': $source = $tag; break;
    case 'destin': $destin = $tag; break;
  }
}

$destin->appendChild($source);
print $doc->saveHTML();
?>

...becomes this....

html source

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>Untitled Document</title>
</head>
<body>

<div id="destin"><div id="source">some content</div></div>
</body>
</html>

tomjohnson 0 Newbie Poster · Answer 19 · 2008-08-23T15:17:15+00:00

I think we're getting close -- which is good because my vision is getting fuzzy....

LOL... I get that all the time too. :)

do you have an html element wrapping these tags, if not can you make one with an id=""
<td id="employee"></td>
<td id="time"></td>
<td id="actions"></td>
<td id="notes"></td>
<td id="report"></td>

Ok, in the report.txt file, no.. I only have straight html code in there. I don't have any php or anything else in that txt file.

Or we can just generate them fresh in the PHP code ...

This sounds like the best option but I don't know how to do that.

Do they need the id="" values, or are they just hooks for the PHP code?

Well, they are only hooks for the dom, I'm not using them for css.

Thanks for sticking with me Langsor..

langsor · Answer 20 · 2008-08-23T15:30:53+00:00

I'm going to have to cash it in here in a minute, do take a look at my attached file in my first or second post of tonight ... it's pretty similar idea I think...

Quick question
$rpt2 is this an array of these values?
4 east 1422 main street los angeles 6 78
... or these value ...
12 8:00 AM Started calls some note morning
... or not an array? ;-\

Not entirely sure I have the clarity to figure this out at the moment.

tomjohnson 0 Newbie Poster · Answer 21 · 2008-08-23T15:35:10+00:00

I'm going to have to cash it in here in a minute, do take a look at my attached file in my first or second post of tonight ... it's pretty similar idea I think...
Quick question
$rpt2 is this an array of these values?
4 east 1422 main street los angeles 6 78
... or these value ...
12 8:00 AM Started calls some note morning
... or not an array? ;-\
Not entirely sure I have the clarity to figure this out at the moment.

Hey Langsor, I understand.. Get some rest. To answer your question, it is the latter. It is an array. I have the results from the db loaded into an array and then call them out with the foreach. The first example (1422 main street) would be drawn from a the $rpt2 database field. Hope that helps. Thanks man!

langsor · Answer 22 · 2008-08-23T15:37:13+00:00

I think this is all I've got for now ... will revisit maybe tomorrow night or Sunday

$morning = $db->Query("SELECT... WHERE... t1.customer_id = '$customer_id' AND t1.location_id = '$location_id' AND t3.report_type = 'morning'");
  foreach($morning as $rpt2)
  {
    $td = $doc->createElement('td');
    switch( $rpt2['report'] ) ) {
      // THIS .. or this kind of thing ...
      case 'location': $td.nodeValue = $rpt2['some_value']; break;
      // OR THIS ... ?
      case 'addr': $td.nodeValue = 2; break;
      // .... the rest of the case lines go here
    }
    $rpt1->appendChild( $td ); // where $rpt1 is a <tr> element you grabbed with an earlier loop through your template $tags
  }

I'll be you can muddle through it but I will be back to help ASAP

Cheers

tomjohnson 0 Newbie Poster · Answer 23 · 2008-08-23T15:44:09+00:00

Thanks for all of your help Langsor! I will review your example and this entire nights postings.