simple analytics script

[Lapixx]

Hi,

I'm creating a simple analytics script to use on my website(s). I started of very simply by just creating a new record for every session, and updating the sessions (i.e. incrementing the number of pageloads for that session) when a new page is loaded. I also save the browser/version and platform the visitor is using.

This works fine as it is, though it is not really efficient in any way. I've tested the script with a few (say 20 or so) records in the table, but I don't think everything will run as smoothly as it does now when the website is being visited more frequently.

Now I have been thinking how to optimize this. The problem I'm facing is: If I were to, for example, only save the number of pageloads and the number of sessions (I could use the 2 numbers to calculate a average pageloads per session), I will be unable to make a graph of pageloads over the last X days.

So my question is: How do I optimize this system in a way that I can still use the data in multiple ways?

Thanks in advance.

[Designer_101]

Interesting concept. Could you not just update the database on each session rather than updating the session itself?

Just a thought? I've never done this myself so don't take my question too in depth.

[Lapixx]

Well, the problem isn't really the updating of the sessions (at least, not the main problem), but rather that I'm afraid that the large number of records (every session/visit is a record) will slow down the script.

I read some things about using an index in the MySQL table but it appears to me that it can only speed things up a little bit and only up to a certain point. Now I'm not really sure about this, since I don't know that much about the working of databases in general.

Does someone have an idea how "big" analytics tools store the data they acquire? Those mostly save data for multiple websites (i.e. Google Analytics) so I assume that it should be possible to make something efficient that is only used by one website.

[Designer_101]

I would keep everything in a cookie. These are stored on the users local machine therefore won't bug everything up or slow anything down for other users. Also, one cookie updating as you go through each page isn't a huge thing. It may sound like it but you'd be suprised how quick processing is. Imagine daniweb for example. The amount of database queries on this single page is quite huge!

This would be your best approach as far as I can see.

Hope this helps.

[Lapixx]

True, but at some point you'll need to save the data serverside. What do you suggest to save in the cookie?

[Lapixx]

Also, I understand that a forum needs to save a lot of data, but I assume *something* is done to make it all run smooth right?

[n_e]

Also, I understand that a forum needs to save a lot of data, but I assume *something* is done to make it all run smooth right?

Big ass databaseservers running Oracle or postgresql are required for a descent-size forum.
If you really need to store a lot of data, you might want to look in to them. But as far as I can see, you only need to update 1 counter and save some userinfo. You're probablt better of with a MySQL DB.

I've tested the script with a few (say 20 or so) records in the table, but I don't think everything will run as smoothly as it does now when the website is being visited more frequently.

If it doesn't run good enough with only 20 records in the DB there could be two things wrong.
1. Your DB-server is really (really) slow and I mean slow in a 486-kinda way:)
2. Something is wrong with your script. You could post it so we can have a look?

[Lapixx]

No, I was trying to say that is DOES run good now, but I'm afraid it won't when there will be more than a few records in the database.

Also, even though I'm aiming for a simple script, I am able to use the saved data to (for example) calculate the bounce rate, show a table with how many people use which browser, and so on.
For this, there need to be run a few queries, and since I store the data in the same table, it needs to search things in a list of [number of visits] records.

The problem with 1 value (the counter itself) is that you can't see how many pageviews you got on a certain day, and you don't know which page was visited.

Since I think it's a bit useless to see the exact time of a visit, I could make a new record for each day and page (so creating a table with 'day','page','pageviews' columns). This would create [number of pages on website] records per day, instead of a record for each visitor/session.