Persisting large objects across requests

[knarfs]

I am developiong a php application which builds several large collections of objects in the course of interacting with the user. The objects need to persist across requests from the user and I've used php sessions to successfully implement this. The objects are persisted until the user is satisfied with what they have done and they request the information represented by the objects to be saved.

In profiling the application I found that the session start takes about half of the total time of a request (about 4 seconds in total for a request). Since not all stored objects are used on each request from the user, I attempted to improve the responsiveness of the application by selectively restoring the objects needed for the request. To do this I have serialized the objects to individual files and only unserialized those required. This change actually blew up the application because the serialize/unserialize consumed huge amounts of memory (and time to run). It surprised me that the session handler could serialize and unserialize all of the objects in less time and lower memory consumption than my attempt to do only a subset. I believe this may be due to the loading of all the serialized object data into memory, but I'm not sure.

Following is the code to serialize and unserialize the objects that I have used:

Code to write the serialized object to files:

public static function writeClose() {
    foreach (self::$activeRegistries as $registryName=>$registry) {
        if (isset($_SESSION["filemap"]) && isset($_SESSION["filemap"]["registries"]) && isset($_SESSION["filemap"]["registries"][$registryName])) {
            $path = $_SESSION["filemap"]["registries"][$registryName];
            if (file_put_contents($path, serialize($registry)) === false) {
                throw new repositoryException("Exception while writing the '$registryName' registry to storage");
            }
        } else {
            throw new repositoryException("Could not find the file path for the '$registryName' registry");
        }
    }
}

Code to retrieve the serialized objects:

private static function getRegistry($registryName) {
    // First check to see if the registry is already active in this request
    if (isset(self::$activeRegistries[$registryName])) {
        $registry = self::$activeRegistries[$registryName];
    } else {
        // The registry is not active, so see if it is stored for the session
        if (isset($_SESSION["filemap"]) && isset($_SESSION["filemap"]) && isset($_SESSION["filemap"]["registries"][$registryName])) {
            $filePath = $_SESSION["filemap"]["registries"][$registryName];
            if (file_exists($filePath)) {
                $registry = unserialize(file_get_contents($filePath));
                self::$activeRegistries[$registryName] = $registry;
            } else {
                throw new repositoryException("Exception while getting serialized object for registry '$registryName'");
            }
        } else {
            // The registry is not saved in the session, so create a new one
            $registry = self::createRegistry($registryName);
            $filePath = "/tmp/" . session_id() . $registryName;
            $_SESSION["filemap"]["registries"][$registryName] = $filePath;
            self::$activeRegistries[$registryName] = $registry;
        }
    }
    return $registry;
}

How can the application be improved over using the php session handler to retrieve all of the collections for each request?

[cereal]

Try igbinary instead of serialize/unserialize, it is faster. Also you can try to store these objects to memory, through Memcached. Since session files will be written to disk you get high I/O activity and this, if session directory is in the same disk of the web server, can decrease performance.

In order to work with these tools you will need to install them in the server, in particular make sure to have a firewall enabled and to filter memcached ports, otherwise you will expose the daemon to everyone.

For more information check:

• https://github.com/igbinary/igbinary
• http://pecl.php.net/package/igbinary
• http://php.net/manual/en/book.memcached.php
• http://memcached.org/

[TomH.PG]

I'd recommend the igbinary implementation if sticking with storage in the session data. Outside of that, you may want to store the objects in the database and store just a reference to it in the session, or use a solution such as Redis http://www.redis.io which can integrate with PHP using a variety of clients (http://www.redis.io/clients) such as phpredis or optionally the Credis wrapper for phpredis