ORM - evil or How I tried to cache Propel in Symfony
Working on one project (social network), I faced the task of “making friends” with the memcache data model. As you already understood from the title, the project is written on the symfony framework, and Propel is used as the ORM.
Why cache the model, you ask, if you can just use the View cache?
As practice has shown (at first it was the view cache that was used) this turned out to be inappropriate. Each time the user changed the data, it was necessary to flush the cache of several cached blocks at once. The number of these blocks with the development of the project is growing, the logic is complicated. To relieve the brain and avoid mistakes, it was decided to cache the model.
As it turned out, Propel is not intended for caching at all, and when the cache-view is disabled, it generates a VERY large number of requests.
My record is 250 !!! Why do you ask? Because Propel implements connections between objects in an interesting way. Instead of taking an object from a single point (for example, using the retriveByPk method), it makes a request to the database !!! And this means that if we have objects of the user, his photos, his blog entries, then from each object we get a request to the database. We have, for example, in a sample of 20 photos - get 20 requests to the database to find out the author name of the photo. And if the photo has another Album? Then 40. And if this is the main page on which there are pictures, blog entries and much more ?! All 250 :)
What is interesting in Propel'e warn about this :)
Here is an example:
And you can just:
In this simple redefinition, you can get rid of "extra" requests .
However, if there are a lot of connections, you will have to pretty dig into the code.
Now let's move directly to caching. I wanted that after calling the retrieveByPK method, an object would be returned from the cache, if it was there, and if not, it would be taken from the database and stored in the cache. Most interestingly, propel has a pool of loaded objects. This pool is just temporary storage.
Here is an example of the generated methods for the user model.
And if you redefine these methods so that objects that fall into the pool are cached in memcache at the same time, then the task of caching model objects will be solved. True, this is not an elegant solution, but in Propel model objects are isolated from each other. And if something can be done, then at a lower level, the level of frameworka. This means that such changes must be taken into account when updating symfony, which is very inconvenient.
Propel should not be used to build large and complex web projects.
UPD
Joins are not suitable, because many join’s are not recommended to be used, this is an additional load on the base, because the request is more complicated. With a heavy load, this is significant. But if you do everything with simple requests and pull the keys on the primary keys, then everything falls into place. Example. A selection of 20 photos, they have albums, authors, author avatars, album avatars. With join, there will be a query connecting 5 tables !!! And if there are 5 such pages per page !? And in my case, that’s it!
UPD2
Read the article carefully))
UPD3
Continuation of the story
How I made friends with memcache and Propel in Symfony
Why cache the model, you ask, if you can just use the View cache?
As practice has shown (at first it was the view cache that was used) this turned out to be inappropriate. Each time the user changed the data, it was necessary to flush the cache of several cached blocks at once. The number of these blocks with the development of the project is growing, the logic is complicated. To relieve the brain and avoid mistakes, it was decided to cache the model.
As it turned out, Propel is not intended for caching at all, and when the cache-view is disabled, it generates a VERY large number of requests.
My record is 250 !!! Why do you ask? Because Propel implements connections between objects in an interesting way. Instead of taking an object from a single point (for example, using the retriveByPk method), it makes a request to the database !!! And this means that if we have objects of the user, his photos, his blog entries, then from each object we get a request to the database. We have, for example, in a sample of 20 photos - get 20 requests to the database to find out the author name of the photo. And if the photo has another Album? Then 40. And if this is the main page on which there are pictures, blog entries and much more ?! All 250 :)
What is interesting in Propel'e warn about this :)
Here is an example:
public function getsfGuardUser(PropelPDO $con = null)
{
if ($this->asfGuardUser === null && ($this->user_id !== null)) {
$c = new Criteria(sfGuardUserPeer::DATABASE_NAME);
$c->add(sfGuardUserPeer::ID, $this->user_id);
$this->asfGuardUser = sfGuardUserPeer::doSelectOne($c, $con);
/* The following can be used additionally to
guarantee the related object contains a reference
to this object. This level of coupling may, however, be
undesirable since it could result in an only partially populated collection
in the referenced object.
$this->asfGuardUser->addPhotos($this);
*/
}
return $this->asfGuardUser;
}
* This source code was highlighted with Source Code Highlighter.
And you can just:
public function getsfGuardUser(PropelPDO $con = null)
{
return sfGuardUserPeer::retrieveByPK($this->getUserId(), $con);
}
* This source code was highlighted with Source Code Highlighter.
In this simple redefinition, you can get rid of "extra" requests .
However, if there are a lot of connections, you will have to pretty dig into the code.
Now let's move directly to caching. I wanted that after calling the retrieveByPK method, an object would be returned from the cache, if it was there, and if not, it would be taken from the database and stored in the cache. Most interestingly, propel has a pool of loaded objects. This pool is just temporary storage.
Here is an example of the generated methods for the user model.
public static function addInstanceToPool(sfGuardUser $obj, $key = null);
public static function removeInstanceFromPool($value);
public static function getInstanceFromPool($key);
public static function clearInstancePool();
* This source code was highlighted with Source Code Highlighter.
And if you redefine these methods so that objects that fall into the pool are cached in memcache at the same time, then the task of caching model objects will be solved. True, this is not an elegant solution, but in Propel model objects are isolated from each other. And if something can be done, then at a lower level, the level of frameworka. This means that such changes must be taken into account when updating symfony, which is very inconvenient.
Propel should not be used to build large and complex web projects.
UPD
Joins are not suitable, because many join’s are not recommended to be used, this is an additional load on the base, because the request is more complicated. With a heavy load, this is significant. But if you do everything with simple requests and pull the keys on the primary keys, then everything falls into place. Example. A selection of 20 photos, they have albums, authors, author avatars, album avatars. With join, there will be a query connecting 5 tables !!! And if there are 5 such pages per page !? And in my case, that’s it!
UPD2
Read the article carefully))
UPD3
Continuation of the story
How I made friends with memcache and Propel in Symfony