Last Updated: February 25, 2016
·
54.64K
· filipc

Avoiding memory leaks in Symfony2 Doctrine EntityManager

Case:

======
Migration script uses lot of Doctrine Queries. In case of lot data to migrate it can fail because of memory limit is reached.

Problem:

======
PHP script is consuming memory after each query and data modification. This mean memory leaks if PHP process is long. When memory limit is reached script throw exception and migration process fails.

Solution:

======
Here are some tips how to control memory usage when using Symfony2 and Doctrine EntityManager. You are able to keep memory consumption of reasonable level.

  • Remember to unset object if not needed

unset($obj);

or

$obj = null;

Both does the same, but unset() does not force immediate memory freeing. PHP's garbage collector will do it when it see fits - by intention as soon, as those CPU cycles aren't needed anyway or as late as before the script would run out of memory, whatever occurs first. Using $obj = null; will rewrite variable's data. It might get memory freed faster, but it may steal CPU cycles from the code that truly needs them sooner, resulting in a longer overall execution time, but lower memory consumption.

  • Define no logging and profilling in config.yml
doctrine:
    dbal:
        connections:
            conn1:
                driver: ...
                ...
                logging: false
                profiling: false
  • Tune php.ini to use all available memory of server.

memory_limit = -1

It can be dangerous. Keep in mind to enable it only for migration process. After that set to some realiable value. It allows you to use large amount of memory if needed to prevent out of memory error.

  • If you are using PHP at least 5.3 you can force garbage collection after destroying object

gc_collect_cycles();

*Remember to call gc_enable() first

  • Use EntityManager cleaning functions
$this->em->detach($obj);
$this->em->flush();
$this->em->clear();
  • Disable SQL Logger

$this->em->getConnection()->getConfiguration()->setSQLLogger(null);

Doctrine by default logs all queries. Turning it off will save extra memory consumption.

6 Responses
Add your response

Thank you !

over 1 year ago ·

very nicely said!

over 1 year ago ·

Hi,
First of all, thank you for the article.
I have a question about setting to null a variable. If a variable is in a function, isn't it destroyed when you are getting out of the function ? So is it necessary to unset a variable inside a small function ?
I'm doing a script for updating datas comming from a Webservice, so I have the same problem, the memory usage goes up to 7Go =)

over 1 year ago ·

Any reason for detaching and clearing? Clearing is just a mass detach.

over 1 year ago ·

After trying this all, replacing createQuery for createQueryBuilder fixed the memory leak problem :)

over 1 year ago ·

I would add a few things.

The detach() step is not necessary, since clear() does that for all entities currently tracked by Doctrine.

I like to flush() then immediately gc_collect_cycles() every X objects in a loop as that keeps the memory usage somewhat constant. Here's an example:

gc_enable();
foreach ($objects as $i => $obj) {
    $em->persist($obj);
    // change 500 depending on object size and available memory
    if ($i % 500 == 0) {
        $em->flush();
        gc_collect_cycles();
    }
}
// Once more for the remainder.
$em->flush();
gc_collect_cycles();
over 1 year ago ·