awzjhw
Last Updated: February 25, 2016
·
34.16K
· filipc
Fcd9b8724d04c501628bb47cc895769b

Avoiding memory leaks in Symfony2 Doctrine EntityManager

Case:

======
Migration script uses lot of Doctrine Queries. In case of lot data to migrate it can fail because of memory limit is reached.

Problem:

======
PHP script is consuming memory after each query and data modification. This mean memory leaks if PHP process is long. When memory limit is reached script throw exception and migration process fails.

Solution:

======
Here are some tips how to control memory usage when using Symfony2 and Doctrine EntityManager. You are able to keep memory consumption of reasonable level.

  • Remember to unset object if not needed

unset($obj);

or

$obj = null;

Both does the same, but unset() does not force immediate memory freeing. PHP's garbage collector will do it when it see fits - by intention as soon, as those CPU cycles aren't needed anyway or as late as before the script would run out of memory, whatever occurs first. Using $obj = null; will rewrite variable's data. It might get memory freed faster, but it may steal CPU cycles from the code that truly needs them sooner, resulting in a longer overall execution time, but lower memory consumption.

  • Define no logging and profilling in config.yml
doctrine:
    dbal:
        connections:
            conn1:
                driver: ...
                ...
                logging: false
                profiling: false
  • Tune php.ini to use all available memory of server.

memory_limit = -1

It can be dangerous. Keep in mind to enable it only for migration process. After that set to some realiable value. It allows you to use large amount of memory if needed to prevent out of memory error.

  • If you are using PHP at least 5.3 you can force garbage collection after destroying object

gc_collect_cycles();

*Remember to call gc_enable() first

  • Use EntityManager cleaning functions
$this->em->detach($obj);
$this->em->flush();
$this->em->clear();
  • Disable SQL Logger

$this->em->getConnection()->getConfiguration()->setSQLLogger(null);

Doctrine by default logs all queries. Turning it off will save extra memory consumption.

Say Thanks
Respond

6 Responses
Add your response

8942
C68ccddba82ebf14bd0b4c7cc8225fe1

Thank you !

over 1 year ago ·
11251
0 rohwyqii6tit wkmyrzzybrmq bj7wkmt0pqyntjmcpdkfbay44b 9zceyfg2ulcajovpaxcuws7

very nicely said!

over 1 year ago ·
11385
B0dc70bbae69d006b03dde0b442cd4ac normal

Hi,
First of all, thank you for the article.
I have a question about setting to null a variable. If a variable is in a function, isn't it destroyed when you are getting out of the function ? So is it necessary to unset a variable inside a small function ?
I'm doing a script for updating datas comming from a Webservice, so I have the same problem, the memory usage goes up to 7Go =)

over 1 year ago ·
13978
4232e4241e8921345f8d986a5173210f

Any reason for detaching and clearing? Clearing is just a mass detach.

over 1 year ago ·
15388
8b923892ad565b8ff7272057b6b54de5

After trying this all, replacing createQuery for createQueryBuilder fixed the memory leak problem :)

over 1 year ago ·
18979
2011 sq normal

I would add a few things.

The detach() step is not necessary, since clear() does that for all entities currently tracked by Doctrine.

I like to flush() then immediately gc_collect_cycles() every X objects in a loop as that keeps the memory usage somewhat constant. Here's an example:

gc_enable();
foreach ($objects as $i => $obj) {
    $em->persist($obj);
    // change 500 depending on object size and available memory
    if ($i % 500 == 0) {
        $em->flush();
        gc_collect_cycles();
    }
}
// Once more for the remainder.
$em->flush();
gc_collect_cycles();
over 1 year ago ·