[prog] Freeing memory in PHP/Apache

[Almut Behrens]

On Sat, May 01, 2004 at 11:20:14AM +1000, Rasjid Wilcox wrote:
> I am using a PHP based CMS (MySource by Squiz.net) and although quite good it 
> is a little heavy handed with the memory usage.
> 
> Upon starting apache, each child uses about 3MB of memory.  However after 
> serving up a page through MySource, the memory usage of the child concerned 
> jumps to about 12MB and stays there.  (Actually, I get two such instances.)  
> Since this is a User-Mode-Linux based virtual server, I only have 96MB ram, 
> and so this was quite a problem.

Firstly, I'd like to say I don't know much about the inner workings of
PHP, so take the following with a grain of salt.

OTH, what you describe seems to be a general problem with most
interpreter languages integrated into apache, like mod_perl and
mod_python. They're all a bit heavy on memory resources.  IIRC, the
issue is that the interpreters have their own memory handling and do
_not_ return memory once it's malloc()ed from the kernel. It does get
reused, though. When some data structure is being freed, the memory
is returned to the interpreter's internal memory pool. As many higher
level languages have reference-count based memory handling, this
happens when the last reference to some variable or data structure is
lost, i.e. when it goes out of scope, normally.

So, memory usage typically goes up rapidly after a new apache child has
been forked. In practice, it should then stay at some maximum value
determined by the largest data structure encountered so far.  If it
keeps growing continuously, you most probably have a memory leak. With
reference-count based garbage collection techniques, this happens when
there are circular data structures. Unfortunately, this is not all that
infrequent when programs are becoming large and complex. The programmer
has to be aware of it, and must break circular data structures herself.
(I'm assuming that PHP works the same way - not 100% sure, though...)

AFAIK, interpreters are using their own memory handling mainly for
two reasons.  One is performance.  The other is to have some last
authoritative handle on the overall memory usage. I.e., independently
of how much the application itself messes up memory (circular data
structures), when the interpreter quits, all memory always gets freed.
The problem is that, _ideally_, an apache child process (containing its
own interpreter instance) is running persistently, and never quits...

Essentially, the only thing you can do is to keep large data structures
outside of the interpreter.  However, this is probably only of academic
interest for you, as (a) you're using some existing application (CMS),
(b) I'm not sure whether PHP generally offers the option to write
extension modules in C.  What you'd have to do is to malloc your own
memory within the extension module, which you can then also free
yourself when no longer needed. In this case it is being returned to
global system memory.  Access to the individual data fields of the
object would typically be realised via accessor functions, which each
only retrieve a tiny fraction of the whole data structure...
I once did this for a mod_perl based web application, for the above
reason. It worked beautifully.

More practical techniques involve setting MaxRequestsPerChild to some
small value (as you did -- '1' seems a bit extreme, though...), or
killing apache child processes automatically when their memory usage
becomes excessive (i.e. memory-usage-based, instead of request-count-
based), or simply having less apache children run in parallel
(-> StartServers, MinSpareServers, MaxSpareServers, MaxClients).
According to some benchmarks I did myself, the latter often results in
a better overall performance when the system is short of memory.

A mod_perl-specific discussion of all this can be found in the mod_perl
guide.  You might still want to read it:

http://perl.apache.org/docs/1.0/guide/performance.html#Preventing_Your_Processes_from_Growing

The following more general discussion of how and when to mix a
lighweight frontend with a heavy-weight backend server might be
useful, too:

http://perl.apache.org/docs/1.0/guide/strategy.html

Also, this topic comes up from time to time on the mod_perl mailing
list (http://perl.apache.org/maillist/modperl.html) - so you might want
to check the archives for further discussion.

Good luck,

Almut