[prog] httpd, accept from forked processes

[Daniel Pittman]

"Sam Watkins" <swatkins at fastmail.fm> writes:

> I'm writing a webserver, it is working well, it uses select / epoll,
> and forks two processes both of which try to accept new connections,
> the idea being that it should be able to take advantage of two
> cpus.

Reasonable enough.  However...

> However it seems that almost all of the connections go to one of the
> processes, which I think is lowering the performance.

...it could be.  In some cases distributing connections to the cache-hot
process will improve, not reduce, performance, especially in the real
world where memory and disk latency is one of the big performance
factors.

> Can anyone suggest how to get the two processes to share the
> connections fairly evenly between them, or why it wouldn't already be
> doing that?

Structure your application in a master/slave arrangement, similar to any
random pre-forking server.  Put a lock around the accept call in the
children, wait for a notification that a connection is ready in the
parent, then signal one child only to accept the request.

Season with enough IPC that you can manage all the other details, like
accept failing and (non-)blocking in the child, if the remote host
disconnects between the wakeup and the accept call.


Um, and FWIW, usually picking the first available child rather than the
next available child does better. :)


If you simply punt this to the kernel you will get uncontrolled
behaviour that varies between kernel versions, ranging from "wake
everything" to "wake one" in whatever sequence is efficient this week.

Regards,
        Daniel