[Techtalk] building a personal cluster

[Daniel Pittman]

Carla Schroder <carla at bratgrrl.com> writes:

> I've been reading up on clusters because I think it would be a fun
> experiment to build one at home. Hey, what self-respecting geek
> doesn't have a houseful of lazy, under-utilitized computers? One
> question I'm having a hard time finding an answer for-- would this
> speed up all computing tasks, or does it only work for applications
> that are specially tweaked?

That depends entirely on the cluster technology you choose, and how much
time you put into it.  The easy options will spread load across machines
only with deliberate action, more complicated options make multiple
machines a transparent non-uniform "single system image".

However, without specific tuning of software you are not going to see a
single process benefit from more than a single system worth of power.

Um, and performance gain ... maybe.  That depends on how independent the
parts of your workload are.

> I do a lot of audio and video editing, and if a little cluster would
> speed that up then I want one now.

Well, transcode is going to benefit a lot more than (say) ffmpeg in this
environment, regardless of how you structure things, because it is a set
of pipelined small tools rather than one big one.


Anyway, the easy options are not really any different to wiring together
a bunch of machines with ssh and ad-hoc shell scripts, really.  The
benefits you pick up come from automatic batch scheduling and shared
filesystem images.

The only really live single system images option still remaining seems
to be OpenSSI: http://openssi.org/cgi-bin/view?page=openssi.html

I was also going to list OpenMOSIX, but it has officially shut down.
http://openmosix.sourceforge.net/

Regards,
        Daniel