[prog] Passing data between modules

Jacinta Richardson jarich at perltraining.com.au
Thu Jul 31 14:27:54 EST 2003


Part II of my thoughts on this matter.

You mention that you want to do this in C and a few people have suggested
that this might not be the best idea.  You've countered sensibly that you
want to use C because it's fast and because that's what you and your
work mates know.  You've also said that you can probably count on *NIX
systems maintaining a C compiler but not necessarily having
Perl/Python/Java.....

I'd like to offer some areas of thought about this.

* C is fast.  That's indisputable.  Higher level languages have great
difficulty being as fast as C in all areas.  Since many higher level
languages are written in C, being faster than C is just impossible.

However, most higher level languages are _faster_ to write in than C is.
Many businesses value programmer time over execution time.  If it takes 3
hours to write and 3 minutes to run, rather than 3 days to write and 2
minutes to run, many bosses will rather the longer run time.  If it's a
process that's expected to run over night, a runtime of 6 hours instead of
2 is still forgiveable if it takes much less time to write and debug and
maintain.

Remember that premature optimisation does nothing but cause headaches
for everyone.


* You don't know <programming language X>, neither do the other
programmers and you don't want to have to teach it to all incomming staff.
What is more, a huge project written by a team of novices in that
programming language is bound to be inelegant and a big ball of mud.
 
In my opinion the best thing about the IT industry is that there is always
more stuff to learn.  I personally believe that the best tool for the job
should be the one that is used, even if I don't know how to use it right
now.  I have enough experience in programming to pick up another
programming language.  I'm sure you do too.  There is often disagreement
as to which programming language is the best for the job (particularly
since some people defend their favourite choice religiously ;) ) but being
open to more than one option is always a good start.


* Will you be able to compile your code?  C executables are inherently
platform specific, as they should be.  Writing your program in C but
keeping the source means that you can compile it on new platforms as they
come up.

C will probably stick around for a very long time.  So will shell, and so
far as I can tell so will most of the other system administration tools
popular at the moment.  I'd be mildly surprised if you didn't have Perl
installed somewhere on every single *NIX box you have.  Probably even in
/usr/bin/  Python's worming it's way in, but it's not there yet.




Now, I come from a Perl shop.  I know C and C++ and Java quite well.  I
personally think they are all too verbose.  I'm not a one liner Perl
hacker, I write long, well documented code, but at least to me, it's still
faster to write than in those other languages.  I believe that Python and
Ruby are also fast to write in.

I do feel however that a pure Perl solution will probably prove to _run_
too slow for you.  Perl is not fast at tonnes of number crunching.  It's
not bad, but because Perl doesn't make distinctions between floats and
ints and longs etc, there's a little additional processing for each and
every number.  The important thing is that I don't think this needs to
matter. 

Perl is very happy for you to embedd your C code within Perl, or link your
shared objects (compiled from C code) into your Perl programs.  Perl is
flexible.  (Python and Ruby might do this too, but I don't know enough to
say).

This means that you could write in both Perl and C and have them work
nicely.  

Why would you want to write in Perl at all?  Because Perl will take care
of all the memory management for you (at least in the Perl bits).  This
means no mallocing structures and being off by one in their size, no
concerns about freeing the memory twice, no need to worry about what the
greatest data size is going to be.  Perl just makes it work.  

Perl is great with databases, and if you later want to change which
database you're using you just need to change your "connect" line.  If you
want to have more than one open "cursor" to the database at a time, do so. 
If you need to clean up your data, Perl's regular expressions make that
trivial.

In a simple version of your code I imagine you'd do something like:

PERL:	connect to database
PERL:	select relevant data
PERL:	pass data to processing function (possibly in a loop over said
data)
C:	process data return results
PERL:	do something with results (possibly munge, write out into Excell
spreadsheet...)

I'm not sure whether Perl is as fast at talking to databases as C is, but
the fact is, when talking to databases the main bottleneck is the database
response.  A few milliseconds more from using Perl really won't be
measurable, although the new ease in talking to databases will certainly
be measureable, as will the ease of changing over to a new db if
necessary.

The biggest issue to cover here is your lack of experience with Perl.  No
matter how good a programmer you are, writing a large, important project
in a brand-new (to you) programing language is not only daunting, it's
fraught with risks.  Some of these can be alleviated by hiring at least
one good Perl programmer possibly on a contractual basis, to help smooth
out the bumps, others can be alleviated by training all the programming
staff in Perl.  The rest will have to be dealt with with confidence. 

There are a number of good Perl trainers out there.  You can probably find
one for your country by heading over to www.perltraining.org.  Your local
Perl Mongers (www.pm.org) group could probably help you out too.

I'm very happy to discuss ideas with you about this on this list too if
you're interested.

	All the best,

		Jacinta Richardson

--
   ("`-''-/").___..--''"`-._          |  Jacinta Richardson         |
    `6_ 6  )   `-.  (     ).`-.__.`)  |  Perl Training Australia    |
    (_Y_.)'  ._   )  `._ `. ``-..-'   |      +613 9354 6001         |  
  _..`--'_..-_/  /--'_.' ,'           | contact at perltraining.com.au |
(il),-''  (li),'  ((!.-'              |   www.perltraining.com.au   |




More information about the Programming mailing list