[Courses] [C] Overreaching myself with arrays...

[Conor Daly]

Hi all,

More in the way of a programming question than a course topic (though arrays
in C could be a nice one to address at some point).  I'm writing something 
that looks like getting horribly complex based on juggling arrays and I'm 
wondering am I going the wrong way about it.  Situation is as follows:

I have a database with a number of tables containing data located in
space-time (ie. year, month, day, hour, location).  The tables have
different temporal resolutions (eg. an hourly table, a daily table and a
monthly table) and contain different data elements (eg. rainfall, temperature
(max / min) and so on).

I'm trying to assemble data from a number of these tables to generate a set
of monthly figures (eg. mean, max, min, counts) and I have a number of
options:

1. Use the database's internal functions to generate the figures.  This is
   the easy option but it's slow and may give trouble if this is to be used
   in real time.  It also won't be able to generate all of the required
   figures.

2. Read the data from the database into a set of arrays / structs tailored to 
   the layout of the tables (The database utilities include one to generate
   a struct definition based on the table definition.  This is fairly easy
   as far as generating the sql is concerned and the processing of the data
   is considerably quicker in C.  It does have the drawback though that the
   data will be in a set of disparate arrays in the program and will need to
   be bashed about to turn it into something coherent.

3. Read the data from the database into a single (enormous) array reading
   only those parameters that are of interest rather than the full dataset
   as specified in (2.).  This means some greater complexity in the sql but
   less handling of arrays involved.  This array then needs to be bashed
   around to generate the desired figures but it's only one array
   (admittedly in about 6 dimensions) and therefore the indexing should be
   that bit easier.

I have some ideas along the lines of using an enumeration to hold the names
of arrays, using arrays of structs to hold the actual data elements and
using an array of data element vs table name as the initial reference point.
However, this looks like getting needlessly complex and likely to jump up
and bite me at a later stage.  Oh, did I mention that this is not simply a
matter of juggling numbers, I also need to do some quality control to accept
or reject data elements based on what comes back from the database!

I'm trying to put together a coherent question based on all that now...

Anyone care to comment at this point?  I'll probably add more tomorrow...

Conor
-- 
Conor Daly 
Met Eireann, Glasnevin Hill, Dublin 9, Ireland
Ph +353 1 8064276 Fax +353 1 8064247
------------------------------------
bofh.irmet.ie running RedHat Linux  3:15pm  up 1 day,  4:45,  9 users,  load average: 0.14, 0.71, 1.05


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept 
for the presence of computer viruses.


**********************************************************************