[Courses] Try out a few different designs and prototype them; try using multiple structures

[Brian Masinick]

Personally, I would define a structure that represents the actual 
contents of the database, then I would use smaller subsets with 
different characteristics to overlay and/or describe each scenario that 
you intend to address.  Is the data fairly static or is is continually 
changing?  That may affect what you want to do also.  The quantity of 
data and the performance of your hardware certainly affects whether a C 
program, some other program, or a series of SQL statements would be the 
most appropriate solution.  Given that there are so many questions to 
answer, I'd consider quickly prototyping two or three different ideas to 
see which one best approximates the kind of eventual solution you'll 
want to take.

Better to prototype now than screw up your design and forever end up 
patching it, or having to redesign it.

-- 
Brian Masinick
mailto:masinick at yahoo.com


>Message: 1
>Date: Thu, 18 Jul 2002 16:11:24 +0000
>From: Conor Daly <conor.daly at met.ie>
>To: Linuxchix Courses <courses at linuxchix.org>
>Subject: [Courses] [C] Overreaching myself with arrays...
>
>Hi all,
>
>More in the way of a programming question than a course topic (though arrays
>in C could be a nice one to address at some point).  I'm writing something 
>that looks like getting horribly complex based on juggling arrays and I'm 
>wondering am I going the wrong way about it.  Situation is as follows:
>
>I have a database with a number of tables containing data located in
>space-time (ie. year, month, day, hour, location).  The tables have
>different temporal resolutions (eg. an hourly table, a daily table and a
>monthly table) and contain different data elements (eg. rainfall, temperature
>(max / min) and so on).
>
>I'm trying to assemble data from a number of these tables to generate a set
>of monthly figures (eg. mean, max, min, counts) and I have a number of
>options:
>
>1. Use the database's internal functions to generate the figures.  This is
>   the easy option but it's slow and may give trouble if this is to be used
>   in real time.  It also won't be able to generate all of the required
>   figures.
>
>2. Read the data from the database into a set of arrays / structs tailored to 
>   the layout of the tables (The database utilities include one to generate
>   a struct definition based on the table definition.  This is fairly easy
>   as far as generating the sql is concerned and the processing of the data
>   is considerably quicker in C.  It does have the drawback though that the
>   data will be in a set of disparate arrays in the program and will need to
>   be bashed about to turn it into something coherent.
>
>3. Read the data from the database into a single (enormous) array reading
>   only those parameters that are of interest rather than the full dataset
>   as specified in (2.).  This means some greater complexity in the sql but
>   less handling of arrays involved.  This array then needs to be bashed
>   around to generate the desired figures but it's only one array
>   (admittedly in about 6 dimensions) and therefore the indexing should be
>   that bit easier.
>
>I have some ideas along the lines of using an enumeration to hold the names
>of arrays, using arrays of structs to hold the actual data elements and
>using an array of data element vs table name as the initial reference point.
>However, this looks like getting needlessly complex and likely to jump up
>and bite me at a later stage.  Oh, did I mention that this is not simply a
>matter of juggling numbers, I also need to do some quality control to accept
>or reject data elements based on what comes back from the database!
>
>I'm trying to put together a coherent question based on all that now...
>
>Anyone care to comment at this point?  I'll probably add more tomorrow...
>
>Conor
>  
>