[prog] fopen and large files (RH9)

Conor Daly conor.daly at cod.utvinternet.com
Mon Jan 17 11:18:47 EST 2005


On Sun, Jan 16, 2005 at 09:55:27PM +0100 or so it is rumoured hereabouts, 
Almut Behrens thought:
> On Fri, Jan 14, 2005 at 06:49:16PM +0000, Conor Daly wrote:
> >        O_LARGEFILE
> >               On  32-bit  systems  that  support the Large Files System, allow
> >               files whose sizes cannot be represented in 31 bits to be opened.
> >  
> > large filesystems?  Is there a call to make I can use to get open(2) to use
> > the O_LARGEFILE flag?
> 
> normally, you compile a program with LFS (large file support) by adding
> two macros at the top of the source, _before_ including header files.
> You can either write:
> 
> #define _LARGEFILE_SOURCE
> #define _FILE_OFFSET_BITS 64
> 
> #include <stdio.h>
> 
> int main() {
>     FILE *f;
>     f = fopen("./file", "r");
> }
> 
> This transparently replaces all relevant functions and data structures
> under the hood with the appropriate 64-bit versions (fopen64(), etc.).

Of course, _finding_ the top of the sources is a bit of a problem here.
It's a very big database system which has gone opensource only a few
months ago {Computer Associates' Ingres 3 DBMS} and the sources consist of
a few hundred interdependent files.  It might suffice to do this for
testing purposes but it's too big for me to test thoroughly.
 
> Or, if you think that's too much magic, you can use the following pair
> of macros, and then explicitly call fopen64(), and whatever other
> file IO routines you need:
> 
> #define _LARGEFILE_SOURCE
> #define _LARGEFILE64_SOURCE
> 
> #include <stdio.h>
> 
> int main() {
>     FILE *f;
>     f = fopen64("./file", "r");
> }
> 
> 
> IOW, you could simply try adding this, and recompile the sources -- and
> hope it doesn't have any side effects... ;-)

That might be worth trying in the first instance.  Even for a test
program.
 
> Unfortunately, there are a number of pitfalls, if the original source
> had not been written with LFS in mind.  Feeling a bit lazy at the
> moment to write it all down myself, I quickly googled and found

Given that this system runs on all major platforms, *NIX, VMS and MS and
happily handles large files on Solaris {where we run the production DBMS},
I'm reasonably confident that it is written with proper 64bit support
built in.  Since we have a support contract with them for Ingres 2.6, I'll
try to raise a ticket on the subject and see if they'll answer Ingres 3
questions on our 2.6 contract...
 
> http://www.unet.univie.ac.at/aix/aixprggd/genprogc/prg_lrg_files.htm#AppAn169stcl
> 
> This is AIX specific, but some of the warnings hold for Linux, too, at
> least the general idea behind them.  Maybe you can google up a better
> document yourself...

I'll certainly poke at it...
 
> Also see "info libc 'Feature Test Macros'" for the meaning of those
> (and similar) macros.

Going to do that now...

Thanks,

Conor
-- 
Conor Daly <conor.daly at oceanfree.net>

Domestic Sysadmin :-)
---------------------
Faenor.cod.ie
 12:29am  up 8 days, 19:38,  0 users,  load average: 0.87, 0.64, 0.32
Hobbiton.cod.ie
 00:06:22  up 8 days, 19:37,  1 user,  load average: 0.00, 0.02, 0.00


More information about the Programming mailing list