fwrite fread for large files

Light Poster
2Jun2009,07:04   #1
ccoolgoose's Avatar
Hi,

I am trying to write a 1 GB file in blocks of 100KB and read them back in 100KB blocks.
First, I would like to write the whole 1 GB in consecutive blocks of 100kb and then read them back in 100kb consecutive blocks.
How do I get the pointer set to the correct value to read the next block? Could anyone please help?

Thank you!
~ Б0ЯИ Τ0 С0δЭ ~
2Jun2009,07:27   #2
SaswatPadhi's Avatar
Assuming your file pointer is pFile; you can use :

Code: C
for(int block = 0; block < MAX_BLOCKS; ++block)
{
      // We have to move cursor by block*100*1024 bytes.
      fseek(pFile, block*102400, SEEK_SET);
      .
      .
}

Last edited by SaswatPadhi; 2Jun2009 at 07:55..
Light Poster
2Jun2009,07:47   #3
ccoolgoose's Avatar
But, does SEEK_SET set the pointer to the beginning of the file. Will I get a 1GB in such a case.

I was thinking of something like this:
long int size;
file *f;
int *buff[102400];
if(size<1024000000)
{
fwrite(buff, 1, 102400, file);
size+=102400;
}
Please suggest. Thanks!
~ Б0ЯИ Τ0 С0δЭ ~
2Jun2009,07:59   #4
SaswatPadhi's Avatar
Yeah you are right. The value of block * 102400 will be huge.
[ Please ALWAYS post code inside code-blocks]

So, you can use this :

Code: C
fseek(pFile, 0, SEEK_SET);
for(int block = 0; block < MAX_BLOCKS; ++block)
      fwrite(buff, 1, 102400, file);
I think the write cursor will automatically move till the end of the written block.
So your pointer is auto-adjusted.

Last edited by SaswatPadhi; 2Jun2009 at 08:02..
Light Poster
2Jun2009,08:08   #5
ccoolgoose's Avatar
so, what value is max_blocks set to? Thanks a ton!
Light Poster
2Jun2009,08:14   #6
ccoolgoose's Avatar
also, is fseek really necessary..since we already have FILE *fp=NULL
~ Б0ЯИ Τ0 С0δЭ ~
2Jun2009,08:16   #7
SaswatPadhi's Avatar
MAX_BLOCKS = Number of 100 kB blocks inside 1 GB = 1 GB / 100 kB = ~10485

Quote:
Originally Posted by ccoolgoose View Post
also, is fseek really necessary..since we already have FILE *fp=NULL
Just for making sure. Not really necessary !
Light Poster
2Jun2009,08:33   #8
ccoolgoose's Avatar
Awesome!

So, do you know if there is anyway for me to write these 10485 blocks not consecutively....i mean not one by one..just randomly..but need to make sure that i write all the 10485 blocks

I think may be for loop doesn't work because"for" loop does each block one by one. What do u suggest?
~ Б0ЯИ Τ0 С0δЭ ~
2Jun2009,08:47   #9
SaswatPadhi's Avatar
Sorry, I was busy creating another thread. So, I didn't see your post.
Hmmm.. I'll think for a while and reply.

OK. Randomly in the sense ?!?
After you write a block, your pointer stays there till you manually move it by reading writing or seeking.
So, you can write a block. Take a break . Write 100 more. Take another break. Then write 10000 more. And then after final break you may write the rest 384 blocks !

I don't exactly understand what do you mean by RANDOMLY ?!

Last edited by SaswatPadhi; 2Jun2009 at 08:50..
~ Б0ЯИ Τ0 С0δЭ ~
2Jun2009,09:00   #10
SaswatPadhi's Avatar
If you mean, you want to write to random positions of the file, then I think you can make a func like this :

Code: c
void SeekTo_NthBlock(FILE *pFile, int n, int blocksize)
{
      // Note that this func treats the first block as Block 0, second as Block 1 and so on ..

      fseek(pFile, 0, SEEK_SET);
      for(int i = 0; i < n; ++i)
            fseek(pFile, blocksize, SEEK_CUR);
}

After defining the above func, you can use it like this :
Code: C
// Write to 100th block
SeekTo_NthBlock(fp, 99, 102400);
fwrite(buf, 1, 102400, fp);

// Write to 10001st block
SeekTo_NthBlock(fp, 10000, 102400);
fwrite(buf, 1, 102400, fp);

Is that what you wanted ?