Go4Expert - fwrite fread for large files

Page 1 of 2

Show 40 post(s) from this thread on one page

Go4Expert (http://www.go4expert.com/)

- C (http://www.go4expert.com/forumdisplay.php?f=20)

- - fwrite fread for large files (http://www.go4expert.com/showthread.php?t=17822)

ccoolgoose

2Jun2009 07:04

fwrite fread for large files

Hi,

I am trying to write a 1 GB file in blocks of 100KB and read them back in 100KB blocks.
First, I would like to write the whole 1 GB in consecutive blocks of 100kb and then read them back in 100kb consecutive blocks.
How do I get the pointer set to the correct value to read the next block? Could anyone please help?

Thank you!

SaswatPadhi

2Jun2009 07:27

Re: fwrite fread for large files

Assuming your file pointer is pFile; you can use :

Code: C

for(int block = 0; block < MAX_BLOCKS; ++block)
{
      // We have to move cursor by block*100*1024 bytes.
      fseek(pFile, block*102400, SEEK_SET);
      .
      .
}

ccoolgoose

2Jun2009 07:47

Re: fwrite fread for large files

But, does SEEK_SET set the pointer to the beginning of the file. Will I get a 1GB in such a case.

I was thinking of something like this:
long int size;
file *f;
int *buff[102400];
if(size<1024000000)
{
fwrite(buff, 1, 102400, file);
size+=102400;
}
Please suggest. Thanks!

SaswatPadhi

2Jun2009 07:59

Re: fwrite fread for large files

Yeah you are right. The value of block * 102400 will be huge.
[ Please ALWAYS post code inside code-blocks]

So, you can use this :

Code: C

fseek(pFile, 0, SEEK_SET);
for(int block = 0; block < MAX_BLOCKS; ++block)
      fwrite(buff, 1, 102400, file);

I think the write cursor will automatically move till the end of the written block.
So your pointer is auto-adjusted.

ccoolgoose

2Jun2009 08:08

Re: fwrite fread for large files

so, what value is max_blocks set to? Thanks a ton!

ccoolgoose

2Jun2009 08:14

Re: fwrite fread for large files

also, is fseek really necessary..since we already have FILE *fp=NULL

SaswatPadhi

2Jun2009 08:16

Re: fwrite fread for large files

MAX_BLOCKS = Number of 100 kB blocks inside 1 GB = 1 GB / 100 kB = ~10485 :smile:

Quote:

Originally Posted by ccoolgoose (Post 48808)

also, is fseek really necessary..since we already have FILE *fp=NULL

Just for making sure. Not really necessary !

ccoolgoose

2Jun2009 08:33

Re: fwrite fread for large files

Awesome!

So, do you know if there is anyway for me to write these 10485 blocks not consecutively....i mean not one by one..just randomly..but need to make sure that i write all the 10485 blocks

I think may be for loop doesn't work because"for" loop does each block one by one. What do u suggest?

SaswatPadhi

2Jun2009 08:47

Re: fwrite fread for large files

Sorry, I was busy creating another thread. So, I didn't see your post.
Hmmm.. I'll think for a while and reply.

OK. Randomly in the sense ?!?
After you write a block, your pointer stays there till you manually move it by reading writing or seeking.
So, you can write a block. Take a break :). Write 100 more. Take another break. Then write 10000 more. And then after final break you may write the rest 384 blocks !

I don't exactly understand what do you mean by RANDOMLY ?!

SaswatPadhi

2Jun2009 09:00

Re: fwrite fread for large files

If you mean, you want to write to random positions of the file, then I think you can make a func like this :

Code: c

void SeekTo_NthBlock(FILE *pFile, int n, int blocksize)
{
      // Note that this func treats the first block as Block 0, second as Block 1 and so on ..

      fseek(pFile, 0, SEEK_SET);
      for(int i = 0; i < n; ++i)
            fseek(pFile, blocksize, SEEK_CUR);
}

After defining the above func, you can use it like this :

Code: C

// Write to 100th block
SeekTo_NthBlock(fp, 99, 102400);
fwrite(buf, 1, 102400, fp);

// Write to 10001st block
SeekTo_NthBlock(fp, 10000, 102400);
fwrite(buf, 1, 102400, fp);

Is that what you wanted ?

All times are GMT +5.5. The time now is 05:44.

Page 1 of 2

Show 40 post(s) from this thread on one page