fwrite fread for large files

Hi,

I am trying to write a 1 GB file in blocks of 100KB and read them back in 100KB blocks.
First, I would like to write the whole 1 GB in consecutive blocks of 100kb and then read them back in 100kb consecutive blocks.
How do I get the pointer set to the correct value to read the next block? Could anyone please help?

Thank you!

 

Assuming your file pointer is pFile; you can use :

Code:

for(int block = 0; block < MAX_BLOCKS; ++block)
{
      // We have to move cursor by block*100*1024 bytes.
      fseek(pFile, block*102400, SEEK_SET);
      .
      .
}

 

But, does SEEK_SET set the pointer to the beginning of the file. Will I get a 1GB in such a case.

I was thinking of something like this:
long int size;
file *f;
int *buff[102400];
if(size<1024000000)
{
fwrite(buff, 1, 102400, file);
size+=102400;
}
Please suggest. Thanks!

 

Yeah you are right. The value of block * 102400 will be huge.
[ Please ALWAYS post code inside code-blocks]

So, you can use this :

Code:

fseek(pFile, 0, SEEK_SET);
for(int block = 0; block < MAX_BLOCKS; ++block)
      fwrite(buff, 1, 102400, file);

I think the write cursor will automatically move till the end of the written block.
So your pointer is auto-adjusted.

 

so, what value is max_blocks set to? Thanks a ton!

 

also, is fseek really necessary..since we already have FILE *fp=NULL

 

MAX_BLOCKS = Number of 100 kB blocks inside 1 GB = 1 GB / 100 kB = ~10485 :smile:

Just for making sure. Not really necessary !

 

Awesome!

So, do you know if there is anyway for me to write these 10485 blocks not consecutively....i mean not one by one..just randomly..but need to make sure that i write all the 10485 blocks

I think may be for loop doesn't work because"for" loop does each block one by one. What do u suggest?

 

Sorry, I was busy creating another thread. So, I didn't see your post.
Hmmm.. I'll think for a while and reply.

OK. Randomly in the sense ?!?
After you write a block, your pointer stays there till you manually move it by reading writing or seeking.
So, you can write a block. Take a break . Write 100 more. Take another break. Then write 10000 more. And then after final break you may write the rest 384 blocks !

I don't exactly understand what do you mean by RANDOMLY ?!

 

If you mean, you want to write to random positions of the file, then I think you can make a func like this :

Code:

void SeekTo_NthBlock(FILE *pFile, int n, int blocksize)
{
      // Note that this func treats the first block as Block 0, second as Block 1 and so on ..

      fseek(pFile, 0, SEEK_SET);
      for(int i = 0; i < n; ++i)
            fseek(pFile, blocksize, SEEK_CUR);
}

After defining the above func, you can use it like this :

Code:

// Write to 100th block
SeekTo_NthBlock(fp, 99, 102400);
fwrite(buf, 1, 102400, fp);

// Write to 10001st block
SeekTo_NthBlock(fp, 10000, 102400);
fwrite(buf, 1, 102400, fp);

Is that what you wanted ?

 

Not a problem..I mean...not consecutive blocks...but want to write/read the whole file. not 1,2,3...something like...1,50,2000, etc....till 1 GB

 

Did you see my last post ?
You can use the func SeekTo_NthBlock to write to any block at any time.

Code:

// Writing randomly
short IsWritten[10485];
memset(IsWritten, 0, 10485*sizeof(short));
int TotalWritten = 0;
int WriteToBlock;

while (TotalWritten < 10485 )
{
      WriteToBlock = rand() % 10485;
      if ( IsWritten[WriteToBlock] )      continue;
      SeekTo_NthBlock(fp, WriteToBlock, 102400);
      fwrite(buff, 1, 102400, fp);
      IsWritten[WriteToBlock] = 1;
      ++TotalWritten;
}

Is it okay now ?

 

Random ...i meant not sequential...but i wanted to write all 10485 blocks. how do i make sure i have written all the blocks

 

I guess you are able to read the post just above yours.
What does while(TotalWritten < 10485) mean ??

 

I was trying to post...but I don't know it doesn't allow me

 

Sorry...somehow I couldn't see your posts. Thank you very much!

 

You are welcome ! :biggrin: