1. We have moved from vBulletin to XenForo and you are viewing the site in the middle of the move. Though the functional aspect of everything is working fine, we are still working on other changes including the new design on Xenforo.
    Dismiss Notice

Split a huge file in to equal parts

Discussion in 'Java' started by naimish, Jul 6, 2009.

  1. naimish

    naimish New Member

    Introduction



    How to split a file in to equal parts without knowing the number of records in the input file?

    Background



    Suppose you don't know how many records are in a dataset, but you want to divide the records as equal as possible among the output datasets. We can split a file into equal number parts by using ICETOOL with a small keyword SPLIT / SPLITBY. The all output files will have same number of records. The below code split the file into three equal parts.

    The code



    Code:
    //SPLIT JOB TEST,'SPLIT', 
    // MSGCLASS=X,MSGLEVEL=(1,1), 
    // CLASS=B,TIME=1440,NOTIFY=&SYSUID 
    //JOBLIB DD DISP=SHR,DSN=SYS3.MVSV5.RUNTIME 
    //PSTY0010 EXEC PGM=ICETOOL, 
    // PARM='RC16=ABE', 
    // COND=(0,LT) 
    //INFILE DD DSN=< INPUT FILE >,
    // DISP=(SHR,KEEP,KEEP) 
    //OUTFILE1 DD DSN=< OUTPUT FILE -1 >,
    // DISP=(NEW,CATLG,CATLG), 
    // DATACLAS= 
    // DCB= 
    //OUTFILE2 DD DSN=< OUTPUT FILE -2 >,
    // DISP=(NEW,CATLG,CATLG), 
    // DATACLAS=
    // DCB= 
    //OUTFILE3 DD DSN=< OUTPUT FILE -3 >,
    // DISP=(NEW,CATLG,CATLG), 
    // DATACLAS= 
    // DCB= 
    //TOOLIN DD * 
    COPY FROM(INFILE) TO(OUTFILE1,OUTFILE2,OUTFILE3) USING(CTL1) 
    //CTL1CNTL DD * 
    OPTION COPY 
    OUTFIL FNAMES=(OUTFILE1,OUTFILE2,OUTFILE3),SPLIT 
    //DFSMSG DD SYSOUT=* 
    //TOOLMSG DD SYSOUT=* 
     
  2. shabbir

    shabbir Administrator Staff Member

Share This Page