1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Split a huge file in to equal parts

Discussion in 'Java' started by naimish, Jul 6, 2009.

  1. naimish

    naimish New Member

    Joined:
    Jun 29, 2009
    Messages:
    1,046
    Likes Received:
    18
    Trophy Points:
    0
    Occupation:
    Software Engineer
    Location:
    On Earth

    Introduction



    How to split a file in to equal parts without knowing the number of records in the input file?

    Background



    Suppose you don't know how many records are in a dataset, but you want to divide the records as equal as possible among the output datasets. We can split a file into equal number parts by using ICETOOL with a small keyword SPLIT / SPLITBY. The all output files will have same number of records. The below code split the file into three equal parts.

    The code



    Code:
    //SPLIT JOB TEST,'SPLIT', 
    // MSGCLASS=X,MSGLEVEL=(1,1), 
    // CLASS=B,TIME=1440,NOTIFY=&SYSUID 
    //JOBLIB DD DISP=SHR,DSN=SYS3.MVSV5.RUNTIME 
    //PSTY0010 EXEC PGM=ICETOOL, 
    // PARM='RC16=ABE', 
    // COND=(0,LT) 
    //INFILE DD DSN=< INPUT FILE >,
    // DISP=(SHR,KEEP,KEEP) 
    //OUTFILE1 DD DSN=< OUTPUT FILE -1 >,
    // DISP=(NEW,CATLG,CATLG), 
    // DATACLAS= 
    // DCB= 
    //OUTFILE2 DD DSN=< OUTPUT FILE -2 >,
    // DISP=(NEW,CATLG,CATLG), 
    // DATACLAS=
    // DCB= 
    //OUTFILE3 DD DSN=< OUTPUT FILE -3 >,
    // DISP=(NEW,CATLG,CATLG), 
    // DATACLAS= 
    // DCB= 
    //TOOLIN DD * 
    COPY FROM(INFILE) TO(OUTFILE1,OUTFILE2,OUTFILE3) USING(CTL1) 
    //CTL1CNTL DD * 
    OPTION COPY 
    OUTFIL FNAMES=(OUTFILE1,OUTFILE2,OUTFILE3),SPLIT 
    //DFSMSG DD SYSOUT=* 
    //TOOLMSG DD SYSOUT=* 
     
  2. shabbir

    shabbir Administrator Staff Member

    Joined:
    Jul 12, 2004
    Messages:
    15,283
    Likes Received:
    364
    Trophy Points:
    83

Share This Page