Split a huge file in to equal parts

Discussion in 'Java' started by naimish, Jul 6, 2009.

  1. naimish

    naimish New Member

    Joined:
    Jun 29, 2009
    Messages:
    1,043
    Likes Received:
    18
    Trophy Points:
    0
    Occupation:
    Software Engineer
    Location:
    On Earth

    Introduction



    How to split a file in to equal parts without knowing the number of records in the input file?

    Background



    Suppose you don't know how many records are in a dataset, but you want to divide the records as equal as possible among the output datasets. We can split a file into equal number parts by using ICETOOL with a small keyword SPLIT / SPLITBY. The all output files will have same number of records. The below code split the file into three equal parts.

    The code



    Code:
    //SPLIT JOB TEST,'SPLIT', 
    // MSGCLASS=X,MSGLEVEL=(1,1), 
    // CLASS=B,TIME=1440,NOTIFY=&SYSUID 
    //JOBLIB DD DISP=SHR,DSN=SYS3.MVSV5.RUNTIME 
    //PSTY0010 EXEC PGM=ICETOOL, 
    // PARM='RC16=ABE', 
    // COND=(0,LT) 
    //INFILE DD DSN=< INPUT FILE >,
    // DISP=(SHR,KEEP,KEEP) 
    //OUTFILE1 DD DSN=< OUTPUT FILE -1 >,
    // DISP=(NEW,CATLG,CATLG), 
    // DATACLAS= 
    // DCB= 
    //OUTFILE2 DD DSN=< OUTPUT FILE -2 >,
    // DISP=(NEW,CATLG,CATLG), 
    // DATACLAS=
    // DCB= 
    //OUTFILE3 DD DSN=< OUTPUT FILE -3 >,
    // DISP=(NEW,CATLG,CATLG), 
    // DATACLAS= 
    // DCB= 
    //TOOLIN DD * 
    COPY FROM(INFILE) TO(OUTFILE1,OUTFILE2,OUTFILE3) USING(CTL1) 
    //CTL1CNTL DD * 
    OPTION COPY 
    OUTFIL FNAMES=(OUTFILE1,OUTFILE2,OUTFILE3),SPLIT 
    //DFSMSG DD SYSOUT=* 
    //TOOLMSG DD SYSOUT=* 
     
  2. shabbir

    shabbir Administrator Staff Member

    Joined:
    Jul 12, 2004
    Messages:
    15,375
    Likes Received:
    388
    Trophy Points:
    83

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice