Data split according to header's and footer's details



IBM's flagship sort product DFSORT for sorting, merging, copying, data manipulation and reporting. Includes ICETOOL and ICEGENER

Data split according to header's and footer's details

Postby Swapnilkumar » Mon Aug 27, 2012 11:19 am

There were 42 different headers having a common footer text "COND CODE" and LRECL=110. We had to split this concatenated data into different files as per report titles(Headers), so that we can process it. say


0HEADER CONFIRMATION                                                 
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE1
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE2
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE3
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE4
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE5
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE6
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE7
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE8
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE9
0-------------------------------------------------------------------------------
1                        REPORT: DAILY REPORTS                    PAGE 00001
                   WORK OF DATE: 10 30 2008                                     
                      COND CODE: ABCD                                           
0HEDER LETTERS                                           
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE1
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE2
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE3
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE4
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE5
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE6         
0-------------------------------------------------------------------------------
1                        REPORT: DAILY REPORTS                    PAGE 00002   
                   WORK OF DATE: 10 30 2008     
                      COND CODE: EFGH     
0HEADR NEXT LEVEL                                           
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE1
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE2
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE3
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE4
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE5
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE6         
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE7
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE8         
0-------------------------------------------------------------------------------
1                        REPORT: DAILY REPORTS                    PAGE 00003   
                   WORK OF DATE: 10 30 2008     
                      COND CODE: IKJK     
0HEADER CONFIRMATION                                                 
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE1
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE2
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE3
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE4
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE5
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE6
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE7
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE8
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE9
0-------------------------------------------------------------------------------
1                        REPORT: DAILY REPORTS                    PAGE 00004
                   WORK OF DATE: 10 30 2008                                     
                      COND CODE: LMN     
0HEADR NEXT LEVEL                                           
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE1
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE2
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE3
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE4
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE5
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE6         
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE7
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE8         
0-------------------------------------------------------------------------------
1                        REPORT: DAILY REPORTS                    PAGE 00005   
                   WORK OF DATE: 10 30 2008     
                      COND CODE: OPQR     


In this data we needed to split the data as per the header along with footer line and data lines into the respective header file details as

File 1 should contain for header "HEADER CONFIRMATION"



0HEADER CONFIRMATION                                                 
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE1
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE2
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE3
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE4
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE5
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE6
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE7
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE8
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE9
0-------------------------------------------------------------------------------
1                        REPORT: DAILY REPORTS                    PAGE 00001
                   WORK OF DATE: 10 30 2008                                     
                      COND CODE: ABCD       

0HEADER CONFIRMATION                                                 
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE1
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE2
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE3
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE4
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE5
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE6
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE7
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE8
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE9
0-------------------------------------------------------------------------------
1                        REPORT: DAILY REPORTS                    PAGE 00004
                   WORK OF DATE: 10 30 2008                                     
                      COND CODE: LMN 


File 2 should contain for header "HEDER LETTERS"


0HEDER LETTERS                                           
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE1
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE2
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE3
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE4
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE5
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE6         
0-------------------------------------------------------------------------------
1                        REPORT: DAILY REPORTS                    PAGE 00002   
                   WORK OF DATE: 10 30 2008     
                      COND CODE: EFGH   


File 3 should contain for header "HEADR NEXT LEVEL"


0HEADR NEXT LEVEL                                           
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE1
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE2
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE3
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE4
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE5
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE6         
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE7
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE8         
0-------------------------------------------------------------------------------
1                        REPORT: DAILY REPORTS                    PAGE 00003   
                   WORK OF DATE: 10 30 2008     
                      COND CODE: IKJK     
0HEADR NEXT LEVEL                                           
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE1
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE2
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE3
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE4
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE5
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE6         
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE7
DATA LINE FOR THE DIFFERNT TYPS OF HEADER LINE8         
0-------------------------------------------------------------------------------
1                        REPORT: DAILY REPORTS                    PAGE 00005   
                   WORK OF DATE: 10 30 2008     
                      COND CODE: OPQR 


How we approached for this in single step as mentioend below.

//STEP01 EXEC PGM=ICETOOL
//SYSOUT    DD SYSOUT=*                           
//SYSPRINT  DD SYSOUT=*                           
//DFSMSG    DD SYSOUT=*                           
//TOOLMSG   DD SYSOUT=*                           
//SYSUDUMP  DD SYSOUT=*                           
//SORTIN    DD DSN=Input data,DISP=SHR   
//SORTO01   DD DSN=output file for first header,     <<HEADER CONFIRMATION
//            DISP=(,CATLG,DELETE),         
//            UNIT=SYSDA,                   
//            DCB=*.SORTIN,                 
//            SPACE=(CYL,(50,20),RLSE)       
//*                                         
//SORTO02   DD DSN=output file for second header,     <<HEDER LETTERS 
//            DISP=(,CATLG,DELETE),         
//            UNIT=SYSDA,                   
//            DCB=*.SORTIN,                 
//            SPACE=(CYL,(50,20),RLSE)
//*   
//SORTO03   DD DSN=output file for third header,     <<HEADR NEXT LEVEL 
//            DISP=(,CATLG,DELETE),         
//            UNIT=SYSDA,                   
//            DCB=*.SORTIN,                 
//            SPACE=(CYL,(50,20),RLSE)
.
.
.
//*   
//TOOLIN DD *                               
 COPY FROM(SORTIN) TO(SORTO01) USING(CP01)   
 COPY FROM(SORTIN) TO(SORTO02) USING(CP02)   
 COPY FROM(SORTIN) TO(SORTO03) USING(CP03)
.
.
.
 COPY FROM(SORTIN) TO(SORTO03) USING(CP42)
/*     
//*   
//CP01CNTL DD *                                                         
 OPTION COPY                                                             
 INREC IFTHEN=(WHEN=GROUP,BEGIN=(2,19,CH,EQ,C'HEADER CONFIRMATION'),             
    END=(23,10,CH,EQ,C'COND CODE:'),PUSH=(111:ID=1))                     
 OUTFIL INCLUDE=(111,1,CH,NE,C' '),BUILD=(1,110)                         
/*                                                                       
//CP02CNTL DD *                                                         
 OPTION COPY                                                             
 INREC IFTHEN=(WHEN=GROUP,BEGIN=(2,13,CH,EQ,C'HEDER LETTERS'),             
    END=(23,10,CH,EQ,C'COND CODE:'),PUSH=(111:ID=1))                     
 OUTFIL INCLUDE=(111,1,CH,NE,C' '),BUILD=(1,110)                         
/*                                                                       
//CP03CNTL DD *                                                         
 OPTION COPY                                                             
 INREC IFTHEN=(WHEN=GROUP,BEGIN=(2,16,CH,EQ,C'HEADR NEXT LEVEL'),             
    END=(23,10,CH,EQ,C'COND CODE:'),PUSH=(111:ID=1))                     
 OUTFIL INCLUDE=(111,1,CH,NE,C' '),BUILD=(1,110)                         
/*                 
.
.
.



This information is for sharing, please let us know if you have any different method to carry this out ;)
____________________________________________
Regards,
- Swapnilkumar.
Swapnilkumar
 
Posts: 12
Joined: Tue Aug 07, 2012 10:05 pm
Location: Pune, India
Has thanked: 0 time
Been thanked: 0 time

Re: Data split according to header's and footer's details

Postby enrico-sorichetti » Mon Aug 27, 2012 11:44 am

Kolusu will certainly give better explanation,
I might be wrong , but why 42 passes ( reads ) ove the input data.

try something along the lines of
 OPTION COPY                                                             
 INREC IFTHEN=(WHEN=GROUP,BEGIN=(2,6,CH,EQ,C'HEADER'),             
    END=(23,10,CH,EQ,C'COND CODE:'),PUSH=(111:ID=2))  <== works for 99 groups
 OUTFIL FNAMES=OUT01,INCLUDE=(111,2,CH,EQ,C'01') BUILD=(1,110)   
 OUTFIL FNAMES=OUT02,INCLUDE=(111,2,CH,EQ,C'02') BUILD=(1,110)   
 ...
 OUTFIL FNAMES=OUT42,INCLUDE=(111,2,CH,EQ,C'42') BUILD=(1,110)   
cheers
enrico
When I tell somebody to RTFM or STFW I usually have the page open in another tab/window of my browser,
so that I am sure that the information requested can be reached with a very small effort
enrico-sorichetti
Global moderator
 
Posts: 3003
Joined: Fri Apr 18, 2008 11:25 pm
Has thanked: 0 time
Been thanked: 164 times

Re: Data split according to header's and footer's details

Postby Swapnilkumar » Mon Aug 27, 2012 11:52 am

Enrico,
its a good suggestion from your end; thank you for the reply. What you suggested certainly work for the first 6 characters matching with "HEADER" string :)

Please have a look at the headers again, those are unique and hence the line
WHEN=GROUP,BEGIN=(2,6,CH,EQ,C'HEADER'),             
can not be a generic one for all the 42 different headers :)
____________________________________________
Regards,
- Swapnilkumar.
Swapnilkumar
 
Posts: 12
Joined: Tue Aug 07, 2012 10:05 pm
Location: Pune, India
Has thanked: 0 time
Been thanked: 0 time

Re: Data split according to header's and footer's details

Postby enrico-sorichetti » Mon Aug 27, 2012 12:05 pm

did You try it ?
from Your description nothing prevent using just HEADER as group separator

tested and working
cheers
enrico
When I tell somebody to RTFM or STFW I usually have the page open in another tab/window of my browser,
so that I am sure that the information requested can be reached with a very small effort
enrico-sorichetti
Global moderator
 
Posts: 3003
Joined: Fri Apr 18, 2008 11:25 pm
Has thanked: 0 time
Been thanked: 164 times

Re: Data split according to header's and footer's details

Postby enrico-sorichetti » Mon Aug 27, 2012 2:20 pm

:oops: Just curious... for what reason the different spelling for HEADER, .... .....
anyway You can achieve the same result just with the END clause

 INREC IFTHEN=(WHEN=GROUP,
    END=(23,10,CH,EQ,C'COND CODE:'),PUSH=(111:ID=2))  <== works for 99 groups


the rest is the same as my previous suggestion
cheers
enrico
When I tell somebody to RTFM or STFW I usually have the page open in another tab/window of my browser,
so that I am sure that the information requested can be reached with a very small effort
enrico-sorichetti
Global moderator
 
Posts: 3003
Joined: Fri Apr 18, 2008 11:25 pm
Has thanked: 0 time
Been thanked: 164 times

Re: Data split according to header's and footer's details

Postby Swapnilkumar » Mon Aug 27, 2012 2:37 pm

Enrico,

Actually headers are unique, the mentioned headers viz,
1) HEADER CONFIRMATION
2) HEDER LETTERS
3) HEADR NEXT LEVEL
had actually different values :!:
The above mentioend solution is executed well and then posted on to publish ;)
____________________________________________
Regards,
- Swapnilkumar.
Swapnilkumar
 
Posts: 12
Joined: Tue Aug 07, 2012 10:05 pm
Location: Pune, India
Has thanked: 0 time
Been thanked: 0 time

Re: Data split according to header's and footer's details

Postby enrico-sorichetti » Mon Aug 27, 2012 3:08 pm

The above mentioend solution is executed well and then posted on to publish


looks like there is a language barrier here ...
You posted a slotion for some problem... it worked but
with 42 passes over input data

did You try my last suggestion with the GROUP on the ending <separator>
ONE PASS OF THE DATA
cheers
enrico
When I tell somebody to RTFM or STFW I usually have the page open in another tab/window of my browser,
so that I am sure that the information requested can be reached with a very small effort
enrico-sorichetti
Global moderator
 
Posts: 3003
Joined: Fri Apr 18, 2008 11:25 pm
Has thanked: 0 time
Been thanked: 164 times

Re: Data split according to header's and footer's details

Postby skolusu » Mon Aug 27, 2012 10:01 pm

Swapnilkumar wrote:The above mentioend solution is executed well and then posted on to publish


If you are presenting DFSORT Solutions , then you need to present an optimized solution and not one that does a 42 passes of data. As enrico pointed it can be DONE in a single pass of data, even when you have varying headers. Please try the solution enrico posted. Alternatively you can actually push the header(longest header is 20 bytes) on to every record and then validate that string to your different header value to split on OUTFIL like this
INREC IFTHEN=(WHEN=GROUP,BEGIN=(2,6,CH,EQ,C'HEADER'),PUSH=(111:2,20))
OUTFIL BUILD=(1,110),INCLUDE=(111,20,CH,EQ,C'HEADER CONFIRMATION')     
OUTFIL BUILD=(1,110),INCLUDE=(111,20,CH,EQ,C'HEDER LETTERS')
OUTFIL BUILD=(1,110),INCLUDE=(111,20,CH,EQ,C'HEADER CONFIRMATION')     
Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
skolusu
 
Posts: 586
Joined: Wed Apr 02, 2008 10:38 pm
Has thanked: 0 time
Been thanked: 39 times


Return to DFSORT/ICETOOL/ICEGENER

 


  • Related topics
    Replies
    Views
    Last post