Merge multiple fixed length files of different record length

[corvette1982]

I would like to merge multiple files of different record length into a variable length file. The input files are either 88, 104, or 163 record (fixed) length. I am trying to merge them into a single variable length file of 1004.

I tried using IEBGENER where I concatenated the files as a part of the SYSUT1 statement. (there are 30+ files to merge, so I was trying to keep this simple with a single JCL step and concatenating the files to merge)..

My SYSUT2 statement defines a VB file of lrecl 1004.

When I check the JCL, I get the following errors:

***ERROR - DSS4880E - LRECL " 104" CANNOT BE CONCAT. WITH LRECL " 88"
***ERROR - DSS4880E - LRECL " 163" CANNOT BE CONCAT. WITH LRECL " 88"

I suppose I could IEBGENER each of the 30+ files into a VB file of length 1004, then merge all those at the end. This would require many more steps though.... Thoughts? Many thanks!

[willy jensen]

Are those datasets (please don't call them files, you will upset some folks in this forum) large?
If not, I would write a small REXX pgm to do it. EXECIO is pretty good at record format conversion, you can read all input datasets to the stack and then write them in one go.
If they are big, I would look at SORT with the FB-to-VB option.
By the way, when you say 'merge' do you really mean that, or will a concatenation be ok?

[corvette1982]

Hi Willy

Thank you for the clarifications. Sorry to all about using the wrong terminology... been 20 years since I have been on the mainframe

I did mean datasets. There are 37 datasets which fall in three different record lengths. I went ahead and used a 4 step JCL process, each step using IEBGENER. The first three concatenated the datasets of similar length in the SYSUT1 into a VB temp dataset for SYSUT2. The last step then did the same for all three temp datasets into the final dataset required. Worked like a charm.

[sergeyken]

Do you really need TO MERGE your datasets (by merge key field), or you only need TO CONCATENATE datasets with different LRECL/RECFM, e.g. place records of the second DSN all after records of the first one?

[corvette1982]

Concatenate. Sorry again for the terminology.

[sergeyken]

In order to do this CONCATENATION(!) in one step, you can use MULTIIN option of SYNCSORT utility (PARM=‘MULTIIN’j:

1) use several //SORTINnn DD concatenations, each of them combining those DSNs with equal LRECL

2) use SORT statement

Code: Select all

INREC IFTHEN=(WHEN=(&MULTIINDD,EQ,C’nn’),  input from lrecl #1?
              BUILD=(1,lrecl1,lllX)),      Extend record to the full desired size
         . . . . . .

3) if needed, convert the result to RECFM=VB:

Code: Select all

OUTFIL FTOV,... - with extra conversion, truncation, whatever...

[willy jensen]

Away for 20 years? Welcome back.
So assuming reasonably small datasets, then consider this REXX (from the top of my head, I haven't tested it).

Code: Select all

/*  rexx */                                                    
 ids.1='input.data.set.name.1'                                  
 ids.2='input.data.set.name.2'                                  
 ids.0=2     /* number of input datasets */                                                  
 ods  ='output.data.set.name'                                  
 dd='A'space(translate(time(),' ',':'),0)                      
 "newstack"                                                    
 do n=1 to ids.0   /* load all */                              
   cc=bpxwdyn('alloc da('ids.n') shr dd('dd') reuse')          
   if cc<>0 then zz=Quit('alloc' ods 'rc' cc)                  
   "execio * diskr" dd "(finis)"                                
   if rc<>0 then zz=Quit('read' ids.n 'rc' rc)                  
 end                                                            
 cc=bpxwdyn('alloc da('ods') shr dd('dd') reuse')   /* write */
 if cc<>0 then zz=Quit('alloc' ods 'rc' cc)                    
 "execio" queued() "diskw" dd "(finis)"                        
 if rc<>0 then zz=Quit('write' ods 'rc' rc)                    
 zz=Quit('All done')                                            
Quit:                                                          
 "delstack"                                                    
 say arg(1)                                                    
 cc=bpxwdyn('free dd('dd')')                                    
 exit 0  

[sergeyken]

Sorry, for MULTIIN option the used DD names must be:
//SORTMI01 DD
//SORTMI02 DD
. . . .
//SORTMInn DD

[willy jensen]

Slight modification to handle bigger datasets.....

Code: Select all

/*  rexx */                                                        
 ids.1='input.data.set.name.1'                                      
 ids.2='input.data.set.name.2'                                      
 ids.0=2                                                            
 ods  ='output.data.set.name'                                      
 ddi  ='I'space(translate(time(),' ',':'),0)                        
 ddo  ='O'space(translate(time(),' ',':'),0)                        
 cc=bpxwdyn('alloc da('ods') shr dd('ddo') reuse')   /* output */  
 if cc<>0 then zz=Quit('alloc' ods 'rc' cc)                        
 "newstack"                                                        
 do n=1 to ids.0   /* load all */                                  
   cc=bpxwdyn('alloc da('ids.n') shr dd('ddi') reuse')              
   if cc<>0 then zz=Quit('alloc' ods 'rc' cc)                      
   "execio * diskr" dd "(finis)"                                    
   if rc<>0 then zz=Quit('read' ids.n 'rc' rc)                      
   "execio" queued() "diskw" ddo                                    
   if rc<>0 then zz=Quit('write' ods 'rc' rc)                      
 end                                                                
 zz=Quit('All done')                                                
XMsg:                                                              
 "execio 0 diskw" ddo "(finis)"       /* close */                  
 "delstack"                                                        
 say arg(1)                                                        
 cc=bpxwdyn('free dd('ddi')')                                      
 cc=bpxwdyn('free dd('ddo')')                                      
 exit 0