Removing duplicates from sorted dataset when using JOINKEYS
Posted: Wed May 11, 2011 8:12 pm
I'm using JOINKEYS to keep all records from a dataset (F1) that have a paired record in (F2). I'd like to remove duplicated keys from F2 to avoid the generation of duplicated F1 records in the output file, and since F2 has been previously sorted, I have specified the SORTED option for F2 and I'm using the SUM FIELDS=NONE statement to remove duplicates before the JOIN phase.
MERGE is selected by DFSORT from the F2 subtask, but it seems to ignore the SUM FIELDS=NONE statement. If I don't use the SORTED option it behaves as expected, but I'll like to avoid sorting an already sorted dataset. Is there a way to remove duplicates from F2 without using another job step?
Job step
Actual OUTPUT
Expected OUTPUT
MERGE is selected by DFSORT from the F2 subtask, but it seems to ignore the SUM FIELDS=NONE statement. If I don't use the SORTED option it behaves as expected, but I'll like to avoid sorting an already sorted dataset. Is there a way to remove duplicates from F2 without using another job step?
Job step
//STEP010 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//MASTER DD *
1AAAAAAAAA
5BBBBBBBBB
3CCCCCCCCC
4DDDDDDDDD
//PULL DD *
3
5
5
5
7
//SORTOUT DD SYSOUT=*
//DFSPARM DD *
* CONTROL STATEMENTS FOR JOINKEYS APPLICATION
JOINKEYS F1=MASTER,FIELDS=(1,1,A)
JOINKEYS F2=PULL,FIELDS=(1,1,A),SORTED
REFORMAT FIELDS=(F1:1,10)
* CONTROL STATEMENT FOR MAIN TASK
OPTION COPY
//JNF2CNTL DD *
* CONTROL STATEMENT FOR SUBTASK2 (F2)
SUM FIELDS=NONE
/*
//*
//SYSOUT DD SYSOUT=*
//MASTER DD *
1AAAAAAAAA
5BBBBBBBBB
3CCCCCCCCC
4DDDDDDDDD
//PULL DD *
3
5
5
5
7
//SORTOUT DD SYSOUT=*
//DFSPARM DD *
* CONTROL STATEMENTS FOR JOINKEYS APPLICATION
JOINKEYS F1=MASTER,FIELDS=(1,1,A)
JOINKEYS F2=PULL,FIELDS=(1,1,A),SORTED
REFORMAT FIELDS=(F1:1,10)
* CONTROL STATEMENT FOR MAIN TASK
OPTION COPY
//JNF2CNTL DD *
* CONTROL STATEMENT FOR SUBTASK2 (F2)
SUM FIELDS=NONE
/*
//*
Actual OUTPUT
3CCCCCCCCC
5BBBBBBBBB
5BBBBBBBBB
5BBBBBBBBB
5BBBBBBBBB
5BBBBBBBBB
5BBBBBBBBB
Expected OUTPUT
3CCCCCCCCC
5BBBBBBBBB
5BBBBBBBBB