IBM Mainframe Forum

Hi,

we are trying to implement a functionality using JCL instead of a cobol program.
functionality is:
need to remove all the occurences of duplicate records and output file should have only the records which are unique.

i know, it is possible to implement using ICETOOL but as per the project rules, we shouldnt use ICETOOL.

I tried implementing using SORT but using SORT one occurence of the duplicate record is appearing in the file. we dont that one occurence also.

Could you please suggest me if it is possible to implement using SORT?

Thanks

Is your data already sorted?

What level of DFSORT do you have (post the ICE201I message, please)?

Hi BillyBoyo,

yeah. file is sorted. here the interface is SYNCSORT itseems. so, by default SYNCSORT will be invoked.
please find the below screen shot.

OK. Not the best of news. If you have it working with the "Tool", can you post the JCL and control cards, in the Code tags, not as a screen print.

Hi,

Below is the JCL using ICETOOL which is working fine. but as per project standards, usage ICETOOL is not allowed.
//STEP1 EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=AUTSO.INPUT.FILE1,DISP=SHR
// DD DSN=AUTSO.INPUT.FILE2,DISP=SHR
//NODUPS DD DSN=AUTSO.OUTPUT.FILE,DISP=SHR
//TOOLIN DD *
SELECT FROM(SORTIN) TO(NODUPS) ON(1,10,CH) NODUPS
/*

example:
input1:
0912345678900000100
1012345678900000200
0912123278900000300

input2:
0912345678900000100
0212125678900000200

output:(all occurences of duplicate records are removed)
0212125678900000200
0912123278900000300
1012345678900000200

below is the JCL using SORT.
//STEP1 EXEC PGM=SORT
//SORTIN DD DSN=AUTSO.INPUT.FILE1,DISP=SHR
// DD DSN=AUTSO.INPUT.FILE2,DISP=SHR
//SORTOUT DD DSN=AUTSO.OUTPUT.FILE,DISP=SHR
//SYSOUT DD SYSOUT=*
//SYSIN DD *
SORT FIELDS=(1,10,CH,A)
SUM FIELDS=NONE
/*

Example:
input1:
0912345678900000100
1012345678900000200
0912123278900000300

input2:
0912345678900000100
0212125678900000200

output:(one occurence of duplicate record appears in output file but as per requirement, all occurences of duplicate records should be removed )
0212125678900000200
0912123278900000300
0912345678900000100
1012345678900000200

This is one of those "funny" situations. The Tool solution provides a "natural" answer, in that the Tool has functions to provide it with no complexity. But, due to site standards, you can't use the Tool code for production.

It is not a "natural" solution in Sort.

I don't have Syncsort or Syncsort documentation, so I can't code this out or test it.

I'd suggest you look at a JOINKEYS application, if your level of sort has that function.

Specify the same DSN for both of the joinkeys files.

Specify a CNTL file for one of the JOINKEYS files.

In the CNTL file append a sequence number, with a RESTART on the key.
With OUTREC INCLUDE only include those records with a sequence number of 2 (which gives you a key containing duplicates) and BUILD the original record.

In the JOIN you want records which are unmatched against what is now a key file of duplicate-only.

This is a more complex solution than the Tool code you have,

If you want a less complex (in terms of code) solution, you probably need multiple steps. Let us know.

If you go with the joinkeys, and get stuck, let us know.

If you get a solution, please post it to aid others with a similar problem in the future.

Hi BillyBoyo,

Thanks for your reply!
i am sorry. i dont have much knowledge on JOINKEYS in JCL. I tried to write a JCL like below. but is ABENDing with code S000 U0016.

//STEP1 EXEC PGM=SORT
//SORTJNF1 DD DSN=AUTSO.INPUT.FILE1,DISP=SHR
//SORTJNF2 DD DSN=AUTSO.INPUT.FILE2,DISP=SHR
//SORTOUT DD DSN=AUTSO.OUTPUT.FILE,DISP=SHR
//SYSOUT DD SYSOUT=*
//SYSIN DD *
JOINKEYS FILE=F1,FIELDS=(1,10,A)
JOINKEYS FILE=F2,FIELDS=(1,10,A)
REFORMAT FIELDS=(F1:1,19,F2:1,19)
SORT FIELDS=COPY
/*

Could you please let me know if the syntax is wrong anywhere?
the key in both the files is from 1,10

Thanks

I think it llooks like a basic Join. Syncsort will be much better at telling you, in the messages produced from your run.

From what I was saying, your two input filenames can be the exact same dataname, you don't have to copy one to another dataset.

You need a bit of this:



Which is a DD where you can put control cards to process the second file "before" the Join. Each output record from here is processed by the Join as though it had just come from the input fle.



On each of your Joinkeys you should add SORTED,NOSEQCHK, because the files are sorted and you don't need sequence to check that the keys are in sequence.

Then your output wants to be only those from the first file which do not match the second file. The second file by this time is a list of keys (and data, you can chop it shorter if the records are long) which are from two or more input records with the same key.

Hi BillyBoyo,

I am sorry for asking multiple questions.

i have writtent the JCL like below.
//STEP1 EXEC PGM=SORT
//SORTJNF1 DD DSN=AUTSO.INPUT.FILE1,DISP=SHR
//SORTJNF2 DD DSN=AUTSO.INPUT.FILE2,DISP=SHR
//JNF2CNTL DD *
INREC OVERLAY=(81,SEQNUM,7,ZD=7,RESTART=(1,10,CH))
SORT FIELDS=(1,10,CH,A,81,7,CH,D)
OUTFIL INCLUDE=(81,7,ZD,EQ,2)
/*
//SORTOUT DD DSN=AUTSO.OUTPUT.FILE,DISP=SHR
//SYSOUT DD SYSOUT=*
//SYSIN DD *
JOINKEYS F1=SORTJNF1,FIELDS=(1,10,A),SORTED,NOSEQCHK
JOINKEYS F2=SORTJNF2,FIELDS=(1,10,A),SORTED,NOSEQCHK
/*

the error message says,
SYNCSORT LICENSED FOR CPU SERIAL NUMBER 28D36, MODEL 2817 705
SYSIN :
JOINKEYS F1=SORTJNF1,FIELDS=(1,10,A),SORTED,NOSEQCHK
*
JOINKEYS F2=SORTJNF2,FIELDS=(1,10,A),SORTED,NOSEQCHK
*
WER268A JOINKEYS STATEMENT: SYNTAX ERROR
WER268A JOINKEYS STATEMENT: SYNTAX ERROR
WER211B SYNCSMF CALLED BY SYNCSORT; RC=0000
WER449I SYNCSORT GLOBAL DSM SUBSYSTEM ACTIVE

You changed from your previous post. I can't check this syntax, but FILE=F1 or SORTJNF1 or FILES=F1 or SORTJNF1 should be OK.