IBM Mainframe Forum

by **Nara_513** » Sun Apr 29, 2012 4:33 am

Sorry ...the file shave diff record count:

Input sort file: 26652426
output file after sort: 26651978

by **dick scherrer** » Sun Apr 29, 2012 7:46 am

Hello,

How many duplicates were eliminated? Is this the same number as the difference in the record counts? It should be.

If one file is the original and the other has the duplicates removed, there Should be a difference.

What should happen to the values that are not the key for the duplicate records?

Do you now have the result you want/need or is there still something that needs to be changed?

by **BillyBoyo** » Sun Apr 29, 2012 12:09 pm

OK, so the confusion grows.

If your original source was a VSAM KSDS, which must have unique keys, so therefore unique "records", how come

Select all

SUM FIELDS=NONE

would lead to the deletion of duplicate records?

Can you provided us with a LISTCAT of the original KSDS, and let us know how you copied it to the flat file in the first place, please?

by **NicC** » Sun Apr 29, 2012 2:02 pm

What is your 'normal sort step' to copy a VSAM to PS? What is the RECFM of the PS file? If it is VB then your SORT control cards are wrong - they should be sorting from position 5 as the first 4 bytes are the RDW which is NOT part of your data.

by **BillyBoyo** » Sun Apr 29, 2012 2:49 pm

That's what I was thinking. Either the KSDS must contain records of different lengths, or the copy is a VB through accidental commision, but for whatever reason the PS is VB.

Then, the Sort starts on the record-length of the RDW. Thus the last two bytes of the actual key are ignored, so keys are "duplicate" where the first 13 digits are equal, and the records in question are an equal length.

So, can you provided us with a LISTCAT of the original KSDS, and let us know how you copied it to the flat file in the first place, please? Show the LISTCAT and the JCL in the Code tags for readability.

by **Nara_513** » Mon Apr 30, 2012 6:07 am

Hi All,

Please find the following:

MY Original KSDS file info:

Select all

Data Component Information:
Device type: 3390
Organization: KSDS EXT-ADDR COMP
KSDS key length: 23
KSDS key location: 0
Average record size: 73
Maximum record size: 73
Allocated Space: Unit Primary Secondary
Data: CYLINDERS 796 199
Index: TRACKS 67 17

Sort step used for copying the VSAM to flat file:

Select all

//STEP01 EXEC SORTD5
//SORTIN DD DSN=XXXXX.VSAM.MINDEX,
// DISP=SHR
//SORTOUT DD DSN=XXXX.FLAT.MINDEX.BKP,
// DISP=(NEW,CATLG,DELETE),
// UNIT=SYSDA,SPACE=(CYL,(800,200),RLSE),
// DCB=(*.SORTIN)
//SYSIN DD *
SORT FIELDS=COPY
/*

Flat file properties:

Select all

General Data Current Allocation
Management class . . : MCTSTSTD Allocated cylinders : 2,320
Storage class . . . : SCBASE Allocated extents . : 9
Volume serial . . . : TSA009
Device type . . . . : 3390
Data class . . . . . : DCBASE
Organization . . . : PS Current Utilization
Record format . . . : FB Used cylinders . . : 2,320
Record length . . . : 73 Used extents . . . : 9
Block size . . . . : 27959
1st extent cylinders: 800
Secondary cylinders : 200 Dates
Data set name type : Creation date . . . : 2012/04/27
SMS Compressible. . : NO Referenced date . . : 2012/04/27
Expiration date . . : ***None***

I used the below sort card to get the count of the files:

Select all

//SYSIN DD *
SORT FIELDS=COPY
OUTFIL NODETAIL,REMOVECC,
TRAILER1=(COUNT=(M10,LENGTH=15))
/*

I have the answer for the queries u have posted:

My VSAM file can have duplicates values for the column 1 to 15, but the original VSAM file with key length 23 doesnt have any duplicates cos the next coulmn values from 16 to 23 makes the record unique.

Since i defined the VSAM with 15 as the Key, when i tried to copy it with the flat file(unsorted) it failed.
So after sorting and by removing the duplicates(first column of lenght 15), the REPRO command worked as the duplicates were now removed.

I used the belwo JCL to get the dup count:

Select all

//STEP040 EXEC PGM=ICETOOL,REGION=128M
//INPUT DD DSN=XXXX.FLAT.MINDEX.BKP,DISP=SHR
//UNIQUE DD DSN=XXXXXX.MINDEX.UNIQUE,
// DISP=(NEW,CATLG,DELETE),
// UNIT=SYSDA,
// SPACE=(CYL,(500,200),RLSE),
// DCB=(*.INPUT)
//OUTPUT DD DSN=XXXXX.MINDEX.DUP,
// DISP=(NEW,CATLG,DELETE),
// UNIT=SYSDA,
// SPACE=(CYL,(500,200),RLSE),
// DCB=(*.INPUT)
//TOOLIN DD *
SELECT FROM(INPUT) TO(OUTPUT) -
ON(01,15,CH) -
ALLDUPS DISCARD(UNIQUE)
//SYSOUT DD SYSOUT=*
//SSMSG DD SYSOUT=*
//SYSPRINT DD SYSOUT=*
//TOOLMSG DD SYSOUT=*

I have checked the record count of the dup file and it is 896, on making it half we get 448 which is the difference between the original file count and the sorted file count:

At last i had the result that i need , the duplicate records that are removed ...though are unique , they are not in my processing list of records :-)

by **BillyBoyo** » Mon Apr 30, 2012 11:53 am

You gave us some bad information earlier in relation to Robert's question about whether the second part of the key was to ensure that the first part was unique.

Instead of sorting your data to get rid of the duplicates, why don't you use SELECT, ensuring that it does not "sort"? You should be able to include all the unique records and the first duplicate at one go, without sorting your file (as it is already in sorted order, as it is in key order).

by **halfteck** » Mon Apr 30, 2012 8:03 pm

I might suggest another option. If you still require the 23 byte key on the KSDS, and the 15 buye key as a second verson. You could maybe build an alternate index to the cluster, where duplicate keys are allowed.

by **Nara_513** » Mon Apr 30, 2012 8:10 pm

BillyBoyo wrote:You gave us some bad information earlier in relation to Robert's question about whether the second part of the key was to ensure that the first part was unique.

Instead of sorting your data to get rid of the duplicates, why don't you use SELECT, ensuring that it does not "sort"? You should be able to include all the unique records and the first duplicate at one go, without sorting your file (as it is already in sorted order, as it is in key order).

I apoligize for that..its my mistake..

by **BillyBoyo** » Mon Apr 30, 2012 10:16 pm

OK. Just take care. It makes your own job easier, and easier for us if you come here.

I used to outline everything in pencil on an old listing, Gives you a good grasp of everything, and how to follow through with keys and counts, etc. I think pencil and paper would still work these days :-)

IBM Mainframe Forum

VSAM fiel copy with different key lengths

Re: VSAM fiel copy with different key lengths

Re: VSAM fiel copy with different key lengths

Re: VSAM fiel copy with different key lengths

Re: VSAM fiel copy with different key lengths

Re: VSAM fiel copy with different key lengths

Re: VSAM fiel copy with different key lengths

Re: VSAM fiel copy with different key lengths

Re: VSAM fiel copy with different key lengths

Re: VSAM fiel copy with different key lengths

Re: VSAM fiel copy with different key lengths