Want to remove duplicate records



IBM's flagship sort product DFSORT for sorting, merging, copying, data manipulation and reporting. Includes ICETOOL and ICEGENER

Want to remove duplicate records

Postby pahi » Mon Aug 09, 2010 5:06 pm

Hi ,

I would like to sort the below file based on column 1 to 3 & column 21 to 25 by eliminating the duplicates record so that in my out put file must have only column 1to 3 & column 21 to 25.
I have used the below JCL but still I’m getting the duplicate records. Could some body help on this?

i/p file:

350100202 ¿F àä 49050 åc*50 N ° î< o N
350100202 ¿F àä 49050 åc*50 N ° î< o N
850100202 ¿F àä 49050 åc*50 N ° î< o N
850100202 ¿F àä 49050 åc*50 N ° î< o N
850100206 ¿F Ì k<45770 Í 50 N ß N
850100206 ¿F Ì k<45770 Í 50 N ß N
850100206 ¿F Ì k<45770 Í 50 N ß N

o/p required:

35049050
85049050
85045770

Jcl used:

//SORT0Q   EXEC PGM=SORT,COND=(0,LT)                           
//SYSOUT   DD  SYSOUT=*                                         
//SORTIN   DD  DSN=xxx,DISP=SHR     
//SORTOUT  DD  DSN=yyy,DISP=OLD   
//SYSIN    DD  *                                               
   SORT FIELDS=COPY                                             
    OUTREC FIELDS=(1:1,3,4:21,5)                               
    SUM FIELDS=NONE                                             
/*                                                             
//*
pahi
 
Posts: 27
Joined: Wed Apr 01, 2009 3:27 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Want to remove duplicate records

Postby NicC » Mon Aug 09, 2010 5:45 pm

From the DFSort User Guide
You can use COPY with all of the other DFSORT control statements except SUM.


I suggest you use SORT FIELDS= and provide sort fields (you stated that you wanted to sort in your post so why use COPY?

Also, what is wrong with using the DFSORT section? This is the JCL section. Someone will probably move it to the correct area so do not repost!
The problem I have is that people can explain things quickly but I can only comprehend slowly.
Regards
Nic
NicC
Global moderator
 
Posts: 3025
Joined: Sun Jul 04, 2010 12:13 am
Location: Pushing up the daisies (almost)
Has thanked: 4 times
Been thanked: 136 times

Re: Want to remove duplicate records

Postby NicC » Mon Aug 09, 2010 5:55 pm

BTW - your second field appears to start in column 18, not 21. Sorting on 1,3 and 18,5 produces the output that you show.
The problem I have is that people can explain things quickly but I can only comprehend slowly.
Regards
Nic
NicC
Global moderator
 
Posts: 3025
Joined: Sun Jul 04, 2010 12:13 am
Location: Pushing up the daisies (almost)
Has thanked: 4 times
Been thanked: 136 times

Re: Want to remove duplicate records

Postby Frank Yaeger » Mon Aug 09, 2010 11:16 pm

Here's a DFSORT job that will do what you asked for:

//S1 EXEC PGM=SORT                                       
//SYSOUT DD SYSOUT=*                                     
//SORTIN DD *                                             
350100202  ¿F   Ã Ã¤  49050  Ã¥c*50  N ° î<    o N           
350100202  ¿F   Ã Ã¤  49050  Ã¥c*50  N ° î<    o N           
850100202  ¿F   Ã Ã¤  49050  Ã¥c*50  N ° î<    o N           
850100202  ¿F   Ã Ã¤  49050  Ã¥c*50  N ° î<    o N           
850100206  ¿F  ÃŒ k< 45770  Í 50  N  ß      N             
850100206  ¿F  ÃŒ k< 45770  Í 50  N  ß      N             
850100206  ¿F  ÃŒ k< 45770  Í 50  N  ß      N             
//SORTOUT DD SYSOUT=*                                     
//SYSIN DD *                                             
  INREC BUILD=(1:1,3,4:21,5)                             
  SORT FIELDS=(1,3,CH,A,4,5,CH,A)                         
  SUM FIELDS=NONE                                         
/*


SORTOUT will have:

35049050 
85045770 
85049050 


I would like to sort the below file based on column 1 to 3 & column 21 to 25


Note that this output is actually sorted by these columns whereas the output you show is not.
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort
User avatar
Frank Yaeger
Global moderator
 
Posts: 1079
Joined: Sat Jun 09, 2007 8:44 pm
Has thanked: 0 time
Been thanked: 15 times

Re: Want to remove duplicate records

Postby pahi » Tue Aug 10, 2010 4:04 pm

Thanks a lot
pahi
 
Posts: 27
Joined: Wed Apr 01, 2009 3:27 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Want to remove duplicate records

Postby Melinda » Fri Jan 06, 2012 12:32 am

Here is the solution in this link:
Advertising link removed.
Melinda
 
Posts: 1
Joined: Fri Jan 06, 2012 12:31 am
Has thanked: 0 time
Been thanked: 0 time

Re: Want to remove duplicate records

Postby dick scherrer » Fri Jan 06, 2012 12:39 am

Hello and welcome to the forum,

Please keep in mind that this is a mainframe forum.

We do not post/host advertisements from members - even for mainframe products (and this is a pc product that does not deal with the question of this topic :? ).

d
User avatar
dick scherrer
Global moderator
 
Posts: 6268
Joined: Sat Jun 09, 2007 8:58 am
Has thanked: 3 times
Been thanked: 93 times


Return to DFSORT/ICETOOL/ICEGENER

 


  • Related topics
    Replies
    Views
    Last post