Merge records of the input file using SORT



IBM's flagship sort product DFSORT for sorting, merging, copying, data manipulation and reporting. Includes ICETOOL and ICEGENER

Re: Join two files using DFSORT

Postby skolusu » Tue Jun 03, 2008 2:43 am

pulcinella,

If you have duplicates in both files then you need to handle it differently as you need to be clear as to which data from file 2 needs to be picked as there are duplicate keys

ex: file 1

key1   record -01
key1   record -02
key1   record -03
key1   record -04


File 2

key1   aaaaaaaaaa
key1   bbbbbbbbbb
key1   cccccccccc


You see key1 in file 2 has 3 different values , So which value do you need to pick and tag it to File1 ?

key1   record -01  aaaaaaaaaa
key1   record -02  aaaaaaaaaa
key1   record -03  aaaaaaaaaa
key1   record -04  aaaaaaaaaa


or

key1   record -01  bbbbbbbbbb
key1   record -02  bbbbbbbbbb
key1   record -03  bbbbbbbbbb
key1   record -04  bbbbbbbbbb


or
key1   record -01  cccccccccc 
key1   record -02  cccccccccc
key1   record -03  cccccccccc 
key1   record -04  cccccccccc


or

key1   record -01  aaaaaaaaaa 
key1   record -02  bbbbbbbbbb
key1   record -03  cccccccccc 
key1   record -04       


Tell me the rules and i will show you a way to do it
Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
skolusu
 
Posts: 586
Joined: Wed Apr 02, 2008 10:38 pm
Has thanked: 0 time
Been thanked: 39 times

Re: Join two files using DFSORT

Postby pulcinella » Tue Jun 03, 2008 11:37 am

Are you reason skolusu the example is not clear. I explain better

The keys of the two files are 54 positions but, the first 9 position can be duplicates and the next 45 can't be duplicates

(I only put 16 position of key but really are 54 = 9(duplicates) + 45(not duplicates). The other fields are not important for comparation)

file1
AAAAAAAAA1111111....a......
AAAAAAAAA1111112...t.......
AAAAAAAAA2222222......h....
AAAAAAAAA3333333....v......
BBBBBBBBB1111112.c.........
BBBBBBBBB1111113....vv.....
BBBBBBBBB1111114........b..
CCCCCCCCC2222221........b..
CCCCCCCCC4444442.....c.....
FFFFFFFFF2222222...b.......
FFFFFFFFF2222224a..........
FFFFFFFFF2222225g..........

file2
AAAAAAAAA1111111axxxxxxxxxx
AAAAAAAAA3333333xxxxxrxxxxx
AAAAAAAAA5555555xxxxxrxxxxx
BBBBBBBBB1111113xxxxcxxxxxx
BBBBBBBBB1111114xxxxyxxxxxx
CCCCCCCCC2222221xxxxnxxxxxx
FFFFFFFFF2222222xxxxoxxxxxx
FFFFFFFFF2222225xxxxpxxxxxx

output file (only the coincidence)

AAAAAAAAA1111111axxxxxxxxxx....a......
AAAAAAAAA3333333xxxxxrxxxxx....v......
BBBBBBBBB1111113xxxxcxxxxxx....vv.....
BBBBBBBBB1111114xxxxyxxxxxx........b..
CCCCCCCCC2222221xxxxnxxxxxx........b..
FFFFFFFFF2222222xxxxoxxxxxx...b.......
FFFFFFFFF2222225xxxxpxxxxxxg..........
pulcinella
 
Posts: 114
Joined: Mon Dec 10, 2007 10:18 pm
Has thanked: 0 time
Been thanked: 0 time

Re: Join two files using DFSORT

Postby skolusu » Wed Jun 04, 2008 1:41 am

pulcinella,

I really am confused with your posts. One post you say you have duplicates and in other you say you don't. So I am not sure which one is valid. Do you want to match both the files on ful 54 bytes or just 9 bytes ?
Kolusu - DFSORT Development Team (IBM)
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
skolusu
 
Posts: 586
Joined: Wed Apr 02, 2008 10:38 pm
Has thanked: 0 time
Been thanked: 39 times

Re: Join two files using DFSORT

Postby pulcinella » Wed Jun 04, 2008 11:49 am

Skolusu,
really I need compare with this 54 position and extract the record which keys are the same in both files
(If the first file for example has the key: "AAAAxxxx" and second file has the key "AAAAyyyy" i don't interested
to extract this record... If the first file has the key "AAAAxxxx" and second file has "AAAAxxxx" I need extract this
record)...
Of this 54 position only 9 can be duplicate more times in the files but the other positions (45) can't be duplicates and
must be unique for only file
pulcinella
 
Posts: 114
Joined: Mon Dec 10, 2007 10:18 pm
Has thanked: 0 time
Been thanked: 0 time

Previous

Return to DFSORT/ICETOOL/ICEGENER

 


  • Related topics
    Replies
    Views
    Last post