Performance in replacing strings



IBM's flagship sort product DFSORT for sorting, merging, copying, data manipulation and reporting. Includes ICETOOL and ICEGENER

Performance in replacing strings

Postby Ron Mascarenhas » Wed Sep 02, 2009 4:40 pm

We have a file in which we need to replace strings . There are about 300 replacement strings. The existing editing tool takes around a minute for the file. However in DFSORT it takes about 1 hour. Is there any other way I can code the parameters in DFSORT?

We tried 2 types of replacement parameters:

Example 1:

OPTION COPY
INREC IFTHEN=(WHEN=INIT,
FINDREP=(IN=C':اسم صاحب العمل',
OUT=C' اسم صاحب العمل')),
IFTHEN=(WHEN=INIT,
FINDREP=(IN=C'مركزد.منصور العرف الطبي',
OUT=C'مركزد منصور العرف الطبي')),
IFTHEN=(WHEN=INIT,

FINDREP=(IN=C'د0غسان عبدالله العثمان استشاري أطفال',
OUT=C'د غسان عبدالله العثمان استشاري أطفال')),
etc....


Example 2:

OPTION COPY
INREC FINDREP=(INOUT=(C':اسم صاحب العمل',
C' اسم صاحب العمل',
C'مركزد.منصور العرف الطبي',
C'مركزد منصور العرف الطبي',
C'د0غسان عبدالله العثمان استشاري أطفال',
C'د غسان عبدالله العثمان استشاري أطفال',
etc.....
Ron Mascarenhas
 
Posts: 15
Joined: Mon Nov 03, 2008 2:04 am
Has thanked: 0 time
Been thanked: 0 time

Re: Performance in replacing strings

Postby Frank Yaeger » Wed Sep 02, 2009 8:38 pm

What is the RECFM and LRECL of the input file?

What is the approx. number of records in the input file?

Can each string be anywhere in the records or can each string only be at a fixed location?

What is the existing editing tool? Does it run in the foreground or in batch?

Do Example1 and Example2 take the same amount of time?
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort
User avatar
Frank Yaeger
Global moderator
 
Posts: 1079
Joined: Sat Jun 09, 2007 8:44 pm
Has thanked: 0 time
Been thanked: 15 times

Re: Performance in replacing strings

Postby Ron Mascarenhas » Thu Sep 03, 2009 3:07 pm

Sorry I was misinformed about the time. It does take 1 hour sometimes (maybe external issued involved), however the average time now is about 15-20 mins. Still it would be nice to further tune DFSORT to near the previous utility time of 1 min.

To answer your questions:

RECFM=FB LRECL=133

About half a million records

The strings can be in any position of the record

The existing tool is SELCOPY and runs in batch.

Both examples take approximately the same amount of time.
Ron Mascarenhas
 
Posts: 15
Joined: Mon Nov 03, 2008 2:04 am
Has thanked: 0 time
Been thanked: 0 time

Re: Performance in replacing strings

Postby Frank Yaeger » Thu Sep 03, 2009 9:12 pm

Well, search and replace for each of 300 strings in 500,000 records of 133 bytes is pretty labor intensive. I don't know anything about SELCOPY so I can't comment on what it does in this case. If you want, you can add

//SORTDIAG DD DUMMY

to the job to get diagnostic messages, rerun it and send me (yaeger@us.ibm.com) the complete JES output and I'll see if I can spot anything that might help. Please send the JES messages as plain text and put "DFSORT" somewhere in your Subject line to catch my attention.
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort
User avatar
Frank Yaeger
Global moderator
 
Posts: 1079
Joined: Sat Jun 09, 2007 8:44 pm
Has thanked: 0 time
Been thanked: 15 times


Return to DFSORT/ICETOOL/ICEGENER

 


  • Related topics
    Replies
    Views
    Last post