ICEETOOL - Memory Objects/Hiperspaces/Dataspaces



IBM's flagship sort product DFSORT for sorting, merging, copying, data manipulation and reporting. Includes ICETOOL and ICEGENER

ICEETOOL - Memory Objects/Hiperspaces/Dataspaces

Postby jpc » Thu Jan 31, 2008 5:19 pm

You can tune the dfsort in order to use memory in several ways.

From the z/os 64 bit architecture, all the storage is central.
As I understant Hiperspace services are re-implemented to use central storage instead of expanded storage.

I tested the same sort several times using hiperspaces (HIPRMAX=OPTIMAL,MOSIZE=0) and using memory objects (HIPRMAX=0,MOSIZE=MAX).
The job lates 10 seconds more using memory objects (30seconds) than hiperspace(20seconds).

Is there any that could explain me that?

Thanks,
Josep
jpc
 
Posts: 2
Joined: Thu Jan 31, 2008 4:57 pm
Has thanked: 0 time
Been thanked: 0 time

Re: ICEETOOL - Memory Objects/Hiperspaces/Dataspaces

Postby Frank Yaeger » Thu Jan 31, 2008 11:47 pm

I asked the DFSORT Performance Team leader to provide a response to your question. Here's what he said:

The sort algorithm used by DFSORT for memory object sorting is different from the one used for Hiperspace. Therefore you may see some cpu variances between the two methods. However, keep in mind, that if you are just looking at TCB time, you are not getting a full picture of the cpu time for a sort using Hiperspace. You must also factor in the hiperspace movement time which is captured in SMF type 30 field SMF30HPT. If you factor in the Hiperspace time, you'll find that the variance is not really that high.

DFSORT evaluates many factors as part of its decision on whether to use memory object, Hiperspace, Data space, etc. These include the available resources, file size, and DFSORT functions used as well as many others. Therefore, we generally recommend you not try to force it to use one or the other but rather leave the installation faults set to allow DFSORT to make the decision on which method to use.
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort
User avatar
Frank Yaeger
Global moderator
 
Posts: 1079
Joined: Sat Jun 09, 2007 8:44 pm
Has thanked: 0 time
Been thanked: 15 times

Re: ICEETOOL - Memory Objects/Hiperspaces/Dataspaces

Postby jpc » Fri Feb 01, 2008 2:44 pm

The problem is that when I use only memory objects, DFSort also uses sortwks when they have enough memory in order to avoid excps. (OPTION DSPSIZE=0,HIPRMAX=0,MOSIZE=MAX )
STEPNAME PROCSTEP RC EXCP CONN TCB SRB CLOCK SERV
TOOLRUN 00 1007 14173 .01 .00 .3 196K

ICE090I 0 OUTPUT LRECL = 1200, BLKSIZE = 27600, TYPE = FB
ICE055I 0 INSERT 0, DELETE 0
ICE054I 0 RECORDS - IN: 105800, OUT: 105800
ICE134I 0 NUMBER OF BYTES SORTED: 126960000
ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 3060 , TRACKS USED: 2400
ICE199I 0 MEMORY OBJECT STORAGE USED = 7M BYTES
ICE180I 0 HIPERSPACE STORAGE USED = 0K BYTES
ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES

If I use hiperspaces, they don't use sortworks:
STEPNAME PROCSTEP RC EXCP CONN TCB SRB CLOCK SERV
TOOLRUN 00 366 7154 .00 .00 .2 192K

(OPTION DSPSIZE=0,HIPRMAX=OPTIMAL,MOSIZE=0)
ICE090I 0 OUTPUT LRECL = 1200, BLKSIZE = 27600, TYPE = FB
ICE055I 0 INSERT 0, DELETE 0
ICE054I 0 RECORDS - IN: 105800, OUT: 105800
ICE134I 0 NUMBER OF BYTES SORTED: 126960000
ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 0 , TRACKS USED: 0
ICE199I 0 MEMORY OBJECT STORAGE USED = 0M BYTES
ICE180I 0 HIPERSPACE STORAGE USED = 128496K BYTES
ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES

Regards,
Josep
jpc
 
Posts: 2
Joined: Thu Jan 31, 2008 4:57 pm
Has thanked: 0 time
Been thanked: 0 time

Re: ICEETOOL - Memory Objects/Hiperspaces/Dataspaces

Postby Frank Yaeger » Fri Feb 01, 2008 9:54 pm

The DFSORT performance team leader has offered to analyze your jobs, but he needs more information. Please add the following to the jobs and rerun them:

//SORTDIAG DD DUMMY

This will give us diagnostic information. Then send the complete JES and message output for both jobs to betten@us.ibm.com and he'll take a look and respond offline.
Frank Yaeger - DFSORT Development Team (IBM) - yaeger@us.ibm.com
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
=> DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort
User avatar
Frank Yaeger
Global moderator
 
Posts: 1079
Joined: Sat Jun 09, 2007 8:44 pm
Has thanked: 0 time
Been thanked: 15 times


Return to DFSORT/ICETOOL/ICEGENER