How much Sortwork for this dfsort



IBM's flagship sort product DFSORT for sorting, merging, copying, data manipulation and reporting. Includes ICETOOL and ICEGENER

Re: How much Sortwork for this dfsort

Postby BillyBoyo » Tue Dec 30, 2014 9:24 pm

//DFSPARM DD DSN=&VVENVIR..PARAM(PSGDRL6),DISP=SHR
//SORTCNTL DD DSN=&VVENVIR..PARAM(PSGDRL7),DISP=SHR
//SGDTRI DD DSN=&&SGDTRI,DISP=(,DELETE),SPACE=(TRK,1),UNIT=DISK,
// DCB=(LRECL=121,BLKSIZE=27951,RECFM=FB)

OK. An internal SORT. Need to see the contents of the above three (members for the first two).
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: How much Sortwork for this dfsort

Postby samb01 » Wed Dec 31, 2014 2:46 pm

i just wanted to know how can i do to calculate exactly the number of sortwork i need for this volume.
samb01
 
Posts: 431
Joined: Mon Nov 16, 2009 7:24 pm
Has thanked: 1 time
Been thanked: 0 time

Re: How much Sortwork for this dfsort

Postby BillyBoyo » Wed Dec 31, 2014 3:10 pm

That may be so, but if you don't provide the information, what would you like us to do? Take a blind guess?

It looks like you have an internal SORT (which is why the JCL doesn't look so much like a SORT). It may be that DFSORT itself is not reading the files, which makes its estimates go up in smoke. It may be that you have some other problem, that is transient. By the time you re-run, the problem has gone, so it looks like increasing the workspace does it for you. It may be... etc. "May be"s are pretty useless to us.

By far the easiest way to minimise space for "sortwork" is to use dynamic allocation. For an internal SORT, you may well have to give SORT some assistance, like an estimate of the number of records (and an average record-length for variable-length records, which you don't have).

However, with no further information from yourself...

Perhaps if you don't want to tell us, send all the information to the DFSort e-mail address. You're paying for their support already. You may as well use it if you don't want to do anything else.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: How much Sortwork for this dfsort

Postby samb01 » Wed Dec 31, 2014 3:31 pm

Hello, i can't andersand why you are tellin me i didn't give you the number of records..
Have a look ti my first message :

BTHDATE= 12/12/2012 VENDOR = BLANKS   COUNT  = 00313    TWERRC = 00000       
DATECLN= ZEROS      USECLN = 00000    CLNCNT = 000      TRERRI = 00000       
VOLSEQ = 0001       ROBTY  = VIBM     ROBID  = 002      TWERRI = 00000       
1STVOL = 409267     NEXTVOL= 410205   PREVVOL=          PRERRC = 00000       
NUMDSNB= 00000      1STDSNB= 00000000 LSTDSNB= 00000000 PWERRC = 00000       
LABEL  = SL         DEN    = 38KC     TRTCH  = 36X2     PRERRI = 00000       
RECFM  = FB         LRECL  = 001500   BLKSIZE= 027000   PWERRI = 00000       
AUDATE = 30/12/2014 AUTIME = 0051     BESKEY = 00000    BLKCNT = 0000929241   
AUCODE = 01         AUFLAG1= 00       CPUID  = CPU1     USERID = OPC         


I think it's quite clear :

LRECL : 1500
BLKSIZE= 027000
and
BLKCNT = 0000929241

you don't have enought ?
samb01
 
Posts: 431
Joined: Mon Nov 16, 2009 7:24 pm
Has thanked: 1 time
Been thanked: 0 time

Re: How much Sortwork for this dfsort

Postby NicC » Wed Dec 31, 2014 3:38 pm

It is not 'quite clear' as the information is 'hidden away' amongst all the other information. And in this case the SYSOUT from the successful run will show the exact number of records processed whereas using the info from LRECL, BLKSIZE and block count only gives an approximation (which you could use to provide the extimated count to sort).
The problem I have is that people can explain things quickly but I can only comprehend slowly.
Regards
Nic
NicC
Global moderator
 
Posts: 3025
Joined: Sun Jul 04, 2010 12:13 am
Location: Pushing up the daisies (almost)
Has thanked: 4 times
Been thanked: 136 times

Re: How much Sortwork for this dfsort

Postby BillyBoyo » Wed Dec 31, 2014 5:39 pm

I didn't say you didn't give us the number of records. I said you didn't tell SORT the (estimated) number of records.

Email the stuff you won't tell us to the DFSORT address. Remember the time of year and the timezone. Can't think of anything else to suggest without seeing what you don't want to show us.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: How much Sortwork for this dfsort

Postby samb01 » Wed Dec 31, 2014 6:19 pm

May be you want this ?

$MOT INFORMATION SGD : NUMERO DE DIFFERE RECUPERE = 0009               
IEC502E K 0D94,409267,SL,EPTBENQ2,SGD,PROBPT.EPTBENQ.ENRPART           
TMS014  IEC502E K 0D94,409267,SL,EPTBENQ2,SGD,PROBPT.EPTBENQ.ENRPART   
IEC501A M 0D94,410205,SL,COMP,EPTBENQ2,SGD.S010,PROBPT.EPTBENQ.ENRPART 
**** CONTINUOUS WAIT TIME LIMIT EXPIRED **** JOB : EPTBENQ2             
IEC502E K 0D94,410205,SL,EPTBENQ2,SGD,PROBPT.EPTBENQ.ENRPART           
TMS014  IEC502E K 0D94,410205,SL,EPTBENQ2,SGD,PROBPT.EPTBENQ.ENRPART   
IEC501A M 0D94,411737,SL,COMP,EPTBENQ2,SGD.S010,PROBPT.EPTBENQ.ENRPART 
IECTMS9 0E3F,206788,EPTBENQ2,SORTOUT ,03/02/2016,00001,T.EPTBENQ.ENREPBE
IEC705I TAPE ON 0E3F,206788,SL,COMP,EPTBENQ2,SGD.S010,PROBPT.EPTBENQ.ENR
IEC502E K 0E3F,206788,SL,EPTBENQ2,SGD,PROBPT.EPTBENQ.ENREPBE           
TMS014  IEC502E K 0E3F,206788,SL,EPTBENQ2,SGD,PROBPT.EPTBENQ.ENREPBE   
IEC501A M 0E3F,PRIVAT,SL,COMP,EPTBENQ2,SGD.S010,PROBPT.EPTBENQ.ENREPBE 
IECTMS9 0E3F,383856,EPTBENQ2,SORTOUT ,03/02/2016,00001,T.EPTBENQ.ENREPBE
IEC705I TAPE ON 0E3F,383856,SL,COMP,EPTBENQ2,SGD.S010,PROBPT.EPTBENQ.ENR
IEC502E K 0E3F,383856,SL,EPTBENQ2,SGD,PROBPT.EPTBENQ.ENREPBE           
TMS014  IEC502E K 0E3F,383856,SL,EPTBENQ2,SGD,PROBPT.EPTBENQ.ENREPBE   
IEC501A M 0E3F,PRIVAT,SL,COMP,EPTBENQ2,SGD.S010,PROBPT.EPTBENQ.ENREPBE 
IECTMS9 0E3F,403423,EPTBENQ2,SORTOUT ,03/02/2016,00001,T.EPTBENQ.ENREPBE
IEC705I TAPE ON 0E3F,403423,SL,COMP,EPTBENQ2,SGD.S010,PROBPT.EPTBENQ.ENR
IEC205I SORTOUT,EPTBENQ2,SGD,FILESEQ=1, COMPLETE VOLUME LIST,  310     
DSN=PROBPT.EPTBENQ.ENREPBE,VOLS=206788,383856,403423,                   
TOTALBLOCKS=2297017                                                     
$MOT PD823 COMPTAGE                                                     
GFAEXCI    ANCRAGE : PROB                                               
GFAEXCI    PARTENAIRE ====>    PROBTP                                   
GFAEXCI    ==> INIT AUTOMATIQUE                                         
GFAEXCI    OPEN_PIPE CALL        EPTBENQ2/CAX1GFA0  OK                 
$MOT  008 CADIF-FIN   DIFFERE PTBENQ   ETAPE 010                       
IEF234E K 0D94,411737,PVT,EPTBENQ2,SGD                                 
TMS014  IEF234E K 0D94,411737,PVT,EPTBENQ2,SGD                         
IEF234E K 0E3F,403423,PVT,EPTBENQ2,SGD                                 
TMS014  IEF234E K 0E3F,403423,PVT,EPTBENQ2,SGD                         
-IEFACTRT S010     SGD         00   255K   1.16    .06  77.09 48264K   0
-IEFACTRT S010     COND0    FLUSH      0    .00    .00    .00      0   0
IEF233A M 0E3F,206788,,EPTBENQ2,SGD,PROBPT.EPTBENQ.ENREPBE             



i have this sysout too (of the suxccessfull job)

ICE201I 0 RECORD TYPE IS F - DATA STARTS IN POSITION 1                         
ICE751I 0 C5-I20731 C6-BASE   C7-K96411 C8-I15445 E4-BASE   C9-BASE   E5-I18897
ICE143I 0 BLOCKSET     SORT  TECHNIQUE SELECTED                               
ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES A
ICE000I 0 - CONTROL STATEMENTS FOR 5650-ZOS, Z/OS DFSORT V2R1  - 11:06 ON TUE D
             MSGDDN=SGDTRI                                                     
ICE140I 0 END OF PARAMETERS FROM DFSPARM  - SYSIN OR SORTCNTL/PARAMETER LIST CO
             OPTION DYNALLOC=(3390,6)                                         
ICE146I 0 END OF STATEMENTS FROM SORTCNTL - PARAMETER LIST STATEMENTS FOLLOW   
          SORT FIELDS=(1,57,CH,A,75,8,BI,A,61,14,CH,A)                         
          RECORD TYPE=FB,LENGTH=(500)                                         
          OPTION MAINSIZE=MAX,SORTDD=SORT                                     
ICE074I 0 RECORD LENGTH L1 OR L3 OVERRIDDEN                                   
ICE193I 0 ICEAM2 INVOCATION ENVIRONMENT IN EFFECT - ICEAM2 ENVIRONMENT SELECTED
ICE252I 1 PARMLIB OPTIONS WERE MERGED WITH INSTALLATION MODULE DEFAULTS       
ICE088I 1 EPTBENQ2.S010    .SGD     , INPUT LRECL = 1500, BLKSIZE = 27000, TYPE
ICE093I 0 MAIN STORAGE = (MAX,134217728,134217728)                             
ICE156I 0 MAIN STORAGE ABOVE 16MB = (134422512,134160368)                     
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZE
ICE128I 0 OPTIONS: SIZE=134217728,MAXLIM=1048576,MINLIM=450560,EQUALS=Y,LIS
ICE129I 0 OPTIONS: VIO=Y,RESDNT=ALL ,SMF=NO   ,WRKSEC=Y,OUTSEC=Y,VERIFY=Y,C
ICE130I 0 OPTIONS: RESALL=131072,RESINV=131072,SVC=109 ,CHECK=Y,WRKREL=N,OU
ICE131I 0 OPTIONS: TMAXLIM=8388608,ARESALL=131072,ARESINV=131072,OVERRGN=16
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE    ,
ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,
ICE235I 0 OPTIONS: NULLOUT=RC0                                             
ICE236I 0 OPTIONS: DYNAPCT=10 ,MOWRK=Y,TUNE=OLD ,EXPMAX=MAX    ,EXPOLD=MAX
ICE084I 0 EXCP ACCESS METHOD USED FOR SORTOUT                             
ICE084I 0 EXCP ACCESS METHOD USED FOR SORTIN                               
ICE750I 0 DC 0 TC 62019459000 CS DSVVV KSZ 83 VSZ 83                       
ICE752I 0 FSZ=41346306 RC  IGN=0 E  AVG=1504 0  WSP=80767424 C  DYN=0 0   
ICE751I 1 DE-BASE   D5-I18897 D3-BASE   ED-BASE   E8-I12417               
ICE090I 0 OUTPUT LRECL = 1500, BLKSIZE = 27000, TYPE = FB                 
ICE055I 0 INSERT 0, DELETE 0                                               
ICE054I 0 RECORDS - IN: 41346292, OUT: 41346292                           
ICE134I 0 NUMBER OF BYTES SORTED: 62019438000                             
ICE253I 0 RECORDS SORTED - PROCESSED: 41346292, EXPECTED: 41346306         
ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 2040000 , TRACKS USED: 1200450
ICE199I 0 MEMORY OBJECT USED AS MAIN STORAGE = 0M BYTES                       
ICE299I 0 MEMORY OBJECT USED AS WORK STORAGE = 0M BYTES                       
ICE180I 0 HIPERSPACE STORAGE USED = 0K BYTES                                   
ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES                                   
ICE052I 0 END OF DFSORT                                                       
ICE052I 0 END OF DFSORT                                                       



Sorry about my undersatnding...

My english is not really good...
samb01
 
Posts: 431
Joined: Mon Nov 16, 2009 7:24 pm
Has thanked: 1 time
Been thanked: 0 time

Re: How much Sortwork for this dfsort

Postby BillyBoyo » Wed Dec 31, 2014 7:52 pm

The second file you showed is the interesting one of the two. Here is the workspace allocated and used in that step.

ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 2040000 , TRACKS USED: 1200450


You could take the "bytes sorted":

ICE134I 0 NUMBER OF BYTES SORTED: 62019438000


And come up with a ratio. Add a little bit to be safe, then you'll have something which will probably mostly work.

This, since the processed and expected are so close, tells you that DFSORT is reading the file:

ICE253I 0 RECORDS SORTED - PROCESSED: 41346292, EXPECTED: 41346306


So a good reason to use dynamic allocation of workspace.

This is interesting:

ICE074I 0 RECORD LENGTH L1 OR L3 OVERRIDDEN


There's still a lot unexplained, but you should have your answer now.
That means the 500 which was specified was ignored. Lucky for you.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Re: How much Sortwork for this dfsort

Postby samb01 » Wed Dec 31, 2014 7:59 pm

OK.

I didn't anderstand the story of 500 which is specified... Where it is specified and why i a m lucky ?

Thank's for your help.
samb01
 
Posts: 431
Joined: Mon Nov 16, 2009 7:24 pm
Has thanked: 1 time
Been thanked: 0 time

Re: How much Sortwork for this dfsort

Postby BillyBoyo » Wed Dec 31, 2014 8:37 pm

ICE146I 0 END OF STATEMENTS FROM SORTCNTL - PARAMETER LIST STATEMENTS FOLLOW   
          SORT FIELDS=(1,57,CH,A,75,8,BI,A,61,14,CH,A)                         
          RECORD TYPE=FB,LENGTH=(500)                                         
          OPTION MAINSIZE=MAX,SORTDD=SORT


There it is. Where it came from I can't say (you won't show the rest to see if it is there, or whether it comes from a program).

Lucky that SORT won't let you do that, since each 1500-byte record would be treated as three records of 500 bytes each. If you want to see the chaos that would cause, take your test file, and code an LRECL of 500 for the input dataset and look at the output :-)

For a non-VSAM and non-exit file, the RECORD statement is not needed. To put it another way, it is only needed if SORT cannot determine the LRECL. Best for it not to exist except when needed.
BillyBoyo
Global moderator
 
Posts: 3804
Joined: Tue Jan 25, 2011 12:02 am
Has thanked: 22 times
Been thanked: 265 times

Previous

Return to DFSORT/ICETOOL/ICEGENER

 


  • Related topics
    Replies
    Views
    Last post