How to find space utilization of a Data Set?



JES, JES2, JCL utilities, IDCAMS, Compile & Run JCLs, PROCs etc...

How to find space utilization of a Data Set?

Postby ITConsultant » Thu Nov 25, 2010 1:05 am

I need to find space utilization for thousands of data sets then adjust the space parameter accordingly. It seems like we are requesting too much space for some of them which translates into delays.

On the same subject, is it possible to find out the space utilization of a data set created in a previous step and use this information in proceeding step(s)?
ITConsultant
 
Posts: 15
Joined: Wed Mar 10, 2010 4:28 am
Has thanked: 0 time
Been thanked: 0 time

Re: How to find space utilization of a Data Set?

Postby enrico-sorichetti » Thu Nov 25, 2010 1:08 am

ISMF or DCOLLECT for the <collect> phase
the issue has been discussed a few times
cheers
enrico
When I tell somebody to RTFM or STFW I usually have the page open in another tab/window of my browser,
so that I am sure that the information requested can be reached with a very small effort
enrico-sorichetti
Global moderator
 
Posts: 3006
Joined: Fri Apr 18, 2008 11:25 pm
Has thanked: 0 time
Been thanked: 165 times

Re: How to find space utilization of a Data Set?

Postby steve-myers » Thu Nov 25, 2010 2:02 am

As enrico suggests, you can use DCOLLECT, though using its output may be difficult. I'm not sure what use could be made of the output after the dataset is allocated in a subsequent step.

What you can, and should, do is specify the RLSE parameter when you allocate the data set. There are some situations where this won't work, and other situations where it is not advisable, but for the most part any excess space is released, which is probably what you really want.
steve-myers
Global moderator
 
Posts: 2105
Joined: Thu Jun 03, 2010 6:21 pm
Has thanked: 4 times
Been thanked: 243 times

Re: How to find space utilization of a Data Set?

Postby enrico-sorichetti » Thu Nov 25, 2010 2:48 am

see here for dfsort dcollect examples
http://www-01.ibm.com/support/docview.w ... g3T7000081

• ICESTGEX - Storage Administrator Examples

The DFSORT product tape contains a set of illustrative DFSORT/ICETOOL examples of interest to Storage Administrators and others who analyze data created by DFHSM, DFSMSrmm, DCOLLECT and SMF. The source for the following examples are available in sample job ICESTGEX:
DCOLEX1 - DCOLLECT Example 1: VSAM report.
DCOLEX2 - DCOLLECT Example 2: Conversion reports.
DCOLEX3 - DCOLLECT Example 3: Capacity planning analysis and reports.
DFHSMEX1 - DFHSM Example 1: Deciphering Activity Logs.
DFHSMEX2 - DFHSM Example 2: Recover a DFHSM CDS with a broken index.
RMMEX1 - DFSMSrmm Example 1: SMF audit report.
RMMEX2 - DFSMSrmm Example 2: Create ADDVOLUME commands.
Download a ZIP file containing the ICESTGEX Examples (9KB)
ftp://ftp.software.ibm.com/storage/dfso ... estgex.zip
The ICESTGEX examples can also be obtained from the DFSORT product tape.
cheers
enrico
When I tell somebody to RTFM or STFW I usually have the page open in another tab/window of my browser,
so that I am sure that the information requested can be reached with a very small effort
enrico-sorichetti
Global moderator
 
Posts: 3006
Joined: Fri Apr 18, 2008 11:25 pm
Has thanked: 0 time
Been thanked: 165 times

Re: How to find space utilization of a Data Set?

Postby steve-myers » Thu Nov 25, 2010 8:57 am

ITConsultant wrote:I need to find space utilization for thousands of data sets then adjust the space parameter accordingly. It seems like we are requesting too much space for some of them which translates into delays. ...
How do you propose to "adjust the space parameter"?
ITConsultant wrote:... is it possible to find out the space utilization of a data set created in a previous step and use this information in proceeding step(s)?
And how do you propose to do this?
steve-myers
Global moderator
 
Posts: 2105
Joined: Thu Jun 03, 2010 6:21 pm
Has thanked: 4 times
Been thanked: 243 times

Re: How to find space utilization of a Data Set?

Postby expat » Thu Nov 25, 2010 12:38 pm

ITConsultant wrote:It seems like we are requesting too much space for some of them which translates into delays.

Would you please care to explain this statement, because I for one would be extremely interested in this.
expat
 
Posts: 459
Joined: Sat Jun 09, 2007 3:21 pm
Has thanked: 0 time
Been thanked: 8 times

Re: How to find space utilization of a Data Set?

Postby ITConsultant » Mon Nov 29, 2010 10:47 pm

Expat,

I have some sort steps that split the input DS into 20+ DSs based on record types. Before every sort step, I am allocating the DS (+1 GDG) using IEFBR14 step. Even though the space being requested is not that big (100 to 200 cylinders) but I see this message in the log, 'WAS ALLOCATED TO A SUBSEQUENT MULTI-TIERED STORAGE GROUP'.

We average around 40 seconds for allocation per dataset. With many sort steps in the job and each steps requiring so many new DS allocations, the job runs quite a while.

In steps that follow the sorts (COBOL), we are again allocating the same space for the output DS translating into the same message and delay again.

Thus two part inquiry, to analyze the thousands of DS and adjust the space allocation in IEFBR14 steps and then to find out the utilization from sort steps in the COBOL steps.
ITConsultant
 
Posts: 15
Joined: Wed Mar 10, 2010 4:28 am
Has thanked: 0 time
Been thanked: 0 time

Re: How to find space utilization of a Data Set?

Postby enrico-sorichetti » Mon Nov 29, 2010 11:24 pm

the multitiered message is related to the allocation strategy setup by Your storage support

You can specify Multi-Tiered SG (Y) in the storage class to enable the multi-tiered storage group function. When you specify this function, SMS allocates to enabled pool storage groups in the order they are specified in the ACS storage group selection routines.

Example:
Multi-Tiered SG (Y) is specified in the storage class. An ACS storage group routine assigns the following three similar storage groups:
SET &STORGRP = 'SG1', 'SG2', 'SG3'
Result:
SMS selects the enabled volumes that are below the high threshold in SG1 before selecting volumes in the SG2 storage groups that are listed second, third, and so forth, can also be used depending on the status of the underlying volumes when:
All SG1 enabled volumes exceed high threshold, then SMS selects the enabled volumes in SG2.
All SG1 and SG2 enabled volumes exceed the high threshold, SMS selects the enabled volumes in SG3.
All enabled volumes exceed the high threshold, SMS selects the quiesced volumes in the same storage group order.
Only volumes that reside in the first storage group assigned by the ACS routines (SG1 in the example) are eligible for the primary list.


the long delay is probably due to the scrooge attitude of Your organization,
keep the space utilization high on each volume... and that results in a longer elapsed time
in finding a volume with the available free space.

seems that nobody in Your organization is aware that everything has a cost ...
they save couple of dasd <units> and waste on allocation delays
really 40 seconds elapsed for allocating a 100/200 cyls dataset seems pretty high anyway
cheers
enrico
When I tell somebody to RTFM or STFW I usually have the page open in another tab/window of my browser,
so that I am sure that the information requested can be reached with a very small effort
enrico-sorichetti
Global moderator
 
Posts: 3006
Joined: Fri Apr 18, 2008 11:25 pm
Has thanked: 0 time
Been thanked: 165 times

Re: How to find space utilization of a Data Set?

Postby steve-myers » Tue Nov 30, 2010 11:06 am

Actually, "space utilization" has several meanings. A one track data set (e.g., trivial in terms of required space) has very poor space utiliztion if it has RECFM=F, LRECL=80, BLKSIZE=80 because most of the space on the track is occupied by inter-record gaps, so you should be looking for data sets with inappropriate DCB parameters (like my example) as well data sets that are badly over-allocated.

Another potential space waster is PDS data sets with substantial space occupied by deleted members. The only way to find this is to open and read through the data set with a program that can analyze space utilization in the data set.
steve-myers
Global moderator
 
Posts: 2105
Joined: Thu Jun 03, 2010 6:21 pm
Has thanked: 4 times
Been thanked: 243 times


Return to JCL

 


  • Related topics
    Replies
    Views
    Last post