IBM Mainframe Forum

by **Aki88** » Fri Oct 21, 2016 12:07 pm

Hello,

Apologies for a non-informative subject-line, but I was unsure what to quote there.

An abend during the write of a KSDS (in an application program); below is the message populated in JESMSGLG; the application program abend handling routine returned a generic- S000 U0999:

ABNDRTN- Custom ABeND RouTiNe
DSACSRTN- Custom DtaSet-ACceSs RouTiNe
APPLPGM- APPLication ProGraM

Select all

IEA794I SVC DUMP HAS CAPTURED:  467                            
DUMPID=003 REQUESTED BY JOB (XXXXXXXX)                         
DUMP TITLE=VSAM DYNAMIC RPL DUMP - IDA019RI+075E FEEDBACK CODE:
            AA0800D4                                           
...
...
REQ=14 STA=90 --> 90 is the KSDS file-status
 

From the CEEDUMP:

Select all

Call-Chain:

   Traceback:
     DSA   Entry       E  Offset  Statement   Load Mod             Program Unit                   Service  Status
     1     CEEHDSP     +00004A4C              CEEPLPKA             CEEHDSP                        HLE77A0  Call
     2     IGZCFCC     +000C4212              IGZCPAC              IGZCFCC                                 Exception
     3     ABNDRTN     +00001BCE              ABNDRTN              SSSABEND                                Call
     4     IGZCFCC     +000002FC              IGZCPAC              IGZCFCC                                 Call
     5     DSACSRTN    +00001FC0              DSACSRTN             DSACSRTN                                Call
     6     IGZCFCC     +000002FC              IGZCPAC              IGZCFCC                                 Call
     7     APPLPGM     +00009C44              APPLPGM              APPLPGM                                 Call
 

Select all

       Machine State:
         ILC..... 0002    Interruption Code..... 000D
         PSW..... 078D1000 900EBFFA
         GPR0..... 00000000_80000000  GPR1..... 00000000_800003E7  GPR2..... 00000000_1009F40C  GPR3..... 00000000_101FA12C
         GPR4..... 00000000_100A2868  GPR5..... 00000000_00000000  GPR6..... 00000000_100A1BB0  GPR7..... 00000000_00000000
         GPR8..... 00000000_00000002  GPR9..... 00000000_100A1A30  GPR10.... 00000000_1009E100  GPR11.... 00000000_90027DE8
         GPR12.... 00000000_900EBFE8  GPR13.... 00000000_100A2878  GPR14.... 00000000_900280E6  GPR15.... 00000000_900EBFE8
         FPC...... 00000000
         FPR0..... 00000000  00000000            FPR1..... 00000000  00000000
         FPR2..... 00000000  00000000            FPR3..... 00000000  00000000
         FPR4..... 00000000  00000000            FPR5..... 00000000  00000000
         FPR6..... 00000000  00000000            FPR7..... 00000000  00000000
         FPR8..... 00000000  00000000            FPR9..... 00000000  00000000
         FPR10.... 00000000  00000000            FPR11.... 00000000  00000000
         FPR12.... 00000000  00000000            FPR13.... 00000000  00000000
         FPR14.... 00000000  00000000            FPR15.... 00000000  00000000
       ABEND code: 000003E7 Reason code: 00000000
 

From whatever little I could discern from IDA019RI and the feedback code was that something went wrong while trying to write the record into the CA which didn't allow a further split. The record being written had the 'exact key incremented by 1', which would mean the next calculated RBA would have attempted an insert in the same block or nearby.

Relevant portions of LISTCAT of the erroneous dataset:

Select all

Data Component:

      ATTRIBUTES
        KEYLEN---------------136     AVGLRECL-------------136     BUFSPACE-----------18432     CISIZE--------------4096
        RKP--------------------0     MAXLRECL-------------136     EXCPEXIT----------(NULL)     CI/CA----------------180
        STRIPE-COUNT-----------1
        SHROPTNS(2,3)      SPEED     UNIQUE           NOERASE     INDEXED       NOWRITECHK     UNORDERED          REUSE
        NONSPANNED      EXTENDED     EXT-ADDR
      STATISTICS
        REC-TOTAL-------31369375     SPLITS-CI----------21924     EXCPS------------2916863
        REC-DELETED-------170867     SPLITS-CA------------305     EXTENTS---------------13
        REC-INSERTED-----1963385     FREESPACE-%CI---------40     SYSTEM-TIMESTAMP:
        REC-UPDATED------------0     FREESPACE-%CA---------40          X'D18612827A501FCA'
        REC-RETRIEVED---31594717     FREESPC-------4826906624
      ALLOCATION
        SPACE-TYPE------CYLINDER     HI-A-RBA-----11649024000
        SPACE-PRI-----------1000     HI-U-RBA-----11449221120
        SPACE-SEC------------200


Index Component:

      ATTRIBUTES
        KEYLEN---------------136     AVGLRECL---------------0     BUFSPACE---------------0     CISIZE-------------10240
        RKP--------------------0     MAXLRECL-----------10233     EXCPEXIT----------(NULL)     CI/CA------------------5
        SHROPTNS(2,3)   RECOVERY     UNIQUE           NOERASE     NOWRITECHK     UNORDERED     REUSE           EXTENDED
        EXT-ADDR
      STATISTICS
        REC-TOTAL----------15755     SPLITS-CI------------305     EXCPS------------5864492     INDEX:
        REC-DELETED------------0     SPLITS-CA-------------88     EXTENTS---------------27     LEVELS-----------------4
        REC-INSERTED-----------0     FREESPACE-%CI----------0     SYSTEM-TIMESTAMP:            ENTRIES/SECT----------13
        REC-UPDATED--------34745     FREESPACE-%CA----------0          X'D18612827A501FCA' SEQ-SET-RBA----------------0
        REC-RETRIEVED----------0     FREESPC----------1996800                              HI-LEVEL-RBA-------119808000
      ALLOCATION
        SPACE-TYPE---------TRACK     HI-A-RBA-------163328000
        SPACE-PRI------------201     HI-U-RBA-------161331200
        SPACE-SEC-------------41
 

It'd be great if someone can help me understand what really went down in this case and if my understanding is correct. Also, it'll be very helpful if some reading material is recommended to better understand these errors, the VSAM modules in question - like IDA019RI (I had some difficulty in getting around the feedback code).

Grateful for any guidance.

Thank you.

by **Robert Sample** » Fri Oct 21, 2016 6:44 pm

reading material is recommended to better understand these errors, the VSAM modules in question - like IDA019RI

To a large degree, the only way to find out details about internal modules like IDA019RI is to go to work for IBM and get assigned to the appropriate development group.

Google is your friend. I googled ida019ri and got back 61 results, and the first one is titled "Damaged indexes - IBM" which does not bode well for your data set. If I were you, I would unload it, delete / define it (with a larger index CI size), and reload it. This quote from APAR OA07821 appears relevant to me and I emphasized the important points:

PL/I application program failed with msgIBM0811S ONCODE=1011
An I/O error occurred. Subcode1=2 Subcode2=AA0800D4
This occurred when a sequence set ci becomes full and VSAM is
unable to complete a CA split because the split point has
shifted to leftmost (highest key) entry and would causes an
empty CA to be written out.
The error code was introduced by OZ82854.
When this happens, the customer has to redefined the KSDS with
a larger index CI SIZE CISZ specified. Since we cannot predict
how effeciently the key is compressed, we're unable to
anticipate when it occurs nor to completely prevent it from
occuring.

The suggested rule of thumb of half the key length times the number of data ci per ca (mentioned in OA07821) yields a value of 12240 for the index CI size. I'd seriously consider 16384 or 18432 for the index CI size as the key compression can vary and 12240 may not be big enough.

by **Aki88** » Fri Oct 21, 2016 7:06 pm

Hello Robert,

As a resolution for the abend earlier on, the KSDS was delete/defined/reloaded.

The first few Google results were the ones (along with this link) I'd referred to as well, during the primary analysis; but I was unclear on the intricate details of the modules hence the understanding drawn was not very clear.

CISIZE was kept small because the DS is 85% dynamic (random + sequential; majoring on random) processing in CICS, though will definitely revisit it.

Apart from this do you see anything else that could've caused this error; or any other recommendations.

Thank you.

by **Robert Sample** » Fri Oct 21, 2016 7:53 pm

CISIZE was kept small because the DS is 85% dynamic

The rule for using a small CISIZE with dynamic access applies to the DATA component, not the INDEX component! With 4 index levels, I'd want at least 10 index buffers to hold the sequence set and the index levels -- more would probably help. From the VSAM Demystified Redbook:

An unnecessary large number of buffers can degrade the performance. For random, have
all the index CIs and many data CIs in the buffer pool. For sequential, have one index CI
buffer (sequence set) and many data CIs buffers for the look ahead.

Apart from this do you see anything else that could've caused this error; or any other recommendations.

I think the APAR is pretty clear about the cause -- the index CISIZE was too small and hence a CA split could not be accommodated with the given index CISIZE. Since it is a boundary condition, I would not expect the error to occur until some number of CA splits have already occurred, but once it occurred you may be okay for some CA splits but not for others trying to split at or near the same spot where the error occurred.

by **Aki88** » Fri Oct 21, 2016 8:27 pm

Thank you Robert, that cleared the air for me.

One last query, and I agree this is a done-to-death-one, though a one liner expert opinion will do me lot-good - 'how many' is MANY and a definite RED signal for a VSAM DS in terms of CA splits for data and CI splits for index, also some insight on CI/CA corresponding to this split value and levels would help me gauge future errors.

Thank you for the help.

by **Robert Sample** » Fri Oct 21, 2016 8:50 pm

For NSR access -- which I would NOT recommend for this data set (LSR appears to be better) -- page 152 of VSAM Demystified gives a formula for calculating the BUFNI value based on LISTCAT output. For your data set, when I ran the numbers, it said BUFNI of 227 would be appropriate. The Redbook suggests system managed buffers (SMB) or LSR (BLSR for batch) for randomly accessed VSAM KSDS data sets, and strongly recommends SMB for extended VSAM data sets.

In general, CA and CI splits are not considered as critical as they used to be since disk drives are now virtualized in things like the DS8000. However, since you mentioned CICS, CA splits in particular are not good for transaction response times -- since the system is moving about half the CA to a new location, you're talking a lot more processing than a normal write to the data set. If this data set is not normally unloaded, delete / defined, and reloaded regularly it might be a good idea to start doing so (although my calculations indicate it may not have been that long ago that it was redefined). Certainly when you start getting growth in the CA splits, it's time to consider a reorganization to keep down the CICS impact.

by **Aki88** » Fri Oct 21, 2016 9:35 pm

Thank you again Robert, a lot; that was insightful.
I WILL revisit VSAM Demystified, I seem to be skipping pages these days to have overlooked the information. You're correct on the point of a recent del/def- about a couple of weeks back.

Is it possible that you can share the 'calculations', plain simple to help me get some understanding for future reference; I know I am really pushing my luck here

IBM Mainframe Forum

VSAM Write Error

VSAM Write Error

Re: VSAM Write Error

Re: VSAM Write Error

Re: VSAM Write Error

Re: VSAM Write Error

Re: VSAM Write Error

Re: VSAM Write Error