IBM Mainframe Forum

Hi All,

I have a production transaction issue, need suggestions for the fix :

We have transaction QQQQ which gets triggered from MQ as soon as the MQ receives a new message in it.
The transaction QQQQ reads the message using the below options :



So here it clearly states that the messahe will get deleted when teh tran QQQQ executes SYNCPOINT .

After it reads the messages it do some processing of the data and it will update in the production VSAM files .

Now here comes the issue :

The transaction is looping quite often nowadays consuming high CPU . when analysed what we found is :

The transaction will be calling a program ' AAAAA' whcih updates the VSAM files, part of the data is sent by the QQQQ by placing some of the data into the Transient Storage queue and other data is sent by reference to the called program 'AAAAA' .

At times the tran is going into infinite loop as it was not able to update the VSAM files thru AAAAA .
Any update the tran do has to be done to all the files or no update should be done .So if concurrent update happens , then SYNCROLLBACK issued to undo the changes to the files.

But at atimes the SYNCROLLBACK is not functioning properly and in one of the file the record update is not undone and its getting commited . Since Rollback issued the tran again processes the message , but unable to update the files since one of the file alreadys has got a record updated in previous try , which results in duplicate record insertion .

So the AAAAA program returns control back to tran QQQQ, the code at tran QQQQ is define din such a way that , it will try for ten times to update files in case of concurrent update or duplicate record found :




here i have few queries :

1. How a record written into the file is not undone when ROLLBACK command is issued .
2. is their a chance to write same record twice , because the problematic file is getting data from Transient storage queue .
3. Why a SYNCPOINT is not able to remove the message from the MQ . IS it possible ?

I do came up with aplan to fix this , but i feel its not optimal solution, so can u let me know what i can do to fix the issue . Please let me know in case you need any other details regarding this

Thanks for your time !!
Nara

Hello,

What have you come up with? If you explain, we can offer comments/suggestions.

I suspect your problem is caused by increased volume causing more bottlenecks. The current design is quite prone to this happening.

There may be some alternative ways to get around this, but they depend on a better knowledge of "what is really going on". How many of these transactions will run concurrently? How many users might need to use this transactikno in a day? How many messages need to be dealt with in one iteration?

Hi Dick,

Please find the details :

The transaction QQQQ runs in four CICS regions in production(all sharing same set of files) , we have transaction TTTT (Runs in four CICS regions) which sends the request to vendor and the transaction QQQQ processes response sent by vendor . Since we have four regions , we have four specific response queues each one specific to a region where the vendor can place the response . Based up on the queue , the tran will get triggered in the corresponding region.

So , the tran QQQQ can run simultaneously in more than one region but processing different data .

My issue is the tran QQQQ is looping quite often ...where it is processing the same message from the queue and is not able to update the message details in the VSAM files .
When checked what we found is : Two issues , when tran executes ROLLBACK and SYNCPOINT --> the changes we expect as per the design is not happening.

The tran QQQQ will run untill all the messages in the queue(which triggers the tran) are processed . On an average 30,000 users will be using this tran.


Thanks Nara

Hi All,

We have worked with our IBM team and further looked into the CICS trace . What we have found is the transaction QQQQ calls a program to update two files . So because of some issue , the first file is getting updated and the second file is not getting updated so a ROLLBACK is issued as we are not able to update second file , when ROLLBACK is issued the first file content is not getting rolled back instead it is gettign committed .
Since ROLLBACK is issued, control comes to the calling tran QQQQ and the tran tries to process the message from the queue for ten times , each time it tries , a ROLLBACK is issued by the UPDATE program as it tries to write a duplicate record into the file (Cos the first file data is not undone when ROLLBACK is issued at the first instance). So after TEN tries the tran QQQQ issues a SYNCPOINT , but since a ROLLBACK happened(Update program) beofre the SYNCPOINT, the current message we are trying to process is not deleted when a SYNCPOINT is issues .

hence as per the design of the tran QQQQ, it will again read the same message from the MQ and tries to process it untill ten times(with TEN ROLLBACKS issues by update program ) and then SYNCPOINT , which is causing the infinite loop of teh transaction QQQQ.

I just have a question over here ...what exactly happens when we issue a SYNCPOINT COMMIT after a ROLLBACK .

As a work Around, we are setting up the " Harden Back Counter " for teh MQ, and doing the code changes to the tran QQQQ to read the back count value , so that in case if a ROLLBACK happens, for a couple of times, the tran QQQQ will check the back count value and remove the message from the queue from the MQ .

This solves our problem (even though the real root cause is not fixed : ROLLBACK unable to remove the inserted record into first file )

Thansk Nara

This is ringing a very faint little bell from way back in 1990. We had something like this and it turned out one of the program was not marked as...logged? Logable? log-using? un-logged? Something that meant it just accepted the fate of the run even if it didn't work.

This let things get out of sync. Maybe your program that is not backing out changes has this flag set. Maybe it's not really a subroutine, but a separate message with the CICS 'commit right-away' option set. (In IMS we call it EXPRESS). This would move it outside the influence of the main program's commit/rollback process.