IBM Mainframe Forum

I have a task which involves
comparison of million records each from 2 files .
I get File A as an input and File B is the unload of a table .
Whatever is in File A if not in B , I have to insert those records into Table .
Whatever is in File B which is not in File A , would be updated .
Additional row each would be created for such rows.

Now Performance wise which would be faster ?

Doing it in COBOL OR Using IBM Sort/SyncSort to find the differences and using BMCHigh speed engine to do table operations ?

Your valuable inputs are appreciated .

Hello,

Whatever is in File B which is not in File A , would be updated .

Updated in the database table?

Suggest you run some timing tests to see which approach works best for your specific situation. One consideration is to make sure that any bit of data is read once rather than many passes of all of the data. . .

From what you have posted, i don't know enough to say more. If you post a more complete description of the process, maybe someone will have a more useful suggestion.

Thanks Dick !

Yes ..Number of reads on same data is important consideration .

The updates and insert will be done on Table .

While thinking for number of reads , we do not really know how many reads tools like DFSORT and SYNCSORT do .
You,Frank or Allisa can give me a better idea on that .

But doing it COBOL puts a big questionmark on why should we write something which is already available through some utility.

Thanks for your inputs and time again !

Hello,

But doing it COBOL puts a big questionmark on why should we write something which is already available through some utility.

If some utility or sort can do 100% of the requirement there is not much question (providing your organization permits using the full features of the sort or utility). I have been asked several times to look into "poor performance" and found that "someone" had implemented a process by using 5 or more sort/utility processes to do what one actually coded program could do. This caused hundreds of millions of records to be passed multiple times.

Someone who is familiar with both the requirement and the tools available should participate in the implementation decision. There is no one "right answer" - it depends on the specifics.

Much (actually most) of the time one sort or utility step does not meet the full requirement. Some things shold be done with code.

The updates and insert will be done on Table .

How are these updates to be accomplished? Is this possibly a typo? What % of the rows in the table will typically be inserted/updated? You mention what shouold happen if a value isin one file and not the other. What happens when the value is in both files? What about duplicatges? Etc, etc, etc. . .

While thinking for number of reads , we do not really know how many reads tools like DFSORT and SYNCSORT do .
You,Frank or Allisa can give me a better idea on that .

Your question is way too vague for me to answer it.

Thanks Dick ...You have provided a lot of valuable information .
And yes..even I execute sort more than once ...as I can't get all things done in just one execution .
Sort can definitely help reduce the development and testing time ..but execution time will really depend upon how we implemented it .

As far as my Update and Insert SQL are concerned ,I construct those using Sort utility and by using data in files .
And using BMC high speed engine I can execute these SQLs within JCL .
In both the files we won't have any duplicates.If record exists in both the files then there won't be insert or update for that record .

Frank,
I was asking about the number of times/passes a data being read when sort tool compares 2 files , only to understand the performance .
Please let me know if you need more information .

Thanks to both of you again for your time and guidence .

If you use DFSORT's JOINKEYS function, then the data from each file will only be read once.