Moving data from mainframe to *nix boxes



Discuss about QA in Mainframe, Back-End Test, Offline code testing and Testing Tools

Moving data from mainframe to *nix boxes

Postby v1gnesh » Sat Jul 14, 2012 12:09 am

Hello,

I need some help in moving data which contains packed decimal data(not the only content in it). I need to preserve the EBCDIC code so that the content doesn't go unreadable/lose alignment while receiving at the *nix end. I've tried using the iconv command
iconv -f ibm-1047 -t iso8859-1 filename > filename1
. The source is an MVS dataset that I copied to OMVS using OCOPY. Any help is highly appreciated. Thanks!

Thanks!
Vignesh
boyo
v1gnesh
 
Posts: 72
Joined: Wed Sep 28, 2011 8:24 pm
Has thanked: 1 time
Been thanked: 0 time

Re: Moving data from mainframe to *nix boxes

Postby Robert Sample » Sat Jul 14, 2012 12:23 am

There are two ways -- and ONLY two ways -- to handle packed decimal data on a *nix machine:

1) unpack the packed decimal fields before transferring the data set to the *nix box
2) transfer the data set as binary and work with it on the *nix box as EBCDIC data.

iconv translates characters. which is NOT what you want. The reason is that the value 97994 (for example) in packed decimal with a positive sign is X'97994C' in EBCDIC. Character-wise, this is a lower-case p, a lower-case r and the < symbol. Translate these characters to ASCII and you're looking at X'70723C' for these three characters. If you translate the data, either through iconv or during the FTP or in any other way, without first unpacking the packed decimal fields, the translate will completely change the packed decimal values.
Robert Sample
Global moderator
 
Posts: 3720
Joined: Sat Dec 19, 2009 8:32 pm
Location: Dubuque, Iowa, USA
Has thanked: 1 time
Been thanked: 279 times

Re: Moving data from mainframe to *nix boxes

Postby dick scherrer » Sat Jul 14, 2012 1:17 am

Hello,

The easiest way my teams have used is to create the file on the mainframe containing nothing but text. Numeric fields are punctuated so that precision is not lost (i.e. the decimal point and the +/- sign - commas not needed). The fields in the download file are tab-delimited (because the tab character (x'05') will not be in text data) and tab-delimited files are easily usable on both *nix and Win-based targets.
Hope this helps,
d.sch.
User avatar
dick scherrer
Global moderator
 
Posts: 6268
Joined: Sat Jun 09, 2007 8:58 am
Has thanked: 3 times
Been thanked: 93 times

Re: Moving data from mainframe to *nix boxes

Postby steve-myers » Sat Jul 14, 2012 1:40 am

Mr. Scherrer is right on. About 10 years ago I was involved in analyzing SMF data. I had worked up a good mainframe report, but it appeared desirable to ship the stuff to Windoze to feed into Windoze spreadsheets for graphing. It turned out it needed sorting differently than the mainframe reports. After the sort I wrote the data into CSV format - note, no binary or packed decimal data, just text. The text CSV data was sent to Windoze, and loaded into a spreadsheet, and with minor work in the spreadsheet, produced excellent graphs. By using CSV or tab delimited data, the spreadsheet import realigns the data into spreadsheet cells. The spreadsheet import also converted date and time data into spreadsheet usable date & time data.
steve-myers
Global moderator
 
Posts: 2105
Joined: Thu Jun 03, 2010 6:21 pm
Has thanked: 4 times
Been thanked: 243 times

Re: Moving data from mainframe to *nix boxes

Postby v1gnesh » Sat Jul 14, 2012 3:24 am

Robert Sample wrote:2) transfer the data set as binary and work with it on the *nix box as EBCDIC data.


Would this require any software on the *nix box?
boyo
v1gnesh
 
Posts: 72
Joined: Wed Sep 28, 2011 8:24 pm
Has thanked: 1 time
Been thanked: 0 time

Re: Moving data from mainframe to *nix boxes

Postby v1gnesh » Sat Jul 14, 2012 3:26 am

dick scherrer wrote:Hello,

The easiest way my teams have used is to create the file on the mainframe containing nothing but text. Numeric fields are punctuated so that precision is not lost (i.e. the decimal point and the +/- sign - commas not needed). The fields in the download file are tab-delimited (because the tab character (x'05') will not be in text data) and tab-delimited files are easily usable on both *nix and Win-based targets.



I'm not quite sure I follow you about the 'nothing but text'.. Are you suggesting that I remove the packed decimal data before I send it over? If so, that's fine too because I just want to get the data(everything other than the packed decimal content) across.
boyo
v1gnesh
 
Posts: 72
Joined: Wed Sep 28, 2011 8:24 pm
Has thanked: 1 time
Been thanked: 0 time

Re: Moving data from mainframe to *nix boxes

Postby dick scherrer » Sat Jul 14, 2012 9:15 am

Hello,

Robert Sample wrote:
2) transfer the data set as binary and work with it on the *nix box as EBCDIC data.

Would this require any software on the *nix box?
It is a LOT of work on the *nix box. Not to be undertaken lightly. . . At least this was true a while back . . .

Are you suggesting that I remove the packed decimal data before I send it over?
Yes, in the process that creates the download file, generate edited numeric values for any packed-decimal or binary values.
Hope this helps,
d.sch.
User avatar
dick scherrer
Global moderator
 
Posts: 6268
Joined: Sat Jun 09, 2007 8:58 am
Has thanked: 3 times
Been thanked: 93 times

Re: Moving data from mainframe to *nix boxes

Postby v1gnesh » Sat Jul 14, 2012 8:48 pm

dick scherrer wrote:It is a LOT of work on the *nix box. Not to be undertaken lightly. . . At least this was true a while back . . .


Can you please explain a little bit on this..?

dick scherrer wrote:Yes, in the process that creates the download file, generate edited numeric values for any packed-decimal or binary values.


How do I do this? I'm not the one creating the data - I'm asking this here on behalf of a client.

Thank you very much..
boyo
v1gnesh
 
Posts: 72
Joined: Wed Sep 28, 2011 8:24 pm
Has thanked: 1 time
Been thanked: 0 time

Re: Moving data from mainframe to *nix boxes

Postby Robert Sample » Sat Jul 14, 2012 10:53 pm

You have to write a program or use a utility (such as SORT) to convert the packed decimal data to zoned decimal data. It doesn't matter if your client does this or you do this -- but if you want the data to be valid and usable on the *nix machine, SOMEBODY has to convert the packed decimal data.
Robert Sample
Global moderator
 
Posts: 3720
Joined: Sat Dec 19, 2009 8:32 pm
Location: Dubuque, Iowa, USA
Has thanked: 1 time
Been thanked: 279 times

Re: Moving data from mainframe to *nix boxes

Postby dick scherrer » Sat Jul 14, 2012 11:53 pm

Hello,

Can you please explain a little bit on this..?
Not really a good topic for our type of forum. . .

If you do not already understand what this requires, it is probably not an alternative to consider. . .

What is needed is to download the native mainframe data to the target in binary (the easy part). Then the target has to have code to convert/translate the binary to something usable on the target. Which means (among other things) the target has to know the field type, position, and length, of every field in the download file. Then the target system code has to be able to properly convert each field from the binary file to whatever is needed. If there are hundreds or thousands of files, the time required to do this individually would be prohibitive. If you don't already have this on the target, the cost of writing generic code to do this would also be prohibitive.
Hope this helps,
d.sch.
User avatar
dick scherrer
Global moderator
 
Posts: 6268
Joined: Sat Jun 09, 2007 8:58 am
Has thanked: 3 times
Been thanked: 93 times

Next

Return to Application Testing

 


  • Related topics
    Replies
    Views
    Last post