IBM Mainframe Forum

Hello,

First of all, sorry to bothering you in this forum, as I am *really* not an IBM guy, but I need your help, since I found this forum.

The question is simple:

I need to compute a file full of type80 records. I know the file is binary and I was able to find his structure (http://publib.boulder.ibm.com/infocente ... e80fmt.htm)

So I wrote a simple C program just to look at the file content. Looking at the record structure I noticed that at the 6th byte, its contents *must* be 0x50 (because it's type80). But the weird thing is that I can't notice any 0x50 on the file, when I print the bytes to the stdout. So I guess I'm looking in the bad way to that file.

So, how can I correctly see the file contents? Is there any mismatch between little-endian or big-endian (as I'm working under a little-endian architecture, and I guess the file was written under big-endian)?

The Mainframe admin said that this file is the original binary one.

Sorry again to bothering you guys, and I hope you understand the problem.

Kind Regards

How was the file transferred?

If you are not seeing the x'50' then the file is probably irredeemably garbled. If you look at the data with something, can you "read" any of it, ie can you see any reasonable amount of normal characters? If Yes, then something somewhere has converted it to ASCII, and you have the effect of changing all the letter e's in a document to a's and then changing all the a's to e's.

First of all, thanks for your help BillyBoyo.

If I look at the data with a text editor or print it to stdout, I can't read anything so I guess the fill is still in binary.

Just for the record and if it helps, I can show the first couple of bytes of the file:


(HEX – BIN)
1e-00011110
02-00000010 (1E in ASCII it's "Record Separator character" and 02 is "Start of Text"..weird or coincidence?)
00-00000000
28-00101000
7b-01111011
bc-10111100
01-00000001
11-00010001
31-00110001 (Somewhere behind this...the 0x50 had to appear, right?)
5f-01011111
d4-11010100
e5-11100101
e2-11100010
c1-11000001
de-11011110
1e-00011110
00-00000000
00-00000000
4b-01001011
3c-00111100
01-00000001
11-00010001
31-00110001
5f-01011111
d4-11010100
e5-11100101
e2-11100010
c1-11000001
d1-11010001
c5-11000101
e2-11100010
c1-11000001
00-00000000
01-00000001
00-00000000
00-00000000
00-00000000
b0-10110000
00-00000000
26-00100110
00-00000000
01-00000001
00-00000000
00-00000000
00-00000000
d6-11010110
00-00000000
ba-10111010
00-00000000
01-00000001



I don't know if I'm dealing with big/little endian issues and, as I said, I don't know if I'm understand the file in the right way.


Regards,

Showing the bit patterns is not required -- we understand ASCII and EBCDIC.

The file you have displayed is EBCDIC -- as evidenced by the MVSA in bytes 11 through 14 of your record (the X'D4' through X'C1'). However, the header does not reflect an SMF 80 type record -- are you sure that the record extraction routine only selected record type 80 for your file? My first thought is that you have a raw SMF file which has many, many, many other record types besides what you want to look at.

There is no little endian - big endian issue that I'm aware of. And the 1E / 02 is pure coincidence -- the fields you are looking at are part of the system data, not application data and have nothing to do with ASCII.

Hi Robert Sample, and thanks for your reply.

Robert Sample wrote:Showing the bit patterns is not required -- we understand ASCII and EBCDIC.

Yes, absolutely. I just copy-pasted a print of my test routine.

Robert Sample wrote:The file you have displayed is EBCDIC -- as evidenced by the MVSA in bytes 11 through 14 of your record (the X'D4' through X'C1'). However, the header does not reflect an SMF 80 type record -- are you sure that the record extraction routine only selected record type 80 for your file? My first thought is that you have a raw SMF file which has many, many, many other record types besides what you want to look at.

Well...at least I've asked the mainframe admin to give me a file with just Type80 Records. I've trusted him

Robert Sample wrote:There is no little endian - big endian issue that I'm aware of. And the 1E / 02 is pure coincidence -- the fields you are looking at are part of the system data, not application data and have nothing to do with ASCII.

Yes, I was almost sure of that. Now I'm 100% sure.

But you are saying that those aren't Type 80 Records. Are you saying that because the lack of that X'50' that I was expecting too? Or can I look to some other clues to conclude that?


Thank you very much, again.

Kind Regards.

I just created a file of nothing but type 80 recordsd from yesterday's SMF data at our site. Here's what the data in the file starts off with:I think that it is obvious by comparing this to your data that you do not have a file of just type 80 records.

cortex wrote:[...]
(HEX – BIN)
1e-00011110
02-00000010 (1E in ASCII it's "Record Separator character" and 02 is "Start of Text"..weird or coincidence?)
00-00000000
28-00101000
7b-01111011
bc-10111100
01-00000001
11-00010001
31-00110001 (Somewhere behind this...the 0x50 had to appear, right?)
5f-01011111
d4-11010100
e5-11100101
e2-11100010
c1-11000001

Likely record ends here.

The MVS1 Robert mentions is not in the "correct" position, it is four bytes too early - looks likely that the RDW (Record Descriptor Word) is not present in your data that you have shown us. This would make this the SMF record type "02", which you are not expecting.

The 0111315F is a date, the 315th day of the 11th year of the the 2nd millenium, ie 315th day of 2011. The "F" is the sign indicator (unsigned) in a "packed" decimal field. The four bytes in front of that are a binary number indicating hundredths of a second since midnight.

Following that, again with no RDW, it looks like an SMF type 30 record.

cortex wrote:de-11011110
1e-00011110
00-00000000
00-00000000
4b-01001011
3c-00111100
01-00000001
11-00010001
31-00110001
5f-01011111
d4-11010100
e5-11100101
e2-11100010
c1-11000001
d1-11010001
c5-11000101
e2-11100010
c1-11000001
00-00000000
01-00000001
00-00000000
00-00000000
00-00000000
b0-10110000
00-00000000
26-00100110
00-00000000
01-00000001
00-00000000
00-00000000
00-00000000
d6-11010110
00-00000000
ba-10111010
00-00000000
01-00000001



I don't know if I'm dealing with big/little endian issues and, as I said, I don't know if I'm understand the file in the right way.


Regards,

So, problem number one, without the RDW you won't know where one record starts and finishes, as this contains the record-length.

Problem number two, you are getting records other than those you expect. Maybe you have your 80s somewhere, but you'd have to work very hard to find them, and it might not even be possible.

You have to go back to those who supplied the data, again requesting only the data that you require. The type 80s are variable-length records. You'd be better off by having the RDW included, as it is perfectly possible that actual data within the record could look like any delimiting character(s) for a record you are expecting (the time since midnight could give you any byte value possible). You'd have to read the entire file byte-by-byte, or in chunks of the file, to not be tripped by the time.

Even better would be for the IBM guys to give you readable data, ie all textual, then your problems probably go away.

EDIT to correct "thousandths" to "hundredths" for the time since midnight.

I wonder why such a thoughtless requirement to process SMF data on a PC

Hi to all, and thanks for the explanations and examples.

So..I'm sure that I was wasting my time with this file. I'll have to talk with the admin. And to be honest, I had a bad feeling about this when I didn't find that x'50'.

But looking to Robert Sample's example, I have some simple questions for you guys:

What is that smaller record in the beginning? (Seems I have it to on my file too).

I'm trying to read and understand that example by mapping the values to the correspondent field (Based on the format of type 80). I can see there the x'50'. Do I need to read that from left to right or top to down? Because x'50' it's suppose to be the 6th byte (before, it must be the RWD (4 bytes) and 1 byte for flag, right?).


Finally, I still don't know the main goal of this task of processing SMF records on a PC. I just know that we have to visualize those records, and based on its contents produce a textual report (i.e, ASCII). I heard they want to do this with many other records (102, for instance) and in the mainframe they aren't able to produce this textual representation of every record types.


Thanks again, you all!

Regards.

Big meeting. Information is needed. Boss(es) on IBM side say they have no programming capacity. Boss of unix/pc says he has a good guy twiddling his thumbs, just give us the data. Everyone agrees. Wheels start falling off instantly. Getting the correct data to the other system takes four times as long as producing a report on the mainframe and ends with a report being produced and downloaded to the unix/pc to be processed... er, printed. Ah... someone wanted it in a spreadsheet...

Curious, as I was posting this, I see the latest response...