Mainframe Copybook

[Carmen Kalua]

TheTransform Message component provides settings for handling the COBOL copybook format. For example, you can import a COBOL definition into the Transform Message component and use it for your Copybook transformations.

Copybook definitions must always begin with a 01 entry. A separate recordtype is generated for each 01 definition in your copybook (there must be atleast one 01 definition for the copybook to be usable, so add one using anarbitrary name at the start of the copybook if none is present). If there aremultiple 01 definitions in the copybook file, you can select whichdefinition to use in the transform from the dropdown list.

When you import the schema, the Transform component converts the copybook fileto a flat file schema that it stores in the src/main/resources/schema folderof your Mule project. In flat file format, the copybook definition above lookslike this:

After importing the copybook, you can use the schemaPath property to reference the associated flat file through the output directive. For example:output application/flatfile schemaPath="src/main/resources/schemas/mailing-record.ffd"

OCCURS DEPENDING ON with controlVal property in schema. Note that if thecontrol value is nested inside a containing structure, you need to manuallymodify the generated schema to specify the full path for the value in theform "container.value".

REDEFINES facilitates dynamic interpretation of data in a record. When you import acopybook with REDEFINES present, the generated schema uses a special groupingwith the name '*' (or '*1', '*2', and so on, if multiple REDEFINES groupings are presentat the same level) to combine all the different interpretations. You use thisspecial grouping name in your DataWeave expressions just as you use any othergrouping name.

REDEFINES requires you to use a single-byte-per-character character encoding forthe data, but any single-byte-per-character encoding can be used unless BINARY(COMP), COMP-5, or PACKED-DECIMAL (COMP-3) usages are included in the data.

The most common issue with copybook imports is a failure to follow the COBOLstandard for input line regions. The copybook import parsing ignores thecontents of columns 1-6 of each line, and ignores all lines with an '*'(asterisk) in column 7. It also ignores everything beyond column 72 in each line.This means that all your actual data definitions need to be within columns 8through 72 of input lines.

Indentation is ignored when processing the copybook, with only level-numberstreated as significant. This is not normally a problem, but it means thatcopybooks might be accepted for import even though they are not accepted byCOBOL compilers.

Both warnings and errors might be reported as a result of a copybook import.Warnings generally tell of unsupported or unrecognized features, which might ormight not be significant. Errors are notifications of a problem that means thegenerated schema (if any) will not be a completely accurate representation ofthe copybook. You should review any warnings or errors reported and decide onthe appropriate handling, which might be simply accepting the schema asgenerated, modifying the input copybook, or modifying the generated schema.

lenient: Line break is used, but records can be shorter or longer than the schema specifies. Do not use lenient if your payload lacks line breaks. The other options to recordParsing support records that lack line breaks.

Line break for a record separator. Valid values: lf, cr, crlf, none. Note that in Mule versions 4.0.4 and later, this is only used as a separator when there are multiple records. Values translate directly to character codes (none leaves no termination on each record).

So, for that 01 within the existing copybook, you can replace the data definitions after the 01 with a COPY ... REPLACING ... to give the same prefix (assuming the data-names have prefixes....) and then create your new copybook with adjusted level-numbers if needed (for instance, the example show level-numbers 02, which is always silly, but this is probably not the only example of that in the world). The different level numbers will not affect any existing code (as the compiler normalises all level numbers anyway, so the compiler will always treat the lowest level number after the 01 as 02, the second-lowest as 03, etc).

If you really cannot change the copybook (some odd diktat, happens at times) then perhaps the best bet would be to make a new copybook which is the same, but with different prefixes, and without the 01-level, for flexibility.

As Bruce Martin asked, knowledge of which compiler and OS you are using would be useful. Some, not all, COBOL compilers support nested copy statements. You could replace the layout of your record in the original copybook with a copy statement for a new copybook which contained the layout.

You'd have a minor issue that you'd need the 01 itself to be outside the copybook, and you'd need to allow a sufficient gap in the level-numbers to allow for your table definition to include the new copybook.

The highest-level data definition(s) in the copybook would have to begin with level-numbers greater than 05. This is not much of a problem to achieve. COBOL compilers will "normalise" the level-numbers anyway, and the chance of you making something less flexible by doing this is almost nil. It is your best solution if your compiler supports nested copy statements.

If not, consider doing the same thing, but removing that particular layout from the existing copybook and simply including the new copybook after the original copy statement(s). Whether you are able to do this will depend on how much the copybook is used elsewhere. Take it to your analyst/boss.

If that is not possible, make a new copybook for the table, and use comments and other documentation available to you to establish a relationship between the two data-definitions. Not ideal, but a common way that it is done.

Another possibility is to simply define areas within the table, and use the record-layout, via a MOVE to the record-layout. This is another common way, which does need documentation and checks for the lengths in the table/record-layout and is an ungainly/inefficient way to do it. Again, you'll come across that way, probably.

I actually ended not having to create a table from copybooks but still needed the 01 groups variables to be their own copybooks to create multiple instances of variables with the same structure just with different 01 group names and variable names.

Note: this is not strictly related to a copybook, it is the same when used in any data description entry in the program's source.

For documentation (I assume you meant an IBM mainframe here) see the appropriate entry in the IBM COBOL Language Reference.

This pattern provides code samples and steps to help you build an advanced tool for browsing and reviewing your mainframe fixed-format files by using AWS serverless services. The pattern provides an example of how to convert a mainframe input file to an Amazon OpenSearch Service document for browsing and searching. The file viewer tool can help you achieve the following:

Retain the same mainframe file structure and layout for consistency in your AWS target migration environment (for example, you can maintain the same layout for files in a batch application that transmits files to external parties)

An input file and its corresponding common business-oriented language (COBOL) copybook (Note: For input file and COBOL copybook examples, see gfs-mainframe-solutions on the GitHub repository. For more information about COBOL copybooks, see the Enterprise COBOL for z/OS 6.3 Programming Guide on the IBM website.)

AWS Lambda is a compute service that helps you run code without needing to provision or manage servers. It runs your code only when needed and scales automatically, so you pay only for the compute time that you use. In this pattern, you use Lambda to implement core logic, such as parsing files, converting data, and loading data into OpenSearch Service for interactive file access.

Amazon OpenSearch Service is a managed service that helps you deploy, operate, and scale OpenSearch Service clusters in the AWS Cloud. In this pattern, you use OpenSearch Service to index the converted files and provide interactive search capabilities for users.

AWS Step Functions is a serverless orchestration service that helps you combine Lambda functions and other AWS services to build business-critical applications. In this pattern, you use Step Functions to orchestrate Lambda functions.

Add a Python dependency to your Lambda environment. Important: To use the s3toelasticsearch function, you must add the Python dependency because the Lambda function uses Python Elasticsearch client dependencies (Elasticsearch==7.9.0 and requests_aws4auth).

In the Step Functions console, review the workflow execution in the Graph inspector. The execution run states are color coded to represent execution status. For example, blue indicates In Progress, green indicates Succeeded, and red indicates Failed. You can also review the table in the Execution event history section for more detailed information about the execution events.

Compare the input file against the formatted output file (indexed document) in OpenSearch Dashboards. The dashboard view shows the added column headers for your formatted files. Confirm that the source data from your unformatted input files matches the target data in the dashboard view.

I need a help. I have a mainframe flat file and its layout (copybook). I can open it in file manager to see the data in readable format. But I want to copy this file to an output with the layout and convert few packed decimal fields into numeric format. I payed with file manager and no luck. Can someone share some sudo code how to copy a flat file to table formatted output file using input file and its copybook layout. Appreciate your feedback. Thanks

The safest way is to use a BINARY file transfer from mainframe to ASCII computer, then use S370 INFORMATs on ALL data columns being read in, including text. These transform the EBCDIC data in the file to ASCII format.

3a8082e126