Data Access
Data Access
No part of these materials may be reproduced, stored in a retrieval system, or transmitted, in any form or
by any means (photocopying, electronic, mechanical, recording, or otherwise), without permission in
writing from the publisher, except by a reviewer who may quote brief passages in a review.
ACL 9
December 2007
Printed in Canada
ACL, the ACL logo, the ACL logo with the text, “ACL Data you can trust. Results you can see.” and Audit
Command Language are trademarks or registered trademarks of ACL Services Ltd.
Microsoft, Windows and Windows Server are trademarks or registered trademarks of Microsoft Corpo-
ration. z/OS is a registered trademark of IBM Corporation. SAP, R/2 and R/3 are trademarks or registered
trademarks of SAP AG. Crystal Reports is a trademark or registered trademark of Business Objects SA. All
other trademarks are the property of their respective owners.
iv
C HAPTER 1
I NTRODUCTION 1
Use this guide to help you access different types of In this chapter...
data with ACL software. Data analysts must gain Finding data
access to diverse data sources from both inside and Acquiring data
outside of their organizations. They must also Reading data
understand the related technical issues. This guide
addresses both concerns.
Much of the material for this publication was drawn
from the collective knowledge of the ACL Help Desk
team. The expertise of these individuals derives from
their own specialized training and from their daily
contact with people who use ACL in the workplace.
Because of this contact, technical support staff also
know what questions people ask about data access.
This guide attempts to answer the most common of L To find definitions of unfamiliar
those questions. And it attempts to provide a terms, select Help » Index and look up
“terminology” in ACL Help.
reference for some of the more obscure challenges of
data access.
1
Chapter 1: Introduction
Find the data that you need and become acquainted with it
3
Chapter 2: Locate the data
ODBC
Working with an ODBC-compliant structure requires little effort L Find a structure that works for
from you. ACL recognizes and creates a table from both you and IS staff.
ODBC-compliant data. In ODBC, you can also choose specific
tables and set a filter to obtain only those records that you need.
But because of security and performance concerns, IS staff
sometimes hesitate to grant ODBC access. One concern is that
you can sometimes circumvent the built-in security features of
the software that reads the database. Also, unregulated access to
a system’s data tables can slow down daily operations, especially
if requests come at peak hours. Requests for large amounts of
data create considerable demands on the system’s resources.
However, if you follow regulated procedures, you can help to
alleviate such concerns and obtain the access that you need.
dBASE
When you can, request data in fixed-length, dBASE output.
Fixed-length files contain records of equal length where the same
field appears in the same location of every record. ACL
recognizes dBASE files and easily creates tables from them.
Flat files
Often, IS staff can extract data to a flat file that contains only the ? To learn more, select Help » Index
data and a CR/LF character at the end of each record. Flat files and look up “flat file”.
store information in simple records and fields, rather than in
complex database structures. When ACL encounters a flat file, it
automatically detects that the CR/LF character defines the end of
a record. To create a table, you can use the Data Definition
Wizard to help you define the fields manually with the aid of a
record layout.
Print-image files
Sometimes, the only practical way to get data from a legacy
system is to generate electronic report files, known as
print-image files. ACL can read electronic report files. It is
recommended you use the Data Definition Wizard to define a
print-image file. The wizard creates an IMPORT statement that
you can reuse for other print-image files with the same format.
Contact information
Data particulars
Storage medium CD
Additional requirements
Record layout that includes field names, field start positions, field lengths, N/A
data types, formats of numeric and date fields, and field descriptions
Transfer time
The time required to transfer data varies according to the
method that you use. Transfer speeds range from 6 kilobytes to
10,000 kilobytes per minute for PC3270 emulation to 10
megabytes per second for a LAN connection.
A client/server configuration transfers data at a comparable
speed to FTP. In-house benchmark tests indicate that a terminal
emulation transfer utility can take up to thirty times longer than
either a client/server configuration or FTP.
Download precautions
With emulation software, you can choose to transfer data in
ASCII or EBCDIC format. Most mainframes store data in
EBCDIC format, and most PCs store it in ASCII format. But ACL
can process data on the PC in either format. If you have a choice,
always process data in its native format. Any form of data
manipulation, including conversion, introduces the risk of data
Drive mapping
IS staff can map a drive between a mainframe and a PC, but must
use external software, such as Samba, to do so. Drive mapping
from one PC to another provides an easy way for members of a
workgroup to share files. However, all processing goes through a
network, which limits the rate at which data can transfer.
From PC to PC
To work with data from a personal computer, you can:
■ Transfer data files to the personal computer on which
ACL is installed. Copy files to any disk media such as
floppy disks, CDs, or Zip disks.
■ Install ACL on the personal computer that contains the L Also see “Drive mapping” on
data files. page 12.
■ Set up a shared folder or drive. See Help in Windows
Explorer for information about shared folders.
ACL connects to ACL Server Edition across a network to operate ? To learn more, select Help » Index
in a client/server architecture. You can process server-based data and look up “accessing data with
on the server or on the desktop. ACL Server Edition”.
System components
To set up a client/server environment for ACL, you need:
■ One or more clients that run ACL with the client/server
option enabled.
■ One or more servers that run ACL Server Edition.
■ A TCP/IP network connection to link the clients to the
servers. The connection can be direct, on a LAN, or by
dial-up connection on a WAN.
ACL reads data from systems that range from legacy In this chapter...
mainframe systems to modern relational databases. The process of data access
As a read-only application, ACL never alters the The function of a table layout
source data. How ACL reads data
15
Chapter 3: Access the data
■ Create a table
When you access a data source, ACL presents the data in the form
of a table. For this reason, you can also describe the process of
accessing data as the process of creating a table.
Table layouts
The ACL table contains information about the structure and
content of the source data. Such information includes location,
path, record length, field names, field lengths, field positions,
and data type. ACL stores information about the data contents
and the way that the data is displayed in a table layout.
To examine the layout of any table, select Edit » Table Layout.
Methods of access
To access data, ACL either imports the data as a flat file and ? Also see “Obtain a record layout” on
copies it to a local drive or reads the data directly in its native page 5.
format.
Manual layout
Data source
Import data
■ Direct access
ACL can access some data sources directly, without the need to
import or copy the data. ACL creates a table with varying degrees
of automation, depending on how much layout information the
source contains.
Import layout
Record-layout or
data-dictionary
document
Manual layout
Import layout
23
Chapter 4: Sources of data
■ ACCPAC
ACCPAC is a PC-based application that stores data in
transaction files that are associated with master files. When
acquiring ACCPAC data, request both the master and the
transaction files from the data provider. Transaction data is
stored in proprietary fixed-length packed fields.
ODBC Link
You can use the third-party software, ODBC Link, to connect to ? For information on ODBC Link, visit
ACCPAC data. the ACCPAC website.
External Definition
Access the external definition file using the Data Definition
Wizard.
On the Select Data Source screen of the wizard, select External
Definition. On the next screen, select COBOL. Then browse to
locate the file. ACL creates a table layout using the external
definition, then prompts you to save it. If prompted, browse to
locate the data file. ACL creates the table.
Manual layout
If you cannot obtain a copybook file, obtain a printed or
electronic copy of the layout information, and create the table
layout manually.
Reference information
Each line in a copybook represents a field definition. All COBOL ? To learn more, select Help » Index
field definitions contain a level, a field name, and a PIC clause and look up “OFFSET( )”.
that specifies the data type. The data dictionary often contains
more detailed information.
PIC clauses
The PIC clause identifies the data type. The table below lists and
describes common COBOL data types, and provides the
equivalent ACL data type, where applicable.
Redefines
REDEFINES is the COBOL equivalent of the ACL overlapping ? To learn more, select Help » Index
field definition. ACL creates overlapping fields, but you must and look up “data filters”.
manually apply the necessary filter.
Occurs
OCCURS specifies the number of times that a field occurs. Only ? To learn more, select Help » Index
the first occurrence of the field appears in ACL. If the field occurs and look up “bucketed files”, “LOOP”, or
a fixed number of times in each record, manually set up the “OFFSET( )”.
remaining occurrences, or use the OFFSET( ) function.
For example:
05 CHANGE-DATE PIC 9(6) OCCURS 5 TIMES.
If the number of OCCURS is based on a field, rather than a L Contact ACL Help Desk at
record, as in a bucketed file, use the LOOP command and the [email protected] to obtain a standard
OFFSET( ) function in an ACL script. script to remove or process segments of
bucketed files.
■ DB2
DB2 is a relational database management system (RDBMS)
developed by IBM. The software is most commonly installed on
IBM mainframes, but is also available for the Unix and Windows
operating systems.
■ dBASE-compatible
The dBASE format for storing data has become an industry
standard, and most database and spreadsheet applications can
import and export data in this format. dBASE files have a dbf
extension.
ACL recognizes dBASE files and easily creates tables from the
record-layout information that they contain. The resulting tables
will contain a field called Record_deleted. You can delete this
field from the table layout.
Reference information
Many applications can export data in a dBASE format. However,
the conversion process may not support all the features of the
source application. In addition, conversion errors can occur that
prevent you from reading the file with ACL.
■ Delimited
Delimited text files are commonly used to import and export
data from spreadsheet and database applications. Delimited text
files represent each database or spreadsheet row as a row in the
delimited text file. In a delimited text file, each record is
separated by a line separator. The valid line separators are:
carriage-return (CR), line-feed (LF), and the standard
DOS/Windows character sequence CRLF (both). Fields in each
record are separated by a field separator character. There are
three main types of delimited files that are categorized based on
the field separator character they use:
■ Comma separated values (.csv) - Commas are used to
delimit the fields in each record.
■ Tab-separated values (.tsv) - Tabs are used to delimit the L Tab-separated value files can
fields in each record. easily become corrupted. If you are
working with tab-separated values you
■ Text files (.txt) - May be comma or tab delimited, or may
should never edit the files by hand. Tabs
use another field separator. Other common field can be mistaken for spaces and when
separators used are spaces, pipes (|), and semicolons (;). the files are manually edited the tabs
can be accidentally deleted.
If the field separator character is used in the data, a text qualifier
(typically double-quotes (" ") or single- quotes (' ')) is used to
indicate that the character is a part of the value, and not a field
separator. If you use a comma as your field separator, values that
use commas need to be qualified with quotes. For example,
$1,000 would be interpreted as two fields ($1 and 000) unless the
field uses a text qualifier ("$1,000").
Once you import data from a delimited text file, you can update
the table whenever necessary without going through the Data
Definition Wizard again. Note, however, that you need to ensure
that the field lengths are large enough to import any new data.
The field length is set based on the longest value in the field, or
the field title if it is longer. You can modify this default value by
editing the field length in the table layout. Any field values that
are longer than field length will be truncated.
Example
The following example shows the first four records from a
delimited text file. The file uses commas as the field separator
and double-quotes as the text qualifier. Each row includes seven
fields separated by commas. The first row identifies the field
names in the table. The last field includes a text qualifier, so that
the comma in the dollar value is not interpreted as field
separator.
First_Name,Last_Name,CardNum,EmpNo,HireDate,Salary,Bonus_2007
Lila,Remlawi,8590122497663807,000008,12/28/1997,52750,"$1,405.40"
Vladimir,Alexov,8590122281964011,000060,10/5/1997,41250,"$4,557.43"
Alex,Williams,8590124253621744,000104,8/12/2001,40175,"$7,460.02"
■ HTML
HTML (HyperText Markup Language) defines the structure and
layout of World Wide Web documents. Markup tags, embedded
in the text, specify structural elements.
In your browser, a document displays similarly to the image on
the left. But the data that you import consists only of HTML
markup, and looks similar to the image on the right.
Once you import the document into ACL, you might find that
the text no longer lines up properly. The way that your document
looks depends upon the markup tags that were used when the
document was created.
■ IBM variable-length
To save space, many IBM mainframe computers use
variable-length records to store records of different types and
different lengths in one file. The file contains a field that stores
the length of every record.
Every IBM variable-length record begins with a four-byte field.
These four bytes consist of a two-byte header field that stores the
record length in binary format and another two-byte field that
you can ignore for ACL purposes. ACL automatically calculates
the record length from the two-byte header field, which is called
a Record Descriptor Word (RDW).
IBM variable-length files also contain a four-byte
block-description field, called a Block Descriptor Word (BDW),
at the beginning of the file and at regular intervals throughout
the file. If the block-size value is 48000, for example, another
Block Descriptor Word occurs 48000 bytes later. ACL hides this
field so you never see it. However, this field must be present for
ACL to recognize the file as IBM variable-length.
This sample IBM variable-length file contains two records with
lengths of 5800 and 5100 bytes respectively:
Sample IBM variable-length file
Name Company Position
Although you never actually see the four-byte BDW, the hidden
structure of the sample file looks like this:
BDW
RDW Record 1, length 5800
RDW Record 2, length 5100
RDW
BDW
RDW
Reference information
Sometimes the download software strips off both the RDW that
contains the record length and the four-byte BDW. When this
occurs, you must reacquire the data or use an ACL expression to
calculate the record length.
If you lose only the four-byte BDW, you can define the first two ? Also see “Avoid record-length
bytes of the record as a binary field called Record_length. Once problems” on page 12.
you close and reopen the file, the fields line up correctly.
■ IMS
The IMS transaction management system from IBM stores data
in a hierarchical format with levels or segments of data, each
dependent on the higher level. To access data at a lower level in
the hierarchy, you follow a hierarchical path from the root to the
segment that you need. The database description (DBD) defines
the order of access.
ODBC
You can also access IMS data through ODBC. See “ODBC” on
page 56.
■ Microsoft Access
ACL can import data from Microsoft Access directly. You do not L Access 2007 data can only be
need Microsoft Access installed on your PC to import the data imported using Windows XP or
into ACL. Windows Vista operating system.
Earlier versions of Windows do not
If the file is password-protected, ensure you have the correct support Office 2007 formats.
password before importing the file.
Numeric Access data with a large number of digits may be
rounded automatically during the import, particularly when
your data was created by an earlier version of Access.
Other methods
To import files from versions of Access earlier than 2000, use
ODBC or save the data in a different format.
The ODBC interface reads Access data and translates it into a
standard ODBC format that can be imported into ACL.
To import Access data through ODBC, open the Data Definition
Wizard and select ODBC in the Source Data screen. The ODBC
wizard is launched. See “ODBC” on page 56 for more details.
You can also save the data in another format that ACL can read. ? For more information, see
Access files can be saved as delimited, XML, dBASE, text, or “Delimited” on page 33.
print-image files.
Tables created from an Access file can be updated to include
changes in the source. To update a table use the Refresh from
Source File button or the REFRESH command.
■ Microsoft Excel
ACL can read Microsoft Excel data directly. You do not need
Microsoft Excel installed on your PC to import Excel files into
ACL. Excel files that are password-protected cannot be
imported.
A successful import of data from Excel into ACL depends on a L Excel 2007 data can only be
number of factors, particularly on the type of data contained in imported using Windows XP or
each column. In Excel, columns can contain various types of Windows Vista operating system.
data. In the same column, one cell might contain a date, another Earlier versions of Windows do not
a number, and another a text string. These inconsistencies may support Office 2007 formats.
cause loss of data during the import.
When a column has different types of data, Excel samples the
first eight cells and determines the data type based on the format
of the majority. For example, if the first few cells in a column
contain numbers, the column is considered a numeric column.
Then, when importing this data into ACL, cells containing
non-numeric data will appear empty. Similarly, if the first few
cells contain text, subsequent cells that contain numbers will be
considered as character data in ACL.
To get complete and correct data from Excel ensure that:
■ Data in a column is of the same type and has the same
format.
■ The data area contains no blank rows or columns.
You can also import Excel data from within a named range.
Other methods
To import files from versions of Excel earlier than 3.0 use ODBC
or save the file in a different format.
ODBC interface reads the data and translates it into a standard
ODBC format that can be imported into ACL.
To import Excel data through ODBC, open the Data Definition
Wizard and select ODBC in the Source Data screen. The ODBC
wizard is launched. See “ODBC” on page 56 for more details.
Alternatively, you can save the Excel file in another format that ? For more information, see
ACL can read. Excel files can be saved as delimited and other text “Delimited” on page 33.
file formats.
Tables created from an Excel file can be updated to include
changes in the source. To update a table use the Refresh from
Source File button or the REFRESH command.
■ Microsoft Word
Microsoft Word documents are saved under a proprietary
format with a doc extension.
■ Multiple-record-type data
Multiple-record-type files contain more than one record type,
and can consist of fixed-length or variable-length records.
You first identify individual record types, then define the fields in
ACL. You then have several options for analyzing the data:
■ One record type at a time by creating a filter in the view.
For example, if your file contains records of type A and B,
use a filter to display only records of one type or the other.
■ More than one record type at a time by using a group in
a script or by using conditional or static-conditional
fields. This is a more complex but versatile approach.
Print-image files
Electronic report files, also known as print-image files, are
multiple-record-type files, and commonly include header, detail,
and trailer records. See “Print-image (Report files)” on page 64.
CR/LF files
In CR/LF files, a carriage return (CR) and a line feed (LF)
character, or sometimes just one or the other, indicate the end of
a line. ACL automatically identifies a CR/LF file and processes it
in the same way that it processes a fixed-length file.
Segmented files
Segmented files, also called bucketed files, are a type of
variable-record-length file in which a set of header fields is
followed by one or more sets of data fields.
Segmented files store data efficiently because each record is only
as long as it needs to be and contains no empty or repeating
fields. COBOL files often contain fields or tables that can occur a
varying number of times, depending on the value in the header
field, as defined by an OCCURS clause.
Here is an example of a segmented file.
Header fields Segment Segment Segment Segment
Segments Emp_ID Emp_Name Year Sales Year Sales Year Sales Year Sales
3 101 Andersen 2002 58500 2001 60400 2000 63000
1 102 Guterriez 2002 60700
2 105 Schmidt 2002 62500 2001 58300
4 107 Lapierre 2002 63300 2001 59500 2000 67500 1999 63500
1 108 Ko 2002 56000
The header fields might contain a field that indicates the number
of segments that follow, as in the example. If this information is
not present, you might be able to calculate the number of
segments in a script by using other information such as the
segment length.
■ ODBC
Open Database Connectivity (ODBC) is a data access
technology you can use to import data from a variety of data
sources into ACL for analysis. ODBC enables you to import data
from any ODBC-compliant data source on your local
workstation, or from any network location that you have the
appropriate security rights to access.
ODBC provides a middle layer, called a database driver, between
the data source and ACL. This layer translates ACL’s queries into
commands that the data source understands. ODBC drivers exist
for almost all relational databases, and are also available for
many non-relational data sources such as spreadsheets and text
files. In order to access a particular data source using ODBC you
must have the correct driver for the data source installed. For
example, in order to access a Microsoft SQL Server database you
must have the appropriate SQL Server ODBC driver installed. In
some cases, the required ODBC drivers are installed with the
operating system. In other cases, you need to install the ODBC
driver that is provided with you application or available from a
third-party vendor. Contact your systems administrator if you
need assistance locating and installing an ODBC driver.
When you access a data source through ODBC, ACL imports the
data you select in the Data Definition Wizard into an ACL table.
You can import both physical database tables and views. To
import an ODBC table into ACL, you need to select the data
source to connect to, and then select the particular table and
fields to import. You can also filter the records in the table you are
importing using a WHERE clause. After you import the data,
you can modify the data type and formatting for individual
columns.
■ Oracle
Oracle is a relational database management system (RDBMS)
developed by Oracle Corporation. The software is available for
the Linux, Unix, and Windows operating systems.
External definition
Access the external definition file using the Data Definition
Wizard.
On the Select Data Source screen of the wizard, select External
Definition. On the next screen, select OS400 FDF, then browse
to locate the file. ACL creates a table layout using the external
definition, then prompts you to save it. If prompted, browse to
locate the data file. ACL creates the table.
ODBC
When you use ODBC, ACL imports the database as a flat,
fixed-length, sequential file and automatically creates a table.
See “ODBC” on page 56.
Other methods
If you cannot obtain an FDF file, obtain a printed or electronic
copy of the layout information, and create the table layout
manually.
You can also save the data in dBASE, text, delimited, or
print-image format in the source application. Then access the
data by the method applicable to that particular file type. ACL
Reference information
The structure of your OS/400 FDF must resemble the structure
of the sample FDF below. With the FDF in ASCII format, you can
compare the ACL table to the FDF, and make changes to the table
layout, if necessary.
1 ASCII text
2 DOS random
3 BASIC sequential
4 BASIC random
6 No-conversion file
1 ASCII
2 NUMERIC
3 ASCII
4 BINARY
5 ZONED
6 PACKED
7 MICRO
8 FLOAT
9 FLOAT
10 EBCDIC
11 ZONED
12 PACKED
■ PDF
You can import data from PDF files into ACL. ACL can read files
created with Acrobat 5 and later versions.
PDF documents can contain images and text, which may appear
in paragraphs, tables, headers, footers, and other structures. ACL
extracts text data only.
A PDF file may be password-protected. Depending on the type L Before importing a PDF file,
of security applied, you may be required to provide a document verify the security settings. If the file is
open password (to open the file) or a permissions password (to password-protected, ACL will prompt
copy the content). PDF files with Certificate Security (public key you to enter the appropriate password.
encryption) cannot be accessed by ACL.
PDF files keep spacing and font information from the original
application. Extra spaces or inconsistent fonts may cause data to
be misaligned during the file definition. If possible, ensure that
the spaces and fonts in the original file are consistent before
creating a PDF file.
Manual layout
If you cannot obtain a copybook file, obtain a printed or
electronic copy of the layout information, and create the table
layout manually.
Reference information
Here is a sample PL/1 copybook.
DECLARE
1 ZDCBHDRD UNALIGNED BASED(ZDCBHDR_PTR),
3 HDR_RECORD,
5 DC_CONTROL,
7 RECORD_STATUS CHAR(1),
7 STATUS_DATE CHAR(6),
7 HDR_TYPE CHAR(1),
7 ENTRY_DETAIL,
9 POINT_OF_ENTRY,
11 OPER_ID CHAR(4),
11 TERMINAL_ID CHAR(8),
11 ENTRY_DATE BIN FIXED(15),
9 BATCH_BALANCE,
11 OPER_ID CHAR(4),
11 TERMINAL_ID CHAR(8),
11 ENTRY_START_DATE BIN FIXED(15),
11 ESTIME BIN FIXED(31),
11 ENTRY_STOP_DATE BIN FIXED(15),
11 ENTRY_STOP_TIME BIN FIXED(31),
9 CORRECTION,
11 OPER_ID CHAR(4),
11 TERMINAL_ID CHAR(8),
11 CORR_START_DATE BIN FIXED(15),
11 CORR_START_TIME BIN FIXED(31),
11 CORR_STOP_DATE BIN FIXED(15),
11 CORR_STOP_TIME BIN FIXED(31),
In addition, you can define fields that span more than one line,
such as an address block or a multi-line description. And you can
define variable-height fields and create multiple header or footer
type records.
Alternatively, you can also define your file in the Table Layout
window. select either the Skip to Finish or the Skip Field
Identification option in the wizard.
Save as text
To save RTF as text
1 From the File menu, select Save As.
2 From the Save as type drop-down list, select Text Only
(*.txt).
You can open the file in a text editor to look at the record layout.
Then import the data through Data Definition Wizard.
Print to file
As another option, you can use the Windows Add Printer ? For assistance on how to add a
Wizard to add a Generic/Text Only driver. printer, contact your systems
administrator.
To print to a text file
1 In the Print dialog box, select Generic/Text Only from the
Name drop-down list and select the Print to file check box.
2 Click OK, and at the prompt, save the file.
You can open the file in a text editor to look at the record layout.
Then import the data through Data Definition Wizard.
■ SAP
You can access SAP® data through Direct Link™. Or you can
download the data from SAP queries, reports, or the data
browser in a format that ACL can read. As a final option, you can
print your report to a file.
Direct Link
Direct Link is a complementary product that provides the ability ? To learn more about Direct Link, visit
to link directly to SAP data from within ACL. the ACL website.
First, you open Direct Link and log onto your SAP system. Then
you build queries to select the data that you want to download.
When you next run the queries, ACL automatically downloads
the source data and creates a table that contains the data that you
queried.
■ SMF
System Management Facility (SMF) data sets collect and record
system and job-related information. Use this information to:
■ Invoice for system use
■ Report reliability
■ Analyze the configuration
■ Schedule jobs
■ Summarize direct access volume activity
■ Evaluate data set activity
■ Profile system resource use
■ Maintain system security
When a particular event occurs, SMF writes a certain type of
record to the data sets. Such events include job starts, job abends,
data set deletions, and initial program loads (IPLs).
By studying trends from historical SMF data, you can evaluate
changes in configuration, workload, or job scheduling
procedures. Similarly, you can analyze SMF data to find
inefficient operational procedures and programming
conventions.
OPEN SMFLAYOUT
COMMENT
******** This filters out records based on their record type.
SET FILTER HEX(REC_TYPE) = 'FA'
COMMENT
******** This will create a an ACL table layout file with multiple IBM record
types.
ELSE
END
Once you run the script, add the FILE_NAME card to the first row
of the table layout file SMF80.FMT.
Finally, import the format file into ACL with the Import from
Server File option, then link the resulting ACL table layout to the
SMF data file.
Reference information
This section provides examples of SMF record types and shows
what types of reports you can create from SMF data. It also offers
suggestions for ways to use these reports.
Multiple extents
An installation can check the number of extents field in the
UCB section of SMF type 14 and 15 records. From this field, the
installation can identify direct access data sets that have exceeded
their primary allocation and used their secondary allocation.
Secondary allocation, while useful, can affect system
performance and fragment the space on direct access volumes.
Check for multiple extents to avoid an X37 abend in a
production job the next time it runs. While the job may get the
space it needs this time, the next allocation may be on a volume
that only has enough space for the primary allocation. The
attempt at secondary allocation will fail and waste resources in
reruns.
Open/close activity
When EOV closes or processes a data set, SMF writes a type 14 or
15 record. The installation can count type 14 and 15 records to
find how many times EOV closed or processed a given data set.
For this kind of report, exclude SMF records for programs such
as sorts, where you know in advance that the open and close
activity is significant.
Blocking factors
From the block size and logical record length fields in SMF type
14 and 15 records, a system can identify data sets that are
processed with ineffective block factors. For example, a 10,000
record data set is processed, unblocked, with a logical record
length of 80, using a 3380 device. When the system encounters
such a data set, it can increase its block size to 6160 or 77 records.
Increased block size minimizes I/O-processing overhead and
reduces direct-access storage requirements.
Optional services
Although they are useful, optional services can hinder system
performance if they are used inappropriately. For example, the
write-validity check (OPTCD=W) service requires an additional
disk rotation to reread the data that is written for each output
block. Similarly, a data set that overuses the chained scheduling
(OPTCD=C) service might monopolize the available time on a
channel. The system can use the SMF type 14 and 15 records to
ensure that applications that use an optional service are
authorized or justified in using the service.
■ SQL Server
Microsoft SQL Server is a relational database management
system (RDBMS) developed by Microsoft. The software is only
available for the Windows operating system.
■ UNIX-based data
The UNIX family of operating systems was one of the first
portable operating system that could be installed on different
hardware platforms, from PCs to mainframes. Originally free,
many businesses and organizations have developed customized,
proprietary versions of the original operating system. AIX is a
version of UNIX produced by IBM. Linux is an open-source
operating system derived from UNIX.
Like Microsoft Windows, UNIX typically uses the ASCII
character set. This means you do not need to convert data from a
computer that runs UNIX to use the data on a PC. However,
some numeric data types that are used on AIX and Linux
systems may be unreadable on a PC.
Network Access
You can access files on a network-enabled computer that runs a
UNIX-based operating system just as you access files on a
network-enabled PC, provided you have the appropriate access
rights.
To access a UNIX file system as a mapped drive or network share ? See also “Drive mapping” on
in Windows, the UNIX system must run a Server Message Block page 12.
(SMB) to provide Windows network share support. Samba
software is commonly used for this purpose.
FTP
You can retrieve files from a UNIX computer using FTP.
ODBC
You can access ODBC-compatible databases that run on a UNIX
system through ODBC. For information see “ODBC” on
page 56.
Direct Link
You can access SAP data on a UNIX system using Direct Link or
the methods of access described in “SAP” on page 67.
■ VSAM
ACL handles Virtual Storage Access Method (VSAM) files
similarly to the way it handles IBM variable-length files. You
specify variable length as a file property when you define VSAM
data.
VSAM Entry Sequence Data Sets (ESDS) are read in the order
that the records were written to the file. VSAM Key Sequence
Data Sets (KSDS) are processed in the sequence defined by the
VSAM index. VSAM Relative Record Data Sets (RRDS) are
processed according to the relative record number sequence. In
VSAM files, the data always starts in position 5.
VSAM consists of two major parts:
■ Catalog management
VSAM maintains extensive information about data sets
and direct access storage space in a catalog. The catalog
can be either an integrated catalog facility, which is
recommended, or a VSAM catalog. The catalog’s
collection of information about a particular data set
defines that data set's characteristics. Every VSAM data
set must be defined in a catalog.
■ Record management
You can use VSAM to organize records into four types of
data sets: key-sequenced, entry-sequenced, linear, or
relative record. The way in which records are stored and
accessed accounts for the difference between these types
of data sets.
Define VSAM data sets by using either access-method-services
commands or Job Control Language (JCL) dynamic allocation.
■ XBRL 2.1
eXtensible Business Reporting Language (XBRL) is an
XML-based standard for defining and exchanging business and
financial data. XBRL can be used to submit financial data to
regulators, and to transfer data between companies or between
systems in a company. ACL supports the current XBRL 2.1
standard.
An XBRL instance document is an XML file that uses XBRL
elements and conforms to the standard. Examples of XBRL 2.1
instance document types include balance sheets, general ledgers,
and financial statements.
An XBRL 2.1 document is a valid XML file that typically uses the
standard XML file extension (.xml). The root element, or first
element in the file, identifies the file as an XBRL file. For more
information on XML files, see “XML” on page 79.
■ XML
eXtensible Markup Language (XML) is a markup language
commonly used to transmit data between computer systems or
applications. ACL can import any well-formed XML document.
A well-formed document is one that follows XML syntax rules.
XML files typically use a standard file extension (.xml). In some
cases, other file extensions are used, and the first line of the
document identifies it as an XML file. If a non-standard file
extension is used, you will need to manually select the XML file
format in the Data Definition Wizard.
XML files are structured in a standard way and support any
number of hierarchy levels. Hierarchy levels are represented by
nesting elements within other elements. When you attempt to
import an XML file, ACL identifies any data structure in the file
that can be imported into ACL tables. A data structure is a group
of elements that can be mapped to an ACL table. Each data
structure is identified with a table icon .
When you import a simple XML document with only one data
structure, it can be mapped directly to an ACL table. If you have
a more complex XML file with several data structures, you can
import each data structure into a separate table. You can then
define relationships between tables using the Relate Tables
command in ACL.
XML uses elements and attributes to identify the structure and
content of data. ACL can import both elements and attributes.
An element is a unit of XML data delimited by tags, and each
XML element can enclose other elements. In the following
example, the name element defines the value "John Smith":
<name>John Smith</name>
You can also import XBRL 2.1 files using the Data Definition
Wizard. XBRL 2.1 is an XML-based standard used by the
financial community. ACL has specific built-in support for
handling XBRL 2.1 files. For more information on XBRL 2.1
files, see “XBRL 2.1” on page 77.
I NDEX
A C D
Access. See Microsoft Access Categories Data
Accessing data data access 16 finding sources of 5
ACL methods 16 CDs integrity
ACCPAC data storage media 3 testing for 9
accessing 24 Character data missing
ACL file dBASE format 31 Microsoft Excel 45
definition of 17 Character set preparing for transfer 10
ACL Server Edition conversion problems 7, 11 Data access
DB2 30, 31, 37, 40, 41, 47, 48, Client Access ACL methods 16
56, 62, 68, 74, 76 download utility 56 impediments to 4
ACL table layout Client/server environment stages of 2
about 16 data transfer option 13 Data browser
ACL Server Edition DB2 30, 31, 37, 40, 41, 47, 48, SAP 67
about 12 56, 62, 68, 74, 76 Data conversion problems
ActiveX Data Objects Oracle 55 avoiding 7
and Microsoft Access 43 processing data in 12 Data dictionary
and Microsoft Excel 45 Client/server. See also record-layout document 15
and XML 79 ACL Server Edition Data error
Advanced Options COBOL data and client/server 13
Field Definition dialog box 64 accessing 25 risks in downloading 11
AIX category of access 21 Data requests
support in ACL 13 Column titles formulating 5
AS/400. See OS/400 header records 48 sample form 8
ASCII Command syntax Data set activities
COBOL copybook format 25 dBASE files, OPEN 32 monitoring with SMF 71
data conversion 7, 11 SMF data, reading 69 Data subsets
FDF format 56 Completeness of data extracting 4
Automatic fill checking 9 Data transfer
blank fields 64 Computed fields PC to PC 12
COBOL level 88 28 speeds, compared 11
Control totals standard methods 10
B importance of 5 Data types
Barriers Conversion COBOL and ACL compared 26
data access 4 character set 7 OS/400 codes 58
Blank fields numeric to character, dBASE 31 Database administrator
filling automatically 64 problems, avoiding 11 role of 3
Block factors copybook Databases
identifying with SMF 72 COBOL, interpreting 26 DB2 30
Block-description field PL/1, sample 63 dBASE 6, 31
IBM variable-length data 39 CR/LF IMS 41
Bucketed files. See Segmented files multiple-record-type file 48 Microsoft Access 43
Oracle 55
OS/400 56
DB2 data
81
Index
82
Index
L N
Layout file Native character set P
SMF 68 OS/400 data 56 PC data
Layout. See Record layout using 7 transferring 12
Legacy data Native format PC Support
category of access 20 reading data directly 16 download utility 56
print-image format 7 Numeric to character conversion PC-based applications
Length. See Record length dBASE 31 ACCPAC 24
Levels dBASE-compatible 31
COBOL fields 27 PDF documents
Linux 74 O accessing 60
support in ACL 13 OCCURS statement Performance concerns
Local systems defining 29 ODBC 6
as sources of data 5 ODBC PIC clauses
Logical fields security and performance interpreting 26
COBOL level 88 28 concerns 6 PL/1 data
Lost data ODBC Link accessing 62
Microsoft Excel 45 software 24 category of access 21
ODBC-compliant data Platforms
category of access 17 support in ACL 13
M IMS 42 Print files. See Print-image
Mainframe data Microsoft Excel 46 Print Image File Definition screen
transferring to PC 10 Oracle 55 features 64
Manual definition OS/400 56 Print to text file
methods of access 16 UNIX systems 74 HTML files 38
print-image files 64 OFFSET( ) function PDF documents 62
record-layout information 15 processing multiple OCCURS 25 RTF documents 66
Master file Open and close activity Print-image files
ACCPAC 24 monitoring with SMF 72 accessing 64
Methods Opening problems category of access 18
data access 16 dBASE files 32 Print-image format
Microsoft Access Operating systems legacy data 7
accessing 43 support in ACL 13 Microsoft Access 44
category of access 17 Optional services SAP unconverted output 67
Microsoft Excel monitoring with SMF 72 Print-spool files. See Print-image
accessing 45 Oracle data Private file format
category of access 17 accessing 55 SAP 67
Microsoft Word category of access 19
accessing 47 date formats in ODBC 55
Missing OS/390
83
Index
84
Index
U
UNIX
support in ACL 13
UNIX-based data
accessing 74
V
Variable-length files. See IBM
variable-length data
VSAM data
accessing 75
category of access 19
W
Word. See Microsoft Word
X
XML data
accessing 79
category of access 17
Z
z/OS
importing SMF data 68
85
Index
86
© 2007 ACL Services Ltd.