Informatica Powermart / Powercenter 6.X Upgrade Features: Ted Williams
Informatica Powermart / Powercenter 6.X Upgrade Features: Ted Williams
PowerCenter 6.x
Ted Williams
Upgrade Features
Sources
Heterogeneous
Sources
Repository
Server
Heterogeneous
Targets
Repository
Agent
Repository
Repository Server
Each repository has an independent architecture for the
management of the physical repository tables
Components: one Repository Server and a Repository
Agent for each repository
Server
Repository
Server
Repository
Repository
Agent(s)
Manager
Client overhead for repository management is
greatly reduced by the Repository Server
Repository
3
Repository Server
The Repository Server manages connections to the
repository from client applications
The Repository Server can manage multiple repositories
on different machines on a network
For each repository it manages, the Repository Server
uses one process, called the Repository Agent, to insert,
update, and fetch objects from the repository database
tables
The Repository Server maintains object consistency by
controlling object locking
The Repository Server runs on the same system
running the Repository Agent
4
Designer Enhancements
Heterogeneous Targets
Sorter Object
Compare Objects
Compare Mappings
Workflow Manager
Variables/
Events
Tasks
Links
Work
Folders
Connection Dialogs
Replace Connections
Register Server
Assign to Workflows
9
Connection Dialogs
Type Filter
Copy As
Environmental
Options
10
Replace Connections
Replace
Connections
11
Register Server
12
Assign to Workflows
13
Global Tasks
Sessions
Local Tasks
Commands
Sessions
Commands
Email
Decision
Assignment
Timer
Control
Event Raise
Event Wait
14
Sessions (Updates)
General
Updated
parameters
First page allows for options to treat conditional links attached to the object as AND/OR
functionality. Also control option to fail the parent (container) if task fails or does not run.
Disabling a task in a workflow allows the task to be skipped instead of having to remove it.
15
Sessions (Updates)
Updated
connection
resolution
Properties
Addition to this page is the option to choose a different connection to resolve $Source and
$Target used in lookups.
**Limitation - You can use $DBConnectionVariable but it will only resolve if the parameter is
evaluated with the appropriate Source or Target Connection
If used when sourcing and targeting flat-files the parameter will not resolve.
16
Sessions (Updates)
Drop-down for
common
settings
Config Object
This area is where typical allocation of memory occurs, log file options and error handling
strategies. The upgrade here is with the concept of maintaining common setting in a config
object which is setup separately and chosen from a drop-down list.
17
Config Object
Setup for reusable
Configuration object
Config Object
By creating a session configuration object you can cut down on repetitively assigning memory
and log options. When migrating sessions you can optionally migrate configurations or maintain
environment specific settings to be reassigned.
18
Sessions (Updates)
Filter
Sources
This area has primarily just been subdivided into sub tabs. The main addition is the filter dropdown that allows for either viewing all source object settings or by individual source object.
Instead of filelist option it is either direct or indirect (filelist)
19
Sessions (Updates)
Filter
Targets
Same subdivisions as sources. However in the Properties sub tab is where the Target Options
are defined. The writers sub tab preserves the legacy operation of writing to a flat-file while using
a database specific target (as opposed to the new flat-file target).
20
Connections
The drop-down list allows to select any available connection as well as specifically define the use
of a Connection Variable
21
Sessions (Updates)
Choice of
reusable or
local command
Components
The area where commands or email unique to this object can be defined. You can alternately
select a reusable task to use as well.
22
Non-Reusable Commands
Option for local
or reusable
Name of
command
object
Components
Regardless of reusable or non-reusable it is necessary to name the object since there is
potential to promote it.
23
Non-Reusable Commands
Components
The properties tab allows for error control for commands/tasks
24
Non-Reusable Commands
Command editor
Components
The Command tab allows for editing of commands (obviously)
25
Reusable Commands
Reusable
Command
Selection Dialog
Components
A non-reusable command can be promoted to reusable or else you can select one from available
tasks.
26
Reusable Commands
Limitation
Components
Limitation a non-reusable and reusable command can resolve Server Variables but only from
within the Components tab of the session. A command can NOT resolve Server Variables if
used in an external Command Task. Understand before promoting.
27
Sessions (Updates)
Filter for modifiable
transformations
Instance Filter
Transformations
The transformations area preserves all the previous capabilities. The update is the addition of
the main instance filter and another filter button that only displays transformation where overrides
can be performed. Also this is where you can input the NEW pre/post SQL commands.
28
Sessions (Updates)
Add Partition points
Partitions
New partitioning scheme allows for repartitioning after Source Qualifier at almost any other
transformation object in the mapping. There are four main partition types Pass Through, Round
Robin, Hash Auto Keys, Hash User Keys.
29
Sessions (Updates)
Metadata column
Promote to
reusable
Metadata Extensions
Metadata extensions are available to individual object/tasks or can be made reusable and global.
These columns can not be auto-populated with functions but they do allow the extendibility for
more reporting options (e.g., Developer Name, Version, Creation Date, etc.)
30
Command Tasks
Created in Task
Developer
Command
The command object can be created globally under the Task Developer. It can also be promoted
here from within a mapping.
31
Command Tasks
Process Control
for multiple
commands
Command
The properties section homes the ability to either run all commands regardless or run them if
each previous command completes. Commands tab is where the actual commands are created.
One command per line.
32
Email Tasks
Email text
creation dialog
Built-in
Variables
Email
Email task is very similar to the command task since it can be either created in the Task
Developer or promoted from a mapping. The properties tab allows for an expression editor for
text creation utilizing the built-in variables.
33
Defiinition
Links and their underlying conditions are what provide process control to the workflow. When an
attached link condition resolves to TRUE then the attached object may begin processing. There
can be no looping and links can only execute once per workflow. However more complex
branching and decisions can be made by combining multiple links to a single object or branching
into decision type paths. Each link has its own expression editor and can utilize upstream
resolved object variables or user-defined variables for its own evaluation.
34
Object Variables
Object Variables
The default set of object variables from a session can provide more information than just a status
of Completed. More complex evaluation can be done for ErrorCode, StartTime,
SrcSuccessRows, etc. In addition to the default object variables, User Defined variables can be
created and populated via parameter files or changed in the workflow via Assignment tasks. Also
any upstream task that has completed can have its variables utilized in downstream link
conditions.
35
Workflow Variables
Variable
Task Type
Datatype
** Supported Status
Returns
Condition
Decision Task
Integer
EndTime
All tasks
Date/time
ErrorCode
All tasks
Integer
DISABLED
ErrorMsg
All tasks
Nstring*
FAILED
FirstErrorCode
Session task
Integer
NOTSTARTED
FirstErrorMsg
Session task
Nstring*
STARTED
PrevTaskStatus
All tasks
Integer
STOPPED
SrcFailedRows
Session task
Integer
SUCCEEDED
SrcSuccessRows
Session task
Integer
StartTime
All tasks
Date/time
Status**
All tasks
Integer
TgtFailedRows
Session tasks
Integer
ABORTED
Pre-defined Variables
This is the list of all pre-defined task level variables available to evaluate upon
TgtSuccessRows
Sessions
Integer
36
Workflow Variables
Edit Variables
User-defined Variables
Variables are created at the container level much like the mappings. (Workflows=Mappings,
Worklets=Mapplets). Once created values can be passed to objects within the same container
for evaluation. (Assignment Task can modify/calculate variables)
37
Workflow Variables
Pre-Defined
Variable
User Defined
Variables
User-defined Variables
A user-defined variable can assist in more complex evaluations. In the above example, an
external parameter file contains the number of expected rows. This in turn is evaluated against
the actual rows successfully read from an upstream session. $ signifies and is reserved for predefined variables. User defined variables should maintain $$ naming.
38
Parameter File
Workflow variables:
[folder name.WF:workflow name]
Worklet variables:
[folder name.WF:workflow name.WT:worklet name]
Assignment Task
Edit Variables
Usage
The assignment task allows for workflow variables to be manipulated, assigned or calculated and
stored (either for the session run or as persistent) for downstream evaluation and condition
resolution. The Expression Section allows for the selection of the appropriate variable to
manipulate along with the assignment through an Expression Editor. In order to use it is
necessary to first create a variable through the Edit Variables dialog
40
Event Task
Edit Events
Usage
Events are similar to variables. They are repository tags that are a little more flexible (cleaner)
than dealing with indicator files (although event watch can be used with indicator files). Before
utilizing the new functionality it is necessary to create these Event tags within the container that
they will be used. Events can not be monitored across workflows or worklets (even if a worklet is
part of the workflow).
41
Event Task
Event Raise
Event Wait
Usage
If using Event tags then an Event Raise is used in conjunction with an Event Wait. In the above
example two branches are executed in parallel. The second session of the lower branch will
remain in stasis until the upper branch completes triggering the event. The lower branches event
wait task recognizes the event and allows for the second session to start.
42
Event Raise
Usage
To configure the Event Raise task the drop-down box allows for selection of the appropriate userdefined Event tag. This will create an entry in the repository for a matching event wait to look for.
43
Event Wait
Indicator File
User Defined
Event
Usage
The event wait allows for configuration for an Event Raise (user-defined event) or existence
check for an indicator file.
44
Event Wait
Resume/Restart
Support
Flat-file Cleanup
Usage
The properties section of the Event Wait task allows for further definition of behavior. If your
workflow has failed/suspended after Event Raise but before the Event Wait has resolved, then
the Enable Past Events is able to recognize that the Event has happened already. If working
with indicator files you have to ability to either delete the file or allow it to stay in case some
downstream Event Waits are also keying on that file.
45
Decision Task
Usage
The decision task allows for True/False based branching of process ordering. The Decision task
can home multiple conditions and therefore downstream links can be evaluated simply upon the
Decision being True or False.
**Note it is possible to have the decision based on SUCCEEDED or FAILED of previous task,
however if workflow is set to suspend on error than that branch is suspended and the decision
wont trigger on a FAILED condition
46
Control Task
Usage
The control task is utilized in a branching manner to present a level of stoppage during the
workflow. Consider if too many sessions have too many failed rows. The options allow for
different levels such as failing at the object level to Aborting the whole workflow.
47
Timer Task
Usage
The timer task has two main ways to be utilized. The first way is by absolute time that is time
evaluated by server time or a user-defined variable (that contains the date/time stamp to start).
48
Timer Task
Usage
The second usage is by Relative time that offers options of time calculated from when the
process reached this (Timer) task, from the start of the container this task, or from the start of the
absolute top-level workflow.
49
Practical
Business Case
Need for three sessions to wait for
indicator file(s) to start each one.
Window of opportunity is only between
10AM and 2AM (next morning). If any
one runs an extra session must run. At
cutoff time there is need to stop the
process (polling - not existing runs) so
that new activity does not continue
between 2AM and 10AM. Workflow is
scheduled to run everyday at 10AM
Objects Used:
Assignment Task Assigns the appropriate cutoff time for logic
File Wait Tasks Polls for the appropriate Indicator files
Timer Task Assigned to start based on the variable assigned by the Assignment task
Command Tasks After cutoff time the commands will put an indicator file to release the polling
Link Logic The remainder of the logic is contained within the links themselves. The main sessions evaluate end
time of file wait tasks to the cutoff time. If within cutoff then sessions will run. If over cutoff sessions will not run. The
cutoff branch also evaluates to see if file wait tasks are running over. If they are still running then the command tasks
will fire.
50
Practical_Descriptive
51
Workflow Monitor
52
53