Professional Documents
Culture Documents
HP Vertica 7.1.x Informatica Plug-In Guide
HP Vertica 7.1.x Informatica Plug-In Guide
Legal Notices
Warranty
The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be
construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.
The information contained herein is subject to change without notice.
Copyright Notice
Copyright 2006 - 2015 Hewlett-Packard Development Company, L.P.
Trademark Notices
Adobe is a trademark of Adobe Systems Incorporated.
Microsoft and Windows are U.S. registered trademarks of Microsoft Corporation.
UNIX is a registered trademark of The Open Group.
Page 2 of 53
Contents
Contents
Overview
8
8
9
11
12
12
13
14
15
16
17
17
20
23
Creating a Workflow
25
29
31
34
34
39
Setting EnableStreamingBatchInsert
43
Page 3 of 53
Enabling SSL
Best Practices
43
44
44
45
47
Modifying powrmart.ini
48
50
53
Page 4 of 53
Use Workflow Manager to create workflows for those sources, targets and mapping you created
in Designer.
Use Repository Manager to manage repository resources, such as moving folders and objects
and managing permissions and users.
The PowerCenter Server enables you to access, read, and write to HP Vertica.
Using this plug-in is the most efficient way you can use HP Vertica as a source or target for
Informatica PowerCenter. If you plan on using HP Vertica as a source or target for PowerCenter,
install and use this plug-in rather than using ODBC.
Page 5 of 53
This manual provides information for installing plug-in components, using the plug-in to access HP
Vertica as source or target database, and implementing and modifying plug-in features.
Overview
The following table provides recommended steps.
Page 6 of 53
Step
Action
Notes
components.
Informatica PowerCenter
Set plug-in features according to your Be aware of situations where the use of one
specific needs. Features are listed
Page 7 of 53
vertica.xml contains the metadata definition needed by the PowerCenter repository to allow
communication between PowerCenter and HP Vertica.
vertica-infa951-x86.reg is a registry file you can use to register the dll on 32 bit machines.
vertica-infa951-x64.reg is a registry file you can use to register the dll on 64 bit machines.
Page 8 of 53
4. In the Domain Navigator, click the entry for the repository that you want to connect to HP
Vertica.
Page 9 of 53
5. Under the Properties tab, click Edit to edit the Repository Properties section.
6. In the Operating Mode list box, choose Exclusive, and then click OK.
Page 10 of 53
The Repository Service may take from a few moments to several minutes to restart and reenable itself.
8. Wait until you see the notice, This PowerCenter Repository Service is available.
2. On the Choose a plug-in field, click Browse and select the vertica.xml in the folder where
Page 11 of 53
3. Under the Repository Authentication section, enter your repository username and password.
Note: If you are upgrading from a previously installed plug-in version, select the checkbox,
Update existing plug-in registration. Otherwise leave the box cleared.
4. Click OK to upload the metadata file. The PowerCenter Administration Console uploads the
metadata file and registers the HP Vertica plug-in data.
Page 12 of 53
C:\Informatica\9.5.1\clients\PowerCenterClient\client\bin
2. When asked if you want to add the contents of the file to the registry, click Yes.
Register the verticacli9.dll Manually
1. Start the registry editor by typing regedit.exe in the Windows Start menu's command run
command box.
2. Navigate to the correct location in the registry.
For 32-bit versions of Windows:
HKEY_LOCAL_MACHINE\SOFTWARE\Informatica\
PowerMart Client Tools\x.x.x\Plugins\Informatica
Page 13 of 53
Where x.x.x is the version of Informatica you are using (for example 9.5.1).
3. Right-click in the right pane of the Registry Editor window. Select New then select String
Value.
4. Change the name of the string value from New Value #1 to VERTICA.
5. Double-click the new VERTICA entry. When prompted for a new value, enter
verticacli9.dll.
6. Exit the registry editor.
Copy the library file to your server's binary directory, which is the \bin\javalib subdirectory in the
PowerCenter server install directory. The full path to this directory for Windows is usually:
C:\Informatica\9.5.1\server\bin\javalib
Page 14 of 53
For all supported server operating systems (Linux, Solaris, AIX, HP-UX), you download and install
the same file, vertica-informatica-plugin-server.jar .
Copy the file to the server/bin/javalib subdirectory of the directory where PowerCenter is
installed.
Page 15 of 53
3. In the Unregister plug-in pop-up, enter your Username and Password to unregister the plugin, and then click OK.
4. Follow the procedure, Switch Back to Normal Mode.
Page 16 of 53
Page 17 of 53
4. From the Import Tables dialog box, choose the name of your ODBC data source, and enter
Username and Password.
Page 18 of 53
6. Choose a table and Click OK. For this example, choose only the table, datatypes_src. (You
can choose a number of tables.)
The table appears in the Source Analyzer panel.
7. Change the table's database type to VERTICA. To do so, double-click the name of the table to
launch the Edit Tables dialog box.
Page 19 of 53
8. From the Table tab, Database type list box, choose VERTICA.
9. Click OK. You have imported a source table. Next, you import a target.
Page 20 of 53
4. From the Import Tables dialog box, choose the name of your ODBC data source, and enter
Username and Password.
Page 21 of 53
6. Choose a table and Click OK. For this example, choose only the table, datatypes_tgt.
The table appears in the Target Designer panel.
7. Change the table's database type to VERTICA. Double-click the name of the table to launch
Page 22 of 53
8. From the Table tab, Database type list box, choose VERTICA.
9. Click OK. You have imported a target table. Next you can make a mapping.
Page 23 of 53
Page 24 of 53
9. Save your work. Next you create a workflow that uses your mapping.
Creating a Workflow
Perform this procedure to create a workflow using the table mapping you previously created.
Page 25 of 53
1. In Informatica PowerCenter, click the Workflow button to launch the Workflow Manager.
3. In the Create Workflow dialog box, enter a name for your new workflow and click OK.
Page 26 of 53
Page 27 of 53
6. In the Mappings dialog box, choose the mapping to associate with the session and click OK.
Page 28 of 53
2. From the Relational Connection Browser dialog box, select your Vertica connection object
and click and click New.
3. From the Select Subtype dialog box, select VerticaConnection and click OK.
Page 29 of 53
4. Fill in the details within the Connection Object Definition dialog box.
a. Fill in the Name, User Name, and Password.
b. Enter the connection string, which must have the format:
jdbc:vertica://<ip>:<port>/<dbname>
c. In the JDBC Driver Name field, enter com.vertica.jdbc.Driver
d. If you are connecting to a database that is running HP Vertica Release 7.1.x, select
EnableStreamingBatchInsert.
Important: If you are running HP Vertica Release 7.x on either source or target
database, enable (check off) EnableStreamingBatchInsert. Note that if you are
running a previous version of HP Vertica you can still check
EnableStreamingBatchInsert; previous versions will not experience the performance
improvements, but setting the option has no detrimental impact.
Note: You must enter a complete path for Trust Store Path (e.g.,
Page 30 of 53
C:\Users\Administrator\Desktop\verticastore).
f. Click OK.
Next you configure your source and target for the workflow.
Page 31 of 53
a. Under the Sources folder, select your source. The VERTICA_READER is listed under
Readers.
b. Click the down arrow icon to bring up the Relational Connection Browser dialog box.
Page 32 of 53
b. Click the down arrow icon to bring up the Relational Connection Browser dialog.
c. Choose your object and click OK. Save your work.
5. While in Workflow Manager, from the Workflows list box, select Start Workflow. (The
Workflow Monitor opens.)
Page 33 of 53
Page 34 of 53
The table that follows lists and describes the plug-in attributes for source/reader.
Page 35 of 53
Attribute
Value
Schema Name
Example:
datatype_src=newschemaname
Page 36 of 53
Attribute
Value
Table Name
tbl1=new-schema-name
tbl1=tbl2
That is, you would not specify tbl2 under the Schema Name
entry.
Select Distinct
Page 37 of 53
Attribute
Value
s1.a.col = s2.b.col
(where s1 and s2 are schema names)
Pre SQL
a.col = b.col
Enter complete SQL statements that run before you read a table.
For example, truncate or add data to a table before you read it.
Post SQL
Enter complete SQL statements that run after you read a table.
Modulus Partitioning
SQL Query
Page 38 of 53
Attribute
Value
Source Filter
Page 39 of 53
The table that follows lists and describes the plug-in attributes for target/writer.
Attribute
Value
Schema Name
Pre SQL
Post SQL
Copy Direct
Specify the name of the file that holds the rejected rows.
There is one log file per partition. If there are multiple log files,
a number is appended to the file names. For example,
rejects.txt would become rejects_01.txt and rejects_
02.txt.
Insert
Delete
Update
Page 40 of 53
Attribute
Value
Page 41 of 53
Attribute
Value
Target Number
Page 42 of 53
Setting EnableStreamingBatchInsert
If you are connecting to a database that is running HP Vertica Release 7.1.x, for best performance,
you should always implement EnableStreamingBatchInsert.
If you are running an earlier version of HP Vertica, setting EnableStreamingBatchInsert has no
impact on performance.
Find the full procedure for accessing the setting in the section, Configuring Your Workflow
Connections. For a workflow that is already set up:
1. While in Workflow Manager, select a task.
2. From the Connections list box, select Relational.
3. Click Edit.
4. In the Connection Object Definition dialog box, check off EnableStreamingBatchInsert.
5. Click OK. Save your work.
Enabling SSL
If your HP Vertica database is SSL enabled, then within the Connection Object Definition dialog,
check EnableSSLConnection, and enter a Trust Store Path and Trust Store Password.
Note: You must enter a complete path for Trust Store Path (e.g.,
C:\Users\Administrator\Desktop\verticastore).
For a simple example of where to access the SSL setting, refer to Configuring Your Workflow
Connections.
Page 43 of 53
Best Practices
This section includes memory requirement considerations and other tips.
Note: Allocate more memory than mentioned here according to your system limitations
and needs. The settings given may not be realistic for the tasks you intend to perform.
Page 44 of 53
at least 25.
Page 45 of 53
Page 46 of 53
c. Click OK.
Set up a public network for import/export and specify an export address for your individual
nodes. See Using Public and Private IP Networks in the Administrators Guide, specifically the
section, Identify the Database or Nodes Used for Import/Export. See also ALTER DATABASE
and ALTER NODE in the SQL Reference manual for subnet and node-related tasks.
Page 47 of 53
The HP Vertica plug-in for Informatica accesses the EXPORT_ADDRESS column of the V_
Catalog schema. (For information on viewing the EXPORT_ADDRESS column, see the SQL
Reference manual, specifically the section NODES in the V_Catalog Schema.)
l
Once the public network is set-up properly, and an export address is assigned to each node,
Informatica can then read and write to an HP Vertica cluster on a different network.
Important: If you are using the HP Vertica plug-in for Informatica on HP Vertica Release 6.x,
the Informatica user must have the PSEUDOSUPERUSER role to access the export
addresses on the different network. For general information on the PSEUDOSUPERUSER
role, see the Administrators Guide, PSEUDOSUPERUSER Role. If you are using HP Vertica
Release 7.0.x, the Informatica user does not need the PSEUDOSUPERUSER role.
Modifying powrmart.ini
When reading or importing a table, you can receive a pop-up warning, such as the following,
concerning a missing DLL.
You can make a correction so that the warning no longer appears by adding the line Vertica
Database=PMODBC.DLL to the powrmart.ini file in the section ODBCDLL. See the following
example.
1. Using the Import Tables option in the Informatica PowerCenter Designer, the system displays
a pop-up warning about a missing DLL. Click OK.
Page 48 of 53
2. The system displays another pop-up letting you know that Informatica is using EXTODBC.DLL
to support Vertica. Click OK.
Page 49 of 53
Note: The two pop-up messages are warnings only and do not affect the import of tables
or the execution of workflows.
Page 50 of 53
2. Also on the target table, from the Edit Tasks dialog, choose the Mapping tab. Under
Properties, check-off Delete.
Page 51 of 53
Page 52 of 53
Page 53 of 53