Professional Documents
Culture Documents
Scribe Insight Tutorial
Scribe Insight Tutorial
Table of Contents
1. Overview ..................................................................................................... 3
Requirements ...................................................................................................................3
Overview To help you get started using Scribe Insight, try the two tutorials included in this guide. The tutorials highlight the major features of the Scribe Workbench. They will show you how to: Connect to the source and target databases Create data links Link source and target data fields Set up insert and update steps to control the data migration Save your work as a Data Translation Specification (DTS) file Test and Run the DTS Identify and fix errors using the Transaction Error Report Throughout this guide you will find references to more advanced functionality that is not included in the tutorials. For more information, see the online help and user guides included in the documentation set. The tutorials involve migrating account, address, and contact data from a text file as the source to a SQL database as the target. Scribe Workbench allows you to create a Data Translation Specification (DTS) file that contains data mappings, parameters, and settings that define the data migration or integration process. A DTS is actually a document that contains a set of instructions for the Scribe Workbench to use in the data integration process. You can save DTS files and use them as templates for future jobs. A DTS includes the following information: Connection information for the source and target data stores. Steps that define the data processing logic to be used when the DTS file is run. A mapping table that links fields in the source to the appropriate field in the target. This can include lookup links, data links, or fixed value links. Formulas used for conversion, parsing, or conditional import of selected source fields.
Requirements
Scribe Insight The Scribe Insight Workbench must be installed before you can use these tutorials. For information about installing Scribe Insight, see the Scribe Insight Installation Guide, which you can download from https://openmind.scribesoftware.com/download/ScribeInsightInstallationGuide. Scribe Sample Text Installing Scribe Insight automatically installs the file c:\Scribe\Samples\textdata\Leads.txt and creates the Scribe Sample Text ODBC Data Source Name. Scribe Sample When installing Insight, you have the option to install the SCRIBESAMPLE SQL Server database and create the corresponding ODBC DSN Scribe Sample. If the sample database is not installed (you will not see the ODBC DSN), you need to install it before starting the tutorial:
1. Navigate to the Scribe program folder.
2. Click on InternalDB.exe to open the Scribe Internal Database Maintenance Utility. 3. Click the Sample Database tab.
Figure 1. Scribe Internal Database Maintenance Utility 4. Click Install Sample Database, and follow the prompts to create the database.
If you have previously run the tutorial and put data into the sample database, click Refresh Sample Data to delete that data and restore the sample database to its original and empty state.
A single text file The result of a SQL query Results returned from a stored procedure An adapter object or related adapter objects To select the source connection for this tutorial:
1. In the Scribe Workbench main window, click the Source Connect button. The Connect
2. Click on ODBC Data Sources to expand the tree, and select Scribe Sample Text, as
The list of data sources included in this tree depends on your specific environment. For example, in your working environment, you might have only a few ODBC data sources displayed.
3. Double-click on Scribe Sample Text. The Configure Source dialog box displays, with
This example uses a single table as the source data object. However, a powerful feature of the Scribe Workbench is the variety of sources that can be configured. Sources include views, the results of any valid SQL query statement, stored procedure result sets and adapter objects.
4. In the Configure Source Data Objects Explorer, expand Tables, then select Leads.
5. Click OK. The list of source data fields displays in the source pane.
Note that each contact field has an associated reference number (the REF field). Youll use this reference number later when you work with functions and formulas.
3. Select Scribe Sample and click OK. The SQL Server login popup is displayed.
Figure 8. Login to the SQL Database 4. Enter the logon ID SCRIBE and password integr8!. The Configure Target dialog box
displays.
Use this dialog box to define the steps to include in the migration. A step is an operation that is performed on a target data object. Steps are performed once for each source row. To define each step, first select a target data object and then define the operation to perform on it. In this case, we want to select the Accounts table to copy the leads data to the Accounts table of the SQL database. As you add steps, each step is listed in order in the Step Order pane on the right side of the Configure Target dialog box. When you run the job, the Scribe Workbench performs the steps in the order they appear here.
5. On the Data Objects tab of the Configure Target dialog box, expand Tables.
Steps can be performed on tables, views, stored procedures, XML objects and adapter objects. The default step operation is Insert (that is, insert the source record in the target). You can select a different operation from the Operation drop-down list.
6. Select the ACCOUNT table.
10
7. From the bottom of the pane, select Update/Insert from the Operation drop-down list at
the bottom of the pane, and click Add Update/Insert Step. Make sure you select Update/Insert, and not Insert/Update. The difference between the two is explained in the Insight User Guide.
Figure 11. First Step Displayed in Configure Target Dialog Box 8. Click Close to close the Configure Target dialog box. The main window displays again,
in which you can view the fields from the ACCOUNT table of the Scribe Sample database in the target pane:
Figure 12. Main Window Containing Field Information from Source and Target
11
Data Link command on the Links menu.) PHONE to PHONE BUSINESS_NAME to ACCOUNTNAME
12
Figure 13. Edit Formula Dialog Box 5. Under Functions by Category pane, open System Functions, and then double-click on
GUID from the functions list. The function will display in the Edit Formula pane. Formula section of ACCOUNTID row of the Data Formulas tab.
6. Click OK to close the Edit Formula window. The GUID() formula now displays in the
The Function Browser contains over 180 functions that you can use to create formulas. Both simple and complex formulas (such as formulas that combine multiple functions and logical IF statements) are supported in creating links to a target. You can save any formula under a user-defined name to use for future translation specifications. Once you save a formula, it is included in the Function Browser under the category User Defined Formulas. Since we are working with an Update/Insert step, we want to ensure that the ACCOUNT ID is assigned only when a new account is inserted. By default, an Update overwrites (that is, updates) data every time you run a job. In this case, using an Update/Insert step, we want a new ACCOUNTID to be generated only when a new account is inserted. When an existing account is updated, we want the ACCOUNTID field to remain unchanged.
7. To change the overwrite status of a field:
b. Move your mouse to the asterisk (*) in the row of the Data Formulas pane where Target field = ACCOUNTID. c. Double-click on the asterisk to turn off the overwrite feature. If an insert is done, the data link is used; if an update is done, the data link will not be used. We turn Scribe Insight Tutorials 13
overwrite off because we do not want the GUID function to generate a new value for ACCOUNTID if it updates a target record.
In general, you want to turn off Overwrite from the Lookup field whenever you are performing an Update/Insert or an Insert/Update step.
the update part of the update/insert step: a In the source field list, select UNIQUE_ID again. b. In the target field list, select the field XREF. c. Click the Lookup Link button.
2. When you are done, select UNIQUE_ID from the source field list, and Show links to
source field from the Links tab at the bottom of the Workbench main window. If the Links tab does not display, click the Splitter button ( the Links, Data Formulas, and Lookup Criteria tabs. ) to display
14
Checkmarks are displayed in both the target field and the selected source field to indicate that the two database fields are linked. As soon as you create a link, a graphical display of the link displays automatically in the Links tab.
Checkmark indicates that the field is linked Circled star indicates that field is part of a unique index
15
display the Test window. The left side of the window shows the source field names and values. In the right side of the window, you can view the Data Links, Lookup Links, and Step Results for each record.
Figure 16. Test Window 2. Click Next to scroll through each of the source rows. 3. When you view the records in the Test window, youll notice that the business names
are all uppercase. While that may be fine for your source data, its unattractive when, for example, printing addresses and is easily changed.
Before you actually run the job, lets change the business names to mixed case. To do so:
1. Open ACCOUNTNAME in the Data Formulas window. 2. Double-click the PROPER function from the TEXT category.
16
3.
After you close the Edit Formulas window, make sure that PROPER(S3) appears in the Formula column next to ACCOUNTNAME.
Figure 17. Data Formulas Pane, Ready to Run 4. Save the DTS as Accounts.dts. 5. If you test the job again, you will notice that the company names are in mixed upper
window displays, you should notice that you have 15 successful inserts, but one failed insert.
Figure 18. Run Complete Window 2. To figure out why one row failed, click on Transaction Errors in the Run Complete
window to open the Transaction Errors report. the Transaction Errors Report Setup window.
3. In this case, you do not need to do any sorting or filtering. Click View Report from
17
Figure 19. Transaction Errors Report 5. Close the report and then close the Run Complete window.
18
3. When you examine the record, youll notice that the value of ACCOUNTNAME is
#NULL!.
For record 14, the ACCOUNTNAME field is NULL
Figure 20. Test Window Showing ACCOUNTNAME Error 4. Close the Test window.
window.
2. In the Edit Formula pane, enter the following formula (note that PROPER(S3) is already
IF(ISERROR(S3),"Unknown"&S1, PROPER( S3 )) This formula checks the BUSINESS_NAME (ref S3) field of each row. If the field is null, it changes the null to Unknown plus the UNIQUE_ID from the source before inserting it into ACCOUNTNAME. This way, we know the account name was not provided in the source data and we can relate it back to the source data with the UNIQUE_ID.
Figure 21. Data Formulas Pane Corrected Formula 3. Save the DTS.
19
This time, the Run Complete window should show that you have 1 insert (the corrected row, for which the ACCOUNTNAME will read Unknown114), and 15 updates. Because of the lookup link between UNIQUE_ID and XREF, the other 15 rows are updated rather than inserted again.
Figure 22. Run Complete Dialog Box 2. To close the Run Complete window, click Close.
This completes the first tutorial. You can proceed to the second tutorial in the following section to learn more about the Scribe Workbench. Tutorial 2 assumes that Tutorial 1 has correctly populated the Scribe Sample database. Without this data, Tutorial 2 will not run correctly.
20
Sample Text.
3. In the Configure Source dialog boxs database tree, expand Tables then select Leads. 4. Click OK.
integr8!).
a. Select the ACCOUNT table and add a Seek step. A Seek step uses lookup links to find rows in the target connection. b. Select the ADDRESS table and add an Insert step. c. Select the CONTACT table and add an Insert step d. Select CONTACT again and add another Insert step.
21
The source data has a contact and alternate contact. We want two CONTACT insert steps so our DTS file will create a contact record in the target for the contact, and where it exists, the alternate contact.
Figure 23. Configuring Multiple Target Steps 5. Close the Configure Target window.
22
This formula checks to see if there is a null in field S16 (that is, ALT_CONTACT). If ALT_CONTACT is null, the SKIPSTEP function causes the step to be skipped. If ALT_CONTACT is not null, the step is executed and the value in ALT_CONTACT is inserted into the target.
Figure 24. Adding a Pre-Operation Step Flow Control Formula 5. Close the Configure Target window.
The first link we want to create is a lookup link on the ACCOUNT Seek step.
1. From the Target Configure drop-down list, select ACCOUNT Seek. 2. Create a lookup link between UNIQUE_ID in the source and XREF in the target.
Because a Seek step does not insert data, notice that you cannot create data links. 23
a. To ensure that the address is for this company is unique, add the GUID() system function to ADDRESSID. Because you have not created a data link to a source field for the ADDRESSID target field, to add a formula, select ADDRESSID in the target pane and click the Formula button between the source and target panes. b. To format the address and city names to be in mixed case, add the PROPER function to ADDRESSLINE1 and CITY c. And finally, lets make sure that we have the correct country code. To the COUNTRYCODE field, add the following formula: IF( ISERROR(S8), "US", DBLOOKUP(S8, "I", "COUNTRY", "COUNTRYNAME", "COUNTRYCODE")) This formula performs a database lookup if the country code is null. If it is null, the country code is set to the default value of US. d. If the country code is not NULL, the DBLOOKUP function takes the value from S8, and uses the connection to the SCRIBEINTERNAL database to perform a database lookup. For this function, I means use the internal database. The DBLOOKUP function does a select from the COUNTRY table where the COUNTRYNAME field is equal to the value in S8. DBLOOOKUP then returns the value from the COUNTRYCODE field in the matching row. In SQL , the formula would be similar to: SELECT COUNTRYCODE FROM COUNTRY WHERE COUNTRYNAME = (value from S8)
4. Next, select CONTACT Insert from the Target Configure drop-down list and create the
24
5. As with the Address ID, we want to ensure that the CONTACTID is unique. In addition,
the CONTACT_NAME (source ref number S2) field in the source needs to be split into two fields; the FIRSTNAME and LASTNAME. However, just to make sure we have the name right, well insert the whole contact name too. To make these changes to your data: a. Create a formula to add the GUID() function to CONTACTID. b. Add the PROPER() function to CONTACTNAME. c. For the FIRSTNAME and LASTNAME fields, you want to parse contact name, and change the case to mixed case. For FIRSTNAME, add the following formula: PROPER(PARSENAME(S2, "F")) d. Do the same for LASTNAME, but enter an L rather than an F: PROPER(PARSENAME(S2, "L"))
6. Finally, select CONTACT(2) Insert from the Target Configure drop-down list and
configure the data links for the CONTACT(2) step similar to how you configured the data links for the CONTACT step. For CONTACT(2), use the source field ALT_CONTACT, (S16), as the field to use in your data links and formulas. The Data Formula tab should be similar to the following.
7. When you are done, set the Target Configure drop-down list back to <View All Steps>.
In the Data Formulas tab, you can sort columns by double-clicking on the column header. In this example, Target Step has been sorted alphabetically. Alternatively, select Sort from the View menu; then select Data Links to display the sort options you can choose on this tab.
25
26
2. Right-click on ACCOUNTID and select Field Properties from the shortcut menu to
Figure 28. ACCOUNTID Properties Window 3. Note that there is an Auto foreign key assignment on where the value for the
ACCOUNTID field in the CONTACT step will be inherited from Step 1, which is a Seek on ACCOUNT for this example. This is how Insight automatically fills in the foreign key to a parent table.
27
Figure 29. Test Target Step Results 3. Note that in the Source Value pane, the ALT_CONTACT field is null. In the Target Step
Results tab, Step 4 is grayed out and arrows indicate that this Insert step is skipped (but that steps 2 and 3 are inserted). different.
4. Check a record for which there is an alternate contact to see how the step results are
In this tutorial, the Jump To and Previous buttons are unavailable. These functions are supported by most Scribe adapters, but are not supported by the ODBC text driver.
5. If you check the Target Lookup Links tab, you will notice that this DTS uses a single
lookup link that you created during the ACCOUNT Seek step.
6. When you are done, close the Test window and save the DTS.
28
Figure 30. Run Complete Window 3. Note that the window shows that there were 16 rows processed, with 40 successful
If your results show 16 successful, 16 failed, and 16 skipped records, then you probably did not run Tutorial 1 first (or you refreshed the database before starting this tutorial). Complete Tutorial 1 before you run Tutorial 2 to get the correct results.
4. If you go back to the Test window and view all the source rows, youll note that there
are 8 records for which ALT_CONTACT is null, which matches the number of skipped records.
This is because there were 16 primary contacts and 8 alternate contacts. The CONTACT table only has a CONTACTNAME field, and does not differentiate between the types of contacts. Therefore, your design requires you to create a new record for each contact, whether it is primary or alternate.
5. In addition, as discussed earlier, the ACCOUNTID field has also been inserted.
29
6. And finally, remember that we assigned the GUID function to the CONTACTID for both
the primary (CONTACT) and alternate (CONTACT 2) steps. Look at the records for John Thibideau and Scott Berman. Youll notice that the CONTACTIDs are different, but the ACCOUNTID is the same.
7. Going back to the Test window, you can see that John Thibideau is the primary contact
for George Tel, Inc., and Scott Berman is the alternate. Therefore, the ACCOUNTID has correctly been assigned to both contacts.
Congratulations!
Youve now successfully completed both tutorials! At this point, you should be ready to apply what you learned to your own data integration scenario. Use the Scribe Insight User Guide and the Scribe Insight online Help for reference as you create your own data translation specifications. At any point, you can re-run either tutorial. However, before you do, you need to refresh the sample database to ensure you get the correct results. To reset the sample database:
1. Navigate to the Scribe program folder. 2. Double-click on the InternalDB.exe file to open the Scribe Internal Database
Maintenance Utility.
3. Click on the Sample Database tab. 4. Click Refresh Sample Data to reset the sample database for the tutorials. 5. Click OK on the Refresh Complete dialog box. 6. Click OK to close the window.
30