Professional Documents
Culture Documents
HP Vertica 7.1.x Flex Tables
HP Vertica 7.1.x Flex Tables
Legal Notices
Warranty
The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be
construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.
The information contained herein is subject to change without notice.
Copyright Notice
Copyright 2006 - 2015 Hewlett-Packard Development Company, L.P.
Trademark Notices
Adobe is a trademark of Adobe Systems Incorporated.
Microsoft and Windows are U.S. registered trademarks of Microsoft Corporation.
UNIX is a registered trademark of The Open Group.
Page 2 of 143
Contents
Contents
Getting Started
10
12
14
15
19
Exploration to Promotion
19
20
21
21
21
24
24
Flex Functions
24
25
License Considerations
25
26
26
26
27
28
28
Page 3 of 143
29
31
32
32
33
34
35
37
37
CFSCOPYLOCALStatement
38
39
39
40
40
42
43
44
45
45
46
Loading JSONData
48
48
48
49
51
51
52
Page 4 of 143
52
52
54
54
55
55
55
56
58
Using COMPUTE_FLEXTABLE_KEYS
58
58
60
Using BUILD_FLEXTABLE_VIEW
60
62
64
Using COMPUTE_FLEXTABLE_KEYS_AND_BUILD_VIEW
65
66
67
67
68
70
71
71
72
72
72
73
Page 5 of 143
73
74
75
75
76
79
79
80
BUILD_FLEXTABLE_VIEW
80
COMPUTE_FLEXTABLE_KEYS
84
COMPUTE_FLEXTABLE_KEYS_AND_BUILD_VIEW
86
MATERIALIZE_FLEXTABLE_COLUMNS
87
RESTORE_FLEXTABLE_DEFAULT_KEYS_TABLE_AND_VIEW
89
91
91
Parameters
91
Examples
92
See Also
94
MAPJSONEXTRACTOR
94
Parameters
95
Examples
95
See Also
98
MAPREGEXEXTRACTOR
98
Parameters
98
Examples
98
See Also
99
101
Page 6 of 143
EMPTYMAP
101
MAPAGGREGATE
103
MAPCONTAINSKEY
104
MAPCONTAINSVALUE
106
MAPITEMS
107
MAPKEYS
111
MAPKEYSINFO
112
MAPLOOKUP
115
116
118
119
121
MAPSIZE
122
MAPTOSTRING
124
MAPVALUES
126
MAPVERSION
127
130
130
Parameters
130
Example
131
See Also
132
FDELIMITEDPAIRPARSER
132
Parameters
132
Example
132
See Also
135
FDELIMITEDPARSER
Parameters
135
136
Page 7 of 143
Example
137
See Also
137
FJSONPARSER
137
Parameters
138
Examples
139
FREGEXPARSER
140
Parameters
140
Example
140
See Also
142
143
Page 8 of 143
Getting Started
This section describes the basics of creating, exploring, and using flex tables. The rest of this guide
presents details beyond this basic section, using simple examples.
2. Load the JSON file you saved, using the flex table parser fjsonparser:
=> copy mountains from '/home/dbadmin/Downloads/mountains.json' parser fjsonparser();
Rows Loaded
------------5
(1 row)
Page 9 of 143
Mt St Helens
Denali
(5 rows)
| volcano | 29029
| mountain | 17000
You have now created a flex table and loaded data. Next, learn more about using flex table data in
your database.
Page 10 of 143
00014000Kilimanjaromountain\003\000\000\000\020\000\000\000\026\000\000\000\032\000\0
00\000heightnametype
[ RECORD 5 ]+--------------------------------------------------------------------------------------__identity__ | 5
__raw__
| \001\000\000\000)
\000\000\000\003\000\000\000\020\000\000\000\024\000\000\000!\000\000\
00050.6Mt
Washingtonmountain\003\000\000\000\020\000\000\000\033\000\000\000\037\000\000\000hik
e_safetynametype
2. Use the mapToString() function (with the __raw__ column of mountains) to inspect the table
contents in readable JSONtext format:
=> select MAPTOSTRING(__raw__) from mountains;
MAPTOSTRING
--------------------------------------------------------------------------------------------{
"hike_safety": "50.6",
"name": "Mt Washington",
"type": "mountain"
}
{
"height":
"29029",
"hike_safety": "34.1",
"name": "Everest",
"type": "mountain"
}
{
"height":
"14000",
"hike_safety": "22.8",
"name": "Kilimanjaro",
"type": "mountain"
}
{
"height":
"29029",
"hike_safety": "15.4",
"name": "Mt St Helens",
"type": "volcano"
}
{
"height":
"17000",
"hike_safety": "12.2",
"name": "Denali",
"type": "mountain"
}
Page 11 of 143
3. Use the view_columns system table to query the column_name and data_type columns for
mountains_view:
Page 12 of 143
Notice the data_type column, its values and sizes. These are calculated when you
compute keys for your flex table with compute_flextable_keys().
Did you also notice the data_type_guess column when you queried the mountains_keys
table after invoking that function?
5. With the data_type information from mountains_view, override the data_type_guess for
hike_safety. Then, COMMIT the change, and rebuild the view with build_flextable_view():
=> update mountains_keys set data_type_guess = 'float' where key_name = 'hike_
safety';
OUTPUT
-------1
(1 row)
=> commit;
COMMIT
=> select build_flextable_view('mountains');
build_flextable_view
-----------------------------------------------------The view public.mountains_view is ready for querying
(1 row)
6. Next, use the view_columns system table. Notice that hike_safety is now cast to a float data
type:
=> select column_name, data_type from view_columns where table_name = 'mountains_
view';
column_name | data_type
-------------+------------hike_safety | float
name
| varchar(26)
type
| varchar(20)
height
| varchar(20)
(4 rows)
Page 13 of 143
3. Query the mountains_hybrid keys table. Notice the data_type_guesses column values
again. The types list the column definitions you declared when you created the hybrid table:
=> select * from mountains_hybrid_keys;
key_name
| frequency | data_type_guess
-------------+-----------+----------------height
|
4 | varchar(20)
name
|
5 | varchar(41)
type
|
5 | varchar(20)
hike_safety |
5 | float
(4 rows)
Page 14 of 143
You can create a hybrid table later if you create a basic flex table and later find you want to promote
one or more columns. See
2. Since you specified three (3) columns to materialize, but the table was created with two real
columns (name and hike_safety), the function promotes only one other column, type. The
example has expanded display to display the columns vertically. Notice the ADDED status for
the column that was just materialized, rather than EXISTS for the two columns you defined
when creating the table:
=> \x
Expanded display is on.
=> select * from materialize_flextable_columns_results where table_name = 'mountains_
hybrid';
-[ RECORD 1 ]-+------------------------------------------------------table_id
| 45035996273766044
table_schema | public
table_name
| mountains_hybrid
creation_time | 2013-11-30 20:09:37.765257-05
key_name
| type
status
| ADDED
message
| Added successfully
-[ RECORD 2 ]-+------------------------------------------------------table_id
| 45035996273766044
table_schema | public
table_name
| mountains_hybrid
creation_time | 2013-11-30 20:09:37.765284-05
Page 15 of 143
key_name
| hike_safety
status
| EXISTS
message
| Column of same name already exists in table definition
-[ RECORD 3 ]-+------------------------------------------------------table_id
| 45035996273766044
table_schema | public
table_name
| mountains_hybrid
creation_time | 2013-11-30 20:09:37.765296-05
key_name
| name
status
| EXISTS
message
| Column of same name already exists in table definition
3. Now list the hybrid table definition, listing the __raw__ column and the three materialized
columns. Flex table data types are derived from the associated keys tables, so you can update
them as necessary. Notice that the __raw__ column has a default NOT NULL constraint:
=> \d mountains_hybrid
List of Fields by Tables
-[ RECORD 1 ]-----------------------------------------------------------Schema
| public
Table
| mountains_hybrid
Column
| __raw__
Type
| long varbinary(130000)
Size
| 130000
Default
|
Not Null
| t
Primary Key | f
Foreign Key |
-[ RECORD 2 ]-----------------------------------------------------------Schema
| public
Table
| mountains_hybrid
Column
| name
Type
| varchar(41)
Size
| 41
Default
| (MapLookup(mountains_hybrid.__raw__, 'name'))::varchar(41)
Not Null
| f
Primary Key | f
Foreign Key |
-[ RECORD 3 ]-----------------------------------------------------------Schema
| public
Table
| mountains_hybrid
Column
| hike_safety
Type
| float
Size
| 8
Default
| (MapLookup(mountains_hybrid.__raw__, 'hike_safety'))::float
Not Null
| f
Primary Key | f
Foreign Key |
-[ RECORD 4 ]-----------------------------------------------------------Schema
| public
Table
| mountains_hybrid
Column
| type
Type
| varchar(20)
Page 16 of 143
Size
Default
Not Null
Primary Key
Foreign Key
|
|
|
|
|
20
(MapLookup(mountains_hybrid.__raw__, 'type'))::varchar(20)
f
f
That's it for getting started with flex table basics, hybrid flex tables, and using the flex functions.
Page 17 of 143
Page 18 of 143
In addition to letting you promote data directly from exploration to operation, these features make
HP Vertica flex tables a key part of your data management toolkit:
l
Ability to load different formats into one flex table lets you handle changing structure over time
Extensive SQL queries and built-in analytics for the data you load
Usability functions explore your unstructured data and then use built-in functions to
materialize the data
Exploration to Promotion
Once you create a flex table, you can quickly load data, including social media content in JSON, log
files, delimited files, and other information. Previously, working with such data required significant
schema design and preparation. Now, you can load and query flex tables in a few steps.
Creating flex tables is similar to creating other tables, except column definitions are optional. When
you create flex tables, with or without column definitions, HP Vertica implicitly adds a real column
to your table, called __raw__. This column stores loaded data. The __raw__ column is a LONG
VARBINARY column with a NOT NULLconstraint, and has the documented limits for its data type (see
Long Data Types in the SQL Reference Manual. The __raw__ column's default maximum width is
130,000 bytes (with an absolute maximum of 32,000,000 bytes). You can change the default width
with the FlexTablesRawSize configuration parameter.
Note: The maximum size of each JSONkey within a __raw__ column is 255 bytes. Loading
JSONdata with keys longer than 255 bytes is unsupported.
Page 19 of 143
If you create a flex table without other column definitions, the table also has a second default
column, __identity__, declared as an auto-incrementing IDENTITY (1,1) column. When no other
columns are defined, flex tables use the __identity__ column for segmentation and sort order.
Loading data into a flex table encodes the record into a VMap type, and populates the __raw__
column. The VMap is a standard dictionary type, pairing keys with string values as virtual columns.
Meaning
VMap
An internal map data format used in the __raw__ column after loading data into
a flex table. VMap data can also exist as the return from a flex-related function,
such as EMPTYMAP or MAPJSONEXTRACTOR.
The VMap consists of a varbinary blob containing raw data that is processed
for structure or interpretation during loading or data extraction. After processing,
the VMap data includes a dictionary-style lookup of key::value pairs. The
keys in VMap data are virtual columns (described next).
Virtual
Undeclared columns that have not been promoted (or materialized) to real
columns
columns. Virtual columns, also called map keys, exist within the __raw__
column of a flex table after loading flex data. They also exist in VMap data
returned from certain flex functions. Virtual columns act as lookups into the
VMap data. While you can query values that exist within virtual columns, you
cannot list virtual columns if you list a flex table with \d flextable.
Restrictions also exist on manipulating virtual columns, as described in
Altering Flex Tables.
Real
columns
Promoted Virtual columns that have been materialized to real, fully-featured columns.
columns
Page 20 of 143
Term
Meaning
Map keys
Within VMap data, Map keys are the virtual column names. For example, by
loading data from a tweet file (JSON) into a FlexTable darkdata, one of the
map keys is "user.lang", which you can query directly:
select "user.lang" from darkdata;
If you do not know what keys exist in the map data, use the Flex data and map
functions to compute them. See the Flex Map Functions Reference and Flex
Data Functions Reference sections.
Page 21 of 143
The flex table has two columns automatically, __raw__ and __identity__. The __raw__ column
exists in every flex table to hold the data you load. The __identity__ column is autoincrementing. HP Vertica uses this column for segmentation and sort order when no other column
definitions exist. The flex keys table (darkdata_keys) has three columns, as shown:
=> select * from darkdata;
__identity__ | __raw__
--------------+--------(0 rows)
=> select * from darkdata_keys;
key_name | frequency | data_type_guess
----------+-----------+----------------(0 rows)
Creating a flex table with column definitions (darkdata1 in the next example) also creates a table
with the __raw__ column, but not an __identity__ column, since specified columns can be used
for segmentation and sort order. Two tables are created automatically, as shown in the following
example:
=> create flex table darkdata1 (name varchar);
CREATE TABLE
Page 22 of 143
Creating a flex table with at least one column definition (darkdata1 in the next example) also
creates a table with the __raw__ column, but not an __identity__ column. The specified columns
are used for segmentation and sort order. Two tables are also created automatically, as shown in
the following example:
=> create flex table darkdata1 (name varchar);
CREATE TABLE
=> \d darkdata1*
List of Fields by Tables
Schema |
Table
| Column |
Type
| Size | Default | Not Null |
Primary Key | Foreign Key
--------+-----------+---------+------------------------+--------+---------+----------+------------+------------public | darkdata1 | __raw__ | long varbinary(130000) | 130000 |
| t
| f
|
public | darkdata1 | name
| varchar(80)
|
80 |
| f
| f
|
(2 rows)
=> \dt darkdata1*
List of tables
Schema |
Name
| Kind | Owner | Comment
--------+----------------+-------+---------+--------public | darkdata1
| table | dbadmin |
public | darkdata1_keys | table | dbadmin |
(2 rows)
Page 23 of 143
Flex Functions
There are three sets of functions to support flex tables and extracting data into VMaps. See the
following sections for more information:
Page 24 of 143
License Considerations
Flex tables are licensed under a Flex Zone license. You can purchase a Flex Zone license as a
standalone product, or in addition to your HP Vertica Enterprise Edition (EE) license. For more
information, see the Managing Licenses section information in the Administrator's Guide.
Page 25 of 143
Selecting from the table before loading any data into it reveals its two real columns, __identity__
and __raw__:
=> select * from darkdata;
__identity__
| __raw__
--------------+--------(0 rows)
Selecting from this table before loading data lists the default __raw__ column, followed by n
columns you defined. No __identity__ column exists, because other columns you specify can be
used for segmentation and sorting:
=> select * from darkdata1;
__raw__ | name
---------+-----(0 rows)
Page 26 of 143
Once flex tables exist, you can add new columns (including those with default derived
expressions), as described in Altering Flex Tables.
GLOBAL TEMP flex tables are supported. Creating a temporary global flex table results in the
flextable_keys table and the flextable_view having temporary table restrictions for their
content.
LOCAL TEMP flex tables must include at least one column definition. The reason for this is that
sequence objects (such as the flex table default __identity__ column) are not supported for
local temp tables. Creating a temporary local flex table results in the flextable_keys table and
the flextable_view existing in the local temporary object scope.
LOCAL TEMP views are supported for flex and columnar temporary tables.
For global local temp flex tables to function correctly, you must also specify the ON COMMIT
PRESERVE ROWS clause. The ON COMMITclause is required for the flex table helper functions, which
rely on commits. Create a local temp table as follows:
=> create flex local temp table good(x int) ON COMMIT PRESERVE ROWS;
CREATE TABLE
After creating a local temporary flex table in this way, you can then load data into the table, create
table keys, and a flex table view, as follows:
=> copy good from '/home/release/KData/bake.json' parser fjsonparser();
Rows Loaded
------------1
(1 row)
=> select compute_flextable_keys_and_build_view('good');
compute_flextable_keys_and_build_view
--------------------------------------------------------------------------------------------Please see v_temp_schema.good_keys for updated keys
The view good_view is ready for querying
(1 row)
However, creating temporary flex tables without an ON COMMIT PRESERVE ROWS clause results in
the following warnings when you create the flex table:
=> create flex local temp table bak1(id int, type varchar(10000), name varchar(1000));
Page 27 of 143
WARNING 5860: Due to the data isolation of temp tables with an on-commit-delete-rows
policy,
the compute_flextable_keys() and compute_flextable_keys_and_build_view() functions cannot
access this
table's data. The build_flextable_view() function can be used with a user-provided keys
table to create
a view, but involves a DDL commit which will delete the table's rows
CREATE TABLE
After loading data into a such a temporary flex table, computing keys or building a view for the flex
table results in the following error:
=> select compute_flextable_keys('bak1');
ERROR 5859: Due to the data isolation of temp tables with an on-commit-delete-rows
policy, the compute_flextable_keys() and
compute_flextable_keys_and_build_view() functions cannot access this table's data
HINT: Make the temp table ON COMMIT PRESERVE ROWS to use this function
Page 28 of 143
For example, you can create a columnar table from a flex table darkdata, selecting two virtual
columns (user.lang and user.name) for the new table. Use a command such as the following,
which casts both columns to varchars for the new table:
=> create table darkdata_full as select "user.lang"::varchar, "user.name"::varchar from
darkdata;
CREATE TABLE
=> select * from darkdata_full;
user.lang |
user.name
-----------+--------------------en
| Avita Desai
en
| The End
en
| Uptown gentleman.
en
| ~G A B R I E L A
es
| Flu Beach
es
| I'm Toaster
it
| laughing at clouds.
tr
| seydo shi
|
|
|
|
(12 rows)
As with other flex tables, creating an external flex table produces two regular tables: the named
table and its associated _keys table. The keys table is not an external table:
=> \dt mountains
List of tables
Schema |
Name
| Kind | Owner | Comment
--------+-----------+-------+---------+--------public | mountains | table | release |
(1 row)
Page 29 of 143
1. Check the keys from the _keys table for the results of running the helper application:
=> select * from appLog_keys;
key_name
| frequency |
data_type_
guess
----------------------------------------------------------+-----------+----------------contributors
|
8 | varchar(20)
coordinates
|
8 | varchar(20)
created_at
|
8 | varchar(60)
entities.hashtags
|
8 | long varbinary
(186)
.
.
.
retweeted_status.user.time_zone
|
1 | varchar(20)
retweeted_status.user.url
|
1 | varchar(68)
retweeted_status.user.utc_offset
|
1 | varchar(20)
retweeted_status.user.verified
|
1 | varchar(20)
(125 rows)
Note: While external tables are fully supported for both flex and columnar tables, using
external flex (or columnar) tables is less efficient than using their resident counterparts, whose
data is stored in the HP Vertica database. Any data maintained externally must be loaded each
time you query such data.
Page 30 of 143
Page 31 of 143
FCEFPARSER
FDELIMITEDPAIRPARSER
FDELIMITEDPARSER
FJSONPARSER
FREGEXPARSER
All flex parsers store the data as a single-value VMap in the VARBINARY __raw__ column, which is
a real column with a NOT NULLconstraint. The VMap is encoded into a single binary value for
storage in the __raw__ column. The encoding places the value strings in a contiguous block,
followed by the key strings. Null values within the VMap are supported for keys with NULL-specified
columns. The key and value strings are the virtual columns and their values in your flex table.
If a flex table data row is too large to fit in the VARBINARY __raw__ column, it is rejected. By default,
the rejected data and exceptions files are stored in the standard CopyErrorLogs location, a
subdirectory of the catalog directory:
v_mart_node003_catalog\CopyErrorLogs\trans-STDIN-copy-from-exceptions.1
v_mart_node003_catalog\CopyErrorLogs\trans-STDIN-copy-rejections.1
Flex tables do not copy any rejected data, due to disk space considerations. The rejected data file
exists, but contains only a new line character for every rejected record. The corresponding
exceptions file lists the reason why records were rejected.
To specify a different path and file for the rejected data and exceptions files, use the COPY
parameters REJECTED DATA and EXCEPTIONS, respectively. You can also save load rejections and
exceptions in a table, as described in the Bulk Loading section of the Administrator's Guide
(Capturing Load Rejections and Exceptions).
Page 32 of 143
Rows Loaded
------------12
(1 row)
Note: You can use many additional COPY parameters as required, but not all are supported.
Loads the keys and values as part of the VMap data in the __raw__ column
Automatically populates real columns with the values from their virtual column counterparts
Note: Subsequent data loads continue loading same-name key-value pairs into both the __
raw__ column and the real column. Over time, storing values in both column types could
impact your licensed data limits. For more information about Enterprise Edition and Flex Zone
licenses, see Managing Licenses in the Administrator's Guide.
For example, continuing with the JSON data:
1. Create a flex table, darkdata1, with a column definition of one of the keys in the data you will
load:
=> create flex table darkdata1 ("user.lang" varchar);
CREATE TABLE
3. Query the user.lang column of darkdata1. Loading the JSON data file populated the column
Page 33 of 143
you defined:
=> select "user.lang" from darkdata1;
user.lang
----------es
es
tr
it
en
en
en
en
(12 rows)
Empty column rows indicate NULL values. For more information about how NULLs are
handled in flex table, see ().
4. You can query for other virtual columns (such as "user.name" in darkdata1), with similar
results as for "user.lang":
=> select "user.name" from darkdata1;
user.name
--------------------I'm Toaster
Flu Beach
seydo shi
The End
Uptown gentleman.
~G A B R I E L A
Avita Desai
laughing at clouds.
(12 rows)
Note: While the results for these two queries are similar, the difference in accessing the
keys and their values is significant. Data for "user.lang" has been materialized into a real
table column, while "user.name" remains a virtual column. For production-level data
usage (rather than test data sets), materializing flex table data improves query
performance significantly.
Page 34 of 143
the next example, the user.lang column declares another virtual column (user.name) as its default
value:
=> create flex table table darkdata1 ("user.lang" long varchar default "user.name");
When you load data into your flex table, COPY uses values from the flex table data, ignoring the
default column definition. Loading data into a flex table uses MAPLOOKUP to find keys that match
any real column names. If a match exists (the incoming data has a virtual column with the same
name as a real column), COPYpopulates the column with values. COPYreturns either a value or
NULL for each row, so real columns always have values.
For example, after creating the darkdata1 flex table, described above, load data with COPY:
=> copy darkdata1 from '/test/vertica/flextable/DATA/tweets_12.json' parser fjsonparser
();
Rows Loaded
------------12
(1 row)
Querying the darkdata1 table after loading data shows that the values for the user.lang column
were extracted from the data being loaded (values for the user.lang virtual column, or with NULL for
rows without values). The table column default value for user.lang was ignored:
=> select "user.lang" from darkdata1;
user.lang
----------it
en
es
en
en
es
tr
en
(12 rows)
Page 35 of 143
Rows Loaded
------------12
(1 row)
=> select "user.lang" from darkdata1;
user.lang
--------------------laughing at clouds.
Avita Desai
I'm Toaster
Uptown gentleman.
~G A B R I E L A
Flu Beach
seydo shi
The End
(12 rows)
A different behavior exists if you specify default values when adding columns, as described in
Altering Flex Tables. For more information about using COPY, its expressions and parameters, see
Bulk Loading Data in the Administrator's Guide and COPY in the SQL Reference Manual.
Page 36 of 143
After you configure CFS and connect it to your HP Vertica database, the connector monitors the
repository for changes and deletions to loaded documents, and for new files not previously added to
the server. CFSthen updates its server destinations automatically.
To achieve the best query results with ongoing CFSupdates and deletes, Hewlett-Packard
recommends using live aggregate projections and top-K projections. For more information about
how these projections work, and for examples of using them, see Working withProjections in the
Administrator's Guide.
Page 37 of 143
[vertica]
IndexerType = Library
ConnectionString
=Driver=Vertica;Server=123.456.478.900;Database=myDB;UID=fjones;PWD=fjones_password
TableName = marcomm.myFlexTable
LibraryDirectory = ./shared_library_indexers
LibraryName = verticaIndexer
For more information about ODBCconnection parameters, see ODBC Configuration Parameters.
CFSCOPYLOCALStatement
CFS first indexes and processes metadata from a document repository to add to your database.
Then, CFSuses the Indexing information you added to the configuration file to create an
ODBCconnection. After establishing a connection, CFS generates a standard COPY LOCAL
statement, specifying the fjsonparser. CFSloads data directly into your pre-existing flex table
with a statement such as the following:
=> COPY myFlexTable FROM LOCAL path_to_compressed_temporary_json_file parser fjsonparser
();
When your IDOL metadata appears in a flex table, you can optionally add new table columns, or
materialize other data, as described in Altering Flex Tables.
Page 38 of 143
Loading JSONData
2. Enter some JSONvalues using the fjsonparser(), and query the results:
=> copy super from stdin parser fjsonparser();
Enter data to be copied followed by a newline.
End with a backslash and a period on a line by itself.
>> {"age": 5, "name": "Tim"}
>> {"age": 3}
>> {"name": "Fred"}
>> {"name": "Bob", "age": 10}
>> \.
=> select * from super;
age | name
-----+-----| Fred
10 | Bob
5 | Tim
3 |
(4 rows)
Page 39 of 143
Notice that both JSON data and delimited data are saved in the same columnar table, super.
Page 40 of 143
2. Load the sample CEF file, using the flex parser fcefparser:
=> copy logs from '/home/dbadmin/data/CEF_sample.cef' parser fcefparser();
Rows Loaded
------------1
(1 row)
3. Use the maptostring()function to see the contents of the logs flex table:
=> select maptostring(__raw__) from logs;
maptostring
------------------------------------------------------------------------------------{
"_cefver" : "0.1",
"agentzoneuri" : "/All Zones/ArcSight System/Private Address
Space Zones/RFC1918: 888.99.0.0-888.200.255.255",
"agt" : "888.99.100.1",
"ahost" : "SKEELES10",
"aid" : "3DxKlG0UBABCAA0cXXAZIwA==",
"art" : "1396328241038",
"at" : "testalertng",
"atz" : "Australia/Sydney",
"av" : "6.0.3.6664.0",
"c6a4" : "fe80:0:0:0:495d:cc3c:db1a:de71",
"c6a4label" : "Agent IPv6 Address",
"cat" : "/Agent/Started",
"catdt" : "Security Mangement",
"categorybehavior" : "/Execute/Start",
Page 41 of 143
"categorydevicegroup" : "/Application",
"categoryobject" : "/Host/Application/Service",
"categoryoutcome" : "/Success",
"categorysignificance" : "/Normal",
"cs2" : "<Resource ID=\"3DxKlG0UBABCAA0cXXAZIwA==\"/>",
"cs2label" : "Configuration Resource",
"deviceproduct" : "ArcSight",
"deviceseverity" : "Warning",
"devicevendor" : "ArcSight",
"deviceversion" : "6.0.3.6664.0",
"devicezoneuri" : "/All Zones/ArcSight System/Private Address Space
Zones/RFC1918: 888.99.0.0-888.200.255.255",
"dtz" : "Australia/Sydney",
"dvc" : "888.99.100.1",
"dvchost" : "SKEELES10",
"eventid" : "1",
"filetype" : "Agent",
"mrt" : "1396328238973",
"name" : "Agent [test] type [testalertng] started",
"rt" : "1396328238937",
"severity" : "Low",
"signatureid" : "agent:030",
"version" : "0"
}
(1 row)
2. Load the sample file into col_logs as you did for the flex table:
=> copy col_logs from '/home/dbadmin/data/CEF_sample.cef' parser fcefparser();
Page 42 of 143
Rows Loaded
------------1
(1 row)
3. Query the table. You can find the identical information in the flex table output.
=> \x
Expanded display is on.
VMart=> select * from col_logs;
-[ RECORD 1 ]-+---------------------------------------version
| 0
devicevendor | ArcSight
deviceproduct | ArcSight
deviceversion | 6.0.3.6664.0
signatureid
| agent:030
name
| Agent [test] type [testalertng] started
severity
| Low
2. Query the logs_keys table to see what the function computed from the sample CEF data:
=> select * from logs_keys;
key_name
| frequency | data_type_guess
----------------------+-----------+----------------c6a4
|
1 | varchar(60)
c6a4label
|
1 | varchar(36)
categoryobject
|
1 | varchar(50)
categoryoutcome
|
1 | varchar(20)
categorysignificance |
1 | varchar(20)
cs2
|
1 | varchar(84)
Page 43 of 143
cs2label
deviceproduct
deviceversion
devicezoneuri
dvchost
version
ahost
art
at
cat
catdt
devicevendor
dtz
dvc
filetype
mrt
_cefver
agentzoneuri
agt
aid
atz
av
categorybehavior
categorydevicegroup
deviceseverity
eventid
name
rt
severity
signatureid
(36 rows)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
varchar(44)
varchar(20)
varchar(24)
varchar(180)
varchar(20)
varchar(20)
varchar(20)
varchar(26)
varchar(22)
varchar(28)
varchar(36)
varchar(20)
varchar(32)
varchar(24)
varchar(20)
varchar(26)
varchar(20)
varchar(180)
varchar(24)
varchar(50)
varchar(32)
varchar(24)
varchar(28)
varchar(24)
varchar(20)
varchar(20)
varchar(78)
varchar(26)
varchar(20)
varchar(20)
Page 44 of 143
2. Use the delimiter=',' parameter when loading some CEF data into the table:
=> copy CEFData3 from stdin parser fcefparser(delimiter=',');
Enter data to be copied followed by a newline.
End with a backslash and a period on a line by itself.
>> eventId=1,location=California
>> eventId=2,location=New Mexico
>> eventId=3,location=Arizona
>> \.
Use fdelimitedpairparser when the data specifies column names with the data in each row.
Use fdelimitedparser when the data does not specify column names, or has a header row for
column names.
This section describes using some options that the fdelimited parser supports.
Page 45 of 143
Note: For the fdelimitedparser, the entire load is rejected if any duplicate is found. The
exceptions file contains the reason for the rejection. The rejected data file includes the
duplicate record that caused the load to fail.
Has a key that cannot be coerced into the materialized column's data type
Suppose the flex table has a materialized column, OwnerPercent, declared as a FLOAT. Trying to
load a row with an OwnerPercent key that has a VARCHARvalue causes fdelimitedparserto
reject the data row.
The following examples illustrate setting this parameter.
1. Create a table, reject_true_false, with two real columns:
=> create flex table reject_true_false(one varchar, two int);
CREATE TABLE
Page 46 of 143
2. Load JSON data into the table (from STDIN), using the fjsonparser with reject_on_
materialized_type_error=false. While false is the default value, the following example
specifies it explicitly for illustrative purposes:
=> copy reject_true_false from stdin parser fjsonparser(reject_on_materialized_type_
error=false);
Enter data to be copied followed by a newline.
End with a backslash and a period on a line by itself.
>> {"one": 1, "two": 2}
>> {"one": "one", "two": "two"}
>> {"one": "one", "two": 2}
>> \.
Page 47 of 143
6. Call maptostring to display the table contents, noting only two rows were loaded, rather than
three from the previous results:
=> select maptostring(__raw__), one, two from reject_true_false;
maptostring
| one | two
---------------------------------------+-----+----{
"one" : "1",
"two" : "2"
}
| 1
|
2
{
"one" : "one",
"two" : "2"
}
| one |
2
(2 rows)
Loading JSONData
You can load JSONdata into flex or columnar tables. This section describes some examples of
using the fjsonparser with several of the parser options.
"id_str": "257
Note: The maximum size of each JSONkey within a __raw__ column is 255 bytes. Loading
JSONdata with keys longer than 255 bytes is unsupported.
Default
Change Default
Page 48 of 143
flatten_maps
flatten_maps=FALSE
flatten_arrays
flatten_arrays=TRUE
You control the default the behavior by using one or both flatten_ parameters.
For JSONmaps, the parser flattens all submaps, separating the levels with a period (.). Consider
the following input data with a submap:
{ grade: { level: 4 } }
Note: Using the parameters flatten_maps and flatten_arrays is recursive, and affects all
data.
Page 49 of 143
2. Load alphanums.json into start_json using the fjsonparser without any parameters:
=> COPY start_json FROM '/home/dbadmin/data/flex/alphanums.json' PARSER fjsonparser
();
Rows Loaded
------------1
(1 row)
4. Truncate the table start_json and load alphanums.json with the start_point parameter:
=> TRUNCATE TABLE start_json;
TRUNCATE TABLE
=> COPY start_json FROM '/home/dbadmin/data/flex/alphanums.json' PARSER fjsonparser
(start_point='B');
Rows Loaded
------------1
(1 row)
5. Next, call maptostring again to compare the results of loading alphanums.jsonfrom start_
point='B' with the initial test:
Page 50 of 143
The fjsonparser ignores the first two start_point keys and loads data from the third occurrence.
Parsing continues from the third start_point key to the end of the start_point value. The
fjsonparser ignores any subsequent instances of the start_point key, even if that key appears
multiple more times in the input file.
Page 51 of 143
Please see
Page 52 of 143
Has a key that cannot be coerced into the materialized column's data type.
2. Load JSON data into the table (from STDIN), using the fjsonparser with reject_on_
materialized_type_error=false. While false is the default value, the following example
specifies it explicitly for illustration:
=> copy reject_true_false from stdin parser
-> fjsonparser(reject_on_materialized_type_error=false);
Enter data to be copied followed by a newline.
End with a backslash and a period on a line by itself.
>> {"one": 1, "two": 2}
>> {"one": "one", "two": "two"}
>> {"one": "one", "two": 2}
>> \.
5. Reload the same data again, but this time, set reject_on_materialized_type_error=true:
Page 53 of 143
6. Call maptostring to display the table contents. Only two rows were loaded, whereas the
previous results had three rows:
=> select maptostring(__raw__), one, two from reject_true_false;
maptostring
| one | two
---------------------------------------+-----+----{
"one" : "1",
"two" : "2"
}
| 1
|
2
{
"one" : "one",
"two" : "2"
}
| one |
2
(2 rows)
Such rows are invalid for SQL. To prevent this situation, you can control the behavior for empty
rows, either rejecting or omitting them. You do so using two boolean parameters for the parsers
FDELIMITEDPARSER or FJSONPARSER:
l
reject_on_empty_key
omit_empty_keys
Page 54 of 143
2. Use COPY to load parts of a log file using the sample regular expression, with the
fregexparser:
Page 55 of 143
Page 56 of 143
Page 57 of 143
COMPUTE_FLEXTABLE_KEYS
COMPUTE_FLEXTABLE_KEYS_AND_BUILD_VIEW
Using COMPUTE_FLEXTABLE_KEYS
Call this function with a flex table argument to compute a list of keys from the map data:
Calling the function
Results
compute_flextable_keys ('flex_table')
Page 58 of 143
----------------------------------------------------------+-----------+--------------------contributors
|
8 | varchar(20)
coordinates
|
8 | varchar(20)
created_at
|
8 | varchar(60)
entities.hashtags
|
8 | long varbinary
(186)
entities.urls
|
8 | long varbinary
(32)
entities.user_mentions
|
8 | long varbinary
(674)
.
.
.
retweeted_status.user.time_zone
|
1 | varchar(20)
retweeted_status.user.url
|
1 | varchar(68)
retweeted_status.user.utc_offset
|
1 | varchar(20)
retweeted_status.user.verified
|
1 | varchar(20)
(125 rows)
Description
key_name
frequency The number of times the virtual column occurs in the map.
data_
type_
guess
The data type for each virtual column, cast to VARCHAR, LONG VARCHAR or LONG
VARBINARY, depending on the length of the key, and whether the key includes one or
more nested maps.
In the _keys table output, the data_type_guess column values are also followed by
a value in parentheses, such as varchar(20). The value indicates the padded
width of the key column, as calculated by the longest field, multiplied by the
FlexTableDataTypeGuessMultiplier configuration parameter value. For more
information, see Setting Flex Table Parameters.
Page 59 of 143
Two helper functions create views. Because the second function also computes keys, the hint is to
use that one:
l
BUILD_FLEXTABLE_VIEW
COMPUTE_FLEXTABLE_KEYS_AND_BUILD_VIEW
Using BUILD_FLEXTABLE_VIEW
After computing keys for the Flex Table (Computing Flex Table Keys), call this function with one or
more arguments. The records under the key_name column of the {flextable}_keys table are used
as view columns, along with any values that exist for the key, or NULL if they do not.
Regardless of the number of arguments, calling this function replaces the contents of the existing
view as follows:
Page 60 of 143
Function invocation
Results
build_flextable_view ('flexible_table')
If you do not specify a view_name argument, the default name is the flex table name with a _view
suffix. For example, if you specify the table darkdata as the sole argument to this function, the
default view is called darkdata_view.
You cannot specify a custom view name with the same name as the default view flex_table_
view, unless you first do the following:
1. Drop the default-named view
2. Create your own view of the same name
As with all views, creating a view stores a definition of the column structure at the time you create
the view. If you create a flex table view, and then promote virtual columns to real columns, you
must rebuild the view. After rebuilding a flex table view that has newly-promoted real columns,
queries such as select * from myflextable_view; return values from both virtual columns in
the map data and from real columns.
Page 61 of 143
While a {flextable}_keys table can include duplicate key names with different cases, a view
cannot. Creating a flex table view with either of the helper functions consolidates any duplicate key
names to one column name, consisting of all lowercase characters. The values from all duplicate
keys are saved in that column. For example, if these key names exist in a flex table:
l
test
Test
tESt
The view will include a virtual column test with values from the test, Test, and tESt keys.
For example, consider the following query, showing the duplicate test key names:
=> \x
Expanded display is on.
dbt=> select * from dupe_keys;
-[ RECORD 1 ]---+--------------------------------------------------------------------------------------------------------------------------------key_name
|
TesttestTesttestTesttestTesttestTesttestTesttestTesttestTesttestTesttestTesttestTesttestT
esttestTesttestTesttestTesttestTesttest
frequency
| 2
data_type_guess | varchar(20)
-[ RECORD 2 ]---+------------------------------------------------------------------------
Page 62 of 143
---------------------------------------------------------key_name
|
TesttestTesttestTesttestTesttestTesttestTesttestTesttestTesttestTesttestTesttestTesttestT
esttestTesttestTesttestTesttestTest12345
frequency
| 2
data_type_guess | varchar(20)
-[ RECORD 3 ]---+--------------------------------------------------------------------------------------------------------------------------------key_name
| test
frequency
| 8
data_type_guess | varchar(20)
-[ RECORD 4 ]---+--------------------------------------------------------------------------------------------------------------------------------key_name
| TEst
frequency
| 8
data_type_guess | varchar(20)
-[ RECORD 5 ]---+--------------------------------------------------------------------------------------------------------------------------------key_name
| TEST
frequency
| 8
data_type_guess | varchar(20)
The following query displays the dupe flex table (dupe_view), illustrating the consolidated test and
testtesttest... virtual columns, with all of the test, Test, and tESt virtual column values in the
test column:
=> select * from dupe_view;
test |
testtesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttesttestt
esttesttesttesttesttesttesttesttesttest
--------+--------------------------------------------------------------------------------------------------------------------------------upper2 |
half4 |
lower1 |
upper1 |
half1 |
half4 |
|
lower1 |
half1 |
upper2 |
|
lower2 |
lower3 |
upper1 |
lower2 |
lower3 |
(16 rows)
Page 63 of 143
Querying the view shows the key names you specified, and their values:
=> select * from dd_view;
user.lang |
user.name
-----------+--------------------en
| Uptown gentleman.
en
| The End
it
| laughing at clouds.
es
| I'm Toaster
|
en
| ~G A B R I E L A
|
en
| Avita Desai
tr
| seydo shi
|
|
es
| Flu Beach
(12 rows)
This example calls build_flextable_view with the original table, and the view you previously
created, dd_view:
=> select build_flextable_view ('darkdata', 'dd_view');
build_flextable_view
----------------------------------------------The view public.dd_view is ready for querying
(1 row)
Querying the view again shows that the function populated the view with the contents of the
darkdata_keys table. The next example shows a snippet from the results, with the key_name
columns and their values:
=> \x
Expanded display is on.
Page 64 of 143
user.following
user.friends_count
user.geo_enabled
user.id
user.id_str
user.is_translator
user.lang
user.listed_count
user.location
user.name
.
.
.
|
|
|
|
|
|
|
|
|
|
791
F
164464905
164464905
F
en
4
Uptown..
Uptown gentleman.
When building views, it is important to understand that creating a view stores a definition of the
column structure at the time the view is created. If you promote virtual columns to real columns
after building a view, the existing view definition is not changed. Querying this view with a select
statement such as the following, returns values from only the __raw__ column:
=> select * from myflextable_view;
If you rebuild the view after promoting virtual columns, future queries return values from both virtual
columns in the map data and from real columns.
Using COMPUTE_FLEXTABLE_KEYS_AND_BUILD_
VIEW
Call this function with a Flex table to compute Flex table keys (see Computing Flex Table Keys ),
and create a view in one step.
Page 65 of 143
Page 66 of 143
2. Loading data into a materialized column populates the new column automatically:
=> copy darkdata1 from '/vertica/flextable/DATA/tweets_12.json' parser fjsonparser
();
Rows Loaded
------------12
(1 row)
Page 67 of 143
laughing at clouds.
(12 rows)
1. Create a darkdata1 table with some column definition, but a name that does not correspond to
any key names in the JSON data you'll load. Assign a default value for a column you know
exists in your data ("user.lang"):
=> create flex table darkdata1(talker long varchar default "user.lang");
CREATE TABLE
3. Query the talker column to see that the default value was not used. The column contains
NULL values.
4. Load data again, specifying just the __raw__ column to use the column's default value:
=> copy darkdata1 (__raw__)from '/test/vertica/flextable/DATA/tweets_12.json'
parser fjsonparser();
Rows Loaded
------------12
(1 row)
5. Query to see that the column's default expression was used ("user.lang"), because you
Page 68 of 143
specified __raw__:
=> select "talker" from darkdata1;
talker
-------it
en
es
en
en
es
tr
en
(12 rows)
6. Alter the table to add a row with a key name, assigning the name as the default value
(recommended):
=> alter table darkdata1 add column "user.name" varchar default "user.name";
ALTER TABLE
8. Query the two real columns and see that talker is NULL, since you did not specify the __raw_
_ column. The user.lang column contains values from the data you loaded:
=> select "talker", "user.name" from darkdata1;
talker |
user.name
--------+--------------------| laughing at clouds.
| Avita Desai
| I'm Toaster
|
|
|
| Uptown gentleman.
| ~G A B R I E L A
| Flu Beach
|
| seydo shi
| The End
(12 rows)
9. Load data once more, this time specifying a COPY statement default value expression for
Page 69 of 143
user.name:
=> copy darkdata1 (__raw__, "user.name" as 'QueenElizabeth'::varchar) from
'/test/vertica/flextable/DATA/tweets_12.json' parser fjsonparser();
Rows Loaded
------------12
(1 row)
10. Query once more. talker has its default values (you used __raw__), and the COPY value
expression (QueenElizabeth) overrode the user.name default column value:
=> select "talker", "user.name" from darkdata1;
talker |
user.name
--------+---------------it
| QueenElizabeth
en
| QueenElizabeth
es
| QueenElizabeth
| QueenElizabeth
| QueenElizabeth
| QueenElizabeth
en
| QueenElizabeth
en
| QueenElizabeth
es
| QueenElizabeth
| QueenElizabeth
tr
| QueenElizabeth
en
| QueenElizabeth
(12 rows)
To summarize, you can set a default column value as part of the ALTER TABLE...ADD COLUMN...
operation. For materializing columns, the default should reference the key name of the virtual
column (as in "user.lang"). Subsequently loading data with a COPY value expression overrides
the default value of the column definition.
Changing the configuration parameter affects all flex tables you create after making this change.
Page 70 of 143
To change the size of the _raw_ column in an existing flex table, use the ALTERTABLEstatement
as follows:
=> alter table tester alter column __raw__ set data type long varbinary(120000);
ALTER TABLE
Note: An error will occur if you try reducing the __raw__ column size to a value smaller than
the data that the column already contains.
__raw__
__identity__
Yes
Yes
No
Yes
Create projections
No
Yes
Segment
No
Yes
Partition
No
Yes
No
No
Note: While segmenting and partitioning the __raw__ column is permitted, it is not
recommended due to its long data type. By default, if no real columns exist, flex tables are
segmented on the __identity__ column.
You cannot drop the last column in your flex table's sort order.
If you have not created a flex table with any real columns, or materialized any columns, you
cannot drop the __identity__ column.
Page 71 of 143
SELECT
COPY
TRUNCATE
DELETE
You can use SELECT queries for virtual columns that exist in the __raw__ column, and real
columns in your flex tables. Column names are case insensitive.
The maplookup() function searches the VMap data for the requested key and returns the following
information:
l
Page 72 of 143
COMPUTE_FLEXTABLE_KEYS
COMPUTE_FLEXTABLE_KEYS_AND_BUILD_VIEW
For more information about these and other helper functions, see the Flex Data Functions
Reference
To determine what virtual columns exist:
1. Call the function as follows:
=> select compute_flextable_keys('darkdata');
compute_flextable_keys
-------------------------------------------------Please see public.darkdata_keys for updated keys(1 row)
key_name
| frequency |
data_type_guess
-----------------------------------------------------+-----------+------------contributors
|
8 | varchar(20)
coordinates
|
8 | varchar(20)
created_at
|
8 | varchar(60)
entities.hashtags
|
8 | long varbinary
(186)
.
.
retweeted_status.user.time_zone
|
1 | varchar(20)
retweeted_status.user.url
|
1 | varchar(68)
retweeted_status.user.utc_offset
|
1 | varchar(20)
retweeted_status.user.verified
|
1 | varchar(20)
(125 rows)
Page 73 of 143
The following query uses the COUNT and AVG functions to determine the average length of text in
different languages:
=> select "user.lang", count (*), avg(length("text"))::int from darkdata1 group by 1
order by 2 desc;
user.lang | count | avg
-----------+-------+----en
|
4 | 42
|
4 |
Page 74 of 143
es
it
tr
(5 rows)
|
|
|
2 |
1 |
1 |
96
50
16
The next example queries the same virtual column, casting created_at data to a TIMESTAMP,
resulting in different output and the regional time:
=> select "created_at"::TIMESTAMP from darkdata1 order by 1 desc;
created_at
--------------------2012-10-15 14:41:05
2012-10-15 14:41:05
2012-10-15 14:41:05
2012-10-15 14:41:05
2012-10-15 14:41:05
2012-10-15 14:41:05
2012-10-15 14:41:05
2012-10-15 14:41:04
Page 75 of 143
FlexTableRawSize
FlexTableDataTypeGuessMultiplier
COMPUTE_FLEXTABLE_KEYS
COMPUTE_FLEXTABLE_KEYS_AND_BUILD_VIEW
Page 76 of 143
Page 77 of 143
Page 78 of 143
Description
COMPUTE_
FLEXTABLE_KEYS
BUILD_FLEXTABLE_
VIEW
COMPUTE_
FLEXTABLE_KEYS_
AND_BUILD_VIEW
MATERIALIZE_
FLEXTABLE_
specified.
COLUMNS
RESTORE_
FLEXTABLE_
view, linking both the keys table and the view to the parent flex table.
DEFAULT_KEYS_
TABLE_AND_VIEW
While the functions are available to all users, they are applicable only to flex tables, their associated
flex_table_keys table and flex_table_view views. By computing keys and creating views from
flex table data, the functions facilitate SELECT queries. One function restores the original keys
table and view that were made when you first created the flex table.
Page 79 of 143
While both objects are dependent on their parent table, (flextable), you can drop either object
independently. Dropping the parent table removes both dependents, without a CASCADE option.
BUILD_FLEXTABLE_VIEW
Creates, or recreates, a view for a default or user-defined _keys table, ignoring any empty keys.
Syntax
build_flextable_view('flex_table' [ [,'view_name'] [,'user_keys_table'] ])
Arguments
flex_table
view_name
user_keys_
table
Page 80 of 143
Examples
Following are examples of calling build_flextable_view with 1, 2, or 3 arguments.
Page 81 of 143
(1 row)
2. Alternatively, create a table based on the default keys table, but without content:
=> create table new_darkdata_keys as select * from darkdata_keys LIMIT 0;
CREATE TABLE
kdb=> select * from new_darkdata_keys;
key_name | frequency | data_type_guess
----------+-----------+----------------(0 rows)
3. Given an existing table (or creating one with no data), insert one or more keys:
=> create table dd_keys as select * from darkdata_keys limit 0;
CREATE TABLE
=> insert into dd_keys (key_name) values ('user.lang');
Page 82 of 143
OUTPUT
-------1
(1 row)
=> insert into dd_keys (key_name) values ('user.name');
OUTPUT
-------1
(1 row)
=> select * from dd_keys;
key_name | frequency | data_type_guess
-----------+-----------+----------------user.lang |
|
user.name |
|
(2 rows)
See Also
l
COMPUTE_FLEXTABLE_KEYS
COMPUTE_FLEXTABLE_KEYS_AND_BUILD_VIEW
MATERIALIZE_FLEXTABLE_COLUMNS
RESTORE_FLEXTABLE_DEFAULT_KEYS_TABLE_AND_VIEW
Page 83 of 143
COMPUTE_FLEXTABLE_KEYS
Computes the virtual columns (keys and values) from the VMap data of a flex table and repopulates
the associated _keys table. The keys table has the following columns:
l
key_name
frequency
data_type_guess
Syntax
compute_flextable_keys('flex_table')
Arguments
flex_table The name of a flex table.
Examples
During execution, this function determines a data type for each virtual column, casting the values it
computes to VARCHAR, LONG VARCHAR, or LONG VARBINARY, depending on the length of the key, and
whether the key includes nested maps.
The following examples illustrate this function and the results of populating the _keys table, once
you've created a flex table (darkdata1) and loaded data:
=> create flex table darkdata1();
CREATE TABLE
=> copy darkdata1 from '/test/flextable/DATA/tweets_12.json' parser fjsonparser();
Rows Loaded
------------12
(1 row)
=> select compute_flextable_keys('darkdata1');
compute_flextable_keys
-------------------------------------------------Please see public.darkdata1_keys for updated keys
Page 84 of 143
(1 row)
=> select * from darkdata1_keys;
key_name
| frequency |
data_type_guess
----------------------------------------------------------+-----------+--------------------contributors
|
8 | varchar(20)
coordinates
|
8 | varchar(20)
created_at
|
8 | varchar(60)
entities.hashtags
|
8 | long varbinary
(186)
entities.urls
|
8 | long varbinary
(32)
entities.user_mentions
|
8 | long varbinary
(674)
.
.
.
retweeted_status.user.time_zone
|
1 | varchar(20)
retweeted_status.user.url
|
1 | varchar(68)
retweeted_status.user.utc_offset
|
1 | varchar(20)
retweeted_status.user.verified
|
1 | varchar(20)
(125 rows)
Description
key_name
frequency The number of times the virtual column occurs in the map.
data_
type_
guess
The data type for each virtual column, cast to VARCHAR, LONG VARCHAR or LONG
VARBINARY, depending on the length of the key, and whether the key includes one or
more nested maps.
In the _keys table output, the data_type_guess column values are also followed by
a value in parentheses, such as varchar(20). The value indicates the padded
width of the key column, as calculated by the longest field, multiplied by the
FlexTableDataTypeGuessMultiplier configuration parameter value. For more
information, see Setting Flex Table Parameters.
See Also
l
BUILD_FLEXTABLE_VIEW
COMPUTE_FLEXTABLE_KEYS_AND_BUILD_VIEW
Page 85 of 143
MATERIALIZE_FLEXTABLE_COLUMNS
RESTORE_FLEXTABLE_DEFAULT_KEYS_TABLE_AND_VIEW
COMPUTE_FLEXTABLE_KEYS_AND_BUILD_VIEW
Combines the functionality of BUILD_FLEXTABLE_VIEW and COMPUTE_FLEXTABLE_KEYS
to compute virtual columns (keys) from the VMap data of a flex table, and construct a view.
Creating a view with this function ignores empty keys. If you don't need to perform both operations
together, use one of the single-operation functions.
Syntax
compute_flextable_keys_and_build_view('flex_table')
Arguments
flex_table The name of a flex table.
Examples
The following example calls the function for the darkdata flex table.
=> select compute_flextable_keys_and_build_view('darkdata');
compute_flextable_keys_and_build_view
----------------------------------------------------------------------Please see public.darkdata_keys for updated keys
The view public.darkdata_view is ready for querying
(1 row)
See Also
l
BUILD_FLEXTABLE_VIEW
COMPUTE_FLEXTABLE_KEYS
MATERIALIZE_FLEXTABLE_COLUMNS
RESTORE_FLEXTABLE_DEFAULT_KEYS_TABLE_AND_VIEW
Page 86 of 143
MATERIALIZE_FLEXTABLE_COLUMNS
Materializes virtual columns listed as key_names in the flextable_keys table. You can optionally
indicate the number of columns to materialize, and use a keys table other than the default. If you do
not specify the number of columns, the function materializes up to 50 virtual column key names.
Calling this function requires that you first compute flex table keys using either COMPUTE_
FLEXTABLE_KEYS or COMPUTE_FLEXTABLE_KEYS_AND_BUILD_VIEW
Note: Materializing any virtual column into a real column with this function affects data storage
limits. Each materialized column counts against the data storage limit of your HPVertica
Enterprise Edition (EE) license. This increase is reflected when HP Vertica next performs a
license compliance audit. To manually check your EElicense compliance, call the audit()
function, described in the SQL Reference Manual.
Syntax
materialize_flextable_columns('flex_table' [, n-columns [, keys_table_name] ])
Arguments
flex_table
The name of the flex table with columns to materialize. Specifying only
the flex table name attempts to materialize up to 50 columns of key
names in the default flex_table_keys table, skipping any columns
already materialized, and ignoring any empty keys. To materialize a
specific number of columns, use the optional parameter n_columns,
described next.
n-columns
Page 87 of 143
keys_table_name
Examples
The following example loads a sample file of tweets (tweets_10000.json) into the flex table
twitter_r.
After loading data and computing keys for the sample flex table, the example calls materialize_
flextable_columns to materialize the first four columns:
=> copy twitter_r from '/home/release/KData/tweets_10000.json' parser fjsonparser();
Rows Loaded
------------10000
(1 row)
=> select compute_flextable_keys ('twitter_r');
compute_flextable_keys
--------------------------------------------------Please see public.twitter_r_keys for updated keys
(1 row)
=> select materialize_flextable_columns('twitter_r', 4);
materialize_flextable_columns
------------------------------------------------------------------------------The following columns were added to the table public.twitter_r:
contributors
entities.hashtags
entities.urls
For more details, run the following query:
SELECT * FROM v_catalog.materialize_flextable_columns_results WHERE table_schema =
'public' and table_name = 'twitter_r';
(1 row)
Page 88 of 143
---+-------------------+--------+-------------------------------------------------------45035996273733172 | public
| twitter_r | 2013-11-20 17:00:27.945484-05
| contributors
| ADDED | Added successfully
45035996273733172 | public
| twitter_r | 2013-11-20 17:00:27.94551-05
| entities.hashtags | ADDED | Added successfully
45035996273733172 | public
| entities.urls
| ADDED
45035996273733172 | public
| twitter_r | 2013-11-20 17:00:27.945532-05
| created_at
| EXISTS | Column of same name already exists in table definition
(4 rows)
See Also
l
BUILD_FLEXTABLE_VIEW
COMPUTE_FLEXTABLE_KEYS
COMPUTE_FLEXTABLE_KEYS_AND_BUILD_VIEW
RESTORE_FLEXTABLE_DEFAULT_KEYS_TABLE_AND_VIEW
RESTORE_FLEXTABLE_DEFAULT_KEYS_TABLE_
AND_VIEW
Restores the _keys table and the _view, linking them with their associated flex table if either is
dropped. This function notes whether it restores one or both.
Syntax
restore_flextable_default_keys_table_and_view('flex_table')
Arguments
flex_table The name of a flex table .
Page 89 of 143
Examples
This example invokes the function with an existing flex table, restoring both the _keys table and _
view:
=> select restore_flextable_default_keys_table_and_view('darkdata');
restore_flextable_default_keys_table_and_view
---------------------------------------------------------------------------------The keys table public.darkdata_keys was restored successfully.
The view public.darkdata_view was restored successfully.
(1 row)
This example shows the function restoring darkdata_view, but noting that darkdata_keys does
not need restoring:
=> select restore_flextable_default_keys_table_and_view('darkdata');
restore_flextable_default_keys_table_and_view
-----------------------------------------------------------------------------------------------The keys table public.darkdata_keys already exists and is linked to darkdata.
The view public.darkdata_view was restored successfully.
(1 row)
See Also
l
BUILD_FLEXTABLE_VIEW
COMPUTE_FLEXTABLE_KEYS
COMPUTE_FLEXTABLE_KEYS_AND_BUILD_VIEW
MATERIALIZE_FLEXTABLE_COLUMNS
Page 90 of 143
MAPDELIMITEDEXTRACTOR
MAPJSONEXTRACTOR
MAPREGEXEXTRACTOR
Each function accepts input data from existing database content, from a table, returned from an
expression, or entered directly. The functions do not parse data from an external file source. All
functions return a single VMap value. The extractor functions can return data with NULL-specified
columns.
This section describes each extractor function.
MAPDELIMITEDEXTRACTOR
Extracts data with a delimiter character, and other optional arguments, returning a single VMap
value. The USING PARAMETERS phrase specifies optional parameters for the function.
Parameters
delimiter
VARCHAR
header
VARCHAR
trim
BOOLEAN
treat_empty_
val_as_null
BOOLEAN
Page 91 of 143
Examples
For this set of examples, we are using a short set of delimited data to load into a table with
mapdelimitedextractor:
Name|CITY|New city|State|zip
Tom|BOSTON|boston|MA|01
Eric|Burlington|BURLINGTON|MA|02
Jamie|cambridge|CAMBRIDGE|MA|08
2. Use COPY to load the delim.dat file. Use the flex tables fdelimitedparser with the
header='false' option:
=> copy dflex from '/home/release/kmm/flextables/delim.dat' parser fdelimitedparser
(header='false');
Rows Loaded
------------4
(1 row)
3. Create a columnar table, dtab, with an identity id column, a delim column, and a column to
hold a VMap, called vmap:
=> create table dtab (id IDENTITY(1,1), delim varchar(128), vmap long varbinary(512));
CREATE TABLE
4. Use COPY to load the delim.dat file into the dtab table, using the mapdelimitedextractor
function, specifying delimiter '!' in this example:
=> copy dtab(delim, vmap as mapdelimitedextractor(delim)) from
'/home/release/kmm/flextables/delim.dat'
delimiter '!';
Rows Loaded
------------4
(1 row)
Page 92 of 143
5. Use the maptostring function for the flex table dflex to view the __raw__ column contents as
strings:
=> select maptostring(__raw__) from dflex limit 10;
maptostring
------------------------------------------------------------------------------------{
"ucol0" : "Jamie",
"ucol1" : "cambridge",
"ucol2" : "CAMBRIDGE",
"ucol3" : "MA",
"ucol4" : "08"
}
{
"ucol0"
"ucol1"
"ucol2"
"ucol3"
"ucol4"
:
:
:
:
:
"Name",
"CITY",
"New city",
"State",
"zip"
"ucol0"
"ucol1"
"ucol2"
"ucol3"
"ucol4"
:
:
:
:
:
"Tom",
"BOSTON",
"boston",
"MA",
"01"
"ucol0"
"ucol1"
"ucol2"
"ucol3"
"ucol4"
:
:
:
:
:
"Eric",
"Burlington",
"BURLINGTON",
"MA",
"02"
}
{
}
{
}
(4 rows)
6. Use the maptostring function again, this time with the dtab table's vmap column. Compare
the results of this output, and that from the flex table:
=> select maptostring(vmap) from dtab limit 10;
maptostring
----------------------------------------------------------------------------------{
"ucol0" : "Tom",
"ucol1" : "BOSTON",
"ucol2" : "boston",
"ucol3" : "MA",
"ucol4" : "01"
}
Page 93 of 143
{
"ucol0"
"ucol1"
"ucol2"
"ucol3"
"ucol4"
:
:
:
:
:
"Name",
"CITY",
"New city",
"State",
"zip"
"ucol0"
"ucol1"
"ucol2"
"ucol3"
"ucol4"
:
:
:
:
:
"Eric",
"Burlington",
"BURLINGTON",
"MA",
"02"
"ucol0"
"ucol1"
"ucol2"
"ucol3"
"ucol4"
:
:
:
:
:
"Jamie",
"cambridge",
"CAMBRIDGE",
"MA",
"08"
}
{
}
{
}
(4 rows)
See Also
l
MAPJSONEXTRACTOR
MAPREGEXEXTRACTOR
MAPJSONEXTRACTOR
Extracts content of repeated JSON data objects, including nested maps, or data with an outer list of
JSON elements. The USING PARAMETERS phrase specifies optional parameters for the function.
Empty input does not generate a Warning or Error.
Note: The maximum size of each JSONkey within a __raw__ column is 255 bytes. Loading
JSONdata with keys longer than 255 bytes is unsupported.
Page 94 of 143
Parameters
flatten_maps
BOOLEAN
flatten_
arrays
BOOLEAN
reject_on_
duplicate
BOOLEAN
reject_on_
empty_key
BOOLEAN
omit_empty_
keys
BOOLEAN
[Optional] Omits any key from the load data that does not
have a value (omit_empty_keys=true).
Default value: false
start_point
CHAR
Examples
These examples use the following sample JSONdata:
{
{
{
{
{
"id":
"id":
"id":
"id":
"id":
"5001",
"5002",
"5005",
"5007",
"5004",
"type":
"type":
"type":
"type":
"type":
"None" }
"Glazed" }
"Sugar" }
"Powdered Sugar" }
"Maple" }
Page 95 of 143
2. Use COPY to load the bake_single.json file with the flex tables fjsonparser parser:
=> copy flexjson from '/home/dbadmin/data/bake_single.json' parser fjsonparser();
Rows Loaded
------------5
(1 row)
3. Create a columnar table, coljson, with an identity id column, a json column, and a column to
hold a VMap, called vmap:
=> create table coljson(id IDENTITY(1,1), json varchar(128), vmap long varbinary
(10000));
CREATE TABLE
4. Use COPY to load the bake_single.json file into the coljson table, using the
mapjsonextractor function:
=> copy coljson (json, vmap AS MapJSONExtractor(json)) FROM '/home/dbadmin/data/bake_
single.json';
Rows Loaded
------------5
(1 row)
5. Use the maptostring function for the flex table flexjson to output the __raw__ column
contents as strings:
=> select maptostring(__raw__) from flexjson limit 5;
maptostring
----------------------------------------------------{
"id" : "5001",
"type" : "None"
}
{
"id" : "5002",
"type" : "Glazed"
}
Page 96 of 143
{
"id" : "5005",
"type" : "Sugar"
}
{
"id" : "5007",
"type" : "Powdered Sugar"
}
{
"id" : "5004",
"type" : "Maple"
}
(5 rows)
6. Use the maptostring function again, this time with the coljson table's vmap column and
compare the results. The element order differs:
=> select maptostring(vmap) from coljson limit 5;
maptostring
----------------------------------------------------{
"id" : "5001",
"type" : "None"
}
{
"id" : "5002",
"type" : "Glazed"
}
{
"id" : "5004",
"type" : "Maple"
}
{
"id" : "5005",
"type" : "Sugar"
}
{
"id" : "5007",
"type" : "Powdered Sugar"
}
(5 rows)
Page 97 of 143
See Also
l
MAPDELIMITEDEXTRACTOR
MAPREGEXEXTRACTOR
MAPREGEXEXTRACTOR
Extracts data results from a regular expression and returns the resulting VMap. Use the USING
PARAMETERS phrase to specify the regular expression.
Parameters
pattern=
VARCHAR
use_jit
BOOLEAN
record_terminator
VARCHAR
logline_column
VARCHAR
Examples
These examples use the following regular expression to search an HP Vertica log file for
information that includes the timestamp, date, thread_name, and thread_id.Note that this
regular expression has embedded newlines for illustrative and display purposes. To use this
expression, first remove the newline characters.
^(?<time>\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d\.\d+)(?<thread_name>[A-Za-z ]+):
(?<thread_id>0x[0-9a-f]+)-?(?<transaction_id>[0-9a-f]+)?(?:\[(?<component>\w+)\]
<(?<level>\w+)> )?(?<(?<elevel>\w+)> @\[?(?<enode>\w+)\]?: )?(?<text>.*)
Page 98 of 143
2. Use COPY to load a sample log file, using the flex tables fregexparser with the
header='false' option. This example has embedded newline characters for display
purposes:
=> copy flogs from '/home/dbadmin/data/vertica.log' PARSER fregexparser(pattern=
'^(?<time>\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d\.\d+) (?<thread_name>[A-Za-z ]+):
(?<thread_id>0x[0-9a-f]+)-?(?<transaction_id>[0-9a-f]+)?
(?:\[(?<component>\w+)\] <(?<level>\w+)> )?(?:<(?<elevel>\w+)>
@\[?(?<enode>\w+)\]?: )?(?<text>.*)');
Rows Loaded
------------8434
(1 row)
3. Select some of the elements from the regular expression that returned data in the flogs flex
table:
4. Create a columnar table
=> SELECT component, level, elevel, enode, text FROM flogs order by 1 limit 10;
component | level | elevel | enode |
text
-----------+-------+--------+-------+-------------------------------------------Catalog
| INFO |
|
| Checking for missed restore table events
Catalog
| INFO |
|
| Found no missed restore table events
Catalog
| INFO |
|
| Checking for missed alter partition events
Catalog
| INFO |
|
| Checking for missed replace node events
Catalog
| INFO |
|
| Checking for missed replace node events
Catalog
| INFO |
|
| Checking for missed replace node events
Catalog
| INFO |
|
| Checking for missed alter partition events
Catalog
| INFO |
|
| Checking for missed restore table events
Catalog
| INFO |
|
| Found no missed alter partition events
Catalog
| INFO |
|
| Found no missed replace node events
(10 rows)
See Also
l
MAPDELIMITEDEXTRACTOR
MAPJSONEXTRACTOR
Page 99 of 143
The flex map functions are applicable for use with flex tables, their associated {flextable}_keys
table and automatically generated {flextable}_view views. Using these functions is not
applicable for use with standard HP Vertica tables.
All map functions (except for emptymap() and mapaggregate()), accept either a LONG VARBINARY
or a LONG VARCHAR map argument.
EMPTYMAP
Constructs a new VMap with one row, but without keys or data. Use this transform function to
populate a map without using a flex parser, either from SQL queries, or from map data present
elsewhere in the database.
Syntax
emptymap()
Arguments
None
Examples
To create an empty map:
=> select emptymap();
emptymap
-----------------------------------------------------------------\001\000\000\000\004\000\000\000\000\000\000\000\000\000\000\000
(1 row)
If you create an empty map from an existing flex table, the new map will have the same number of
rows as the table from which it was created. For example, creating an empty map from the
darkdata table, which has 12 rows of JSON data has the following result:
=> select emptymap() from darkdata;
emptymap
-----------------------------------------------------------------\001\000\000\000\004\000\000\000\000\000\000\000\000\000\000\000
\001\000\000\000\004\000\000\000\000\000\000\000\000\000\000\000
\001\000\000\000\004\000\000\000\000\000\000\000\000\000\000\000
\001\000\000\000\004\000\000\000\000\000\000\000\000\000\000\000
\001\000\000\000\004\000\000\000\000\000\000\000\000\000\000\000
\001\000\000\000\004\000\000\000\000\000\000\000\000\000\000\000
\001\000\000\000\004\000\000\000\000\000\000\000\000\000\000\000
\001\000\000\000\004\000\000\000\000\000\000\000\000\000\000\000
\001\000\000\000\004\000\000\000\000\000\000\000\000\000\000\000
\001\000\000\000\004\000\000\000\000\000\000\000\000\000\000\000
\001\000\000\000\004\000\000\000\000\000\000\000\000\000\000\000
\001\000\000\000\004\000\000\000\000\000\000\000\000\000\000\000
(12 rows)
See Also
l
MAPAGGREGATE
MAPCONTAINSKEY
MAPCONTAINSVALUE
MAPITEMS
MAPKEYS
MAPKEYSINFO
MAPLOOKUP
MAPSIZE
MAPTOSTRING
MAPVALUES
MAPVERSION
MAPAGGREGATE
Returns a LONG VARBINARY VMap with keys and value pairs supplied from two VARCHAR input
columns of an existing columnar table. Using this function requires specifying an over() clause for
the source table. The examples here use the over (PARTITION BEST) clause.
Syntax
mapaggregate(source_column1, source_column2)
Arguments
source_column1
Table column with values to use as the keys of the key/value pair of the
returned VMap data.
source_column2
Table column with values to use as the values in the key/value pair of the
returned VMap data.
Examples
This example creates a columnar table btest, with two VARCHAR columns, named keys and
values, and adds three sets of values:
=> create table btest(keys varchar(10), values varchar(10));
CREATE TABLE
=> copy btest from stdin;
Enter data to be copied followed by a newline.
End with a backslash and a period on a line by itself.
>> one|1
>> two|2
>> three|3
>> \.
After populating the btest table, calling mapaggregate() return the raw_map data:
=>
The next example illustrates using MAPTOSTRING() with the returned raw_map from
mapaggregate() to see the values:
=> select maptostring(raw_map) from (select mapaggregate(keys, values) over(PARTITION
BEST) from btest) bit;
maptostring
---------------------------------------------{
"one": "1",
"three":
"3",
"two": "2"
}
(1 row)
See Also
l
EMPTYMAP
MAPCONTAINSKEY
MAPCONTAINSVALUE
MAPITEMS
MAPKEYS
MAPKEYSINFO
MAPLOOKUP
MAPSIZE
MAPTOSTRING
MAPVALUES
MAPVERSION
MAPCONTAINSKEY
Determines whether a VMap contains a virtual column (key). This scalar function returns true (t) if
the virtual column exists, or false (f) if it does not. Determining that a key exists before calling
maplookup()lets you distinguish between NULL returns, which that function uses for both a nonexistent key, and an existing key with a NULL value.
Syntax
mapcontainskey(VMap_data, 'virtual_column_name')
Arguments
VMap_data
Any VMap data. The VMap can exist as the __raw__ column of a flex
table, data returned from a map function such as maplookup(), or other
database content.
virtual_column_name
Examples
This example uses maplookup() and the mapcontainskey() functions. Displaying the results
returned from both functions differentiates whether the empty fields that maplookup() returns
indicate a NULL value for the row (t), or no value (f):
=> select maplookup(__raw__, 'user.location'), mapcontainskey(__raw__, 'user.location')
from darkdata order by 1;
maplookup | mapcontainskey
-----------+---------------| t
| t
| t
| t
Chile
| t
Narnia
| t
Uptown.. | t
chicago
| t
| f
| f
| f
| f
(12 rows)
See Also
l
EMPTYMAP
MAPAGGREGATE
MAPCONTAINSVALUE
MAPITEMS
MAPKEYS
MAPKEYSINFO
MAPLOOKUP
MAPSIZE
MAPTOSTRING
MAPVALUES
MAPVERSION
MAPCONTAINSVALUE
Determines whether a VMap contains a specific value. Use this scalar function to return true (t) if
the value exists, or false (f) if it does not.
Syntax
mapcontainsvalue(VMap_data, 'virtual_column_value')
Arguments
VMap_data
Any VMap data. The VMap can exist as the __raw__ column of a flex
table, data returned from a map function such as maplookup(), or
other database content.
virtual_column_value
Examples
This example shows a flex table (ftest), populated with some virtual columns and values. Both
virtual columns are named one:
=> create flex table ftest();
CREATE TABLE
=> copy ftest from stdin parser fjsonparser();
Enter data to be copied followed by a newline.
Calling mapcontainsvalue() on the ftest map data returns false (f) for the first virtual column,
and true (t) for the second , which contains the value one:
=> select mapcontainsvalue(__raw__, 'one') from ftest;
mapcontainsvalue
-----------------f
t
(2 rows)
See Also
l
EMPTYMAP
MAPAGGREGATE
MAPCONTAINSKEY
MAPITEMS
MAPKEYS
MAPKEYSINFO
MAPLOOKUP
MAPSIZE
MAPTOSTRING
MAPVALUES
MAPVERSION
MAPITEMS
Returns information about items in a VMap. Use this transform function with one or more optional
arguments to access polystructured values within the VMap data. This function requires an over()
clause. The examples here use over(PARTITION BEST).
Syntax
mapItems(VMap_data [, passthrough_arg [,...] ])
Arguments
VMap_data
Any VMap data. The VMap can exist as the __raw__ column of a flex table,
data returned from a map function such as maplookup(), or other database
content.
passthrough_ [Optional] One or more arguments indicating keys within the map data in VMap_
arg
data.
Examples
This example uses a flex table darkmountain, populated with JSON data. This query uses the
count() function to return the number of virtual columns found in the map data:
=> select count(keys) from (select mapitems(darkmountain.__raw__) over(PARTITION BEST)
from darkmountain) as a;
count
------19
(1 row)
This example shows a snippet of the return data querying a flex table of all items in the map data:
=> select * from (select mapitems(darkmountain.__raw__) over(PARTITION BEST) from
darkmountain) as a;
keys
|
values
-------------+--------------hike_safety | 50.6
name
| Mt Washington
type
| mountain
height
| 17000
hike_safety | 12.2
name
| Denali
type
| mountain
height
| 29029
hike_safety | 34.1
name
| Everest
type
| mountain
height
| 14000
hike_safety | 22.8
name
| Kilimanjaro
type
| mountain
height
| 29029
hike_safety | 15.4
name
| Mt St Helens
type
| volcano
(19 rows)
3. Call mapkeys on the flex table's __raw__ column to see the flex table's keys, but not their
submaps. The return values indicate three_Array as one of the virtual columns:
4. Call mapitems on flex table mapper with three_Array as a pass-through argument to the
function to return the array values:
=> select __identity__, mapitems(three_Array) OVER(PARTITION BY __identity__) from
mapper;
__identity__ | keys |
values
--------------+------+-----------1 | 0
| three_One
1 | 1
| three_Two
1 | 2
| 3
1 | 3
| three_Four
(4 rows)
See Also
l
EMPTYMAP
MAPAGGREGATE
MAPCONTAINSKEY
MAPCONTAINSVALUE
MAPKEYS
MAPKEYSINFO
MAPLOOKUP
MAPSIZE
MAPTOSTRING
MAPVALUES
MAPVERSION
MAPKEYS
Returns the virtual columns (and values) present in any VMap data. This transform function
requires an over(PARTITION BEST) clause, as shown in the example.
Syntax
mapkeys(VMap_data)
Arguments
VMap_data
Any VMap data. The VMap can exist as the __raw__ column of a flex table, data
returned from a map function such as maplookup(), or other database content.
Examples
This example uses a flex table darkdata, populated with JSON tweet data. This query returns the
number of virtual columns found in the map data:
=> select count(keys) from (SELECT mapkeys(darkdata.__raw__) over(PARTITION BEST) from
darkdata) as a;
count
------550
(1 row)
This example shows a snippet of the return data querying an ordered list of all virtual columns in the
map data:
=> select * from (SELECT mapkeys(darkdata.__raw__) over(PARTITION BEST) from darkdata) as
a;
keys
------------------------------------contributors
coordinates
created_ at
delete.status.id
delete.status.id_str
delete.status.user_id
delete.status.user_id_str
entities.hashtags
entities.media
entities.urls
entities.user_mentions
favorited
geo
id
.
.
.
user.statuses_count
user.time_zone
user.url
user.utc_offset
user.verified
(125 rows)
See Also
l
EMPTYMAP
MAPAGGREGATE
MAPCONTAINSKEY
MAPCONTAINSVALUE
MAPITEMS
MAPKEYSINFO
MAPLOOKUP
MAPSIZE
MAPTOSTRING
MAPVALUES
MAPVERSION
MAPKEYSINFO
Returns virtual column information from a given map. This transform function requires an over
(PARTITION BEST)clause, as shown in the example.
Syntax
mapkeysinfo(VMap_data)
Arguments
VMap_data
Any VMap data. The VMap can exist as the __raw__ column of a flex table, data
returned from a map function such as maplookup(), or other database content.
Returns
This function is a superset of the MAPKEYS() function. It returns the following information about
each virtual column:
Column
Description
keys
length
The data length of the key name, which can differ from the actual string length.
type_oid
The OID type into which the value should be converted. Currently, the type is always
116 for a LONG VARCHAR, or 199 for a nested map, stored as a LONG VARBINARY.
row_num
field_num
Examples
This example shows a snippet of the return data querying an ordered list of all virtual columns in the
map data:
=> select * from (SELECT mapkeysinfo(darkdata.__raw__) over(PARTITION BEST) from
darkdata) as a;
keys
| length | type_oid | row_num |
field_num
----------------------------------------------------------+--------+----------+--------+----------contributors
|
0 |
116 |
1 |
0
coordinates
|
0 |
116 |
1 |
1
created_at
|
30 |
116 |
1 |
2
entities.hashtags
|
93 |
199 |
1 |
3
entities.media
|
772 |
199 |
1 |
4
entities.urls
|
16 |
199 |
1 |
5
entities.user_mentions
6
favorited
7
geo
8
id
9
id_str
10
16 |
199 |
1 |
1 |
116 |
1 |
0 |
116 |
1 |
18 |
116 |
1 |
18 |
116 |
1 |
18 |
116 |
11 |
18 |
116 |
11 |
9 |
116 |
11 |
9 |
116 |
11 |
18 |
116 |
12 |
18 |
116 |
12 |
9 |
116 |
12 |
9 |
116 |
12 |
.
.
.
delete.status.id
0
delete.status.id_str
1
delete.status.user_id
2
delete.status.user_id_str
3
delete.status.id
0
delete.status.id_str
1
delete.status.user_id
2
delete.status.user_id_str
3
(550 rows)
See Also
l
EMPTYMAP
MAPAGGREGATE
MAPCONTAINSKEY
MAPCONTAINSVALUE
MAPITEMS
MAPKEYS
MAPLOOKUP
MAPSIZE
MAPTOSTRING
MAPVALUES
MAPVERSION
MAPLOOKUP
Returns values associated with a single key. This scalar function returns a LONG VARCHAR with
virtual column values, or NULL if the virtual column does not exist. Column names are case
insensitive.
Before using maplookup, you can use these two functions to find out about your VMap data:
l
You can control the behavior for non-scalar values when loading data with the fjsonparser parser
and its flatten-arrays argument. See Loading JSONData and the FJSONPARSER reference.
Syntax
maplookup (VMap_data, 'virtual_column_name' [USING PARAMETERS [case_sensitive=
{false | true}] [, buffer_size=n] ] )
Any VMap data. The VMap can exist as the __raw__ column of
a flex table, data returned from a map function such as
maplookup(), or other database content.
virtual_column_name
buffer_size
case_sensitive
Examples
This example shows how to return the row values of one virtual column, user.location:
=> select maplookup(__raw__, 'user.location') from darkdata order by 1;
maplookup
----------Chile
Narnia
Uptown
.
.
chicago
(12 rows)
"name": "jen",
"age": "38",
"eyes": "green",
"weapon": "shopping"
}
4. Use maplookup with buffer_size=0 for the logs table name key. All of the values are
returned:
=> SELECT MapLookup(__raw__, 'name' USING PARAMETERS buffer_size=0) FROM logs;
MapLookup
----------sierra
ben
janis
jen
(4 rows)
5. The example continues with buffer_size 3, 5, and 6. Now, maplookup() returns only values
with a byte length less than, or equal to (<=), the specified buffer_size:
=> SELECT MapLookup(__raw__, 'name' USING PARAMETERS buffer_size=3) FROM logs;
MapLookup
----------ben
jen
(4 rows)
=> SELECT MapLookup(__raw__, 'name' USING PARAMETERS buffer_size=5) FROM logs;
MapLookup
----------janis
jen
ben
(4 rows)
=> SELECT MapLookup(__raw__, 'name' USING PARAMETERS buffer_size=6) FROM logs;
MapLookup
----------sierra
janis
jen
ben
(4 rows)
For example, if you use maplookup without first checking the existence of a key, the following
output (12 empty rows) is ambiguous. Does a location key exist, exist with a NULL value in a row,
or exist without a value?
=> select maplookup(__raw__, 'user.location') from darkdata;
maplookup
-----------
(12 rows)
To disambiguate empty rows, use the mapcontainskey() function in conjunction with maplookup
(). When maplookup returns an empty field, the corresponding value from mapcontainskey
indicates t for a NULL value, or ffor no value.
The following vsql example output from both functions annotates rows without values in purple. The
example also shows location Narnia as a value in the user.location virtual column:
=> select maplookup(__raw__, 'user.location'), mapcontainskey(__raw__, 'user.location')
from darkdata order by 1;
maplookup | mapcontainskey
-----------+---------------| t
| t
| t
| t
Chile
| t
Narnia
| t
Uptown.. | t
chicago
| t
| f >>>>>>>>>>No value
| f >>>>>>>>>>No value
| f >>>>>>>>>>No value
| f >>>>>>>>>>No value
(12 rows)
0.name :
0.ppu : 0.55
0.topping :
0.id : 6001
0.type
1.id :
1.type
2.id :
2.type
3.id :
3.type
4.id :
4.type
5.id :
5.type
6.id :
6.type
0.type :
CupCake
: None
6002
: Glazed
6005
: Sugar
6007
: Powdered Sugar
6006
: Chocolate with Sprinkles
6003
: Chocolate
6004
: Maple
Muffin
(1 row)
The next examples illustrate using several invocations of maplookup() to return values from the
map __raw__ column (after creating the table and loading data). Compare the returned results to the
maptostring() output in the previous example.
=> create flex table bake();
CREATE TABLE
kdb=> copy bake from '/vertica/test/flextable/DATA/bake.json' parser fjsonparser(flatten_
arrays=1,flatten_maps=0);
Rows Loaded
------------1
(1 row)
=> select maplookup(maplookup(maplookup(maplookup(maplookup(__raw_
_,'items'),'item.0'),'batters'),'batter.0'),'type') from bake;
maplookup
----------Regular
(1 row)
=> select maplookup(maplookup(maplookup(maplookup(maplookup(__raw_
_,'items'),'item.0'),'batters'),'batter.1'),'type') from bake;
maplookup
----------Chocolate
(1 row)
=> select maplookup(maplookup(maplookup(maplookup(maplookup(__raw_
_,'items'),'item.0'),'batters'),'batter.2'),'type') from bake;
maplookup
----------Blueberry
(1 row)
=> select maplookup(maplookup(maplookup(maplookup(maplookup(__raw_
_,'items'),'item.0'),'batters'),'batter.3'),'type') from bake;
maplookup
--------------
Devil's Food
(1 row)
CREATE TABLE
dbt=> copy dupe from '/home/release/KData/repeated_key_name.json' parser fjsonparser
();
Rows Loaded
------------8
(1 row)
See Also
l
EMPTYMAP
MAPAGGREGATE
MAPCONTAINSKEY
MAPCONTAINSVALUE
MAPITEMS
MAPKEYS
MAPKEYSINFO
MAPSIZE
MAPTOSTRING
MAPVALUES
MAPVERSION
MAPSIZE
Returns the number of virtual columns present in any VMap data. Use this scalar function to
determine the size of keys.
Syntax
mapsize(VMap_data)
Arguments
VMap_data
Any VMap data. The VMap can exist as the __raw__ column of a flex table, data
returned from a map function such as maplookup(), or other database content.
Examples
This example shows the returned sizes from the number of keys in the flex table darkmountain:
=> select mapsize(__raw__) from darkmountain;
mapsize
--------3
4
4
4
4
(5 rows)
See Also
l
EMPTYMAP
MAPAGGREGATE
MAPCONTAINSKEY
MAPCONTAINSVALUE
MAPITEMS
MAPKEYS
MAPKEYSINFO
MAPLOOKUP
MAPTOSTRING
MAPVALUES
MAPVERSION
MAPTOSTRING
Recursively builds a string representation VMap data, including nested JSONmaps. Use this
transform function to display the VMap contents in a readable format. The function returns a LONG
VARCHAR. Use maptostring to see how map data is nested before querying virtual columns with
mapvalues().
Syntax
maptostring(VMap_data [using parameters canonical_json={true | false}])
Arguments
VMap_data
Any VMap data. The VMap can exist as the __raw__ column of a flex table,
data returned from a map function such as maplookup(), or other database
content.
Parameters
canonical_json
=bool
[Optional parameter] Default canonical-json=true
Produces canonical JSON output by default, using the first instance of any
duplicate keys in the map data.
Use this parameter as other UDFparameters, preceded by using
parameters, as shown in the examples. Setting this argument to false
maintains the previous behavior of maptostring(), and returns same-name
keys and their values.
Examples
The following example creates a sample flex table, darkdata. After loading sample JSON data
from STDIN, the example continues by calling maptostring() twice with both values for the
canonical_json parameter, illustrating the different results on the flex table __raw__column data.
1. Create sample table:
=> create flex table darkdata();
CREATE TABLE
3. Call maptostring() with its default behavior using canonical JSONoutput. Notice the flex
table contents. The function returns the first duplicate key and its value ("aaa": "1"), and
omits remaining duplicate keys ("aaa": "2"):
=> select maptostring (__raw__) from darkdata;
maptostring
-----------------------------------------------------------{
"AAA" : "3",
"aaa" : "1",
"bbb" : "aaa\"bbb"
}
(1 row)
"1",
"2",
"3",
"aaa"bbb"
}
(1 row)
See Also
l
EMPTYMAP
MAPAGGREGATE
MAPCONTAINSKEY
MAPCONTAINSVALUE
MAPITEMS
MAPKEYS
MAPKEYSINFO
MAPLOOKUP
MAPSIZE
MAPVALUES
MAPVERSION
MAPVALUES
Returns a string representation of the top-level values from a VMap. This transform function
requires an over() clause. The examples uses an over(PARTITION BEST) clause.
Syntax
mapvalues(VMap_data)
Arguments
VMap_data
The VMap from which values should be returned. The VMap can exist as the __raw_
_ column of a flex table, data returned from a map function such as maplookup(), or
other database content.
Examples
The following example uses mapvalues() with a darkmountain flex table, returning the values.
=> select * from (select mapvalues(darkmountain.__raw__) over(PARTITION BEST) from
darkmountain) as a;
values
--------------29029
34.1
Everest
mountain
29029
15.4
Mt St Helens
volcano
17000
12.2
Denali
mountain
14000
22.8
Kilimanjaro
mountain
50.6
Mt Washington
mountain
(19 rows)
See Also
l
EMPTYMAP
MAPAGGREGATE
MAPCONTAINSKEY
MAPCONTAINSVALUE
MAPITEMS
MAPKEYS
MAPKEYSINFO
MAPLOOKUP
MAPSIZE
MAPTOSTRING
MAPVERSION
MAPVERSION
Returns the version or invalidity of any map data. This scalar function returns the map version (such
as 1), or -1 if the map data is invalid.
Syntax
mapversion(VMap_data)
Arguments
VMap_data
The VMap data either from a __raw__ column in a flex table, or from the data returned
from a map function such as maplookup().
Examples
The following example uses mapversion() with the darkmountainflex table, returning mapversion
1 for the flex table map data:
=> select mapversion(__raw__) from darkmountain;
mapversion
-----------1
1
1
1
1
(5 rows)
See Also
l
EMPTYMAP
MAPAGGREGATE
MAPCONTAINSKEY
MAPCONTAINSVALUE
MAPITEMS
MAPKEYS
MAPKEYSINFO
MAPLOOKUP
MAPSIZE
MAPTOSTRING
MAPVALUES
FCEFPARSER
FDELIMITEDPAIRPARSER
FDELIMITEDPARSER
FJSONPARSER
FREGEXPARSER
All parsers store the data as a single Vmap in the LONG VARBINARY __raw__ column. If a data row is
too large to fit in the column, it is rejected. Null values are supported for loading data with NULLspecified columns.
This section describes each parser's options. For more information about working with flex parsers,
see Using Flex Table Parsers
FCEFPARSER
Parses HPArcSight Common Event Format (CEF) log files. The fcefparser loads values directly
into any table column with a column name that matches a source data key.The parser stores the
data loaded into a flex table in a single VMap.
Parameters
delimiter
CHAR
record_terminator
CHAR
trim
BOOLEAN
reject_on_unescaped_delimiter
BOOLEAN
Example
The following example illustrates creating a sample flex table for CEF data, with two real columns,
eventId and priority.
1. Create a flex table cefdata:
=> create flex table cefdata();
CREATE TABLE
2. Load some basic CEF data, using the flex parser fcefparser:
=> copy cefdata from stdin parser fcefparser();
Enter data to be copied followed by a newline.
End with a backslash and a period on a line by itself.
>> CEF:0|ArcSight|ArcSight|2.4.1|machine:20|New alert|High|
>> \.
3. Use the maptostring()function to see the contents of your cefdata flex table:
=> select maptostring(__raw__) from cefdata;
maptostring
---------------------------------------------------------------{
"deviceproduct" : "ArcSight",
"devicevendor" : "ArcSight",
"deviceversion" : "2.4.1",
"name" : "New alert",
"severity" : "High",
"signatureid" : "machine:20",
"version" : "0"
}
(1 row)
See Also
n
FDELIMITEDPARSER
FDELIMITEDPAIRPARSER
FJSONPARSER
FREGEXPARSER
FDELIMITEDPAIRPARSER
Parses delimited data files. This parser provides a subset of the functionality in the parser
fdelimitedparser.Use the fdelimitedpairparser when the data you are loading specifies pairs
of column names with data in each row.
Parameters
delimiter
CHAR
record_terminator
CHAR
trim
BOOLEAN
Example
The following example illustrates creating a flex table with two real columns, num and val. Then,
loading the following data, saved in a file called, datapairs.dat:
100|200
101|201
102|202
103|203
104|204
105|205
106|206
107|207
108|208
109|209
110|210
3. After loading the sample data file, use maptostring()to display the virtual columns in the __
raw__ column of CEFData:
=> select maptostring(__raw__) from CEFData limit 1;
maptostring
----------------------------------------------------------"agentassetid" : "4-WwHuD0BABCCQDVAeX21vg==",
"agentzone" : "3083",
"agt" : "265723237",
"ahost" : "svsvm0176",
"aid" : "3tGoHuD0BABCCMDVAeX21vg==",
"art" : "1099267576901",
"assetcriticality" : "0",
"at" : "snort_db",
"atz" : "America/Los_Angeles",
"av" : "5.3.0.19524.0",
"cat" : "attempted-recon",
"categorybehavior" : "/Communicate/Query",
"categorydevicegroup" : "/IDS/Network",
"categoryobject" : "/Host",
"categoryoutcome" : "/Attempt",
"categorysignificance" : "/Recon",
"categorytechnique" : "/Scan",
"categorytupledescription" : "An IDS observed a scan of a host.",
"cnt" : "1",
"cs2" : "3",
"destinationgeocountrycode" : "US",
"destinationgeolocationinfo" : "Richardson",
"destinationgeopostalcode" : "75082",
"destinationgeoregioncode" : "TX",
"destinationzone" : "3133",
"device product" : "Snort",
"device vendor" : "Snort",
"device version" : "1.8",
"deviceseverity" : "2",
"dhost" : "198.198.121.200",
"dlat" : "329913940429",
"dlong" : "-966644973754",
"dst" : "3334896072",
"dtz" : "America/Los_Angeles",
"dvchost" : "unknown:eth1",
"end" : "1364676323451",
"eventid" : "1219383333",
"fdevice product" : "Snort",
"fdevice vendor" : "Snort",
"fdevice version" : "1.8",
"fdtz" : "America/Los_Angeles",
"fdvchost" : "unknown:eth1",
"lblstring2label" : "sig_rev",
"locality" : "0",
"modelconfidence" : "0",
"mrt" : "1364675789222",
"name" : "ICMP PING NMAP",
"oagentassetid" : "4-WwHuD0BABCCQDVAeX21vg==",
"oagentzone" : "3083",
"oagt" : "265723237",
"oahost" : "svsvm0176",
"oaid" : "3tGoHuD0BABCCMDVAeX21vg==",
"oat" : "snort_db",
"oatz" : "America/Los_Angeles",
"oav" : "5.3.0.19524.0",
"originator" : "0",
"priority" : "8",
"proto" : "ICMP",
"relevance" : "10",
"rt" : "1099267573000",
"severity" : "8",
"shost" : "198.198.104.10",
"signature id" : "[1:469]",
"slat" : "329913940429",
"slong" : "-966644973754",
"sourcegeocountrycode" : "US",
"sourcegeolocationinfo" : "Richardson",
"sourcegeopostalcode" : "75082",
"sourcegeoregioncode" : "TX",
"sourcezone" : "3133",
"src" : "3334891530",
"start" : "1364676323451",
"type" : "0"
}
(1 row)
4. Select the eventID and priority real columns, along with two virtual columns, atz and
destinationgeoregioncode:
=>
See Also
l
FCEFPARSER
FDELIMITEDPARSER
FJSONPARSER
FREGEXPARSER
FDELIMITEDPARSER
Parses data using a delimiter character to separate values. The fdelimitedparser loads delimited
data, storing it in a single-value VMap. You can use this parser to load data into columnar and flex
tables.
Parameters
delimiter
CHAR
record_terminator
CHAR
trim
BOOLEAN
header
BOOLEAN
omit_empty_keys
BOOLEAN
reject_on_duplicate
BOOLEAN
reject_on_empty_
key
BOOLEAN
reject_on_
materialized_
type_error
BOOLEAN
treat_empty_val_
as_null
BOOLEAN
Example
For example:
1. Create a flex table for delimited data:
=> create flexible table my_test();
CREATE TABLE
2. Use the fdelimitedparser to load the data from a .csv file, specifying a comma (,) column
delimiter:
=> copy my_test from '/test/vertica/DATA/a.csv' parser fdelimitedparser
(delimiter=',');
Rows Loaded
------------3
(1 row)
See Also
l
FDELIMITEDPAIRPARSER
FDELIMITEDPAIRPARSER
FJSONPARSER
FREGEXPARSER
FJSONPARSER
Loads a file of repeated JSON data objects, including nested maps, or with an outer list of JSON
elements. The fjsonparser loads values directly into any table column with a column name that
matches a source data key. The parser stores the data in a single-value VMap.
Note: The maximum size of each JSONkey within a __raw__ column is 255 bytes. Loading
JSONdata with keys longer than 255 bytes is unsupported.
Parameters
flatten_maps
BOOLEAN
flatten_arrays
BOOLEAN
reject_on_
duplicate
BOOLEAN
reject_on_empty_ BOOLEAN
key
omit_empty_keys
BOOLEAN
[Optional] Omits any key from the load data that does
not have a value (omit_empty_keys=true).
Default value: false
reject_on_
materialized_
type_error
BOOLEAN
start_point
CHAR
start_point_
occurrence
INTEGER
suppress_
BOOLEAN
nonalphanumeric_
key_chars
key_separator
CHAR
Examples
Following is a basic example loading JSONdata without optional parameters:
1. Create a flex table, super, with two columns, age and name:
=> create table super(age int, name varchar);
CREATE TABLE
See Also
n
FCEFPARSER
FDELIMITEDPARSER
FDELIMITEDPAIRPARSER
FREGEXPARSER
FREGEXPARSER
Parses a regular expression, matching columns to the contents of the named regular expression
groups.
Parameters
pattern
VARCHAR
use_jit
BOOLEAN
record_terminator
VARCHAR
logline_column
VARCHAR
Example
These examples use the following regular expression, which searches for information that includes
the timestamp, date, thread_name, and thread_id strings.For illustrative purposes, the regular
expression components are shown here on separate lines. Remove any new line characters before
using this example in your own tests.
'^(?<time>\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d\.\d+)
(?<thread_name>[A-Za-z ]+):(?<thread_id>0x[0-9a-f]+)-?(?<transaction_id>[0-9a-f]+)?
(?:\[(?<component>\w+)\] \<(?<level>\w+)\> )?(?:<(?<elevel>\w+)> @\[?(?<enode>\w+)\]?: )?
(?<text>.*)'
1. Create a flex table (vlog) to contain the results of an HP Vertica log file. For this example, we
made a copy of a log file in the directory /home/dbadmin/data/vertica.log:
=> create flex table vlog();
CREATE TABLE
2. Use the fregexparser with the sample regular expression to load data from the log file
(removing line characters first):
=> copy vlog from '/home/dbadmin/data/flex/vertica.log' PARSER FRegexParser(pattern=
'^(?<time>\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d\.\d+) (?<thread_name>[A-Za-z ]+):
(?<thread_id>0x[0-9a-f]+)-?(?<transaction_id>[0-9a-f]+)?
(?:\[(?<component>\w+)\] \<(?<level>\w+)\> )?(?:<(?<elevel>\w+)> @\[?(?<enode>\w+)
\]?: )?(?<text>.*)');
Rows Loaded
------------8434
(1 row)
3. After successfully loading data, query columns directly from the flex table:
=> select text, time, thread_name from vlog limit 4;
text
|
time
|
thread_name
-------------------------------------------------------+-------------------------+------------Starting up Vertica Analytic Database v7.1.0-20140331 | 2014-04-02 04:02:02.613 |
Main
Checking for missed alter partition events
| 2014-04-02 04:02:51.017 |
ManageEpochs
Found no missed alter partition events
| 2014-04-02 04:02:51.017 |
ManageEpochs
Checking for missed replace node events
| 2014-04-02 04:02:51.017 |
ManageEpochs
(4 rows)
4. Use the MAPTOSTRING() function with the table's __raw__column. The four rows (limt 4)
that the query returns are regular expression results of the vertica.log file, parsed with
fregexparser:
=> select maptostring(__raw__) from vlog limit 4;
maptostring
------------------------------------------------------------------------------{
"component" : "Init",
"level" : "INFO",
"text" : "Log /scratch_b/qa/STDB/v_stdb_node0001_catalog/vertica.log opened; #2",
"thread_id" : "0x16321430",
"thread_name" : "Main",
"time" : "2014-04-02 04:02:02.613"
}
{
"component" : "Init",
"level" : "INFO",
"text" : "Project Codename: Dragline",
"thread_id" : "0x16321430",
"thread_name" : "Main",
"time" : "2014-04-02 04:02:02.613"
}
{
"component" : "Init",
"level" : "INFO",
"text" : "Processing command line: /opt/vertica/bin/vertica -D /scratch_
b/qa/STDB/v_stdb_node0001_catalog -C STDB -n v_stdb_node0001 -h 10.20.90.192 -p 6059
-P 6061",
"thread_id" : "0x16321430",
"thread_name" : "Main",
"time" : "2014-04-02 04:02:02.613"
}
{
"component" : "Init",
"level" : "INFO",
"text" : "64-bit Optimized Build",
"thread_id" : "0x16321430",
"thread_name" : "Main",
"time" : "2014-04-02 04:02:02.613"
}
(4 rows)
See Also
l
FDELIMITEDPAIRPARSER
FDELIMITEDPAIRPARSER
FDELIMITEDPARSER
FJSONPARSER