Professional Documents
Culture Documents
Pgintro 1hr
Pgintro 1hr
Introduction to PostgreSQL
The Open Source ObjectRelational Database Management
System
PostgreSQL BSD License
Redistribution and use in source and binary forms,
with or without modification, are permitted provided
that the following conditions are met:
Redistribution in source or binary must maintain copyright and
following disclaimer
Neither the name of the organization nor the names of its
contributors may be used to endorse or promote products.
3
Agenda
PostgreSQL Features
Installation and Configuration
Maintenance and Monitoring
Command Line Interface
Database Basics in PostgreSQL
4
Not the Agenda
Client Interfaces
Inheritance
Comparisons to other Databases
Replication, Point in Time Recovery
Full Text Search
5
History of Postgres
1986 1992 1995
Miró/Illustra I postgres95
UCB
1996 PostgreSQL
IIInformix Global Dev
Informix 2000
IUS 9
2001
IBM
2006
6
What is PostgreSQL?
Relational Database Management System
ObjectRelational Database
Ability to add First Class simple and complex objects,
with methods, that can be used in a Relational
Context (SQL)
7
PostgreSQL Relational Features
Foreign keys
Triggers
Views
Transactional Integrity
ACID compliance
Complex Queries
8
Data Centricity
Data stands on its own
Data is money
Many applications one database
Database centric logic
Integrity cannot be circumvented by applications
9
ACID Compliance
Atomic
transactions seen in full or not at all
Consistent
system enforced contraints
Isolated
transactions do not interfere with each other transactions
Durable
On Commit, result will not be lost
10
MultiVersion Concurrency Control
Snapshot of data for command or transaction
Virtually eliminates need for locking
Reading does not block writing and vice versa
SET TRANSACTION ISOLATION LEVEL
READ COMMITTED
SERIALIZABLE
11
SQL and PostgreSQL
Excellent Standards Compliance
SQL89, SQL92, SQL98, SQL2003
Documentation includes Compliance
Design Issues decided by Standards
12
Object Relational Features
Data types
Functions
Operators
Rules
Aggregates
Index Methods
13
PostgreSQL Queries with Objects
select hotel_name, hotel_address
from hotels h, airports a
where a.name = 'OAK' and
h.loc @ Circle(a.loc, '5 miles');
select name, num_kids from people;
select pdf( doc, '/home/me')
from doc d
where dnameget(doc) = 'myresume';
14
Client GUI Interfaces
PgAdmin III
www.pgadmin.org
phppgadmin
phppgadmin.sourceforge.net
DbVisualizer
www.minq.se/products/dbvis/
Others, e.g. pgaccess
See sourceforge.net
15
Client Programming Interfaces
psql Command Line
libpq – C library
ECPG – Embedded SQL
pgtcl – Tcl binding library
Drivers
JDBC
ODBC
DBI: Perl, Python, PHP, etc.
.NET
16
Server Side Languages
PL/pgsql
SQL
C
Other server side languages
PL/perl, PL/pythonu,
PL/R, PL/Tcl, PL/Ruby,
PL/bash, PL/Java
etc.
17
Downloading PostgreSQL
http://www.postgresql.org
By Source: ftp, bittorrent
By CVS tree
In Packages: RPM, Debian
Company Distributions
18
Operating System Distributions
Most Linux like OS distributions
MacOSX:
www.entropy.ch/software/macosx/ postgresql
8.1 Native Win32 Version
pginstaller at pgfoundry.org
Cygwin:
www.cygwin.com
19
Configuration Points
Build Time
Build directives
Installation directory
PL Language options
Server Environment
postgresql.conf, pg_hba.conf
Runtime/Client Environment
PG environment variables
20
Configuration Points
Build Time
As user postgres ...
$ ./configure \
prefix=/local/pgsql81 \
withperl \
withpython \
withtcl \
enabledepend
$ make
$ sudo make install
21
Initdb D $PGDATA
Creates Data Directory with:
configuration files
postgresql.conf
pg_hba.conf
template databases
template0
template1
super user database
22
Configuration Points
Server Environment
Global User Configuration
$PGDATA/postgresql.conf
Environment variables for server startup
Access Security
$PGDATA/pg_hba.conf
Host, user and database access.
23
Configuration Points
Global User Configuration
Environment Variables for Server Startup
postgresql.conf
See also:
www.varlena.com/GeneralBits/
Tidbits/#Performance
24
Configuration Points
Global User Configuration
V a ria b le D e fa u lt @ 2 G R A M
m ax_ c o n n ec tio n s 100 100
sh ared _ b u ffers 1000 25000
w o rk _ m em 1024 16384
m ain ten an c e_ w o rk _ m em 16384 16384
m ax_ fsm _ p ag es 20000 *
m ax_ fsm _ relatio n s 1000 *
effec tive_ c ac h e_ size 1000 82500
lo g _ d estin atio n std err std err
red irect_ std err o ff on
25
Configuration Points
Global User Configuration
V a ria b le D e fa u lt @ 2 G R A M
lo g _ d irecto ry p g _ lo g /varl/lo g /p g sq l
lo g _ m in _ d u ratio n _ statem en t 1 500
lo g _ lin e_ p refix [% p % t ]
lo g _ statem en t none ddl
stats_ start_ co llecto r on on
stats_ co m m an d _ strin g o ff on
stats_ b lo ck_ level o ff on
stats_ ro w _ level o ff on
au to vacu u m o ff on
26
Configuration Points Basic Security
Different for multiple installations
28
Configuration Points
Session Setting
View: pg_settings
Show values and descriptions
SELECT name, setting, short_desc
FROM pg_settings
ORDER BY name;
What can be set in a session?
SELECT name
FROM pg_settings
where context='user';
29
Housekeeping PostgreSQL Start and Stop
Starting & Stopping PostgreSQL
Installation Specific Script (/etc/init.d)
$ pg_ctl start D $PGDATA
$ pg_ctl stop
Windows PostgreSQL> Programs
30
Housekeeping PostgreSQL Logging
Log Maintenance
Rotate Log Settings in postgresql.conf
Alternative:
$ pg_ctl start D $PGDATA |\
rotatelogs $PGDATA/pglog 86400 2>&1;
Always know where your log file is!
31
Housekeeping PostgreSQL Vacuuming
Autovacuum
Configure in postgresql.conf
Vacuum
$ vacuumdb analyze full
Updates Statistics
Improves Performance
Recovers Disk Space
Frequency tuning required
32
Housekeeping PostgreSQL Backing Up
Backup
pg_dumpall > \
.../`date +%Y%m%d`dump.sql
Restore
psql f 20061231dump.sql
Backup! Now!
No excuses! Really!
33
Monitoring PostgreSQL
Client Server Architecture
pg_stat_activity
Set pg_stats_command in postgresql.conf
ps alx
Log files
check pgfoundry for log parsers
34
Documentation and Help
Online & Downloadable Docs
Mailing Lists: www.postgresql.org
IRC #postgresql freenode.net
PostgreSQL General Bits :)
http://www.varlena.com/GeneralBits
35
Creating Databases
$ createdb accounts
36
Adding Users
$ createuser bob
Shall the new role be a superuser?
(y/n) n
Shall the new role be allowed to create
databases? (y/n) y
Shall the new role be allowed to create
more new roles? (y/n) y
CREATE ROLE
37
psql Basics
Always learn help first.
Command Line options
$ psql help
Backslash Command Help
$ psql db
db=# \?
SQL Help
$ psql db
db=# \help [SQL command]
38
Database Design Elements
Data Types & Sequences
Nulls
Keys
Constraints & Defaults
Triggers, Functions & Operators
Tablespaces
Simple domains
Rules
39
Create Table
AS, LIKE
WITH OIDS
Current default WITH may change
See default_with_oids
Temporary Tables
PRESERVE ROWS, DELETE ROWS, DROP
INHERITS
CONSTRAINTS
TABLESPACE
40
Create Table
CREATE TABLE people (
id SERIAL PRIMARY KEY,
name text,
dept_no int REFERENCES dept(dept_no)
);
CREATE temp TABLE ships_temp as
SELECT ship_id, cargo_no, voyage
FROM ships;
41
Data Types
Integers, big and small Boolean
Serials Geometric
Arbitrary precision–numeric Network Addresses
Floating points Bit Types
Serial Types–Identity Arrays
Character Types Oids
Binary Data, big and small Pseudo Types
Date/Time/Timestamp
42
Data Type Mapping
Integers................. 2, 4, 8 bytes
Serials.................... Identity, Autoincrement
Numeric.................. Money
Floats..................... Arithmetic
Text....................... Character Types
Date/Time/Interval... Dates & Times
Timestamp.............. Timestamps
Boolean................... Boolean
bytea...................... Byte stream, images
43
Keys
Primary Keys
Implemented as BTree Unique indexes
Foreign Keys
Implement Referential Integrity.
A FK in table A says that this value references a unique
value in table B.
Cascading updates, deletes
Nulls OK
44
Defaults & Constraints
Initialize column with constants
Check value for validity
UNIQUE, [NOT] NULL, KEYS
CREATE TABLE players (
nick_name text PRIMARY KEY,
team_name text REFERENCES teams(team_name),
age integer CHECK (age > 15) NOT NULL,
games_played integer DEFAULT 0
);
45
Nulls
A NULL is a NULL is a NULL
NULLS are not equal to each other
NULLS are not equivalent to an empty string
NULLS are not equivalent to 0
NULLS are not indexed
46
TableSpaces
Creating a tablespace
CREATE TABLESPACE bd LOCATION '/bigdisk';
Using a tablespace
CREATE TABLE FOO (...) TABLESPACE bd;
Altering a tablespace
alter owner, alter name
Alter a table's tablespace
ALTER TABLE SET TABLE SPACE TO bd;
47
SELECT
Target List – list of columns to be returned
any expression,
aggregate,
subquery,
function,
columns from FROM clause data sources
48
SELECT
FROM – data sources
Tables,
Views,
Set Returning Functions,
SubQueries,
JOINS,
UNIONS
49
SELECT
WHERE – boolean expression qualifying data
Expressions,
Columns,
Functions,
SubQueries
50
SELECT
GROUP BY – scope of Aggregate
Elements of Target List not involved in aggregation.
Determines Break columns
Conditional Statements
COALESCE
coalesce( description,
short_description, 'N/A')
CASE
(select case when $1 is null then
'#ffffff'
else
'#000000'
end)
NULLIF (value1, value2)
NULL if values are equal else value1
53
SubQuery Expressions
Expressions and Lists
EXISTS
WHERE EXISTS (select id from bigtable)
IN
WHERE thisid IN (select id from bigtable)
ANY (SOME)
name = ANY (select user from users)
ALL
due_date > ALL (select milestones from
projects)
54
UNIONS & JOINS
55
JOINS: ON, USING, WHERE
SELECT ...
FROM matches m JOIN events e
USING (matchid)
SELECT ...
FROM matches m JOIN events e
ON (m.matchid = e.m_id)
SELECT ...
FROM matches m, events e
WHERE m.matchid = e.matchid
56
INSERT
Target Table
(Column Names)
VALUES
(Column Values)
Expressions
DELETE FROM daily_log
where log_ts < (current_date 1)
+ '12:00pm'::time
61
Views
Named Queries
Implemented Using Rules
Can do Updates, Inserts, Deletes via Rules
Usability
CREATE OR REPLACE VIEW phonelist AS
SELECT t.team, p.player, p.name, p.phone
FROM teams t, p.players
WHERE t.team = p.team;
62
Blobs, Slobs and TOAST
Large Objects
special interface lo_
seek, read, write
TOAST
automatic and invisible promotion
INSERT, UPDATE, DELETE
no seek
63
Simple Domains
Subtype Inherits Parent Type
Attributes and
Operators, Functions
May Over Ride
DEFAULT, CHECK
CONSTRAINT, [NOT] NULL
Operators, Functions
64
Simple Domains
May Not Over Ride
Casts
LIKE
AS PRIMARY KEY use UNIQUE INDEX
CREATE DOMAIN degrees float CHECK
(degrees > 180 and degrees <= 180);
65
Builtin Functions & Operators
Logical & Comparison Operators
Math Functions, Aggregates & Operators
Type Conversions
Date, Time & Interval Arithmetic
String and pattern matching
Conditional Statements
66
Functions & Operators
SELECT (('1/1/' || 2006)+ 7*( week 1 ),
SUM(cookies), scout_name
FROM cookie_sales c JOIN scouts s
USING (s.name),
generate_series(1,53) g(week)
WHERE
date_part('week',c.sales_date) = week
GROUP BY week, scout_name;
67
Functions & Operators: Casts
INTERVAL '2 days 3 hours'
TIMESTAMP '12/31/59'
'gotta wanna'::text
16::bigint
'(1.5,2.7)'::point
123.456::numeric(6,3)
68
Input/Output Functions
Output Format
to_char( , text)
timestamp, integer, double precision, numeric
to_char( idate, 'ddMonYYYY');
to_char( price, '999D99');
69
Input/Output Functions
Input Format
to_date(text, text)
to_timestamp(text,text)
to_number(text, text)
to_date( '31 Dec 2006', 'DD Mon YYYY')
to_timestamp( '5/24/06', 'DD/MM/YY');
to_number( '543', '999D99')
70
Functions & Operators
Interval Arithmetic
Regular Arithmetic Expressions
current_date + INTERVAL '5 days'
start_date + duration
Regular Comparison Operators
item_date > due_date
start_date + INTERVAL '5 days' <= due_date
logtime <> last_log
71
Functions & Operators
Date, Time Arithmetic
extract( field FROM src)
extract(epoch FROM
TIMESTAMP '20041231 01:43:03');
extract(hours FROM
INTERVAL '2 days 5 hours');
age( timestamp )
age('12/31/1959');
Functions & Operators
72
Interval Arithmetic
(start, end) OVERLAPS (start2, end2)
(proposed_start, proposed_end)
OVERLAPS
('12/23/06'::date, '1/4/06'::date)
(sessiontime, INTERVAL '1 hour')
OVERLAPS
(breaktime, INTERVAL '15 minutes')
73
Functions & Operators
String and Pattern matching
LIKE, ILIKE or ~~, ~~*
city LIKE 'San_%'
city ~~ 'San_%'
city ILIKE 'oak%'
city ~~* 'oak%'
SIMILAR TO or ~, ~*
name SIMILAR TO
'(Mr.|Ms.) [AZ]([ az])*'
74
Indexing Operators
create index uname_idx
on users (user_name);
create index ttnotes_idx
on trouble_tickets(ticket_id, note_id);
create index range_idx
on cows USING RTREE (range);
75
Functional Indexing
Functional indexes
Result of any immutable procedure
create index tsdate_idx on
log_table date(createtimestamp);
create name_idx on
users lower(user_name);
Expressional indexes
Result of any immutable expression
create overdue_idx
on books duedate + '30 days'
76
Partial Indexing
Indexes over parts of tables
create index active_clients on clients
where status = 'A';
create index currentyear on accounts
where reg_date = '2005';
77
Server Side Languages
PL/pgsql and SQL Primary languages
Query & Trigger enabled
Trusted vs. untrusted languages
Available server side languages
PL/perl, PL/pythonu,
PL/R, PL/Tcl, PL/Ruby,
PL/bash
C, etc.
78
Server Side Functions
CREATE FUNCTION foo(text, integer)
RETURNS integer AS
$$
...
$$
LANGUAGE 'plpgsql' [OPTIONS...]
79
PlPgSQL Trigger Functions
Executes once per row
Often Used for
complex or dynamic defaults
logging
80
Triggers
Function executed per Row
Before or After Event
Insert, Update or Delete
CREATE OR REPLACE FUNCTION lastmod
RETURNS TRIGGER AS $$
BEGIN
NEW.last_modified = now();
RETURN NEW;
END;
$$ LANGUAGE 'plpgsql';
CREATE TRIGGER team_upd
BEFORE INSERT OR UPDATE on teams
FOR EACH ROW EXECUTE PROCEDURE lastmod();
81
Rules
ReWrite a Query
Action On a Table or View
Select Rules Implement Views
Updateable Views Implemented via Rules
82
Rules View
Example View:
CREATE VIEW matches_v
SELECT m.matchname, m.matchid,
t1.team AS team1, t2.team AS team2,
t1.teamid as t1id, t2.teamid as
t2id,
e.eventname, m.eventid
FROM matches m JOIN teams t1 USING
(t1.id=teamid)
JOIN teams t2 USING (t2.id=teamid)
JOIN event e ON (eventid);
83
Rules Implement a View
(Implicit)
CREATE RULE “_RETURN” AS ON
SELECT TO matches_v DO INSTEAD
SELECT...;
84
Rules Implement a View
CREATE RULE upd_matches
AS ON UPDATE TO matches_v
DO INSTEAD
UPDATE matches
SET matchname=NEW.matchname,
eventid=NEW.eventid,
t1id=NEW.t1id, t2id=NEW.t2id
WHERE matchid=OLD.matchid;
85
Rules
CREATE RULE ins_matches
AS ON INSERT TO matches_v
DO INSTEAD
INSERT INTO matches
(matchid, eventid, t1id, t2id,
matchname)
VALUES
(default, NEW.eventid, NEW.t1id,
NEW.t2id, NEW.matchname);
86
Rules
CREATE RULE del_matches
AS ON DELETE TO matches_v
DO INSTEAD
DELETE FROM tmatches
WHERE matchid=OLD.matchid
87
Operators
Create first class operators
Implemented by functions
Use the same way as ordinary builtin operators.
Natural cost overhead.
88
Tuning Queries
The usual suspects
DID YOU VACUUM?
Type mismatch
Indexing Expressions
GUC configurations
Explaining Explain
plpgsql[email protected]
89
Explain
Explain [analyze] [verbose]
OP (cost=n...n rows=n width=n)
(actual time=t..t rows=n loops=n)
OP cond: (...)
> OP (cost=...)(actual time=...)
OP cond: (...)
Look for
Seq Scan, Hash Join,
Subquery, Hash,
Index Scan
Index usage
90
Replication Products
SLONY1
Mammoth Replicator Command Prompt, Inc.
pgpool (client side)
postgresr, dbmirror async, Rserv async, clustgres,
pglcluster, osogres (client side replication)
91
References
www.postgresql.org
www.varlena.com/GeneralBits
Mailing Lists
general, sql, novice, interfaces
hackers
advocacy
performance, bugs
docs
IRC #postgresql freenode.net