1

Introduction to PostgreSQL
The Open Source Object-Relational Database Management
System

Varlena,LLC A.Elein M ustain

www.varlena.com
elein@ varlena.com
2

PostgreSQL BSD License

Redistribution and use in source and binary forms,

with or without modification, are permitted provided
that the following conditions are met:
Redistribution in source or binary must maintain copyright and
following disclaimer
Neither the name of the organization nor the names of its
contributors may be used to endorse or promote products.
3

Agenda

PostgreSQL Features
Installation and Configuration
Maintenance and Monitoring
Command Line Interface
Database Basics in PostgreSQL
4

Not the Agenda

Client Interfaces
Inheritance
Comparisons to other Databases
Replication, Point in Time Recovery
Full Text Search
5

History of Postgres
1986 1992 1995

Miró/Illustra I postgres95
UCB
1996 PostgreSQL
IIInformix Global Dev

Informix 2000
IUS 9
2001

IBM
2006
6

What is PostgreSQL?
Relational Database Management System
Object-Relational Database
Ability to add First Class simple and complex objects,
with methods, that can be used in a Relational
Context (SQL)
7

PostgreSQL Relational Features

Foreign keys
Triggers
Views
Transactional Integrity
ACID compliance
Complex Queries
8

Data Centricity
Data stands on its own
Data is money
Many applications one database
Database centric logic
Integrity cannot be circumvented by applications
9

ACID Compliance
Atomic
transactions seen in full or not at all
Consistent
system enforced contraints
Isolated
transactions do not interfere with each other transactions
Durable
On Commit, result will not be lost
10

Multi-Version Concurrency Control

Snapshot of data for command or transaction
Virtually eliminates need for locking
Reading does not block writing and vice versa
SET TRANSACTION ISOLATION LEVEL
READ COMMITTED
SERIALIZABLE
11

SQL and PostgreSQL

Excellent Standards Compliance
SQL89, SQL92, SQL98, SQL2003
Documentation includes Compliance
Design Issues decided by Standards
12

Object Relational Features

Data types
Functions
Operators
Rules
Aggregates
Index Methods
13

PostgreSQL Queries with Objects

select hotel_name, hotel_address
from hotels h, airports a
where a.name = 'OAK' and
h.loc @ Circle(a.loc, '5 miles');

select name, num_kids from people;

select pdf( doc, '/home/me')

from doc d
where dnameget(doc) = 'myresume';
14

Client GUI Interfaces

PgAdmin III
www.pgadmin.org
phppgadmin
phppgadmin.sourceforge.net
DbVisualizer
www.minq.se/products/dbvis/
Others, e.g. pgaccess
See sourceforge.net
15

Client Programming Interfaces

psql - Command Line
libpq – C library
ECPG – Embedded SQL
pgtcl – Tcl binding library
Drivers
JDBC
ODBC
DBI: Perl, Python, PHP, etc.
.NET
16

Server Side Languages

PL/pgsql
SQL
C
Other server side languages
PL/perl, PL/pythonu,
PL/R, PL/Tcl, PL/Ruby,
PL/bash, PL/Java
etc.
17

Downloading PostgreSQL
http://www.postgresql.org
By Source: ftp, bittorrent
By CVS tree
In Packages: RPM, Debian
Company Distributions
18

Operating System Distributions

Most Linux like OS distributions

MacOSX:
www.entropy.ch/software/macosx/ postgresql
8.1 Native Win32 Version
pginstaller at pgfoundry.org
Cygwin:
www.cygwin.com
19

Configuration Points
Build Time
Build directives
Installation directory
PL Language options
Server Environment
postgresql.conf, pg_hba.conf
Runtime/Client Environment
PG environment variables
20
Configuration Points
Build Time
As user postgres ...
$ ./configure \
--prefix=/local/pgsql81 \
--with-perl \
--with-python \
--with-tcl \
--enable-depend
$ make
$ sudo make install
21

Initdb -D $PGDATA
Creates Data Directory with:
configuration files
postgresql.conf
pg_hba.conf
template databases
template0
template1
super user database
22
Configuration Points
Server Environment
Global User Configuration
$PGDATA/postgresql.conf
Environment variables for server startup
Access Security
$PGDATA/pg_hba.conf
Host, user and database access.
23
Configuration Points
Global User Configuration
Environment Variables for Server Startup
postgresql.conf
See also:
www.varlena.com/GeneralBits/
Tidbits/#Performance
24
Configuration Points
Global User Configuration
V a ria b le D e fa u lt @ 2G RAM
m ax_ c o n n ec tio n s 100 100
sh ared _ b u ffers 1000 25000
w o rk _ m em 1024 16384
m ain ten an c e_ w o rk _ m em 16384 16384
m ax_ fsm _ p ag es 20000 *
m ax_ fsm _ relatio n s 1000 *
effec tive_ c ac h e_ size 1000 82500
lo g _ d estin atio n std err std err
red irect_ std err o ff on
25
Configuration Points
Global User Configuration
V a ria b le D e fa u lt @ 2G RAM
lo g _ d irecto ry p g _ lo g /varl/lo g /p g sq l
lo g _ m in _ d u ratio n _ statem -1 500
lo g _ lin e_ p refix [% p - % t ]
lo g _ statem en t none ddl
stats_ start_ co llecto r on on
stats_ co m m an d _ strin g o ff on
stats_ b lo ck_ level o ff on
stats_ ro w _ level o ff on
au to vacu u m o ff on
26

Configuration Points Basic Security

# Host DB USER ADDRESS METHOD

# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local:
host all all 127.0.0.1/32 trust
# IPv6 local:
host all all ::1/128 trust
# bad bernie
host all bernie 163.555.9.9 reject
# demo
host demo varlena163.555.9.9 trust
# users
host all all 163.555.9.9 md5
27
Configuration Points
Runtime/Client Environment
Environment Variables
PGHOST – default localhost
PGPORT – default 5432
PGUSER – default $USER
PGDATABASE – default $PGUSER

Different for multiple installations

28
Configuration Points
Session Setting
View: pg_settings
Show values and descriptions
SELECT name, setting, short_desc
FROM pg_settings
ORDER BY name;

What can be set in a session?

SELECT name
FROM pg_settings
where context='user';
29

Housekeeping PostgreSQL Start and Stop

Starting & Stopping PostgreSQL
Installation Specific Script (/etc/init.d)
$ pg_ctl start -D $PGDATA
$ pg_ctl stop
Windows PostgreSQL--> Programs
30

Housekeeping PostgreSQL Logging

Log Maintenance
Rotate Log Settings in postgresql.conf
Alternative:
$ pg_ctl start -D $PGDATA |\
rotatelogs $PGDATA/pglog 86400 2>&1;
Always know where your log file is!
31

Housekeeping PostgreSQL Vacuuming

Autovacuum
Configure in postgresql.conf
Vacuum
$ vacuumdb --analyze --full
Updates Statistics
Improves Performance
Recovers Disk Space
Frequency tuning required
32

Housekeeping PostgreSQL Backing Up

Backup
pg_dumpall > \
.../`date +%Y%m%d`dump.sql
Restore
psql -f 20061231dump.sql

Backup! Now!
No excuses! Really!
33
Monitoring PostgreSQL
Client Server Architecture
pg_stat_activity
Set pg_stats_command in postgresql.conf
ps -alx
Log files
check pgfoundry for log parsers
34

Documentation and Help

Online & Downloadable Docs
Mailing Lists: www.postgresql.org
IRC #postgresql freenode.net
PostgreSQL General Bits :-)
http://www.varlena.com/GeneralBits
35

Creating Databases

$ createdb accounts
36

Adding Users
$ createuser bob
Shall the new role be a superuser?
(y/n) n
Shall the new role be allowed to create
databases? (y/n) y
Shall the new role be allowed to create
more new roles? (y/n) y
CREATE ROLE
37

psql Basics
Always learn help first.
Command Line options
$ psql --help
Backslash Command Help
$ psql db
db=# \?
SQL Help
$ psql db
db=# \help [SQL command]
38

Database Design Elements

Data Types & Sequences
Nulls
Keys
Constraints & Defaults
Triggers, Functions & Operators
Tablespaces
Simple domains
Rules
39

Create Table
AS, LIKE
WITH OIDS
Current default WITH may change
See default_with_oids
Temporary Tables
PRESERVE ROWS, DELETE ROWS, DROP
INHERITS
CONSTRAINTS
TABLESPACE
40

Create Table

CREATE TABLE people (

id SERIAL PRIMARY KEY,
name text,
dept_no int REFERENCES dept(dept_no)
);

CREATE temp TABLE ships_temp as

SELECT ship_id, cargo_no, voyage
FROM ships;
41

Data Types
Integers, big and small Boolean
Serials Geometric
Arbitrary precision–numeric Network Addresses
Floating points Bit Types
Serial Types–Identity Arrays
Character Types Oids
Binary Data, big and small Pseudo Types
Date/Time/Timestamp
42
Data Type Mapping
Integers................. 2, 4, 8 bytes
Serials.................... Identity, Autoincrement
Numeric.................. Money
Floats..................... Arithmetic
Text....................... Character Types
Date/Time/Interval... Dates & Times
Timestamp.............. Timestamps
Boolean................... Boolean
bytea...................... Byte stream, images
43

Keys

Primary Keys
Implemented as B-Tree Unique indexes
Foreign Keys
Implement Referential Integrity.
A FK in table A says that this value references a unique
value in table B.
Cascading updates, deletes
Nulls OK
44

Defaults & Constraints

Initialize column with constants
Check value for validity
UNIQUE, [NOT] NULL, KEYS
CREATE TABLE players (
nick_name text PRIMARY KEY,
team_name text REFERENCES teams(team_name),
age integer CHECK (age > 15) NOT NULL,
games_played integer DEFAULT 0
);
45

Nulls
A NULL is a NULL is a NULL
NULLS are not equal to each other
NULLS are not equivalent to an empty string
NULLS are not equivalent to 0
NULLS are not indexed
46
TableSpaces
Creating a tablespace
CREATE TABLESPACE bd LOCATION '/bigdisk';
Using a tablespace
CREATE TABLE FOO (...) TABLESPACE bd;
Altering a tablespace
alter owner, alter name
Alter a table's tablespace
ALTER TABLE SET TABLE SPACE TO bd;
47
SELECT
Target List – list of columns to be returned
any expression,
aggregate,
subquery,
function,
columns from FROM clause data sources
48
SELECT
FROM – data sources
Tables,
Views,
Set Returning Functions,
SubQueries,
JOINS,
UNIONS
49
SELECT
WHERE – boolean expression qualifying data
Expressions,
Columns,
Functions,
SubQueries
50
SELECT
GROUP BY – scope of Aggregate
Elements of Target List not involved in aggregation.
Determines Break columns

select tname, count(match_id)

from tmatches
group by tname;
51
SELECT
HAVING – boolean expression qualifying aggregates
Expressions usually involving aggregates

select team1, count(matid)

from tmatches
group by team1
having count(matid) > 5;
52

Conditional Statements
COALESCE
coalesce( description,
short_description, 'N/A')

CASE
(select case when $1 is null then
'#ffffff'
else
'#000000'
end)

NULLIF (value1, value2)

NULL if values are equal else value1
53
SubQuery Expressions
Expressions and Lists
EXISTS
WHERE EXISTS (select id from bigtable)

IN
WHERE thisid IN (select id from bigtable)

ANY (SOME)
name = ANY (select user from users)

ALL
due_date > ALL (select milestones from
projects)
54

UNIONS & JOINS

55

JOINS: ON, USING, WHERE

SELECT ...
FROM matches m JOIN events e
USING (matchid)

SELECT ...
FROM matches m JOIN events e
ON (m.matchid = e.m_id)

SELECT ...
FROM matches m, events e
WHERE m.matchid = e.matchid
56
INSERT
Target Table
(Column Names)
VALUES
(Column Values)
Expressions

INSERT INTO tmatches

(matid, team1, team2, score1, score2)
VALUES
(DEFAULT, 'Berkeley', 'KC', 40,2);
57
INSERT
Target Table
SubQuery

INSERT INTO events (ename, year, descr)

SELECT lower(ename), 2006, description
FROM events2006
WHERE lower(ename) not in
(select ename from events);
58
UPDATE
Target Table
SET Column_Name = Value,
Column_Name = Value
expression, value from Target Table, FROM list
FROM
Other Tables
WHERE
DON'T FORGET THE WHERE CLAUSE!
59
UPDATE
UPDATE teams
SET descr = nt.longdescr
FROM newteam_names nt
WHERE teams.sname = nt.sname;
60
DELETE
Table Name
USING
Data Sources (i.e. table list)
WHERE
DON'T FORGET THE WHERE CLAUSE!

DELETE FROM daily_log

where log_ts < (current_date -1)
+ '12:00pm'::time
61
Views
Named Queries
Implemented Using Rules
Can do Updates, Inserts, Deletes via Rules
Usability

CREATE OR REPLACE VIEW phonelist AS

SELECT t.team, p.player, p.name, p.phone
FROM teams t, p.players
WHERE t.team = p.team;
62

Blobs, Slobs and TOAST

Large Objects
special interface lo_
seek, read, write
TOAST
automatic and invisible promotion
INSERT, UPDATE, DELETE
no seek
63

Simple Domains
Subtype Inherits Parent Type
Attributes and
Operators, Functions
May Over Ride
DEFAULT, CHECK
CONSTRAINT, [NOT] NULL
Operators, Functions
64

Simple Domains
May Not Over Ride
Casts
LIKE
AS PRIMARY KEY use UNIQUE INDEX

CREATE DOMAIN degrees float CHECK

(degrees > -180 and degrees <= 180);
65

Built-in Functions & Operators

Logical & Comparison Operators
Math Functions, Aggregates & Operators
Type Conversions
Date, Time & Interval Arithmetic
String and pattern matching
Conditional Statements
66

Functions & Operators

SELECT (('1/1/' || 2006)+ 7*( week - 1 ),
SUM(cookies), scout_name
FROM cookie_sales c JOIN scouts s
USING (s.name),
generate_series(1,53) g(week)
WHERE
date_part('week',c.sales_date) = week
GROUP BY week, scout_name;
67

Functions & Operators: Casts

INTERVAL '2 days 3 hours'
TIMESTAMP '12/31/59'
'gotta wanna'::text
16::bigint
'(1.5,2.7)'::point
123.456::numeric(6,3)
68

Input/Output Functions
Output Format
to_char( ----, text)
timestamp, integer, double precision, numeric

to_char( idate, 'dd-Mon-YYYY');

to_char( price, '999D99');
69

Input/Output Functions
Input Format
to_date(text, text)
to_timestamp(text,text)
to_number(text, text)

to_date( '31 Dec 2006', 'DD Mon YYYY')

to_timestamp( '5/24/06', 'DD/MM/YY');
to_number( '543', '999D99')
70
Functions & Operators
Interval Arithmetic
Regular Arithmetic Expressions
current_date + INTERVAL '5 days'
start_date + duration

Regular Comparison Operators

item_date > due_date
start_date + INTERVAL '5 days' <= due_date
logtime <> last_log
71
Functions & Operators
Date, Time Arithmetic
extract( field FROM src)
extract(epoch FROM
TIMESTAMP '2004-12-31 01:43:03');
extract(hours FROM
INTERVAL '2 days 5 hours');

age( timestamp )
age('12/31/1959');
72
Functions & Operators
Interval Arithmetic
(start, end) OVERLAPS (start2, end2)

(proposed_start, proposed_end)
OVERLAPS
('12/23/06'::date, '1/4/06'::date)

(sessiontime, INTERVAL '1 hour')

OVERLAPS
(breaktime, INTERVAL '15 minutes')
73
Functions & Operators
String and Pattern matching
LIKE, ILIKE or ~~, ~~*
city LIKE 'San_%'
city ~~ 'San_%'
city ILIKE 'oak%'
city ~~* 'oak%'

SIMILAR TO or ~, ~*
name SIMILAR TO
'(Mr.|Ms.) [A-Z]([ a-z])*'
74
Indexing Operators
create index uname_idx
on users (user_name);

create index ttnotes_idx

on trouble_tickets(ticket_id, note_id);

create index range_idx

on cows USING RTREE (range);
75

Functional Indexing
Functional indexes
Result of any immutable procedure
create index tsdate_idx on
log_table date(createtimestamp);
create name_idx on
users lower(user_name);
Expressional indexes
Result of any immutable expression
create overdue_idx
on books duedate + '30 days'
76

Partial Indexing
Indexes over parts of tables
create index active_clients on clients
where status = 'A';
create index currentyear on accounts
where reg_date = '2005';
77

Server Side Languages

PL/pgsql and SQL Primary languages
Query & Trigger enabled
Trusted vs. untrusted languages
Available server side languages
PL/perl, PL/pythonu,
PL/R, PL/Tcl, PL/Ruby,
PL/bash
C, etc.
78

Server Side Functions

CREATE FUNCTION foo(text, integer)
RETURNS integer AS
$$
...
$$
LANGUAGE 'plpgsql' [OPTIONS...]
79
PlPgSQL Trigger Functions
Executes once per row
Often Used for
complex or dynamic defaults
logging
80
Triggers
Function executed per Row
Before or After Event
Insert, Update or Delete
CREATE OR REPLACE FUNCTION lastmod
RETURNS TRIGGER AS $$
BEGIN
NEW.last_modified = now();
RETURN NEW;
END;
$$ LANGUAGE 'plpgsql';
CREATE TRIGGER team_upd
BEFORE INSERT OR UPDATE on teams
FOR EACH ROW EXECUTE PROCEDURE lastmod();
81

Rules
Re-Write a Query
Action On a Table or View
Select Rules Implement Views
Updateable Views Implemented via Rules
82

Rules View
Example View:
CREATE VIEW matches_v
SELECT m.matchname, m.matchid,
t1.team AS team1, t2.team AS team2,
t1.teamid as t1id, t2.teamid as
t2id,
e.eventname, m.eventid
FROM matches m JOIN teams t1 USING
(t1.id=teamid)
JOIN teams t2 USING (t2.id=teamid)
JOIN event e ON (eventid);
83

Rules Implement a View

(Implicit)
CREATE RULE “_RETURN” AS ON
SELECT TO matches_v DO INSTEAD
SELECT...;
84

Rules Implement a View

CREATE RULE upd_matches
AS ON UPDATE TO matches_v
DO INSTEAD
UPDATE matches
SET matchname=NEW.matchname,
eventid=NEW.eventid,
t1id=NEW.t1id, t2id=NEW.t2id
WHERE matchid=OLD.matchid;
85

Rules
CREATE RULE ins_matches
AS ON INSERT TO matches_v
DO INSTEAD
INSERT INTO matches
(matchid, eventid, t1id, t2id,
matchname)
VALUES
(default, NEW.eventid, NEW.t1id,
NEW.t2id, NEW.matchname);
86

Rules
CREATE RULE del_matches
AS ON DELETE TO matches_v
DO INSTEAD
DELETE FROM tmatches
WHERE matchid=OLD.matchid
87

Operators

Create first class operators

Implemented by functions
Use the same way as ordinary built-in operators.
Natural cost overhead.
88

Tuning Queries
The usual suspects
DID YOU VACUUM?
Type mismatch
Indexing Expressions
GUC configurations
Explaining Explain
[email protected]
89
Explain
Explain [analyze] [verbose]
OP (cost=n...n rows=n width=n)
(actual time=t..t rows=n loops=n)
OP cond: (...)
-> OP (cost=...)(actual time=...)
OP cond: (...)
Look for
Seq Scan, Hash Join,
Subquery, Hash,
Index Scan
Index usage
90
Replication Products
SLONY-1
Mammoth Replicator Command Prompt, Inc.
pgpool (client side)
postgres-r, dbmirror async, Rserv async, clustgres,
pglcluster, osogres (client side replication)
91

References
www.postgresql.org
www.varlena.com/GeneralBits
Mailing Lists
general, sql, novice, interfaces
hackers
advocacy
performance, bugs
docs
IRC #postgresql freenode.net