Chapter 2 - Communication: Distributed Systems (IT 441)
Chapter 2 - Communication: Distributed Systems (IT 441)
Chapter 2 – Communication
2
Introduction
Interprocess communication is at the heart of all distributed
systems
communication in distributed systems is based on message
passing as offered by the underlying network as opposed to
using shared memory
modern distributed systems consist of thousands of
processes scattered across an unreliable network such as
the Internet
unless the primitive communication facilities of the network
are replaced by more advanced ones, development of large
scale Distributed Systems becomes extremely difficult.
3
2.1 Layered Protocols
two computers, possibly from different manufacturers, must
be able to talk to each other
for such a communication, there has to be a standard
The ISO OSI (Open Systems Interconnection) Reference
Model is one of such standards - 7 layers
TCP/IP protocol suite is the other; has 4 or 5 layers
OSI
Open – to connect open systems or systems that are open
for communication with other open systems using standard
rules that govern the format, contents, and meaning of the
messages sent and received
these rules are called protocols
two types of protocols: connection-oriented and
connectionless
4
layers, interfaces, and protocols in the OSI model
5
Media (lower) Layers
Physical: Physical characteristics of the media
Data Link: Reliable data delivery across the link
Network: Managing connections across the network
or routing
Transport: End-to-end connection and reliability
(handles
lost packets); TCP (connection-oriented),
UDP (connectionless), etc.
Session: Managing sessions between applications
(dialog control and synchronization); rarely
supported
Presentation: Data presentation to applications; concerned
with the syntax and semantics of the
information transmitted
Application: Network services to applications; contains
protocols that are commonly needed by
users;
Host (upper) FTP, HTTP, SMTP, ...
Layers
6
a typical message as it appears on the network
7
Transport Protocols: Client-Server TCP
10
2.2.1 Basic RPC Operation
Conventional Procedure Call, i.e., on a single machine
e.g. count = read (fd, buf, bytes); a C like statement, where
fd is an integer indicating a file
buf is an array of characters into which data are read
bytes is the number of bytes to be read
Stack pointer
Stack pointer
The client program implements the client program and links it with the
client-side stubs.
The stubs are responsible for managing all details of the remote
communication between client and server.
13
RPC Stubs
14
Steps of a Remote Procedure Call
15
16
Steps of a Remote Procedure Call
1. Client procedure calls client stub in the normal way
2. Client stub builds a message and calls the local OS
(packing parameters into a message is called parameter
marshaling)
3. Client's OS sends the message to the remote OS
4. Remote OS gives the message to the server stub
5. Server stub unpacks the parameters and calls the server
6. Server does the work and returns the result to the stub
7. Server stub packs it in a message and calls the local OS
8. Server's OS sends the message to the client's OS
9. Client's OS gives the message to the client stub
10. Stub unpacks the result and returns to client
hence, for the client remote services are accessed by making
ordinary (local) procedure calls; not by calling send and
receive
server machine vs server process; client machine vs client process
17
2.2.2 Parameter Passing
o The function of the client stub is to take its parameters, pack them into a mes
sage, and send them to the server stub. While this sounds straightforward, it is
not quite as simple as it at first appears.
1. Passing Value Parameters
e.g., consider a remote procedure add(i, j), where i and j are
integer parameters
19
original message on the Pentium
(the numbers in boxes indicate the address of each byte)
the message after receipt on the SPARC; wrong integer (5*224) 83886080, but
correct string
20
one approach is to invert the bytes of each word after receipt
the message after being inverted (correct integer but wrong string)
21
2.3 Remote Object (Method) Invocation (RMI)
resulted from object-based technology that has proven its
value in developing nondistributed applications
it is an expansion of the RPC mechanisms
it enhances distribution transparency as a consequence of
an object that hides its internal from the outside world by
means of a well-defined interface
Distributed Objects
an object encapsulates data, called the state, and the
operations on those data, called methods
methods are made available through interfaces
the state of an object can be manipulated only by invoking
methods
this allows an interface to be placed on one machine while
the object itself resides on another machine; such an
organization is referred to as a distributed object
the state of an object is not distributed, only the interfaces
are; such objects are also referred to as remote objects 22
23
24
25
the implementation of an object’s interface is called a proxy
(analogous to a client stub in RPC systems)
it is loaded into the client’s address space when a client
binds to a distributed object
tasks: a proxy marshals method invocation into messages
and unmarshals reply messages to return the result of the
method invocation to the client
a server stub, called a skeleton, unmarshals messages and
marshals replies
26
common organization of a remote object with client-side proxy
27
The Stub and Skeleton
call
skeleton
Stub
RMI Client RMI Server
return
28
the situation when passing an object by reference or by value
29
2.4 Message Oriented Communication
RPCs and RMIs are not adequate for all distributed system
applications
the provision of access transparency may be good but
they have semantics that is not adequate for all
applications
example problems
they assume that the receiving side is running at the
time of communication
a client is blocked until its request has been processed
30
2.4.1 Persistence and Synchronicity in
Communication
assume the communication system is organized as a
computer network shown below
Persistent Transient
Asynchronous
33
persistent asynchronous communication persistent synchronous communication
34
transient asynchronous communication receipt-based transient synchronous
communication
35
delivery-based transient synchronous response-based transient synchronous
communication at message delivery communication
the sender is blocked until the strongest form; the sender is
message is delivered to the blocked until it receives a reply
receiver for further processing; message from the receiver
e.g., asynchronous RPC
36
2.4.2 Message-Oriented Transient Communication
many applications are built on top of the simple message-
oriented model offered by the transport layer
standardizing the interface of the transport layer by
providing a set of primitives allows programmers to use
messaging protocols
they also allow porting applications
1. Berkley Sockets
an example is the socket interface as used in Berkley
UNIX
a socket is a communication endpoint to which an
application can write data that are to be sent over the
network, and from which incoming data can be read.
37
Primitive Meaning Executed by
Socket Create a new communication endpoint; also
reserve resources to send and receive messages both
Bind Attach a local address to a socket; e.g., IP
address with a known port number
Listen Announce willingness to accept connections; for
connection-oriented communication
Accept Block caller until a connection request arrives
servers
Connect Actively attempt to establish a connection; the
client is blocked until connection is set up
Send Send some data over the connection
Receive Receive some data over the connection
Close Release the connection
socket primitives for TCP/IP
38
connection-oriented communication pattern using sockets
39
2.4.3 Message-Oriented Persistent Communication
there are message-oriented middleware services, called
message-queuing systems or Message-Oriented Middleware
(MOM)
they support persistent asynchronous communication
they have intermediate-term storage capacity for messages,
without requiring the sender or the receiver to be active
during message transmission
unlike Berkley sockets and MPI, message transfer may take
minutes instead of seconds or milliseconds
Message-Queuing Model
applications communicate by inserting messages in
specific queues
it permits loosely-coupled communication
the sender may or may not be running; similarly the
receiver may or may not be running, giving four possible
combinations
40
messages are managed by queue managers
they generally interact with the application that sends and
receives messages
some also serve as routers or relays, i.e., they forward
incoming messages to other queue managers
however, each queue manager needs a copy of the queue-
to-location mapping, leading to network management
problems for large-scale queuing systems
the solution is to use a few routers that know about the
network topology
41
the general organization of a message-queuing system with routers
42
2.4 Stream Oriented Communication
until now, we focused on exchanging independent and
complete units of information
time has no effect on correctness; a system can be slow or fast
however, there are communications where time has a critical
role
Multimedia
media
storage, transmission, interchange, presentation,
representation and perception of different data types:
text, graphics, images, voice, audio, video, animation, ...
movie: video + audio + …
multimedia: handling of a variety of representation media
end user pull
information overload and starvation
technology push
emerging technology to integrate media 43
Types of Media
two types
discrete media: text, executable code, graphics, images;
temporal relationships between data items are not
fundamental to correctly interpret the data
continuous media: video, audio, animation; temporal
relationships between data items are fundamental to
correctly interpret the data
a data stream is a sequence of data units and can be applied
to discrete as well as continuous media
stream-oriented communication provides facilities for the
exchange of time-dependent information (continuous media)
such as audio and video streams
44
timing in transmission modes
asynchronous transmission mode: data items are
transmitted one after the other, but no timing constraints;
e.g. text transfer
synchronous transmission mode: a maximum end-to-end
delay defined for each data unit; it is possible that data can
be transmitted faster than the maximum delay, but not slower
isochronous transmission mode: maximum and minimum
end-to-end delay are defined; also called bounded delay
jitter; applicable for distributed multimedia systems
a continuous data stream can be simple or complex
simple stream: consists of a single sequence of data; e.g.,
mono audio, video only
complex stream: consists of several related simple streams
that must be synchronized; e.g., stereo audio, video
consisting of audio and video (may also contain subtitles,
translation to other languages, ...)
45
movie as a set of simple streams
46
a stream can be considered as a virtual connection between a
source and a sink
the source or the sink could be a process or a device
48
Quality of Service (QoS)
QoS requirements describe what is needed from the
underlying distributed system and network to ensure
acceptable delivery; e.g. viewing experience of a user
for continuous data, the concerns are
timeliness: data must be delivered in time
volume: the required throughput must be met
reliability: a given level of loss of data must not be
exceeded
quality of perception; highly subjective
49
QoS Dimensions
timeliness dimensions
latency (maximum delay between consecutive frames)
start-up latency (maximum delay before starting a
presentation)
jitter (delay variance)
volume dimensions
throughput in frames/sec or bits/sec or bytes/sec
reliability dimensions
MTBF (Mean Time Between Failure) of disks
MTTR (Mean Time To Repair)
error rates on the telecommunication lines
50
QoS Requirements
deterministic
precise values or ranges
e.g., latency must be between 45 and 55 ms
probabilistic
probability of the required QoS
e.g., latency should be < 50 ms for 95% of the frames
stochastic distributions
e.g., frame arrival should follow normal distribution with
mean interval-time of 40 ms and 5 ms variance
classes
e.g., guaranteed and best effort
51
QoS Management
can be static or dynamic
Static QoS Management Functions
specification
e.g., deterministic range for timeliness, volume and
reliability categories
negotiation
the application may accept lower level of QoS for
lower cost
admission control
if this test is passed, the system has to guarantee the
promised QoS
resource reservation
may be necessary to provide guaranteed QoS
52
Dynamic QoS Management Functions
monitoring
notices deviation from QoS level
at a certain level of granularity (e.g., every 100 ms)
policing
detect participants not keeping themselves to the contract
e.g., source sends faster than negotiated (e.g., 25 fps)
maintenance
sustaining the negotiated QoS
e.g., the system requires more resources
renegotiation
client tries to adapt – may be can accept lower QoS
53
QoS requirements can be specified using flow specification
containing bandwidth requirements, transmission rates,
delays, ...
e.g. by Partridge (1992)
it uses the token bucket algorithm which specifies how the
stream will shape its network traffic (in fact the leaky
bucket, as used in networking)
the idea is to shape bursty traffic into fixed-rate traffic by
averaging the data rate
packets may be dropped if the bucket is full
the input rate may vary, but the output rate remains
constant
54
the principle of a token bucket algorithm
55
problem in flow specification
an application may not know its requirements
how can a user (human) specify quality using the various
parameters? usually very difficult
may be provide defaults for various streams as high,
medium, low quality
Setting up a Stream
resources such as bandwidth, buffers, processing power
must be reserved once a flow specification is made
on such protocol is RSVP - Resource reSerVation Protocol
it is a transport-level protocol for enabling resource
reservation in network routers
56
the basic organization of RSVP for resource reservation in a distributed system
57
Stream Synchronization
how to maintain temporal relations between streams, e.g., lip
synchronization
two approaches
1. explicitly by operating on the data units of simple
streams; the responsibility of the application
59