Systems Engineering
Systems Engineering
Prepared for
12. Sponsoring Agency Name and Address 13. Type of Report and Period Covered
Department of Transportation
Intelligent Transportation Systems Joint Program Office
400 Seventh Street, SW – Room 3416
Washington, DC 20590 14. Sponsoring Agency Code
HOIT
15. Supplementary Notes
William S. Jones – Task Manager
16. Abstract
This monograph is intended to introduce the topic of systems engineering to managers and staff working on
transportation systems projects, with particular emphasis on Intelligent Transportation Systems (ITS)
projects. Systems engineering is a discipline that has been used for over 50 years and has its roots in the
building of large, complex systems for the Department of Defense. Systems engineering is an approach to
building systems that enhances the quality of the end result and the expectation is that its application to
transportation systems projects will make those projects more effective in developing and implementing the
systems they are intended to build. Although applying systems engineering techniques on a project doesn’t
guarantee success, not following a systems engineering approach is a strong recipe for failure.
This monograph presents a common approach to systems engineering, one that is followed in many
industries and domains, not just by Defense Department contractors. This approach emphasizes the
combination of technical and management activities that produce a disciplined approach to building
systems. And, although anyone from a technical or scientific discipline can learn to practice good systems
engineering techniques, the most effective systems engineers are those who also bring domain knowledge
about the system being built. Since the domain knowledge in transportation systems involves knowledge of
transportation systems, it is important to being familiarizing transportation engineers about this discipline.
This monograph is intended for use in conjunction with the systems engineering courses being offered by
the National Highway Institute.
Section Page
Chapter 1 - Introduction 1
Purpose of This Document 1
Some Conventions Used in the Document 1
Intended Audience 1
What is Systems Engineering? 2
Why is Systems Engineering So Difficult to Define? 3
Why Use Systems Engineering? 4
The Challenges in System Engineering 5
Identifying and Evaluating System Alternatives 5
Managing Uncertainty and Risk in Our Systems 7
Designing Quality into Our Systems 9
Handling Project Management Issues that Arise 11
Our Approach to Illustrating Systems Engineering Concepts 13
iii
Benchmarks 55
Technical Performance Measures 55
Appendix 61
Bibliography 67
iv
List of Tables and Figures
Tables
1 Key Questions 8
2 AVI Tag Trade-Off Study Matrix 23
3 Comparison of Relative Costs by System Life Cycle Stage 29
4 SE-CMM Process Area Groupings 36
5 Alternative Assessment Matrix 49
6 List of Representative Standards 58
Figures
1 Project Resolutions 4
2 Assessment of Importance vs. Uncertainty 7
3 Statement of the Problem 14
4 Concept of Operations Outline 19
5 Sample Operation Scenario 21
6 “V” Diagram 26
7 Sample Work Breakdown Structure 40
8 Requirements Traceability Matrix 44
9 Sample Gantt Chart 51
10 Activity Diagram 51
v
Executive Summary
Systems engineering is an approach to building systems that enhances the quality of the
end result. It has its roots in the development of large, complex systems for the
Department of Defense. Systems engineering has been around for over 50 years and
there are many books and articles that cover it in great detail. Our goal is to introduce
systems engineering to managers and staff working on Intelligent Transportation Systems
(ITS) projects, if they aren’t already familiar with its practice. We believe that they will
become more effective in acquiring, developing, and implementing ITS systems by
following systems engineering practices. We prepared this monograph to introduce
transportation professionals to established systems engineering practices that have proven
successful in other domains.
Although our primary focus is on ITS project managers, we hope our coverage of this
topic encourages others to learn more about this important area. Applying systems
engineering techniques on a project doesn’t guarantee success; not following a systems
engineering approach, however, is a strong recipe for failure.
Those who saw it as a function solely of systems engineers, but consisting of both
technical and management activities
Those who believed that it consisted only of technical activities that anyone from
a technical or scientific discipline could do
vii
Those who saw it as set of technical and management activities that anyone from
a technical or scientific discipline could do
Our position on the subject falls into the last category. As we discuss throughout this
paper, systems engineering combines technical activities and management activities to
produce a disciplined approach to building systems.
Although anyone from a technical or scientific discipline can learn to practice good
systems engineering techniques, to be most effective as a systems engineer, a person
must have domain knowledge about the system being built. Domain knowledge is a
fundamental understanding of the technology and functions involved in the system being
built. In an ITS system, for example, domain knowledge includes transportation
engineering or transit system management. Without domain knowledge, a systems
engineer is not as effective. However, if you have to choose between using systems
engineers who don’t have domain knowledge and not using any systems engineers, use
the systems engineers and complement them with staff who do have domain knowledge.
What every ITS project manager wants is a successful system at the end of the project,
with “success” measured by how well the system satisfies the requirements of the people
who use it. So a goal-oriented ITS project manager wants to use any tools or techniques
that help achieve success. Systems engineering provides those tools and techniques.
Systems engineering helps accomplish four key activities that impact a project’s success.
These are:
viii
the technical, cost, and schedule parameters that we’ve set, it may come down to which
alternative offers the least implementation risk.
Managing risk and uncertainty in our systems development efforts is important because
we want to avoid mistakes or potential problems that threaten the success of our work.
Systems engineering focuses on three aspects of risk management: identification,
analysis, and mitigation.
Designing quality into our systems is done by addressing those factors that can negatively
affect quality. Paraphrasing the International Organization for Standardization (ISO), we
can define quality as “the totality of features of a system that bear on its ability to satisfy
stated or implied needs.” Among the factors that can negatively affect the quality of a
system are its complexity, its inflexibility, its lack of standardized components, and its
reliability and availability. A complex system is hard to use and maintain. While it may
be necessary for a system to be complex, our goal should be to keep it as simple as
possible. An inflexible system doesn’t adapt well to change; when its environment
changes or when we must add to it or modify it to deal with changes in our needs, it may
fail us. Systems that lack standardized components are difficult to maintain. When we
must replace some part of the system, we may not be able to find a replacement part.
This can increase overall system maintenance costs or cause the system to fail while
operating. We can’t use systems effectively if they are not reliable and available. An
unreliable system breaks down while we’re trying to use it; an unavailable system isn’t
there to be used when we need it.
Handling project management issues that arise is easier to do if we have a good project
plan to start with. A good project plan should be complete, comprehensive, and
communicated. We can ensure the completeness and comprehensiveness of the plan by:
We must also communicate the plan to everyone that it affects. We must tell those
people assigned to work on project tasks:
ix
Having done that, we must then track each task, measure its progress, revise the overall
plan if needed, and identify and address any obstacles that impede our progress. These
are standard project management activities, but project management is an important
element in a good systems engineering program.
Systems engineering is a process, not just a set of tools. As such, systems engineering
activities occur throughout the system development life cycle. A common set of stages in
a system development life cycle and the systems engineering technical activities that
accompany them include the following:
Conception. The stage in which the need for a system (or major system
enhancement) is first identified. The principal systems engineering activities in
this stage revolve around feasibility assessment. Is the system feasible? Can a
feasible system be built in a reasonable time and at a reasonable cost? How much
time will we need to build this system? How much should it cost? These are the
types of questions that a systems engineer focuses on during this stage.
Design. During this stage, the systems engineer helps flesh out the details of the
system, helping make decisions about the best way to satisfy the system’s
requirements. To help overcome technical uncertainty, the systems engineer may
conduct trade-off studies, use modeling and simulation to analyze potential
system performance, build prototypes to assess the technical feasibility of a
proposed solution, or perform in-depth analyses of different technologies to assess
their applicability to the system under development. The systems engineer also
conducts design reviews with project stakeholders, to ensure that the design
approach selected is consistent with their needs and expectations.
x
development and ensure that these standards are followed. But the systems
engineering activities aren’t solely directed at software. It is also important to
ensure that hardware developed (or modified) during this stage also matches the
agreed-upon design. Hardware inspections are one technique for quality control
during this stage.
Testing. Systems engineers create complete and comprehensive test sets that
effectively exercise the system and its components. They also analyze test results
and assess the degree to which the system satisfies its requirements.
System Acceptance. Some of the key systems engineering activities during this
stage look at the training of system users and the approach for fielding the system.
New systems, or enhanced versions of existing systems, need to be fielded in a
manner that does not disrupt normal business operations. Systems engineers plan
these “rollout” activities to effect a smooth transition to the new system. This
stage also involves the final testing of the system by its users, to ensure that it
meets their requirements.
In addition to the technical activities that systems engineers perform, systems engineering
also plays an important role in project and program management. Systems engineering
activities in project and program management span multiple system development life
cycle stages. Management activities associated with systems engineering fall into four
major categories:
xi
Project planning and control. Planning is an essential element of systems
engineering. Without good plans, it’s difficult to know what to do and when to do
it. But control is also important. Control involves two key areas: 1) monitoring
project activities to determine task status, and 2) initiating corrective action when
problems arise. To monitor project tasks, the systems engineer sets up feedback
mechanisms that provide accurate and timely status information. These feedback
mechanisms also identify problems that impede progress. When a problem arises,
the systems engineer shifts into analysis, to assess the impact of the problem and
to determine ways to eliminate it or to minimize its effect.
Every ITS project has “success” as a goal. Success is delivering a sound, reliable system
that meets user needs and gets done within schedule and within budget. Your best chance
at achieving success comes from introducing good systems engineering practices on your
project from the start and continuing to use them until the project ends (and beyond).
Let’s consider what some of these practices are.
Planning
Good systems engineering starts with planning. Your plan is your roadmap for getting to
success. Systems engineering looks at the plan from the broadest possible view,
considering all aspects of the project – from onset to operation and maintenance.
xii
Systems engineering plans each step along the way and plans how you’re going to
address the potential road blocks that you may find as you move from beginning to final
delivery. While it isn’t always possible to plan long projects in detail at the beginning,
you can usually plan the early stages of a project in detail and the rest at a higher level.
Then, as you accomplish the tasks planned in detail, you learn enough about how the rest
of the work should be done and can increase the level of detail in subsequent months.
This is a continual planning approach that lets you enhance your chances of planning the
right work and planning the work right.
Your plans should synchronize with your cost estimates for the work to be done. As you
refine your plans, you must refine your cost estimates. Your initial cost estimate for your
project sets a ceiling that likely is difficult to raise. Your re-planning and re-costing
efforts must aim at staying within that cost ceiling. However, if it appears that the
original target was underestimated, your continual re-planning and re-costing gives you
an early indication of cost growth while you can still control the growth.
In systems development, the quality inspection process consists of the reviews and audits
that you perform. You review documents, requirements, designs, program code, test
results, and anything else that might give you an indication of the quality of the system
that you’re building. Audits ensure that you know, at any time, what your system
consists of and where all of its parts reside. Many texts on systems engineering, and even
the systems engineering standards that exist, describe the types of reviews and audits that
you can conduct. As you introduce the practice of systems engineering, these should be
your references.
We can never be certain about the future. However, we must make decisions when
implementing a system that assume some confidence in what the future holds. When
uncertainty is cause by lack of knowledge or lack of information, we can apply systems
engineering practices to help reduce that lack. Systems engineers use mathematical and
software tools and engineering techniques to increase our knowledge and information.
Prototyping, for example, is frequently used to gain a better understanding of what
human-computer interfaces best suit the needs of our system’s users. We can also use it
as a “proof of concept” technique, where we integrate components (hardware and
software) that have not been integrated before. In integrating these components, we learn
xiii
what works and what doesn’t work as an integrated unit. We also learn how to integrate
the components effectively (or at least, how not to).
Another tool that systems engineers use is simulation modeling. A simulation can test
different assumptions about how a system will work, under different conditions of load,
before that system is built. That helps identify the bottlenecks in the system that
negatively affect performance and provides us clues about how to eliminate those
bottlenecks. Simulations can also help us identify when and where processes interfere
with one another. This allows us to revise the processes and improve workflow.
Any tool or technique that reduces our uncertainty helps reduce the risk involved in
building a system. The more we know about the possible outcomes of a decision or
approach we plan to take, the more easily we can avoid the undesirable outcomes. This is
one aspect of risk management. But there are other aspects as well. Risks can come
from “threats,” i.e., outside events that can have a negative impact on our efforts. “Threat
analysis” identifies those potential events that may harm us and allows us to estimate the
potential cost to our project of each event’s occurrence. Once we have identified
“threats,” we can plan our strategy for mitigating their ability to harm us.
A Final Word
Systems engineering is hard work. It brings as a reward the increased probability that we
will successfully implement a useful, reliable system. This paper won’t make you a
systems engineer, but it does introduce the topic and provides you with a framework for
understanding its value. We hope you find it useful.
xiv
1.0 Introduction
Purpose of This Document
This document introduces the concept and practice of systems engineering, and discusses
why it applies to the acquisition, development, and fielding of Intelligent Transportation
Systems (ITS). Systems engineering is a vast topic area and you can find many books
and articles that cover it in detail. You’ll find some of them listed in the References at
the end of this document. Reading this document (or those books and articles) won’t turn
you into a systems engineer; that takes experience as well as more in-depth knowledge of
systems engineering topics. But our goal is to provide you with an overview of the
subject and, we hope, spark your interest in pursuing it in more detail.
Throughout this document, we’ll mark “key” ideas by using a key symbol and some text
that summarizes the idea we want to emphasize. This marker looks like the following
example:
We’ll float the key idea box in the margin next to the text that we are
emphasizing. This should help you extract the important concepts you’ll
use when implementing systems engineering in your projects.
We’ll also use some examples to get our points across. We’ll use examples
This is a that are related to transportation and ITS projects whenever possible, but
“key” may bring in ones from other areas when they help make the point. We’ll
mark our examples with simple graphics as well. When we want to talk
idea
about schedules, costs, and technical areas, we’ll use the following
example graphics:
Intended Audience
1
Project managers of ITS projects
Project team members on ITS projects
Contractors and staff on ITS projects
Anyone else interested in quality results in systems implementation
It is primarily geared, however, to ITS project managers, to help them succeed in their
efforts. Applying systems engineering in a project effort doesn’t guarantee success;
external factors can always cause a project to fail. However, experience shows that not
applying a systems engineering approach when implementing a system is a good way to
fail. So, let’s look at what systems engineering is all about.
Our intent in providing such varying definitions is not to confuse you (or to poke fun at
academia), but to illustrate that there is no one accepted, “perfect” definition of the term.
1
Taken from a University of Virginia Department of Systems Engineering web page,
http://freney.sys.virginia.edu/sys-def.html.
2
Why is Systems Engineering So Difficult to Define?
The term, “systems engineering,” was coined in the 1950s to describe a process used by
engineers working on large and complex systems, usually incorporating computers and
software. A significant body of work in systems engineering derives from the US
military’s efforts to build large command, control, and communications defense systems
and the need to ensure that those systems worked when fielded. Since the Department of
Defense (DoD) hired contractors to build most of those systems, the DoD wanted those
contractors to follow an approach that gave the highest possible likelihood of overall
success, regardless of whom the DoD selected to build a particular system. The DoD
developed systems engineering standards and required contractors to conform to them.
Over time, these standards have evolved and matured and become accepted by more than
just defense contractors.
But systems engineering applies to the building of any large system, regardless of type.
As practitioners began to write textbooks for others to use, some followed the DoD
model and others looked at the basic principles involved, without necessarily using the
formalism of the DoD model. Thus, there are diverse descriptions and definitions
throughout the literature of systems engineering. If you look at any text among the
references in this paper, you will find several with at least two different definitions.
People working in this field don’t always agree on what constitutes systems engineering.
When the International Council on Systems Engineering (INCOSE) was formed in 1990,
all who attended the founding session had to contribute a definition of systems
engineering. No two were the same, but they could all be grouped into one of the
following categories:
Those who saw it as a function solely of systems engineers, but consisting of both
technical and management activities
Those who believed that it consisted only of technical activities that anyone from
a technical or scientific discipline could do
Those who saw it as set of technical and management activities that anyone from
a technical or scientific discipline could do
3
control system, domain knowledge includes understanding what an air traffic controller
does.
A lot of what’s involved in systems engineering is common sense and forethought – and
good engineering principles that you learn in any engineering program. Systems
engineering is practiced along with many other engineering disciplines, whenever
complex systems are involved.
Figure 1
Project Resolutions
The use of systems engineering techniques wasn’t the sole cause for the improvement,
but their use did play a key role.
4
When we apply systems engineering on any project, there are at least four major things
that we want to accomplish. These are:
When building a system, we make choices about how to accomplish our goal. Systems
engineering helps us make better choices by providing a framework within which to
make those choices, one that helps us analyze and understand the consequences of our
choices. If we can compare the consequences of our choices on an even par, we improve
the likelihood that we will make a good choice. Again, we’re not dealing with a crystal
ball that predicts the future accurately. We’re going to use analysis and judgment and
make estimates about what will happen.
5
is the one that gives us the most capacity, because that provides the greatest
flexibility and potential to handle growth, is the most reliable, because we avoid
transmission errors and interruptions, and the most available (which may require
building in some redundancy), since that provides the most consistent service.
However, the best technical solution isn’t necessarily the best solution. To
determine which one is best overall, we also need to consider cost and schedule
factors.
Cost feasibility: This is our judgment on whether the system we want can be built
and operated for the money we intend to spend, using a specific design
alternative. We must consider each design alternative’s life-cycle costs. We have
to look at the initial system development costs, the costs to operate the system
(including people costs), and the costs to maintain it (both hardware and
software), including the cost of supplies associated with the system during its
operation. We evaluate these costs over the expected life of the system, i.e., the
amount of time we expect the system to be in use. If a system can be built for the
development money that we have, but costs too much to operate and maintain, it
does not have cost feasibility. Take the alternative that we assessed as the best
technical solution above, the one with the most capacity. If the life-cycle costs of
that alternative are beyond what we believe we can afford to invest in this
solution, we can’t consider it the “best” solution. Instead, we have to go with the
“better” solution, if that solution is affordable.
Schedule feasibility: This is our judgment on whether the system we want can be
built, using a particular design alternative, within our implementation time frame.
There may be external factors to consider, such as whether a supplier can provide
one or more necessary components on time. There are always internal factors,
such as staff availability for our project and staff skills, that we must consider
when assessing schedule feasibility. If a particular design alternative can’t be
used successfully by our deadline, we have two choices: eliminate it or change the
deadline.
What we’re after in assessing the technical, cost, and schedule aspects of our alternatives
is to choose the alternative that is the best combination of the three. It must be
technically feasible, it must have a reasonable (affordable) cost, and it must be one that
we can build in the required time frame. That’s the solution that our trade-off study
should lead us to.
6
Managing Uncertainty and Risk in Our Systems
If we could accurately predict the future, it would be easy to avoid mistakes and
problems. However, in real life, we need to deal with uncertainty and risk. Systems
engineering focuses on three aspects of uncertainty and risk management:
Identification
Analysis
Mitigation
Identification deals with ferreting out the issues and potential problems that can
affect our project. Table 1 contains a set of questions that you can ask yourself, to
help identify issues and potential problems that you might have. The questions are at
a high level. As you answer them, other, more detailed questions will arise that you
must also address.
Analysis assesses the importance of each issue and problem and determines what we
know about it. One way to analyze the issues and problems is to rank-order them
according to their importance and the degree of uncertainty in our knowledge about
them. This organizes them into a 3x3 matrix, as shown in Figure 2.
Figure 2
Assessment of Importance vs. Uncertainty
The upper right-hand cell of our matrix is the one of greatest concern. These are
highly important issues and problems about which we have a high degree of
uncertainty, i.e., the inverse of knowledge. The lower left-hand cell is of least
significance. The seven other cells are all addressed according to whatever
scheme we deem best, but generally we address the issues in each cell, moving
from left to right, top to bottom, as we move between cells.
7
Table 1
Key Questions
8
Area Key Questions
System Do we have clear criteria for system completion?
Acceptance Have all users agreed with our completion criteria?
Will our customers be satisfied with the system?
Will we have adequate system documentation for all users and maintainers?
Operation and Have we assessed the full life-cycle costs of the system, including training,
Maintenance operation, and maintenance?
Have we identified who will maintain the system?
Do we have the system maintainer on-board?
Do we have a schedule for upgrades and/or enhancements to this system?
What growth in demand have we planned for?
Analysis continues with an evaluation of what it will cost us, in terms of time,
money, and effort, to increase our knowledge about each of these items so that we
are comfortable with the degree of certainty that we have. Once we know that,
we can decide how we want to invest our resources (time, money, and effort) in
mitigation.
Mitigation involves performing some action to gain knowledge and reduce the
risk related to an item. The types of actions that we can perform include:
Prototyping
Modeling and simulation
Experimentation
Trade-off studies
Market assessments
Surveys and questionnaires
While that list is not comprehensive, it illustrates ways we can reduce uncertainty
and mitigate risk.
A US automobile manufacturer once had as its slogan, “Quality is Our Most Important
Product.” For systems engineers, that’s a motto to take to heart. Systems engineering is
all about building quality into systems. Consider the following definition from one of the
world’s most important standards-making bodies, the International Organization for
Standardization (ISO), which defines quality as:
“…the totality of features of a product or service that bear on its ability to satisfy
stated or implied needs.”
If we replace the phrase “product or service” with the word “system,” the definition fits
our needs well.
Systems engineering helps us design quality into our systems by giving us tools to
address factors that can affect quality negatively. Among these are:
9
Complexity: If a system is complex, it consists of many different, dissimilar
parts, each of which has to interoperate with many other parts for the system to
work successfully. If the design of a system is complex, there are many different
parts, each of which has to be well understood to build the system successfully. It
may not be possible to reduce the complexity of the system, but it serves us well
to reduce the complexity of its design. The simpler the design of the system, the
easier it will be to build.
Another key point related to the lack of standardized components relates to the
issue of system expansion or growth. If you are dealing with standard
components and system growth requires you to add more of them, it’s a simpler
acquisition. You’re buying more of a product that you can purchase from
multiple sources, so there’s competition and the likelihood that you’ll get a
reasonable price on those items. On the other hand, if system growth requires you
to add more nonstandard components, you’re faced with a more difficult
acquisition. The components are probably only available from the original
supplier. There’s no competition, so there’s no price break; you’re at the mercy
of the original supplier. Or worse, the original supplier may have gone out of
10
business. This may force you to re-engineer these components and hope that the
re-engineered ones work the same as, and in conjunction with, the remaining
original components.
One place where we can begin building quality into our system is in the Concept of
Operations (see next section). The Concept of Operations represents the “vision” of the
system that is communicated to all “stakeholders,” i.e., individuals, organizations, and
groups that have an interest in seeing the ITS project succeed.
One key to a successful project is a sound project plan. This means that a plan exists that
has:
11
Defined dependencies, where they exist, among tasks to show the order and
precedence of task accomplishment
Defined products or completion criteria for each task
A method of measuring progress against plan
Having a good plan is not enough. The plan must be communicated to and understood by
all of the people working on the project. Everyone who is scheduled to work on a task
needs to know:
The work product(s) of each task for which they are responsible
When the task is scheduled to begin
What the relationship of the task is to other tasks (i.e., on what completed effort
does the task depend, what depends on the successful completion of the task)
Who else will be working on that task
When must the task be completed
What resources other than people are required to perform the task
How one prepares a realistic schedule is beyond the scope of this paper. However, once
the project is underway, there are several important project management functions that
systems engineering can support. These include:
Tracking
Measuring progress
Revising schedules when required
Addressing obstacles to progress
Tracking involves ensuring that tasks begin and end on schedule and, if they don’t,
determining what caused the variance from plan. If the variance is unfavorable, i.e.,
the task begins or ends behind schedule, you must assess what prevents the scheduled
start or completion of a task. It could be the lack of an important resource or the
failure of a related task to start or end on time. Determining the root cause of the
problem, however, is important if you intend to solve it. If the variance is favorable,
i.e., the task starts or completes ahead of schedule, you must still assess the impact of
the early start or finish. It may be possible to shift resources from a task completed
early to work on efforts scheduled later in the project. This may have the overall
effect of shortening the project’s duration. But it is also possible that the early start or
finish of a task only means that some resource is underutilized for a period, since
there is nowhere else on the project where you can use the resource.
12
you’re 30% complete when 15 are installed. It’s harder to determine when you’re
“30% complete” in coding a program.
In the rest of this paper, we’ll cover many key topics in systems engineering, although at
the “30,000 foot” level. To help make these topics clearer, we’ll use the following
artificial example of an ITS project to illustrate what we mean.
Metropolis and Gotham City, two cities less than 50 miles apart in the State of Bliss, are
linked by Interstate Highway 105 (I-105). There are a number of ITS elements deployed
between the two cities, and there is a regional Transportation Management Center (TMC)
that manages these elements. The MetGo County regional planning commission
envisions an upgrade to the ITS capabilities along the Metropolis-Gotham City corridor,
to provide an enhanced infrastructure with the ability to handle both growth and
additional functionality. The regional planning commission is working with the State of
Bliss Department of Transportation (BDOT) and its Chief Traffic Engineer for the
MetGo region, M. D. Mandrake. At the latest planning meeting, Chief Traffic Engineer
Mandrake provided the assessment of existing capabilities and needs shown in Figure 3.
After the Chief Traffic Engineer gives this initial report, she and her staff begin work on
preparing an ITS project to deal with the situation that they’ve assessed. We’ll see how
this works out in the rest of this document.
Well, that’s the approach that we’re going to use. So, let’s get started.
13
Description of the Problem
“One of the major problems we have in managing our ITS elements in this region is the
multiplicity of communications networks that we are using, along with the limited capacity of
some of those networks. We have the following communications networks currently in use:
Our Dynamic Message Signs (DMS) are controlled by voice grade telephone lines
We have separate voice grade telephone lines to use for entering data into our two Highway
Advisory Radio (HAR) sites
A third set of voice grade telephone lines links five existing video detection sites along I-105
We have an additional nine video surveillance sites along I-105 that are linked to our TMC by
analog microwave
This gives us four relatively low-speed communications networks that operate independently, with a
large number of connections required at our regional TMC. In addition, three of the networks are
saturated and cannot handle additional communications traffic. The fourth is close to its rated
capacity.
We have requests from police and fire agencies asking to share the surveillance data from our cameras.
In addition, those agencies have suggested the installation of additional surveillance cameras on
secondary routes feeding to and from I-105, to assist them in responding to incidents in the area.
Agencies that have requested data sharing or additional camera installations are:
In addition to the upgrades for our communications networks, we need to upgrade our computers and
consolidate existing software. At present, each separate system (VMS, HAR, the two video
surveillance groups) is on a separate computer. There is no backup computer available for any of these
systems.
We would like to put all four systems onto a single large server and have a second computer, of similar
capacity, available as a backup to the main processor. If we do this, the software for each of the
existing systems will require some modification to run under the operating system used by the server.
Once we have our communications upgrade completed, we will upgrade our computer facility,
replacing our existing processors with newer, more powerful machines.”
Figure 3
Description of the Problem
14
2.0 The System Life Cycle and Systems Engineering
So when and how does systems engineering fit in as you build a system. Before getting
too far along, we’ll define some terms. Definitions help ensure that the author clearly
communicates what’s meant by key terms. There are three key terms in this discussion:
System
System life cycle
Systems engineering
Definitions
A system is something whose parts work together to achieve a common goal. That’s a
basic definition of the term, but it has all the components of a more formal definition3.
No definition of “system” states what size a system has to be. In fact, the parts of a
system can themselves be systems. When we describe an ITS system, we call it a
“system of systems,” i.e., each item in it is (at a high level) a system itself. For example,
you can find an Advanced Traveler Information System, an Advanced Traffic
Management System, an Emergency Management System, a Transit Fleet Management
System, an Automated Transit Information System, and a Traffic Control System in an
ITS system. Where you draw the boundaries of a system determines what its
components are. However, the wider you draw the boundaries, the more complex the
interactions of the system’s parts become.
A system life cycle starts when the need for the system is conceived and ends when the
system is discarded. The duration of that life cycle varies. Also, the way we break the
life cycle into its component parts varies. Although there are many different ways of
breaking a system life cycle into its component parts, we’re going to use the following
stages or phases in our discussion:
Conception: This begins when you first recognize the need for a system (or new
system version). During this stage, you do two things: 1) perform a needs
analysis, and 2) develop a concept of operations. The needs analysis determines
what problem you are trying to solve; the concept of operations is a user-oriented
view or “vision” of the system that you want to implement. You develop a
concept of operations to help communicate your vision of the system to the other
“stakeholders,” i.e., the other agencies, organizations, and individuals whose
interests the system affects. It also highlights the interfaces that the system has
and ensures, through the publication and discussion of the concept of operations
with others, that you have identified all interfaces. During this stage, you
establish the high-level requirements for the system.
3
Webster’s New Collegiate Dictionary defines ‘system’ as “a regularly interacting or interdependent group
of items forming a unified whole”. The idea of working toward “a common goal” is implied.
15
In some depictions of the system life cycle, this stage is known by the name of
one of its major products, the concept of operations. When we discuss a graphical
depiction of the systems life cycle later on, we’ll use the Concept of Operations
title for this stage.
Frequently, as you’ll see later on, this stage is divided into two major parts, each
identified by the detail level of the requirements developed during each part. The
two major parts are: High-level requirements and Detailed requirements.
Design: This stage involves deciding “how” each requirement in the system is
satisfied. It heavily involves systems engineering, since there are many trade-off
decisions to make. It is also important to ensure, as the design evolves and
becomes more detailed, that the design retains its internal consistency. If groups
working independently on parts of the system design fail to communicate
effectively, it may be difficult to make the system’s pieces mesh during
implementation.
It’s also common to view this stage in two major parts, High-level design and
Detailed design.
Implementation: This stage transforms the design into a product. It may involve
building parts of the system from scratch or integrating the different pieces that
you buy. However it is done, it makes the system real.
Integration and Testing: The testing stage overlaps implementation. It starts with
unit testing, i.e., the testing of individual pieces of the system, as soon as any
pieces are available. It continues with string or integration testing, which ensures
that individual pieces work together as intended and which tests interfaces at the
lowest level within the system. Last is system testing. This involves a full end-to-
end test of the complete system and comes in two major “flavors.” The first is the
final integration test by the system developer (whether that’s you or a contractor
you hired). The second is the “acceptance” test, where the end user(s) of the
system make sure that it does what it’s supposed to do. The acceptance test,
however, is conducted in the next stage.
16
System Acceptance: During this stage, you test the completed system prior to
putting it into production use. Users and operators should participate in the
system acceptance process. During this stage, in preparation for the system
acceptance process, you must train the users and operators of the system so they
understand how to use it to do their jobs.
Operation and maintenance: This should be the longest stage, the one in which
you use and maintain the system for the remainder of its existence. Maintenance
involves all processes that keep the system performing satisfactorily, including
upgrades of equipment and software to later versions to enhance the system’s
performance as its volume grows. When you deal with an upgrade or major
change to a system, the life cycle starts over, for that piece of the system that is
being modified. It’s also important to recognize that you have to keep monitoring
the system’s performance against its original performance requirements. This is
particularly important as demand on the system increases. You may have planned
for some growth in demand; you need to know when you reach that limit.
At a relatively early point in the system life cycle, you make your “build versus buy”
decision. When exactly you make that decision depends on whether you intend to
conduct the implementation with internal staff or with external staff, i.e., contractors. In
most ITS projects, public sector agencies hire contractors to work with them on the
development of the ITS system. If you plan to use contractors, you should bring them on
board before you complete the requirements analysis stage. That way, the contractors
can help develop and refine the requirements and there’s a better chance that everyone
will develop a common understanding of what the requirements mean.
The decision on whether to build or to buy is not simple. Even if you decide to “build”
the system, you may buy equipment and services during its building. If you decide to
“buy” the system, you still must devote resources from your organization to working with
the vendor(s) who sell you the system. “Buy” decisions also involve determining what
type of contract to use. For guidance on acquisitions, refer to The Road to Successful ITS
Software Acquisition, Vols. I and II as well as other DOT guidance (see Where to Go to
Get More Help, at the end of this document).
Systems engineering is the process by which we build quality into complex systems. It
uses a set of management and technical tools to analyze problems and provide structure
to projects involving system development. It focuses on ensuring that requirements are
adequately defined early in the process and that the system built satisfies all defined
requirements. It ensures that systems are robust yet sufficiently flexible to meet a
reasonable set of changing needs during the system’s life. It helps manage projects to
their cost and schedule constraints and keeps realism in project cost and schedule
estimates.
The key to the above definition is that systems engineering is a process, not just a set of
tools. Let’s look at how this process relates to the system life cycle we discussed earlier.
17
Where in the Systems Life Cycle Does System Engineering Fit?
The easy answer to the above question is “everywhere.” Systems engineering should be
an integral part of any system project. After all, it’s the process by which you build
quality into your systems. Let’s go back to the Metropolis-Gotham City example to
illustrate this idea.
Chief Traffic Engineer Mandrake has her marching orders. She kicks off her system
effort with the first stage, Conception.
Conception. This stage begins with a needs analysis. That there is a need is clear
but she must figure out what the system should do. Asking one of the questions
that we listed in Table 1, “What is the problem we’re trying to solve?” is the best
way to start.
The first few sections, Scope and Referenced documents, are mostly “boilerplate,”
necessary, but not exciting. The third section, Current system or situation, deals
with what currently exists. We described that briefly on pages 13 and 14,
although not as thoroughly as the Concept of Operations document might need.
However, let’s assume that the information on those pages is adequate and go on
to the fourth section, Justification for and nature of changes.
This section addresses the shortcomings of the current system that motivate either
the development of a new one or the modification of the existing one. Let’s
review what these shortcomings are, as indicated in Figure 3 on pages 13 and 14:
Three of the four existing networks are saturated and cannot handle
additional communications traffic. That’s important because local public
agencies want additional surveillance cameras that will increase the
communications workload.
18
There are no backup computers for any of the existing systems. If any one
computer goes down, the system that runs on it is unavailable.
19
Each system runs independently on its own computer. This can be viewed
Figure 4
Concept of Operations Outline
as a shortcoming because it means that having a backup system (as of
now) for each system would require four separate backup computers.
20
Those shortcomings are taken from the original problem statement Mandrake
provided. But she knows that there are some other operational issues that need to
be addressed. The seven public safety agencies asking to share data want to
manage their incident response better. To help them do this, the TMC needs to
understand what data they need and when they need to get it. The TMC also
needs to know which agencies to notify and transmit surveillance data to and what
criteria to use in notifying them and sending the data.
The fifth Concept of Operations section, Concepts for the proposed system, gives
Mandrake the opportunity to lay out the system and explain how she thinks it will
work, once it’s in operation. This is a key section, because it shows the intended
users of the system how the new system will affect them. It’s a good tool for
setting or managing user expectation for the new system. Note that one
subsection specifically deals with operational policies and constraints related to
the new system. These might include items such as:
Limits on the hours of operation of the system, or any part of the system
Policies related to system access or access to secure areas of the system
Constraints related to specific hardware that must be used
It’s in this section that Mandrake addresses the operational protocols that must
exist for determining which agencies get data and when and how they get it.
21
The sixth Concept of Operations section, Operational scenarios, allows
Mandrake to describe specific cases of system use. Some like to use flow
diagrams to illustrate operational scenarios; others like to use a narrative
approach, sort of a “Day in the Life of …” approach. We’ve provided a narrative
example operational scenario in Figure 5 below.
Switch any surveillance camera images available on the network to any monitor(s) or display
screen(s) in the TMC
Modify or update special event data on any MetGo County, BDOT, Metropolis, or Gotham City
websites
Retrieve Preventive Maintenance schedules for system fine tuning
Access the TMC information database for update purposes
Change any HAR message within the MetGo County area, selecting from an approved message
list
Access any DMS in the TMC’s operating area and change messages to be displayed, selecting
from an approved message list
View traffic congestion and incident data fro both surface traffic street and freeways
View automatic vehicle locator (AVL) data form the Metropolis and Gotham City Police
Departments and the MetGo County Sheriff’s Department
Alerted by the sudden decrease in vehicle speeds along Palm avenue, Joe selects the video images from
Palm and Date Streets in Gotham City. He notices that the northbound lanes on Palm are backing up due
to a stalled car on the outside lane. The problem is compounded by the fact that a construction crew is
working next to the car, on the inside lane. Looking more closely, he sees the construction crew assisting
the driver out of his car and onto the street. He switches the image to the large screen display – it looks
like a medical emergency.
Joe logs the incident using an Incident Management Report window. He records information about the
incident – reporting agency, contact name, phone, incident location, lanes affected, place on map,
description, notification list, traffic management plan, and alternate routes. The incident report is
immediately and automatically routed to the Gotham City Police Department’s computer-aided
dispatching (CAD) system, alerting operators who can react by dispatching police and emergency
services. Additionally, the report automatically updates local DMSs (already in use for the construction
activity) to warn oncoming traffic of the incident and suggest alternate routes.
Quick action addresses the medical emergency, possibly saving a life. Travelers receive information
about alternates for navigating more efficiently through or around the problem area.
Figure 5
Sample Operational Scenario
The seventh section of the Concept of Operations, Summary of inputs, and the
eighth section, Analysis of proposed system, recap much of the material covered
in earlier sections, but in a more condensed form. The ninth section, Notes, is
22
where you’d put additional material, things that don’t fit anywhere else, but help
the reader understand what’s in the Concept of Operations.
Design. During this stage, you will make some high-level decisions about how
you expect to implement the system. For example, you may decide (or at least
narrow the decision on) where to place the Traffic Management Center you plan
as part of this system. Some key systems engineering activities in this stage are to
identify what trade-off decisions are needed and to conduct the trade-off studies to
provide information on which to base these decisions. As an example of a trade-
off study, let’s say Chief Traffic Engineer Mandrake’s ITS plan includes
Automated Vehicle Identification (AVI) tags to measure and report travel times at
23
congested sites among the major arteries between Metropolis and Gotham City.
Since the type of tag you use has an effect on the specifics of system design, she
must assess the types available to her against her selection criteria. Table 2 is an
example of the results that might occur from a trade-off study, with criteria,
weights assigned to criteria, and relative ratings assigned to each factor assessed.
The trade-off studies clarify the Concept of Operations for the system.
Table 2
AVI Tag Trade-Off Study Matrix
Type of Tag
IVRT 5
Toll Beam
Ranking
Ranking
Ranking
Weight
Score
Score
Score
Criteria Characteristic Characteristic Characteristic
As we flesh out the details of the system in our design, we use system engineering
activities to guide decisions on how to satisfy our requirements. For example, we
use modeling and simulations to test assumptions about a design choice’s ability
to meet our performance requirements. These performance requirements may be
5
Intelligent Vehicle Registration Tag (IVRT)
24
either an ability to handle a specified range of data volume over a specified time
or an ability to process input and return a response in a specific window of time.
In this stage, we’ll also again make use of reviews. Design reviews generally fall
into two major categories: Preliminary Design Reviews and Critical Design
Reviews. You conduct the Preliminary Design Review (PDR) on each
component of the system. The PDR: 1) assesses the progress, technical adequacy,
and risk mitigation involved in the selected design approach; 2) determines
whether the item being reviewed is compatible with defined performance
requirements; 3) estimates the degree of technical risk remaining; and 4) verifies
the existence and compatibility of all interfaces involved. You conduct the
Critical Design Review (CDR) when the design for each component is complete.
The CDR: 1) ensures that the item under review satisfies all requirements
allocated to it; 2) ensures that the item is compatible with all other items in the
system; and 3) assesses whether there is any remaining risk associated with the
item. You also conduct trade-off studies here as part of your risk mitigation
approach, although the trade-offs are more detailed.
Implementation. This is the stage where all components of the system are either
built or brought together into the overall system. Your systems engineering
activities focus on ensuring that what gets built matches what you designed. In
the implementation stage, systems engineering is synonymous with quality
engineering. When you are applying standards as part of your design approach,
systems engineering activities focus on ensuring that the developers adhere to the
standard (or justify a waiver from the standard, if necessary).
Hardware development is done on some ITS projects, at least in some States, but
hardware development is not always done. When your project is developing
hardware, you may want to consider some of the issues associated with
configuration management for hardware. One place to start is another monograph
in this series, The ITS Guide to Configuration Management. That monograph
covers configuration management topics for both hardware and software.
25
activities to creating complete and comprehensive test sets that effectively
exercise the system and its components. In addition, your systems engineers
analyze the test results and assess the degree to which the system satisfies its
requirements.
This stage also deals with the actual installation of the system in pre-production
use. It includes preparing all of the instruction manuals for the system and all
training materials, and training all users of the system. The training conducted
here is the initial system training. Additional training is needed during the next
stage as new users come on board and the system is enhanced with new features.
Your systems engineering activities during this stage focus on ensuring that the
system is fielded so it does not disrupt the organization’s critical business
functions. For example, if you’re installing new sensors throughout an existing
road network, you recognize it may be necessary to close certain travel lanes
while installing them. However, these closures should take place during low
usage times on the affected roadways.
System Acceptance. This is the final step before putting the system into
operation and maintenance. System acceptance implies that the operators and
users take ownership of the system. System engineering activities here focus on
ensuring that all of the needs of the users and operators have been met. Many of
the systems engineering activities relate to developing a sound system acceptance
test process, one that is both fair to the developer and ensures the needs of the
system’s new owner(s) are met.
26
Another type of performance measure relates to the performance of the ITS system
itself, or its components. In this case, you might measure actual network carrying
capacity against planned capacity or actual system throughput against planned
system throughput. You could perform both types of performance measurement
during the operations and maintenance stage. But, since collecting and analyzing
the data required for either type of performance measurement is expensive, you
should only do this if you seriously plan to use the information you get as the
basis for decision-making.
A good way to see how systems engineering and the system life cycle interact is to look
at the life-cycle as what’s known as a “V” model or diagram of the system life cycle.
The “V” (or “VEE”) model is a way of relating the different stages in the system life
cycle to one another. Figure 6 illustrates the “V” model.
Concept
of Operations Operations &
Maintenance
High Level
Requirements System
Acceptance
Detailed
Requirements
High Level
Design Subsystem
Verification
Implementation
T
i
Time m
e
Figure 6
“V” Diagram
One reason for using the “V” model is to emphasize the importance of evaluation in all
stages of a system project. In the early stages of the system life cycle (the left leg of the
“V” model), you are using mostly inspection and analysis as your evaluation tools. In the
later stages of the system life cycle (the right leg of the “V” model), the primary
evaluation tool is testing. Regardless of which leg of the “V” model you’re on, you’re
interleaving evaluation efforts with system development activities.
27
Another reason for using the “V” model is to show an explicit relationship between work
done on each side of the “V.” Let’s look at the relationships between the two legs of the
“V” model to show what this means.
Looking at the explicit relationship between the two stages helps the systems
engineer focus, at the beginning of the project, on issues associated with keeping
the system in operation and effectively maintained once it’s built. This long-
range perspective is important in systems engineering.
It’s important to recognize that you should be able to trace every requirement in
the system to the system component that satisfies it. One way to do this is
through a requirements traceability matrix, which we discuss in Chapter 3.
System Design vs. Integration and Testing. The “V” diagram in Figure 6
actually breaks this down into two components. System Design is subdivided into
“High-Level Design” and “Detailed Design.” Directly across from “High-Level
Design” is “Subsystem Verification,” which is a reasonable correlation. Part of
what’s done in the High-Level Design activity is to break the system down into its
28
subsystems, each of which is assigned a major functional area of the overall
system. In Subsystem Verification, you’re integrating all of the components of
the subsystem and testing the subsystem(s) as units. It’s the appropriate way to
look at how the two stages are correlated.
“Detailed Design” is directly across from the “Integration and Test” component.
Detailed Design relates to the definition of the complete system, the final design
of all of the elements, at the level necessary to build it bottom-up. The Integration
and Test portion of the “V” is the start up on the leg where you bring all of the
pieces together. During this stage, you’ll test individual components as “units”
and begin testing the individual units that interact with one another, preparing to
fit them all together into individual subsystems.
The design stage is the final decomposition stage of the decomposition leg of
system development. It’s the stage where you establish, at the most detailed level,
your plan for how you will accomplish the work of the system. The integration
and testing stage is the first of the major re-composition stages, where you begin
assembling the pieces of the system into an integrated whole.
One activity associated with systems projects that doesn’t fall neatly into only one stage
is deciding when and how you’re going to acquire the system or its components. This is
often categorized as the “make vs. buy” decision, although it’s rarely that simple. Let’s
look at how systems engineering fits in with making the acquisition decision.
29
Table 3
Comparison of Relative Costs
By System Life Cycle Stage
When you’re deciding whether to “make” or to “buy” the system, the decision is rarely
straightforward. But there are several key systems engineering activities that take place
here.
The manner in which you answer each question will have an impact on the overall costs
of the system.
In the early stages of the system life cycle, it’s relatively easy to recognize some
costs, namely those involved in getting to a fielded system. However, it is also
easy to overlook what are probably the more significant costs of the system --
its operational and maintenance costs once it is fielded. It is particularly easy
to overlook the maintenance costs of the software associated with the system,
since software is not what we normally think of as “tangible.” It’s also easy to
overlook staffing costs for the new system. In its early days, automation was touted as a
way to reduce costs by eliminating labor. Machines would do the work of people. We
got past that point many years ago. Now, infusions of technology tend to increase labor
costs, because they require smarter, better-trained workers. ITS systems are no different.
30
If you don’t consider early on how your staffing costs will change once the new system is
implemented, there’s trouble potentially waiting for you.
Another key systems engineering activity associated with the Acquisition Method
Decision is determining what systems engineering activities you will assign to a
contractor, if you use a contractor to build or integrate the system. Once hired, the
contractor (or contractors) is a stakeholder, just like any other stakeholder. The
contractor also wants to see the system succeed because his reputation is on the line.
Some systems engineering activities are best done by the contractor, because the
contractor may have more or deeper expertise in certain areas. You must allocate those
activities and make clear to the contractor, in a statement of work, which ones he is
responsible for.
A good project management plan will include a section devoted to systems engineering.
This section, called the Systems Engineering Master Plan (SEMP), describes how you
organize to conduct systems engineering activities and what systems engineering
activities you plan to conduct during each stage of your ITS project. In the section above,
we were mostly concerned with technical activities. Let’s look at some management
activities associated with systems engineering. These activities will fall into four major
categories:
Project Planning and Control. There are many tools available to help a project
manager develop a detailed project plan and schedule. All of them assume
that the project manager knows, at a sufficient level of detail, what tasks
are necessary to complete the ITS project successfully. Let’s assume
that the project manager can successfully develop a detailed plan that
includes all dependencies among tasks, where all tasks have a reasonable
duration and resources assigned to them, and that is communicated to everyone
responsible for completing a task within the project. Given that, we’ll focus on
the issue of control.
31
an hour, with all key managers where each reports on the status of tasks
for which he or she is responsible. At this time, each manager should
discuss any items impeding progress on his or her tasks. It is critical,
however, that these sessions not become faultfinding or finger-pointing
sessions. If this occurs, it cuts off the open flow of information necessary
for successful program control and participants hunker down into
defensive postures rather than working together to find solutions.
Initiate
corrective No matter what mechanism the program manager uses to get accurate
action if a status information, the next step, initiating corrective action, is essential.
problem Determining what corrective action will succeed entails finding the root
occurs cause of the problem impeding progress. If the problem is a technical
one, finding the root cause may involve significant technical analysis of
the symptoms. Solving it may require analyzing different technical solutions to
decide which is most effective.
Systems engineering is involved in three key ways in the control activity. First, in
defining reasonable measures of progress against tasks so that status can be
measured and reported as objectively as possible. Second, in developing an
information system that captures the objective data with which to measure
progress so managers can effectively report on their status. Third, in analyzing
problems and developing recommendations, for the program manager’s
assessment and approval, on how to solve them.
Technical Planning. Technical planning primarily involves how you set up your
systems engineering organization and what activities you plan to undertake in
each stage of the ITS project. This information is in the SEMP section of the
overall program plan. In addition to the overall technical plan covered in the
SEMP, you must decide what tools to use in your systems engineering
activities. For example, if you plan to use simulation and modeling as a
tool during any stage of the ITS project, you must decide what simulation and
modeling package or language to use and on what physical computer platform to
run it. If you plan to use automated testing tools during the testing stage of the
project, you must decide what specific tools you want to use and, if you don’t
already have them, acquire them and make sure that you train people in their use.
You also must decide what specific skills you must have in your systems
engineering team. Since the major focus of your systems engineering efforts are
to increase the quality of the system by reducing the risk and uncertainty
associated with the project, your team needs to have technical strength and depth
in those areas that you deem the most uncertain (thus, the riskiest).
The third part of technical planning is deciding how you will allocate systems
engineering activities between government and contractor organizations. If the
government organizations involved have little systems engineering expertise, you
may decide to allocate all of these activities to your prime contractor. Another
32
option is to hire a separate, independent systems engineering contractor, who
reports directly to you, not through the prime contractor, to perform the
government’s share of systems engineering activities. Having a separate,
independent contractor is often the more expensive6 of the two options. However,
many practitioners believe it is the better option. It provides the government
project manager with a check and balance system on the prime contractor. But
remember, it also requires cooperation from the prime contractor to be effective.
33
accurately. If possible, develop a range of estimates rather than a single cost
estimate. When a single number is given for a project’s cost, that number
becomes a ceiling that is difficult to exceed. Since cost estimates are just
educated guesses, it is better to give yourself some latitude. As you complete
each stage of the project, you can refine your cost estimates and provide a more
realistic range of costs. It is unlikely that you can do this with great accuracy at
the outset of the project. To protect yourself, you have to set aside contingency
funds as part of your project budget.
34
3.0 Elements of Systems Engineering
System Engineering Standards
There are a number of systems engineering standards that describe how to establish a
systems engineering capability within an organization or a project.8 The Software
Engineering Institute (SEI) has created a Systems Engineering Capability Maturity
ModelSM that describes what any organization should have in its systems engineering
process, without specifying any particular process. It is an excellent reference model for
systems engineering practices. Although the SEI, an organization created to improve
software engineering capabilities in the Defense industry, developed this model, it is
applicable to more than just software and to more than just Defense programs.
Version 1.1 of the SE-CMM lists 18 process areas or practices that make up good
systems engineering. These processes are categorized into three groups: organizational,
project, and engineering. Table 4 below shows what processes fall into each group with
the process areas listed alphabetically within each group. Although the processes are
listed alphabetically, some processes normally precede others. For example, one has to
plan technical effort before one can effectively monitor and control it. But the planning
effort could take place many times during the project’s life cycle.
The Organization processes create the business infrastructure that supports systems
engineering and are performed primarily at the organizational level. The Project
processes focus on the technical management infrastructure needed to develop a system
8
The Electronics Industry Alliance (EIA) published its systems engineering standard, EIA 632, in 1994. It
then joined forces with the American National Standards Institute (ANSI) to make this a US standard. The
revised standard, ANSI/EIA 632, was approved for publication in January 1999. In addition, the
International Organization for Standardization (ISO) is developing its own systems engineering standard,
ISO 15288. The Institute of Electronics and Electrical Engineering (IEEE) has established a systems
engineering standard, IEEE 1220, as one of its standards. These standards are mostly based on the military
standard for systems engineering, MIL-STD-499 (1969) and MIL-STD-499A (1974). A third version of
this military standard, MIL-STD-499B, was prepared in 1994, but never issued. Instead, the DOD opted to
adopt standards developed by standards-making bodies, such as EIA, IEEE, and ISO. MIL-STD-499B
was, however, the starting point for the EIA/IS 632 standard.
35
successfully. And the Engineering processes concentrate on the technical and
engineering aspects of developing a system.
Table 49
SE-CMM Process Area Groupings
Organization Project Engineering
Coordinate with Ensure Quality Analyze Candidate
Suppliers Manage Configurations Solutions
Define Organization’s Manage Risk Derive and Allocate
Systems Engineering Monitor and Control Requirements
Process Technical Effort Evolve System
Improve Organization’s Plan Technical Effort Architecture
Systems Engineering Integrate Disciplines
Processes Integrate System
Manage Product Line Understand Customer
Evolution Needs and Expectations
Manage Systems Verify and Validate
Engineering Support System
Environment
Provide Ongoing
Knowledge and Skills
Let’s start with the process areas in the Organization grouping. These process areas (see
Table 4) are normally performed by the organization (company, agency) in which the
project resides.
An ITS project manager may have little direct involvement with most of these process
areas. However, there are two process areas that are important at the project level:
Coordinate with Suppliers and Manage Systems Engineering Environment. Let’s look at
these in a little more detail.
Coordinate with Suppliers. There are several base practices that are logically part
of government procurement processes. These include:
36
An ITS program manager uses these base practices when deciding what vendor(s)
will supply products to his or her ITS project. They are all practices used in a
Request for Proposals or Request for Quotes procurement. These base practices are
not just performed in a procurement process, however. You should continue timely
two-way communication with any supplier after that supplier is on board, to ensure
that you both clearly understand what is being supplied and what is being bought.
If a contractor performs the primary systems engineering support on your project, you
want to know how the contractor manages his systems engineering support
environment. You may request this information from the bidders who respond to
your RFP.
We’ll pass on the other Organization process areas and move to the Project ones.
There are logical relationships among the five Project process areas (see Table 4), which
we will point out as part of the discussion.
Ensure Quality. This base practice and Manage Risk are related since you
ensure quality by reducing the risk and uncertainty in your design and your
implementation. Another way to improve quality is to inspect the products you
are incorporating in your system. Inspection is done both with physical products
and with software. Good software engineering practice includes the concept of
code “walkthroughs.” “Walkthroughs” are when a programmer convenes a group
of knowledgeable programmers and reviews a section of his or her code, at a very
detailed level, with them. The programmer whose code is reviewed explains what
he or she is trying to accomplish and the reviewers assess whether the code
accomplishes that objective effectively.
37
Reviews can be done on any product. A good practice is to review designs with
stakeholders to ensure that they concur the design accomplishes what they expect.
Mockups of reports (or actual user interfaces) are a good tool to use in a review.
Letting a user get a feel for the actual look of a product helps the user identify
what’s good and what’s bad about the product.
Manage Risk. One risk that you should consider and manage is the risk that
some critical factor will go wrong, e.g., deliveries of key products are late, a key
individual is lost while the project is underway, some external event with a
negative impact on the project occurs. You assess the likelihood that an event
will occur, determine what its cost will be, and determine what the cost of
mitigating it is. If the cost to mitigate the event is greater than the negative cost if
the event occurs, you ignore it. If you can mitigate the risk at a reasonable cost,
this is good project insurance.
There are also technical risks associated with the project. Technical risks include:
interfaces not working as expected, software failing to perform its expected
function, and different communication systems failing to interoperate. Your
systems engineering process must consider these risks and plan to mitigate them.
For example, you may prototype the interface to determine if it functions as
expected. Or you may test the communication systems early on, to determine if
there are interoperability problems.
Monitor and Control Technical Effort. One way to monitor technical effort is
to establish clear exit criteria for each task in the project. Exit criteria are criteria
that clearly and uniquely define when a task is complete. They are effective in
monitoring project task status since they provide a clear and unambiguous way of
determining if the task is done. The monitoring scheme for your project should
tell you when a task may not be completed on schedule or when it is consuming
more resources than you planned for it. Either situation is a warning of a problem
you should resolve. If the task is either on the critical path or one that will
10
“Payroll Systems Flunk First Exams,” Computerworld, October 25, 1999
38
become part of the critical path if delayed sufficiently, you must resolve the
problem affecting it by determining the problem’s root cause and correcting it. If
the problem cannot be fixed, you assess its impact on the overall project. For
example, a delay in a critical path task delays the project’s overall completion; a
task that runs over budget may lead to an overall project overrun.
Plan Technical Effort. One technique used in planning technical effort on ITS
projects is the Work Breakdown Structure. A Work Breakdown Structure is a
hierarchical depiction of a project’s planned work, with each Work Breakdown
Structure element defining a product to be developed. Figure 7 depicts a sample
Work Breakdown Structure for a project named “ORBITS” and only shows one
subsystem at any level of detail.
You assign resources, duration, and costs at the lowest level in a Work
Breakdown Structure. The cost of the element at the next higher level is the sum
of the costs for all of its sub-elements. Thus, if Work Breakdown Structure
element 1.1.1.1 costs $10,000 and 1.1.1.2 costs $20,000, and 1.1.1.3 costs
$50,000, Work Breakdown Structure element 1.1.1 costs $80,000. Estimating the
resource requirements, the expected duration of the effort, and the costs of a Work
Breakdown Structure element is essential in a good technical plan.
In this process area, you determine the technical parameters for your project.
Technical parameters include:
Establishing the technical parameters and their threshold values gives you a way
to measure how well the system meets its performance requirements. If lowest
levels in a Work Breakdown Structure have clear exit criteria, you can use a Work
Breakdown Structure to measure a project’s earned value. (Appendix A discusses
earned value.)
The last set of process areas (see Table 4) is in the Engineering group. Although the SE-
CMM does not specify an order for the process areas, we’ll consider them in the
following order:
39
1.0 ORBITS
Regional Project
1.1 Develop 1.2 Develop 1.3 Develop 1.4 Develop 1.5 Integrate
Trans. Mgt Incident Mgt AVI System Sensor System
Center System Arrays Components
Figure 7
Sample Work
Breakdown Structure
We’ve left the Integrate Disciplines process area out of that list. Let’s discuss it first
and then take the others in order.
40
Integrate Disciplines. In any system development project, there are “specialty11“
engineering disciplines needed during the life of the project. Such “specialty”
disciplines could include:
Network engineering
Aerodynamic engineering
Nuclear engineering
Human factors engineering
Computer security
Software engineering
The customer’s needs and expectations may not be clearly articulated when you start
your effort. Your job (or the system engineer’s job, if that’s not you) is to find out
what they are and to document them so that the customer can understand and agree on
them. In particular, you must determine the technical performance measures for the
system. Technical performance measures are quantitative values used to track the
progress of your design and development to determine how well you meet the
system’s performance requirements. Response time (as perceived by the user) is a
frequent technical performance measure. If you are delivering traffic status
information to the public through a web site, you must ensure that someone looking
for that information can get a response within a reasonable time, or that individual
will stop looking for information there. If your web server can’t handle the volume of
queries it receives, the volume of queries will drop, which isn’t a good sign.
One output of this process is the Concept of Operations document, which describes
what the user wants the system to do and how the user expects it to operate. There
11
“Specialty” engineering disciplines, in this context, are those engineering areas used in some, but not
necessarily all, systems engineering efforts.
41
will be user requirements, expressed as “needs,” in the Concept of Operations
document, but they’ll be high-level ones and will need further detail. The Concept of
Operations document should identify the key users needs, as these are critical to
success or failure.
Derive and Allocate Requirements. This process area logically follows the
Understand Customer Needs and Expectations process area. The whole point of
analyzing requirements is to get to a definition of the specific functions that a system
must perform and to a specification of how well the system must perform these
functions. This lets you design a system to meet these requirements. Usually, a high-
level requirement (or “need”) is stated too broadly to serve as an adequate design
guide. Thus, we have to refine each requirement, get more precise in what we mean.
Take the following high-level requirements statement: The TMC operator shall control
video surveillance cameras from the operator’s console . That’s a reasonably clear
statement of need, isn’t it?
As used in the context of a proposed system, the term could mean all of those
capabilities or merely some of them. It is the system engineer’s job, in this process
area, to determine which capabilities the user wants. In doing so, the system engineer
may determine still more refined requirements. For example, the following
requirement seems straightforward: The TMC operator shall be able to rotate the
camera 360º in either direction (e.g., right or left) from its starting position. But it
isn’t because we need to answer a few questions before we know exactly what’s
wanted. These questions include:
1. Once the rotation has begun, should it continue automatically or must the
operator continue to direct it?
2. If the rotation continues automatically, when does it stop?
- When it reaches its original position?
- When the operator cancels the “rotate” command?
- Automatically after some specified time?
42
3. Is there only one way to stop the camera’s rotation? (E.g., by the operator
canceling the “rotate” command.) If not, what options exist? If yes, what is
it?
4. Must the rotation be a smooth, continuous rotation or should the camera
“jump” to pre-specified points along the circle of rotation?
The above list of questions is just an illustration, to give you a feel for how a system
engineer could derive additional requirements from what looks like a complete
requirements statement. Note, however, that nowhere do we specify how the operator
should “control” the camera’s actions (the “command” can be issued in a number of
ways, ranging from the entry of the word “rotate” as a command to pressing a button
to point-and-click on a virtual button on the console display screen). We don’t want
to over-specify and mandate a “how” answer; we do want to make sure we’ve
considered and included all of the functional requirements.
In addition to deriving requirements, the system engineer also decides how to allocate
them to system components. Camera controls, for example, could be handled by
switches, knobs, buttons, or other hardware devices located on an operator’s console.
Or the controls could be handled in part by hardware and in part by software. There
could also be separate pieces of software for different camera functions. For
example, the software to transmit a camera image to another agency could be separate
from the software used to control the standard camera functions, such as pan, zoom,
and tilt, since the transmission software may be used less frequently.
Prototyping
Simulation
Modeling
Trade-off study
Decision tree
43
Literature search
Review and use of prior analyses
Expert judgment
Process quality improvement
Requirement 2
Requirement 5
Requirement 1
Requirement 3
Requirement 4
Requirement n
...
...
...
...
...
...
...
...
...
...
...
...
Subsystem 1 x x
Subsystem 2 x x
Subsystem 3 x ... ... ... ... ... ... ... ... ... ... ... ...
... ... ... ... ... ... ... ... ... ... ... ... ...
... ... ... ... ... ... ... ... ... ... ... ... ...
... ... ... ... ... ... ... ... ... ... ... ... ...
Subsystem n x
Figure 8
Requirements Traceability Matrix
Evolve System Architecture. The “architecture” in this process area’s title is not the
National ITS Architecture, although one can build a system architecture from the
National ITS Architecture12. “Architecture” describes the overall framework of the
system you are building. It refers to the physical system environment (hardware,
networks, facilities) and the logical constructs (subsystems) that function in the
physical system environment. As a framework, “architecture” provides a blueprint
within the system design effort, giving everyone involved in the system design effort
a clear understanding of what you are trying to accomplish and how. You evolve a
system architecture in a top-down fashion, adding details as you flesh out the system
design. There are multiple levels of architecture within a system. Each subsystem
has its own architecture; each module or component of a subsystem has its own
architecture. In addition, each physical item (equipment) within the system has its
own architecture.
12
The U.S. Department of Transportation (DOT) has published a rule, 23 CFR Parts 655 and 940,
Intelligent Transportation System Architecture and Standards, that defines conformance with the National
ITS Architecture as requiring the development of a regional ITS architecture, i.e., a local implementation of
the National ITS Architecture, within four years after the region’s first ITS project advances to final design,
with subsequent projects having to adhere to that regional ITS architecture. For more information, look for
the architecture conformity section of the DOT’s web site (www.its.dot.gov). We cover the manner in
which the National ITS Architecture can support requirements development for ITS systems in a
companion monograph to this one, Developing Functional Requirements for ITS Projects.
44
The other critical purpose that architecture serves is to identify all of the interfaces
within the system and within each of the system’s parts. The interfaces consist of the
physical connections between components of their system, the logical
connections between the system and other systems, and the data that
flows between components or systems at each interface. You must
document these details and ensure that the system, as it evolves,
continues to address all of your received and derived requirements.
Integrate System. Some base practices in this process area are obvious,
Architecture others aren’t. Although it may seem that integrating a system is like
Establishes putting a jigsaw puzzle together, i.e., you connect the pieces where they
the System fit as they come to you, your system’s assembly may require that the
Framework pieces go together in a specific sequence. For example, when putting
and Identifies together a computer system, you start with the hardware. Then you add
Interfaces! an operating system. After the operating system come any other tools,
such as a database management system, on which you’re going to run
your applications. And so on. If it doesn’t matter which comes first, the chicken or
the egg, you don’t worry about how you assemble your system. When the order
matters, you should prepare an integration strategy and communicate it to the people
who are going to integrate the pieces as you make them available.
Verify and Validate System. There’s an easy way to differentiate between “verify”
and “validate.” You “verify” when you determine that you built the system right, i.e.,
that the system meets the specifications that you laid down for it. You “validate”
when you determine that you built the right system, i.e., that the system meets the
user’s needs and expectations. You verify the system by testing it against criteria you
laid down to ensure it does what you intended. This is your system test plan. To
validate the system, you let your users test it, using whatever criteria they have for
acceptability. This is the system acceptance test plan. When you, as program
manager, represent the user community and a contractor is the system developer, you
approve the contractor’s system test plan. However, the contractor cannot develop
the system acceptance test plan. An independent group (preferably the user or with
heavy user involvement) must prepare and execute the acceptance test plan.
These two tests are the basis for future end-to-end system tests you will conduct as
you enhance the initial system. The details of the tests conducted may change, but
the overall test plans should not.
We’ve discussed the SE-CMM and its process areas. It’s fairly extensive and geared to
organizations that build more than one system or product. If you only expect to build one
system, you just define a systems engineering process for your project. What should that
process look like? James Martin believes that the following are the minimum
requirements for an effective systems engineering process:
45
“In general, every project ought to have a schedule (produced on one of the popular
project management programs such as TimeLine, or Microsoft Project), a requirements
traceability matrix which includes tests (for small enough projects use a commercial
database such as dBase, Paradox, or Access), configuration management, and a
modification request system.
The next section looks at different ways of dealing with risk and uncertainty in the
systems engineering process.
13
Martin, James N. Systems Engineering Guidebook: A Process for Developing Systems and Products
(Systems Engineering Series), CRC Press, Boca Raton: 1996, pp.183-184
46
4.0 Addressing Uncertainty and Risk With Systems
Engineering
Types of Uncertainty and Risk Addressed
One of our major premises is that reducing uncertainty and risk in projects through sound
systems engineering techniques significantly improves the project’s chances for success.
Some of the factors that cause uncertainty and risk in projects include:
Complexity
Processing capacity/communication bandwidth
Rate of technological change
Information imperfections
Complexity. The more parts that a system has, the more complex it is likely to
be. When a system includes software that does more than just trivial processing,
the software alone brings considerable complexity to the system. Complexity in a
system adds uncertainty in two ways. First, it is difficult to visualize a system
when there are multiple interactions among all of its parts. Since we have
difficulty “seeing” it, we may have difficulty in understanding and describing it.
When we can’t create a comprehensive picture of our system, whether with
pictures or words, we may omit some relevant part of it. This, in turn, means that
we may not fully satisfy all of the requirements that our users had in mind.
Second, we may find it difficult to maintain the interfaces and interactions within
the system whenever we change any of its parts. Since some part of the system
will change either while we’re building it or after we field it, change in a complex
system can have unexpected (and undesirable) results.
47
should include computing and telecommunications experts as “specialty”
engineering capability.
Managers are generally averse to making decisions when there is risk about the decision.
Unfortunately, in real life, managers have to make many decisions with imperfect
48
information and high degrees of uncertainty. As a systems engineer, you must help
managers by giving them ways to assess the risks involved in a decisions and the
strategies they can follow to reduce the risk.
The worse possible case for decision-making is the case of complete uncertainty. There
are several strategies available to a manager in this situation, but all require you to be
able to determine the possible results of your actions. Let’s take the case where a
manager is attempting to choose between two alternatives with each alternative having
two possible results. This gives us a 2x2 outcome space, illustrated in Table 5. Let’s
assume that the first alternative, Alternative 1, has two possible outcomes. The first is an
outcome with a positive value of $2,500; the second is a negative outcome, losing $500.
We represent these two outcomes in the first row of Table 5, with the negative outcome, a
loss, shown as -$500. Alternative 2, on the other hand, has the same positive value of
$500 for each of its two outcomes. The second row in Table 5 illustrates this alternative.
Table 5
Alternative Assessment Matrix
Alternative Result 1 Value Result 2 Value
Alternative 1 $2,500 -$500
Alternative 2 $ 500 $500
Maximin: In this strategy, you choose the alternative that optimizes the minimum
result. This is the same as choosing the alternative with the least harmful worse case
scenario. Given the above potential results, you would choose alternative 2.
Maximax: In this strategy, you choose the alternative that optimizes the maximum
result. This is choosing the alternative with the best upside. In the case illustrated by
table 4, you would choose alternative 1.
Equal probability rule: In this strategy, you assume that all alternatives have the same
probability of occurring and choose the one that yields the higher expected value.
Expected value is determined by multiplying the value of each result by the
probability of its occurring. Since we have two results for each alternative, each has a
probability of 50%. The expected value of alternative 1 is 1,000 ((2,500 * .5) + (-500
*.5)) and the expected value of alternative 2 is 500. This strategy would pick
alternative 1.
Two of the three strategies opt for alternative 1 while only one recommends alternative 2.
So which would a manager choose?
In all likelihood, a manager will use the maximin strategy and choose alternative 2.
Why? Because it offers him or her the least risk if he/she is wrong. Either result in
49
alternative 2 guarantees a positive return. Even if alternative 1 offers a 50% probability
that the result may be a greater return than either result in alternative 2, it is the 50%
probability of the negative result that dissuades many managers. The psychological
difference in the two alternatives favors the second.
Our goal in systems engineering is to improve the quality of the information that we can
provide project managers on ITS projects. By doing so, we take the decisions they have
to make out of the realm of complete uncertainty and give them a better chance of
making high quality decisions.
In our discussion, we’ll talk about how these tools address the issues we described.
Gantt charts
Activity networks
Work breakdown structures
Most of the automated tools available today allow you to create and represent your
project schedule in each of these formats, transforming from one format to another
automatically upon your request. Each tool (format) has a particular strength.
Gantt Charts. The Gantt chart, or “bar chart,” is a common way of depicting the
tasks on a project against their expected duration. Figure 9 is a sample Gantt chart.
This figure uses the same tasks as those shown in the sample Work Breakdown
Structure figure (Figure 7) in section 3.
50
The strength of a Gantt chart is that it shows the duration of each task, along with the
expected start and end dates, explicitly. It may or may not show the dependencies
among the tasks, depending on what you use to produce it.
Figure 9
Sample Gantt Chart
TimeProject TasksJan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec1.0 -- ORBITS Regional Project1.1 --
Develop Transportation Management Center1.1.1 -- Develop Roadway Management System1.1.2 -- Develop
Network1.1.2.1 -- Select Network Protocol51.1.2.2 -- Select Vendor1.1.2.3 -- Install Network1.1.2.3.1 -- Install
Physical Cabling1.1.2.3.2 -- Install Routers1.1.2.3.3 -- Test Network1.1.3 -- Develop User Interface1.2 -- Develop
Incident Management System1.3 -- Develop AVI System1.4 -- Develop Sensor Arrays1.5 -- Integrate System
Components
Activity Networks. An activity network shows tasks and the order in which they
should execute. There are different ways to draw activity networks, but we’ll
illustrate one using the “activity on arrow” approach, which lists the tasks on the
arrows connecting the nodes of a network. Figure 10 illustrates this type of network.
S Fi
t ni
a sh
rt
Figure 10
Activity Diagram
51
The strength of an activity network is that it clearly shows the sequence of tasks and
the dependency of any task on another. It frequently is used to show the critical path
of a project. By definition, the critical path is the sequence of tasks in which a delay
in any task causes an overall delay in the completion of the project. A project
manager must monitor tasks on the critical path more closely than tasks that are not.
However, the critical path is not constant. If some tasks not on the critical path are
delayed beyond a reasonable point, they may change the project’s critical path and
cause an unexpected delay. Thus, a project manager has to monitor all tasks in a
project schedule and continually verify that the critical path has not changed.
Work Breakdown Structure. The third type of scheduling and tracking tool is the
Work Breakdown Structure (WBS). We discussed this briefly in section 3. In a
Work Breakdown Structure approach, a project manager divides the project into
“work packages,” or tasks that can be grouped together to yield a distinct product on
the project. On an ITS project, for example, a work product may equate to a market
package in the National ITS Architecture. The effort required to implement one of
these market packages could be part of a work package. The work package can be
broken down into smaller and smaller work packages until we reach the level where
work is being performed by one individual on the project team with a definite,
measurable product at the end of the effort.
The strength of a WBS approach is that it allows you to show the hierarchy of work
clearly. Lower level tasks combine to form higher level tasks. You can easily assign
resources to a task and calculate what the resources will cost you. Then, by adding up
the cost of all of the lowest level tasks, you can get the total cost of a work package or
of the entire system.
These techniques illustrate how one can deal with the complexity in a system. By
dividing the work required to implement the system into small manageable pieces,
regardless of the technique used, and then estimating the effort to produce each piece, a
project manager can create a schedule and track progress against it. Moreover, each
member of the project team can understand his or her role and responsibilities on the
project and focus on the work he or she must accomplish to make the project a success.
These tools, while seemingly mundane, are valuable elements in the systems engineer’s
toolbox.
Trade-off Studies.
Trade-off studies are one way to deal with information imperfections and with rapid
technological change. While the trade-off study is sometimes used to decide whether a
technical solution to a problem exists, a trade-off study can examine different technical
alternative solutions to a problem. Trade-off studies can be paper studies where the
analyst assesses different products or solutions using technical descriptions from the
vendor of the product. In this case, the analyst is assessing how the technical description
of the product maps against the requirements the project has. A paper study, however,
52
relies on someone else’s assessment of the technical capability of a product. If that
assessment is wrong or incomplete, your analysis will suffer. Another approach to
conducting a trade-off study, although this option is not always possible, is to test
competing products in a controlled environment that simulates, as well as possible, the
actual conditions under which the products will be used. This approach can yield the
most reliable results, if the test environment is realistic.
Design reviews help ensure that the system design, as it evolves, continues to meet the
needs of the end-user and all others who interact with it. The basic goals of a design
review are:
Ensure that the evolving system design meets the requirements defined for the
system
Ensure that the evolving system design is understood and agreed to by the
stakeholders affected by the system
Find any flaws in the design, so that they can be corrected before they become
more firmly embedded in the system implementation
In section 3 of this paper we specifically called out three kinds of reviews: System
Requirements Review, Critical Design Review, and Preliminary Design Review. The
Preliminary Design Review and the Critical Design Review are the two key types of
reviews that should be conducted on ITS project design elements.
There are other types of reviews and audits that you can conduct as part of the system
engineering toolbox. You can find them described in the References to this document.
We will not go into more detail in this area.
53
Modeling and Simulation.
Let’s consider how we might use modeling and simulation in an ITS project. Let’s say
that Chief Traffic Engineer Mandrake is considering a project to give traffic signal
priority treatment to buses in the Metropolis metropolitan area. (This capability already
exists for the Metropolis emergency vehicles and it can be expanded to the transit
vehicles.) Giving Metropolis Transit Authority buses traffic signal priority treatment will
help them keep to their schedules and provide better overall service. However, some of
Mandrake’s traffic engineers are concerned about the effect on traffic on cross-streets.
Mandrake agrees that this could cause problems and is only interested in a selective
priority approach, giving priority only to those buses running behind schedule.
To examine the potential congestion problems that could arise, Mandrake commissions a
simulation study. She asks her systems engineers to model the bus routes with the
greatest number of behind-schedule trips over the last year, looking at the potential effect
on schedule performance if buses on those routes get signal priority treatment when
they’re running behind schedule. She wants to know the answers to questions like the
following:
Will more buses be on schedule if they’re granted traffic signal priority when they
fall behind schedule?
How much traffic build-up will occur on cross-streets?
Which cross-streets will have the worst build-ups?
At what times will the worst traffic build-ups most likely occur?
Mandrake assigns some of her staff to collect the necessary data and then to give it to the
modelers among her systems engineers. She tells the modelers what results she wants
and lets them analyze the data and prepare their simulations to answer her questions.
When the modelers finish their work, they’ll present Mandrake with the information she
can use to make her decision. If the selective priority scheme isn’t effective, the
54
modelers can show Mandrake why. If it is, but causes congestion on some cross-streets,
they can tell Mandrake where and when the congestion will most likely occur. Mandrake
may still proceed to implement this priority system, but will do so knowing the
consequences of her action.
Prototyping.
ITS projects have used prototyping in many ways. One key use of prototyping is to
develop system requirements for complex ITS systems. The State of Maryland recently
used prototyping creatively to determine its system requirements for a major ITS effort.
The State hired two vendors to build a prototype of the ITS system that Maryland wanted
to implement. Each vendor had to dig out the critical requirements from the user
community and had to develop approaches to solve the technical problems that arose in
building their prototypes. Maryland was then able to compare the two prototypes and
select the one that they believed had the better chance of providing an acceptable
solution. The winning vendor had already invested considerable effort in building the
prototype and was well positioned to complete a successful build of the overall system.
Prototyping can also be used to create a version of the system that can be used for
benchmarking system performance. The prototype is an operational model of the system,
but in a scaled down format. Prototyping is also valuable in determining whether it is
possible to interface elements of a system that have never been linked together before.
This helps reduce the uncertainty related to the interfaces.
Benchmarks.
Benchmarks are useful when you want to measure the actual performance of a system
and ensure that it can meet its performance requirements. Benchmarks can be conducted
on the implemented version of a system, as part of the system or acceptance testing of the
system. They can also be conducted against a scaled-down version of the system to
assess how well the scaled-down version performs, before investing in the final version
of the system. When you use a scaled-down version of the system in a benchmark, you
must be confident that increasing the scale of the system will not significantly change
system performance. Otherwise, the benchmark may be invalid. Benchmarking is
particularly useful in measuring the capability of telecommunications systems and the
process capability of computing systems.
Technical performance measures are those characteristics of a system that directly impact
its performance. For example, the weight of an airplane directly affects its speed and
cargo or passenger capacity. In an ITS setting, CPU utilization and processing time are
two technical performance measures to consider. Storage capacity, i.e., the amount of
storage necessary to hold the data required for normal system operation, is another
possible technical performance measure to use. Other examples could include: number
of sensors supported in simultaneous operation; total time (e.g., .5 seconds) to process
55
toll collection information from an AVI tag; end-to-end time for signal light control
processing.
Technical performance measures should be established early on in the ITS project. Then,
as the system develops, the actual system values achieved for each of these technical
performance measures is compared against its design goal. For example, if you defined
toll collection processing time goal as .5 seconds, measuring actual processing time at .75
seconds indicates that you exceeded the technical performance measure by 50%, an
undesirable result. You would need to modify the design or its implementation to reduce
response time and achieve the defined goal.
Having discussed some system engineering tools, let’s look at what other sources of help
exist.
56
5.0 Where to Go to Get More Help
Department of Transportation Resources
One Department of Transportation resource is the ITS System Acquisition document set,
The Road to Successful ITS Software Acquisition, Volumes 1 and 2, and its related course.
A second document resource, one that covers basic system engineering concepts in the
context of Transportation Management Centers (TMCs) is the Transportation
Management Center Concepts of Operation Implementation Guide. Other documents
that can provide useful insights into the systems engineering effort are the companion
documents to this one: Developing Functional Requirements for ITS Projects, and The
ITS Guide to Configuration Management.
The National ITS Architecture is another resource that can help you with your systems
engineering needs as it provides an excellent starting point for defining requirements (and
for developing regional ITS architectures). It also provides guidance, through its market
packages, on implementation strategy for ITS systems.
The National Highway Institute is offering two introductory Systems Engineering courses
for transportation engineers and has other, more detailed courses that cover topics related
to systems engineering. Contact the NHI for information on the course numbers and
course schedules.
Standards
There are two existing overview or general standards for systems engineering: the
Electronics Industry Association (EIA) and American National Standards Institute
(ANSI) joint standard, EIA/ANSI 632, and the Institute for Electronic and Electrical
Engineering (IEEE) standard, IEEE 1220. These two standards should be joined by a
third, published by the International Organization for Standardization (ISO), ISO 15228.
The ISO standard is still in the Draft International Standard (DIS) stage as of August
2001. All three standards have a common heritage and are similar, but not identical.
Any of the standards could be used in an ITS project.
57
ISO International Organization for Standardization (www.iso.ch)
Table 6
List of Representative Standards
References
The texts listed in the bibliography to this paper are all useful for systems engineering.
The Handbook of Systems Engineering, while the most expensive, is also the most
58
comprehensive. Also, INCOSE (see below) publishes its Systems Engineering
Handbook, subtitled A “HOW TO” Guide For All Engineers. This is a very practical
handbook and would be a useful addition to any engineer’s library.
Other
There are a number of systems engineering organizations world wide, all of which can
provide help in applying systems engineering practices on an ITS project. The
International Council on Systems Engineering (INCOSE), located in the United States, is
recognized as the leading systems engineering organization in the world. Its web site
(www.incose.org) provides links to a number of other systems engineering resources. Other
systems engineering organizations include:
Also, Southern Methodist University (SMU), in Texas, has 1000+ links to systems
engineering resource sites on a web page, (www.seas.smu.edu/disted/sys/r.html).
59
Appendix
Earned Value Analysis Calculations
61
Earned Value Analysis Calculations
Calculating Earned Value on ITS Projects
With these three values, you can estimate four key factors:
Cost variance (CV): This is the difference between the budgeted cost of work
performed and the actual cost of work performed, using the equation:
CV = BCWP - ACWP
Schedule variance (SV): This is the difference between the budgeted cost of
work performed and the budgeted cost of work scheduled, using the equation:
SV = BCWP - BCWS
ECAC = (ACWP/BCWP) x CB
ECAC = (BCWS/BCWP) x T
We have a project whose original schedule was one year. The original cost estimate for
the project was $500,000. We are now reviewing the project at the end of six months,
and we determine the following facts:
63
Our project has been spending about $40,000 per month and total expenditures for
the first six months are $235,000.
The project has six major work breakdown schedule (WBS) tasks at the second
detail level, and three of them are complete.
The tasks that are complete are tasks 1.1, 1.3, and 1.4.
At the end of six months, you planned to have tasks 1.1, 1.2, and 1.3 completed.
Let’s determine each of the values we need. We already know the actual cost of work
completed ($235,000). We can compute the other two costs.
When we calculate the estimated cost at completion, we see the potential impact of the
cost variance: $500,000 x ($235,000/$215,000) = $546,500 or a 9.3% potential cost
overrun. And calculating the estimated time to complete tells us what kind of extra time
we may need: 12 months x ($240,000/$215,000) = 13.4 months or an 11.7% time
overrun.
Well, the news is not good anywhere. But remember, this is a management tool. If you
know now that there’s a potential to overrun both your costs and your schedule, you still
have time to take corrective action. If you can’t fix the problem that’s causing your
project to overrun, at least you can alert your management ahead of time.
64
Granted, the example given is very simple. And, you could ask, why did we ignore the
work completed in the other tasks, which could have changed the calculations
dramatically? In reality, you’re going to calculate the actual cost and the budgeted costs
at the lowest level of the WBS, the level where the task is either done or not done. (You
should only consider a task “x percent complete” when there is a solid measure that tells
you that the “x” represents a value that you can count on.) You can adjust how you credit
work done and work scheduled, but remember that you’re trying to get a realistic picture
of where you stand on your project, not make yourself look good when the circumstances
don’t warrant it.
So good luck! This may provide one more good tool in your systems engineering tool
bag.
65
Bibliography
______, “Payroll Systems Flunk Final Exam,” Computerworld, October 25, 1999
Anon., Systems Engineering Handbook, A “HOW TO” Guide For All Engineers, Release
1.0, International Council on Systems Engineering (INCOSE), San Francisco Bay Area
Chapter: 1998
Blanchard, Benjamin S. and Wolter Fabrycky, Systems Engineering and Analysis, 3rd ed.,
Prentice Hall, New Jersey: 1998
Blanchard, Benjamin S., Systems Engineering Management, 2nd ed., John Wiley & Sons,
Inc., New York: 1998
Boehm, Barry W., (ed.), Software Risk Management, IEEE Computer Society Press,
Washington, DC: 1989
Charette, Robert N., Software Engineering Risk Analysis and Management, McGraw-Hill
Book Company, New York, 1989
Eisner, Howard, Essentials of Project and Systems Engineering Management, John Wiley
& Sons, Inc., New York: 1997
Hunger, Jack W., Engineering the System Solution: A Practical Guide to Developing
Systems, Prentice Hall, Inc., Englewood Cliffs, NJ: 1995
IEEE Std. 1362-1998: IEEE Guide for Information Technology – System Definition –
Concept of Operations (ConOps) Document, Institute of Electronic and Electrical
Engineers, 31 December 1998
Janis, Irving L. and Leon Mann, Decision Making: Psychological Analysis of Conflict,
Choice, and Commitment, The Free Press, New York: 1977
Johnson, Jim, “Turning Chaos Into Success,” Software Magazine, Dec. 1999 – Jan. 2000
Ould, Martyn A., Strategies for Software Engineering: The Management of Risk and
Quality, John Wiley & Sons, Inc., New York: 1990
Rechtin, Eberhardt and Mark W. Maier, The Art of Systems Architecting (Systems
Engineering Series) CRC Press, Boca Raton: 1997
67
Sage, Andrew P., Systems Engineering, John Wiley & Sons, Inc., New York: 1992
Sage, Andrew P. and William B. Rouse, (ed.), Handbook of Systems Engineering, John
Wiley & Sons, Inc., New York: 1999
68