Professional Documents
Culture Documents
Condition Based Maintenance Theory Metho
Condition Based Maintenance Theory Metho
Application
Amanda Gillespie
Amanda Gillespie
Science Applications International Corporation (SAIC)
1710 SAIC Drive
McLean, VA 22102, USA
e-mail: [email protected]
Gillespie, Page i
SUMMARY AND PURPOSE
Current emphasis on improving maintenance policies and techniques while decreasing overall Life Cycle
Costs (LCCs) has driven industry to develop unique and potential cost saving philosophies such as Condition
Based Maintenance (CBM), a subset of Reliability Centered Maintenance (RCM). As the government has
taken notice, CBM policies have been directed for mission-critical systems. Of particular interest and focus
are the Performance Based Logistics (PBL) contracts, which reward industry for achieving government
requirements by improving profitability. CBM techniques can be utilized inside a PBL contract as a way to
reduce life cycle costs, but incentives for this be long-term goal are minimal for PBL and implementing
CBM to achieve this goal can be cost prohibitive for contractors. This tutorial presents a history of CBM,
tools, techniques, and the elements, presents an example of CBM analysis and its benefits, discusses the
government CBM and PBL contracting policies, and presents a proposed way for government and industry
to both optimize profitability of PBL contracting for industry and decrease LCCs for the government with
the utilization of CBM strategies.
Amanda Gillespie
Ms. Gillespie is a Solution Architect within the Logistics and Supply Chain Service Line of Science
Applications International Corporation (SAIC). She received her BS in Applied Mathematics from the
Georgia Institute of Technology in 2000. Ms. Gillespie received her American Society of Quality (ASQ)
Certified Reliability Engineer (CRE) certificate in January 2011. She has 14 years combined experience in
reliability engineering, maintainability engineering, logistics engineering, Integrated Product Support (IPS),
and statistical analysis. Ms. Gillespie currently works to develop sound, repeatable methodologies and
offerings in the Logistics, Maintenance, and Reliability Engineering fields for with multiple Department of
Defense (DoD), federal, and civilian customers.
Ms. Gillespie spent six years as a reliability engineer for NASA at Kennedy space Center (KSC) performed
reliability, maintainability, and availability (RMA) analysis for various NASA programs projects. Amanda
also developed sampling plans, per ANSI/ASQ Z1.4, to address safety concerns during the Space Shuttle
Program (SSP). Prior to joining SAIC, Amanda was a Senior Systems Analyst, where she specialized in
Range risk management for such things as maximum probable loss, trajectory analysis, casualty expectation,
over-flight risk, and hazardous area determination. Ms. Gillespie started out as a Systems Analyst, where
she performed root cause failure analysis, acceptance testing, and launch monitoring of Range Safety,
Control & Display systems, and Radar systems tracking.
Table of Contents
1. INTRODUCTION .................................................................................................................................. 1
2. BRIEF HISTORY OF RCM AND CBM DEVLOPMENT ................................................................... 1
3. CBM ELEMENTS AND TECHNIQUES .............................................................................................. 2
4. CBM EVOLUTION TO CBM+ ............................................................................................................. 3
5. BENEFITS, DRAWBACKS, AND CHALLENGES OF CBM/CBM+................................................. 6
6. CBM/CBM+ STRATEGIES, ACTIVITIES, AND TIMELINE ............................................................ 6
7. EXAMPLE OF CBM ANALYSIS ...................................................................................................... 13
8. CBM+ AND PBL CONTRACTS ......................................................................................................... 19
9. CONCLUSIONS .................................................................................................................................. 21
10. REFERENCES ..................................................................................................................................... 22
Gillespie, Page ii
2. BRIEF HISTORY OF RCM AND CBM
1. INTRODUCTION
DEVLOPMENT
Maintenance techniques and policies are changing and
Maintenance techniques evolved from the 1940s to today.
evolving, arguably more rapidly than any other discipline. In
CBM is a maintenance technique used to actively manage the
some government programs and industries, maintenance and
health condition of equipment in order to optimize maintenance
repair procedures and policies were developed post-design or
by performing maintenance only when it is needed and at the
only in response to unpredicted maintenance issues. Now,
most opportune times, thus improving overall system and
however, new techniques and changing views on maintenance
equipment availability and safety while decreasing operating
organization, policies, and responsibilities are emerging. This
costs. One cannot talk about CBM without first discussing the
is due, in part, to the management understanding of the impact
concepts and history of RCM maintenance techniques of RCM.
of equipment failures on safety, availability, mission readiness,
and the environment. These changes in attitude have brought 2.1 RCM Concepts & History
about Reliability Centered Maintenance (RCM) policies in an
RCM models were developed as a result of the changing
attempt to address a host of reliability issues in order to balance
world of maintenance, in which managers began to understand
improvement in overall equipment and system effectiveness
and acknowledge the costs of equipment failures on product
and control of the program Life-Cycle Costs (LCC).
quality, availability, and ROI. Figure 1 shows the block
The RCM model uses Preventative Maintenance (PM),
diagram of the RCM model [1].
Run to Failure (RTF), and Condition-based Maintenance
(CBM) techniques to in an integrated manner to increase the Reliability Centered Maintenance
probability that equipment will function in the required manner (RCM)
over its predicted life cycle with a minimum of maintenance.
CBM models can be divided into two elements, predictive
maintenance (PdM) and Real-time Monitoring (RTM), which
are the pillars of CBM techniques. There are difficulties,
Preventative Maintenance Condition-based
Run to Failure (RTF)
however, in implementing CBM, as initial investment costs can (PM) Maintenance (CBM)
Gillespie, Page 1
probability curve for failures based on operating time. This but in decreasing overall system downtime.
failure measure supported the view that systems can fail
2.2 CBM History
“randomly” at any point in their useful life. This represented
the horizontal line in the bathtub curve. CBM is not a new maintenance technique. The first
These types of maintenance techniques, known as PM, generation appeared in the late 1940's, as implemented by the
typically utilizes MTTF, or the inverse of MTTF, failure rate Rio Grande Railway Company. The company achieved
(λ), or Weibull shape (β) and scale (η) parameters, in economic success and reduced engine failure by performing
conjunction with maintenance procedures and the measures of maintenance whenever glycol or fuel was detected in the
maintainability, including Mean time to Repair (MTTR), to engine oil. The U.S. Army was impressed by the relative ease
predict when maintenance activities should take place to with which physical asset availability could be improved and
improve availability, or uptime. In this maintenance technique, adopted those techniques and developed others. During the
overhauling a system, or replacing parts likely to fail after a 1950's, 1960's, and early 1970s, the emerging technology
certain period of time, is intended to optimize and improve industry began providing training, products, and services which
availability at the point where the “bathtub” curve begins to came to be known as "predictive maintenance" [3].
show an increase in failures, or end of life. Performing The next generation of CBM coincided with the dawn of
maintenance or overhauls prior to this point, or during a the "information age.” “Technology entrepreneurs conjectured
system’s useful life, will most likely not optimize the that, if simple physical measurements, such as vibration
availability and can tend to increase maintainability costs. amplitude or oil viscosity, could provide such useful benefits,
Figure 2 is an industry accepted graph showing the qualitative then collecting the data in computers and trending it over time
relationship between the frequency of PM and the total costs would, likely, provide a far deeper insight into the state of a
[2]. Exceptions to these types of PM techniques would be machine's health. Hence, the 1980s and 1990s witnessed a
when the equipment failures have extreme safety soaring rise in the use of computers, software, and data
consequences, like in the nuclear power industry. In these collectors in maintenance shops throughout the industrial world
cases the cost of PM is usually a secondary factor. [3]. This is the current understanding and application of CBM
as it will be discussed in this tutorial.
3. CBM ELEMENTS AND TECHNIQUES
CBM is a failure management strategy for a particular
failure mode that meets criteria such as:
The potential failure is clearly defined
Failure interval is identifiable
The maintenance task interval is less than the failure
interval and physically possible.
The time between the discovery of the potential failure
and the occurrence of the function failure is long
enough for the maintenance action to be taken to
avoid, eliminate, or minimize the consequences of the
Figure 2. Time between Maintenance Intervals versus Cost
failure mode [4].
There can be difficulty in determining the cost/benefit ratio Whereas PM addresses age-dependent failure probabilities,
of PM. Although increasing the maintenance intervals will CBM addresses failures that can be measured by one or several
almost immediately decrease the overall cost, the negative indicators [5]. When applying maintenance efforts (people,
results of increasing that interval will not be seen until processes, and tools) in a CBM environment, maintenance is
equipment replacement and repair costs and downtime based on the actual condition of the equipment versus the age
increases, at a much later time, and erases the short term of the equipment, so that equipment in good condition does not
savings [2]. However, PM techniques alone do contribute to need to be maintained as frequently as equipment that has
improved system availability and tend to minimize reached the predicted age of deterioration. The core of CBM is
maintenance costs; new research has changed our beliefs and using test equipment or statistically modeling data to predict
understanding of age and failure of equipment. It is becoming the condition of equipment [2]. The vision of the CBM
apparent that the connection between the operating age of application is to enable equipment to achieve nearly zero
equipment and how likely they are to fail is diminishing. As breakdowns. This will transform traditional maintenance
our decision support tools and data analysis tools become practices from RTF to PdM and prevention of failures. CBM
increasingly more sophisticated, and our systems are being utilizes failure history to predict breakdowns before they
designed with greater emphasis on high reliability and happen to prevent future failures from occurring in the field
maintainability, our equipment reliability and maintainability where repairs are costly and operations are impacted.
modeling also must evolve. Utilizing CBM techniques can
assist not only in our reliability and maintainability modeling,
Gillespie, Page 2
3.1 CBM Elements: RTM
RTM monitoring is based on the idea that equipment can
be evaluated while remaining in service, which will drive down
the overall cost of maintenance. For example, a generator can
be monitored based on a number of parameters in real time,
such as temperature, cooling gas density, bearing vibration,
lubricating oil condition, and others [6]. Statistical analysis of
the collected data allows diagnosis of impending failures.
These failures, also known as incipient faults, cannot be
predicted by human senses [2]. Other equipment such as
circuit breakers, relays, and switches are not readily assessable
utilizing RTM techniques [6].
The valid candidate for RTM must satisfy both the
following criteria:
1. Equipment must be critical or expensive enough to
warrant the cost, purchase and installation, of
monitoring hardware and software.
2. Analysis of the parameters monitored must provide
meaningful diagnostics and prognostics [2].
3.2 CBM Elements: PdM Figure 3. RCM, CBM, and CBM+ Strategies
PdM trending techniques have been used to confirm acquisition, sustainment, and operations.” CBM+ ranges from
maintenance decisions that would have previously been based addressing challenges, by improving diagnostics and
on expert opinions [6]. PdM uses test results taken from PM prognostics or similar, to fostering and developing new support
techniques. Statistical analysis results are evaluated, and a concepts, such as anticipatory maintenance.
prognosis is developed indicating the need for increased, The range of technologies and strategies included in
decreased, or even elimination of maintenance intervals [2]. CBM+ is broad, which can make it difficult to focus on a core
While these techniques have often found problems that might strategy or goal. The CBM concept encourages the need-
not have been identified, PdM may actually slightly increase driven approach to system support and forms the foundation of
daily maintenance costs for some equipment due to the CBM+. The “plus” is intended to encompass a variety of
additional analysis required [6]. equipment and maintenance process improvements that are
4. CBM EVOLUTION TO CBM+ possible through the following:
More disciplined approach to maintenance planning
The current Department of Defense (DoD) initiative is Development of sensor technologies
called CBM+. The DoD CBM+ “initiatives strive to push Better maintenance collection and analysis techniques
inspection and scheduled replacement failure strategies toward More capable and user friendly maintenance aids
CBM (to improve safety readiness and cost). DoD CBM+ has Integration of maintenance and other logistics systems
broadened this definition to include technologies such as into the program structure
Interactive Training, Item Unique Identification (IUID), and
Integrated Information Systems [4]” 4.1 Essential Elements of CBM+
All activities within CBM+ can be broken into three CBM+ elements can be placed in two primary categories:
categories: CBM analysis tools, like RCM, and Failure Modes, business/management and technical. There are six main
Effects, and Criticality Analysis (FMECA), CBM enablers, subgroups within these categories. The first three listed are in
such as active or passive sensors, and CBM ancillary enablers, the business/management category and the last three listed fall
such as re-designs of systems that remove failure modes that within the technical category.
cannot be monitored.
While RCM uses CBM as a failure management strategy, 4.1.1 Policy & Doctrine
the CBM+ focuses on providing the support required to permit In a memorandum signed by the Deputy Under Secretary
CBM. These RCM, CBM, and CBM+ strategies are shown in of Defense (Logistics and Materiel Readiness) in November of
Figure 3 [4]. 2002, CBM+ was to be “implemented to improve maintenance
CBM+ strategies applied allow for maintenance performed agility and responsiveness, increase operational availability,
based on evidence of need provided by RCM analysis and and reduce life cycle total ownership costs.” DoD activities
techniques and other enabling processes and technologies. should establish a CBM+ environment for the maintenance and
According to the DoD CBM+ guidebook, “CBM+ uses the support of systems by establishing appropriate processes,
systems engineering approach to collect data, enable analysis, procedures, technological capabilities, information systems,
and support the decision-making process for system and logistics concepts.
Gillespie, Page 3
4.1.2 Business Strategy Society of Automotive Engineers (SAE). The Machinery
Information Management Open Systems alliance (MIMOSA)
The basis for CBM+ is a focus on improving the business
also established specifications and data models in support of
process of maintenance with the objective being improved
condition monitoring.
operational performance as a result of increased maintenance
Data strategy is also essential to CBM+. The degree of
effectiveness. Identifying business needs and strategies is the
data management will depend on the amount of health
key to this element of CBM+. Processes should add value,
assessment and predictive activity required. Many possible
prevent unnecessary maintenance activities, and increase
approaches exist to data storage and interchange, and
effectiveness and efficiency. Once the CBM+ business case is
aggregation may occur across the system.
developed, it becomes the essential tool for validating and
Data collection strategies include trend analysis, pattern
supporting the CBM+ requirement.
recognition, tests against limits and ranges, and statistical
4.1.3 RCM Relationship analysis.
As discussed previously, there is a close relationship 4.2 CBM+ Enablers
between RCM and CBM+. The RCM analytical approach
CBM+ enablers are those tools, equipment, and techniques
helps the maintenance and logistics manager identify potential
failures and support the selection of viable courses-of-action. that allow for successful results from CBM+ strategies. There
RCM tools help define the optimal failure management are several examples of these enablers, including sensors and
strategies and provide input to the business case for algorithms, data collections, maintenance information systems,
implementation of CBM+ strategies. information tools, engineering analysis, and system integration,
4.1.4 Hardware & Software Infrastructure Tools as shown in Figure 4.
Gillespie, Page 4
include and quality control issues, support equipment 700-127.
requirements, and warehouse support availability. 2. Material Reliability (MR) – MR is a measure of the
Maintenance information systems network tools identify probability that the system will perform its intended
both up-line reporting/recording and down-line support. In function without failure over a specific interval. MR
other words, implementing tools such as Computerized is generally expressed in terms of MTBF when the
Maintenance Management Systems (CMMS) to track system is in design or non-operational. Once
maintenance and calibration activities enables CBM+ strategies operational, MR can be measured by dividing actual
by tracking PM, unscheduled maintenance, and calibrations. operating hours by the number of failures observed
Support should also include training for CMMS use to the during a specific interval.
maintainers. 3. Ownership Cost (OC) – OC balances the sustainment
Information tools can enable CBM+ strategies by solution by ensuring that Operations and Sustainment
providing interactive electronic technical manuals (IETMs), (O&S) costs associated with MR are considered when
portable maintenance aids (PMA) and computers, automated making design and operational decisions.
identification technology (AIT), and serialized item 4. Mean Down Time (MDT) – MDT is the average total
management (SIM) tolls to maintainers to allow for efficient time required to restore a component to its full
and effective identification of impending failures or operational capabilities. MDT includes the time from
maintenance requirements. reporting of a component failure or Non-Mission
RCM analysis results collected from actual and like Capable (NMC) status to the component being given
systems identify trends to provide maintenance plans. These back to operations or production to operate in a Fully
analyses fall under the engineering analysis umbrella of CBM+ Mission Capable (FMC) status.
enablers. Maintenance plans should be iterative in response to MDT is an important measure in Operational
engineering analyses. Availability (Ao), and, thus, MDT can be extended to
Finally, system integration links logistics with components operating in Partially Mission Capable
maintenance through the increased capabilities of computer (PMC) status while achieving all operational
systems and connectivity tools to provide faster response and requirements. For example, a system with four
better material support. An example of this enabler is the generators may be able to meet mission requirements
linkage of equipment orders for spares and support equipment with only two generators operating, with a third in
with predicted maintenance or operations activities. backup. A failure of the fourth generator would give
These CBM+ enablers allow for CBM+ to be implemented the system PMC status, but not impact the operational
appropriately providing the optimal results. availability, thus allowing the operation to continue.
4.3 CBM+ Objectives and Metrics
The relationship between CBM+ Objectives and these
CBM+ strategies provide the opportunity to improve Life-cycle sustainment metrics is shown in Figure 5.
business processes, specifically improved maintenance
performance to include benefits such as increased productivity,
shorter maintenance cycles, lower costs, increased process
quality, improved system availability, and enhanced reliability
of equipment. With this in mind, the DoD has developed four
metrics to determine the overall life-cycle sustainment impact
and outcome of CBM+. These metrics are:
1. Material Availability (MA) – MA is a measure of the
percentage of the total inventory of a system that is
operationally capable (ready for tasking) of
performing an assigned mission at a given time, based
on material condition. Mathematically, this is
expressed as the number of operational end items
divided by the total population.
MA also indicates the percentage of time a system is
operationally capable of performing an assigned
mission. This is expressed as uptime divided by the
sum of uptime and downtime.
Material, or equipment, availability is a Key
Performance Parameter (KPP) associated with Key
System Attributes (KSAs) of material reliability and Figure 5. CBM+ Objective versus Life-Cycle Sustainment
ownership costs. This is part of an Integrated Product Metrics
Support Planning (IPSP) strategy, or Integrated
Logistics Support (ILS) strategy, Army Regulation
Gillespie, Page 5
5. BENEFITS, DRAWBACKS, AND CHALLENGES OF operational readiness effects, logistics benefits, and ROIs to
CBM/CBM+ justify the cost of CBM+ implementation.
CBM implementation in legacy systems versus new
Arguably, one of the most significant positive impacts in
acquisition systems can pose substantial challenges. Particular
implementing CBM strategies is that accurate, efficient, and
problems include installation of on-board, or embedded,
effective information from failures and maintenance activities
sensors, inadequate existing communications and data
can enable informed and optimal management decision
repository capabilities can impede data collection and failure
making. This usually results in a smaller footprint for the
condition analysis, and off-board, or manual data gathering and
maintenance and logistics support packages with more efficient
analysis, capabilities may not be as comprehensive as required
maintenance, optimal system readiness, and cost savings.
and could burden an already overworked maintenance
CBM+ focuses on applying technology that improves
workforce. Legacy data systems may not be adequate. When
maintenance capabilities and business process, complements
adding CBM to exiting systems, concentration should be
and enhances reliability analysis efforts, involves the
placed on standardizing data management technologies by
integration of support elements to enable enhanced
maximizing application of common health management
maintenance-centric logistics system response, and facilities
software and standardized training, i.e., COTS solutions.
more accurate predictions of impending failures, resulting in
CBM/ CBM+ is focused on determining the need for
dramatic savings and improved system availability.
maintenance before operations are effected, and then having
On the contrary, the most significant drawback of CBM
maintainers respond to that need effectively and efficiently.
can be on the initial bottom line. Costs associated with initial
When implementing CBM strategies, maintainers are the key to
implementation of CBM/ CBM+ strategies can be prohibitive
the successful execution of equipment maintenance.
to some programs. Implementing CBM techniques and
Maintainers need to be well trained, well informed, well
strategies could mean new tools, test equipment, and/or
equipped, and well supported. This is where CBM+ exceeds
embedded on-board diagnostics. Although, initially,
the traditional CBM strategies, by ensuring that maintenance
maintenance and logistics costs may increase in the short-term
training is performed.
(1-2 years), if CBM/CBM+ is implemented correctly, the
Regardless of what maintenance and logistics strategies are
overall cost of the system logistics, operations, and
implemented, RCM, CBM, CBM+, the key to successful
maintenance footprint is less than it would have been without
results are the maintainers. Management can support the
CBM/ CBM+ strategies applied. However, this is a long-term
strategies, maintenance and logistics engineers can recommend
goal in the DoD’s usually short-term budgets, developed one or
maintenance activities, procedures, and work flows, but
two years (or months) at a time in response to yearly allocated
without the buy-in of the maintainers, none of these strategies
funding levels. This makes the initial investment into CBM/
and techniques will be successful.
CBM+ methodologies less palatable to managers. Even with
DoD policies requiring implementation of CBM/CBM+ 6. CBM/CBM+ STRATEGIES, ACTIVITIES, AND
strategies, CBM/ CBM+ has to “buy its way” into the program. TIMELINE
This cost of CBM/CBM+ implementation can be spread
It has been proven repeatedly that the most effective and
throughout an entire fleet of like systems, for example, air craft
efficient maintenance plans are developed during the design
from all services, Army, Air Force, and Navy/Marine Corps, or
and acquisition of a system or equipment, as the correct
even similar aircraft within the service. CBM identifies
processes and technologies are incorporated up front. Re-
maintenance actions based on near-real time assessment of
design efforts are expensive, as are implementation of new
equipment status from diagnostics sensors and equipment.
maintenance plans when a system is operational. However, as
Data collected from sensors, such as health and usage
discussed previously, initial cost impacts to implement new
monitoring systems, are analyzed and translated into predictive
maintenance policies as a result of CBM should not deter any
trends or metrics that can anticipate when component failures
maintenance or logistics practitioner, or manager, from taking
will occur and/or identify components that may require
advantage of CBM strategies and opportunities when possible
redesign or replacement to reduce failure rates. If fleets would
and when the long-term gains exceed the short term
require common use of items and data on like-systems, the
expenditures.
logistics footprints and costs would be greatly reduced.
Figure 6 is depicts CBM implementation in the three
Another implementation challenge of CBM is to develop
phases that complement the overall total system life-cycle
the basis of information, knowledge, and experience to help the
design and acquisition strategy: planning, implementation, and
program and management make the necessary changes for
operations. The timeline for CBM implementation will be
existing programs and future acquisitions and system designs
discussed in this context. It should also be noted that this
and development. Once established, however, a reliability or
tutorial will address the CBM+ methodologies and approaches,
logistics engineer can champion the CBM/ CBM+ insertion in
as it encompasses and, in the author’s opinion, enhances, all
the design phase, recognize opportunities to modify legacy
aspects of sound CBM strategies. CBM will refer to CBM+
systems when cost effective, and better communicate the
methodology from this point forward.
Gillespie, Page 6
Figure 6. Total System Life Cycle Process
6.1 CBM Planning and Technology Selection Phase with this step. A maintenance technician can better quantify
the impact of system downtime, supply chain issues, support
The initial efforts with CBM planning should focus on
equipment issues, and maintenance issues, than a logistics or
CBM concept familiarization, ensuring that management and
maintenance manager. This will also be particularly important
employees at all levels, specifically the maintainers, understand
to the manager’s understanding of the personnel and funding
CBM and are supportive of CBM objectives and planning, and
impacts as the CBM initiative progresses.
developing the basic steps required to initiate the effort. As
with all initial efforts, change management is key, and “buy in” 6.1.2 Perform RCM and Reliability Analyses
from the bottom-up can be a more effective approach than
Reliability analysis should consider trade-offs among time
forcing change from the top-down.
to failure, system performance, and system life-cycle costs.
6.1.1 Obtain Management Support The primary types of analyses needed for CBM include: RCM,
FMECA, and Fault Tree Analysis (FTA). One should also
As CBM strategies and technologies are gaining popularity
consider an approach with Reliability Block Diagrams (RBDs)
and proving their long term impact on operations and
and Importance Measure Analysis (IMA) in order to determine
sustainment costs for many programs, systems, and equipment,
those components contributing the highest to system
it may be easier to obtain this support. Many logistics
unavailability. This methodology is described in a previous
managers accept CBM strategies, to varying degrees, but may
paper [7], and can help quantify component failure
be unfamiliar with the specific changes required to implement
contributions and the impact on the ROI.
these strategies to achieve results.
Although not explicitly called out, a Level of Repair
This puts CBM proponents in the role of a CBM publicist.
Analysis (LORA) can also identify the overall maintenance
Having high quality communication skills, energy, and
structure and determine where an item will be replaced,
knowledge in CBM strategies and its application to the
repaired, or discarded based on cost considerations and
program, are all must haves for the CBM publicist. Ensuring
operational readiness.
that management receives the appropriate amount of briefings
with the right reasoning is essential in getting through this 6.1.3 Form a CBM Team
hurdle.
Throughout DoD and industry, the Integrated Product or
Obtaining maintenance technician “buy-in” first may help
Gillespie, Page 2
Process Team (IPT) is an effective way to gather the personnel requirements.
and skills required to accomplish CBM initiatives. CBM is not
6.1.8 Develop Data Strategy
a one-dimensional discipline, such that brining in only
personnel focused on one aspect of CBM, such as sensor This should define the approach and mechanism for
technology, will not provide the range of expertise needed for managing the condition and related data required to accomplish
effective implementation. A CBM IPT should include, at a condition-based analysis. The architectural interface views for
minimum, systems engineering, reliability engineering, safety, data management, storage, and exchange should be completed.
data management, supply chain management, maintenance and 6.1.9 Develop Architecture
logistics engineering, and training and certification. Personnel
with clear understanding of the costs associated with each and Once the CBM Implementation Plan has been approved,
budgeting and funding should also be included. the architectural views, descriptions and profiles should be
constructed. Managers rely on the architectural representations
6.1.4 Identify CBM Target Application
to identify personnel training topics, assess progress for each
In many cases, as discussed previously, CBM can require process component, reallocate developmental resources,
significant up-front funding and resources from the integrate different process components, and explain details of
maintenance organization. Therefore, CBM proponents should the initiative to outside reviewers.
selectively focus on equipment with an anticipated high ROI in 6.1.10 Set Life-Cycle Metrics
improved performance, improved system life, and more
efficient maintenance capability, and overall reduction of life- Life-cycle sustainment metrics, as described previously in
cycle resource costs. This is where the reliability analysis, Section 4.3, provide the quantitative tools to track CBM
specifically the aforementioned RBD and IMA will provide implementation and operation. High-level diagnostic metrics
significant input, regardless of whether the implementation is must be developed to ensure achievement of overall
for new acquisition or legacy program. Other targeted areas performance and cost goals. Quantitative measures should be
could be in those systems with the greatest maintenance in the areas of meeting the strategic needs of the enterprise,
workload or support equipment requirements. needs of individual customers, addressing internal business
performance, and addressing process improvement initiative
6.1.5 Accomplish Proof-of-Concept
results.
Small scale demonstrations of primary CBM methods and 6.1.11 Develop Deployment & Support Strategy
technologies are advisable to justify time and funding resources
for primary CBM implementation. Demonstration of CBM CBM implementers should define and develop the
planned methods and technologies give managers a higher projected deployment schedule, including the expected training
degree of confidence in the likelihood of future success. and installation dates. A well-documented, yet flexible
Documentation of test results will provide support for future deployment plan is critical to success. The deployment plan
implementation and funding. should be iterative. If installation isn’t working, rollback
efforts can be made to try to install again at a future date. Data
6.1.6 Prepare CBM Implementation Plan
conversion will be a critical task for the deployment of new
The format for a “CBM Plan” should include a capabilities.
comprehensive statement for the planned scope of the CBM 6.1.12 Complete the Business Case
application, general supportability objectives for major
maintenance activities, a description of how initiative goals and A business case analysis (BCA) identifies functional and
objective are to be achieved, requirement statement and supporting technical alternatives and presents economical and
planned design approaches, identification of key factors technical arguments for selecting alternatives over the life cycle
external to the organization and beyond their control that could to achieve the business objectives or management direction.
significantly affect goal and objective achievement, a This is an essential tool to support management decisions and
description of the program evaluation process (including help justify program and budget inputs.
metrics) to be utilized in evaluating progress, and a plan of 6.1.13 Develop Resources Strategy & an Integrated Budget
action and milestones.
It is critical that from the beginning that CBM be marketed
6.1.7 Examine New Technologies
with stakeholders and customers. This requires the
The most difficult task for CBM implementation may be to identification of resources and budget expectations. Depending
correctly match available hardware, software, and supporting upon where the CBM initiative is applied, applicable funding
technology solutions to the requirements of anticipated and may be from research and development, procurement, or
future maintenance processes. A decision analysis, or trade-off operations and maintenance (O&M) appropriations. CBM
analysis, will need to be performed to determine the optimal managers should ensure validated resources are included in
solution for the required capabilities against cost, time, and acquisition requirements documents as early as possible.
implementation difficulty. Decisions on technology selection
should always be made considering the need to meet functional
Gillespie, Page 9
DEVELOP RESOURCES STRATEGY & AN INTEGRATED BUDGET
DEVELOP ARCHITECTURE
Timeline
Gillespie, Page 10
production, communication, storage, access, and use. A 6.2.8 Resolve Performance & Cost Issues
functional demonstration of the data management process to
The demonstration and pilot test efforts provide the input
users should occur as early as possible in the implementation
for modification of performance objective and identify areas
phase. This demonstration should include a review of a
where additional costs or reallocation of resources may be
statistically significant range of data in a database and health
necessary. If resource changes cannot be made, then the
management software test runs against the database. This is a
impact on the CBM Implementation Plan should be determined
building block of the foundation for user confidence in CBM
and communicated with management.
ROI.
6.2.9 Train Stakeholders & Users
6.2.4 Revalidate RCM & Reliability Analysis
Training is a very important part of deployment.
A continuous review process will ensure periodic
Stakeholders and users may need training in both learning how
revalidation of initial reliability assessments. During the
to work with the condition-based applications and on how to
design or acquisition process, this continuous review is
work within the condition-based process. Training plans and
necessary to determine appropriate changes to maintenance
schedules should follow the implementation milestones.
procedures and approaches based on system re-design,
Training plans should maximize the use of new learning
equipment and component modifications, operational and
techniques, simulation technologies, embedded training, and
mission changes, technological advances, improved failure
distance learning systems.
data, and an improved logistics capability.
6.2.10 Revise Implementation Plan
6.2.5 Demonstrate CBM Element Interoperability
The CBM Implementation Plan should be responsive and
This is the demonstration of the effective interoperations
kept current and aligned with management decisions, resource
of such things as hardware, software, data analysis, and failure
availability, acquisition of essential CBM elements, and the
trends. This is an iterative process, as hardware and software
achievement of milestones.
elements of CBM are acquired and the data management
mechanism is put in place, CBM implementers should test the 6.2.11 Update Supportability Strategy
information exchange capabilities using as much of the
System supportability is highly dependent on the
condition data and analytical information derived from sensor maintenance plan. Continuous analysis and revisions to the
sources. maintenance plan can balance logistics support resources.
6.2.6 Demonstrate CBM Functionality Reviews of readiness degraders, equipment maintenance data,
maintenance program schedules and execution, and industrial
Once the component elements of CBM initiative are
coordination to identify and assess new maintenance methods
acquired and tested, the next step is to test and validate system
and technologies are necessary to update supportability
inputs, outputs, and analytical products against approved
strategies.
metrics. Demonstrations should assure the implementers and
users that CBM elements will produce results that are accurate, 6.2.12 Acquire Full Production Capability
timely, and meet the expectations of the user. The full range of planning, acquisition, testing, and
6.2.7 Complete Pilot Program Field Test demonstration actions must be successfully accomplished prior
to approval to acquire the full scope of CBM capabilities.
There is no substitute for process testing in an operational
environment. Pilot testing should be done at an operational 6.2.13 Accomplish CBM Deployment
location that permits the users to participate in the new process Deployment should be executed in accordance with the
in a familiar setting. The pilot test environment should still be Deployment and Supportability Strategy Plan. CBM
more controlled than actual operations, such that a implementers should ensure user organizations are fully
comprehensive test plan structure is followed, test activity and prepared to receive and operate the planned CBM capabilities.
results are tracked and fully documented, including operational
user commands, input and output test data should be screened, 6.2.14 Implementation Phase Timeline
with out-of-tolerance data clearly identified, and specific Most of the implementation activities must be performed
testing timeframe should be established. Documentation of consecutively, with testing, interoperability demonstrations,
these test results help assess whether the results of the pilot field testing, acquisition of technologies and deployment
reliability analysis that determined the maintenance actions are needing to take place at specific times with specific pre-
appropriate for the tested equipment. requisites. However, some activities like iterating CBM
Implementation Plans and Supportability Strategies should be
performed constantly through the implementation phase in
response to equipment and support availability and
management decisions.
Gillespie, Page 11
ACCOMPLISH CBM DEPLOYMENT
Timeline
Gillespie, Page 12
management efforts to evaluate the effectivity and efficiency 6.3.10 Operations Phase Timeline
the CBM initiative.
Unlike the planning and implementation phases of
6.3.5 Refresh Enabling Technologies CBM, the activities in the operations phase should be
continuous. However, the activities will start and stop
Technology advance occurs rapidly and requires CBM
depending upon the resource availability.
implementers to develop a technology refreshment strategy
to provide long-term, cost-effective support and operations 6.4 CBM Strategies, Activities, & Timeline Summary
ahead of the obsolescence curve. A proactive approach to
All strategies and activities described in the previous
managing technology updates include improving technology
subsections are taken from the CBM+ DOD Guidebook;
surveillance, leveraging commercial industry technology
however, all activities can be applied to industry or DoD.
advancements, minimizing product obsolescence impacts,
The CBM activities and timeline approach is similar to that
developing realistic and achievable technology refreshment
of systems engineering timelines and activities, and in
planning schedules for selected system and safety critical
acquisition and design, systems engineering activities should
products, and building and maintaining a knowledge base
be performed concurrently, with CBM activities providing
that contains lessons learned that can easily be accessed.
input to systems engineering activities and vice-versa.
There are downsides to technology refreshment, such as
CBM activities can seem overwhelming, but provide a
expensive modifications and increased configuration
methodology that allows for ROI with sound
management (CM) for multiple versions of hardware and
implementation. Some of the activities can and should be
software.
tailored depending upon the program, systems, and
6.3.6 Revalidate Human Interfaces equipment. However, care should be taken to document
reasoning behind tailoring activities.
DoD has an unfortunate history of mismatches between
All estimated timelines can be shortened by applying
technology capabilities and human operator abilities to
higher levels of resources towards planning and
properly understand and make the best use of these
implementation activities. Likewise, if applying minimal
technologies. Adequate training can help mitigate this risk,
resources, timelines can be extended.
but periodic reviews of manual input and output procedures
will sometimes reveal human interface deficiencies. 7. EXAMPLE OF CBM ANALYSIS
6.3.7 Periodically Update CBM Business Case There are many ways and examples of the CBM
applications, in both hardware and software. These
As the life cycle of the system or equipment progresses,
examples range from simple (using a penny to check your
maintenance managers should revisit the business case to see
tire tread to determine when you need to change your tires)
if the factors validating a particular level of CBM
to more complex (using data analysis to determine when to
implementation are still applicable and provide quantified
update a software algorithm or perform radar maintenance).
justification for revised inputs to budget updates.
6.3.8 Continuously Update Resources Strategy & 7.1 Simple CBM Analysis Case
Integrated Budget It is arguable that most people either already employ or
Resource requirements to maintain and update CBM understand the application of the “tire tread” analysis using a
capabilities will change and new systems and equipment are standard U.S. penny. Rather than using the standard
fielded, maintenance plans are revised, new technologies are maintenance recommendation of “replace tires after ‘x’
developed, and reliability assessments are modified. It is number of miles,” most will watch the tire tread, and
still necessary to market the CBM strategy as stakeholders possibly utilize the “penny” maintenance analysis, to
and customers change to ensure management support. As determine when to replace their tires. That isn’t always the
systems and equipment reaches the end of their operational best way to prevent a tire failure while on the road. Car
life cycle, phasing out investment (financial and resource manufacturers are now adding in dash board warning lights
levels) is prudent. to address low tire pressure, indicating a possible impending
failure due to a foreign object, or other leak. This allows
6.3.9 Optimize Maintenance Strategies many to forgo checking tire pressure on a regular
New policies and procedures, operational experience, maintenance cycle, and, instead, rely on the “low tire
technology updates, mission and organization changes, pressure” warning light to tell them when air needs to be
funding availability, and other factors will necessitate added, or tires need to be checked for leaks and/or foreign
reassessment of a number of the initial approaches. From objects, like nails.
initial deployment to cessation of CBM strategies, This is a simple example of both CBM analysis with no
documenting lessons learned and looking for new ways to software, only using a penny, and CBM analysis with on-
improve CBM methods and adopt updated enabling board sensors that send signals to turn on dashboard warning
technologies will only improve the CBM process. lights. Although the tire pressure sensor is more
complicated to implement, and far more costly, the customer
Gillespie, Page 13
can easily realize its benefits. This, in turn, provides profit
benefit to the manufacturer.
These CBM techniques have been implemented through
the identification of unwanted safety and operational failures
– no one likes to have a flat tire that puts them on the side of
the road, in danger, and/or that prevents them from attending
a meeting or other function. The “low tire pressure”
warning light was implemented after new technologies were
examined, cost of implementation was studied, and CBM
functionality was demonstrated.
7.2 Complex CBM Analysis Case
A more complex case using statistical analysis to
determine when to perform radar maintenance does not lend
itself to on-board sensor technologies. For radars, some
tracking issues can occur with new operators or due to low Figure 9 – Example of Using AER to Establish the
elevations of the target or low return signal of a beacon. Location of a Target
Each radar may need to be studied using statistical analysis
to determine the optimal time to perform maintenance. elevation, and range. The arithmetic average is an estimate
Likewise, a control and display (C&D) tracking system may of the overall system error, or bias. The standard deviation
rely on the accurate estimate of radar performance of the residuals is an estimate of random variation around
parameters and acceptable errors. the bias. The standard deviation is termed the random error.
In this case, a more complex CBM analysis technique These biases and random errors are used as indicators of the
can be used to both determine predicted maintenance and radar’s performance. The individual random error values
update C&D tracking system algorithms using this same from each mission are combined into one number using the
predicted maintenance technique. Implementation of CBM Mean Root Mean Squared (MRMS) value shown in
is also more readily accepted due to the criticality of these Equation 1. Where n is the number of samples, ̅̅̅̅ 𝑥𝑚 is the
systems both for safety and operations. mean residual for sample m, and σm is the standard deviation
of residuals for sample m.
7.2.1 RADAR Operation
x
n
Radar is actually an abbreviation for RAdio Detection
And Ranging; however, we now use this term to describe
2
m m2
many different types of radars. In this case, we are not
talking about weather radars or police speed tracking radars;
MRMS m 1
(1)
rather we are talking about ground based radars to determine
n
the position of a vehicle on a specific trajectory. These large Radar maintenance can be performed via statistical
ground based radars can track in “skin” or “beacon” mode. process control utilizing the MRMS value for multiple
“Skin” mode is where the radar tracks only the shell of the samples.
vehicle, and “beacon” mode is where the radar relies on a 7.2.2 Control & Display Tracking System Operational
beacon located on the vehicle for a more accurate position. Algorithms
Most launch vehicles have a C-band transponder, or beacon,
to increase the range and accuracy of radar ground systems. Some of the more complex C&D tracking systems
The radar determines the position of the vehicle by provide the latest advances in hardware and software in an
measuring azimuth angle, elevation angle, and range, also integrated package. These systems are able to receive and
known as AER (Azimuth-Elevation-Range). The azimuth is process plot data from a multitude of radars and/or other
the horizontal angular distance from true north measured tracking systems like optical trackers and on-board
clockwise. Elevation is the number of degrees the beam is telemetry. Filtering of this source data is desirable in most
transmitted above the horizon, with 0 degrees being parallel cases. A good filtering algorithm can remove noise from the
to the horizon. The range is the distance from the radar site signals while retaining the useful information [9]. For this
to the vehicle. Figure 9 shows the example taken from example, we will use the filtering algorithm known as the
reference [8] Kalman filter.
If one knows the intended trajectory of the vehicle or The Kalman filter provides a feedback control to the
object, the radar’s performance can be evaluated by tracking system. The filter estimates the track at some time,
determining the difference between the radar track data and and then obtains feedback in the form of measurements,
the intended trajectory of the vehicle, which can be termed which are usually noisy. The equations for the Kalman filter
as “residuals.” The arithmetic average and standard can be divided into time update and measurement update
deviation of these residuals are computed for the azimuth, equations. The time update equations are responsible for
Gillespie, Page 14
projecting forward in time the current state and error found in the lot.
covariance estates to obtain the a priori estimates for the The radar measurements are then processed to form a
next time step. These time update equations are considered state estimate with time, positions, velocities, and a
the predictor equations. The measurement update equations covariance estimate. Of those radar state estimates, a C&D
are responsible for the feedback, specifically by tracking system will either select the one that is considered
incorporating a new measurement into the a priori estimate “most accurate” using the Kalman gain matrix. The radar
to obtain an improved a posteriori estimate. These scoring process starts with the source covariance estimate,
measurement update equations are considered the corrector which accounts for some known source errors, like noise.
equations. The final algorithm then resembles a predictor- Rather than measuring the source noise in recent
corrector algorithm for solving numerical problems. One measurements, for radar, the variance value contained in a
can review the reference [10] for specific information configuration file is used. This avoids the possibility of the
regarding the Kalman filter equations. system overreacting to short glitches; however, there is the
For C&D tracking systems, radar, or sensor, data is potential to miss truly bad, but non-characteristic,
processes upon receipt. All data needs range checking, wild performance. Other factors are used to adjust this initial
point editing, and time shifting for epoch agreement. A variance value, including known errors, such as low
more complicated C&D system may have more than one elevation and track mode, and to account for known
Kalman filter, one for each radar. As described previously, operational issues.
Kalman filters are process noise filters, not fading memory This algorithm is described in detail to show how
filters. The basic operation of a C&D Kalman filter is important the accurate estimate of radar AER errors can be
iterative. Data from the previous cycle is used with data on the C&D tracking system. These estimates should be
from the current cycle to produce a new estimate of the state updated based on radar performance; however, updating
vector and its statistics. The state vector is estimated using these estimates too much can allow erroneous radar track
the difference between the new measurement and the data to be displayed. This is why radar maintenance must be
hypothetical measurement perfect radar would return based performed as a result of the statistical analysis, then error
on the predicted value of the state vector. The state vector values verified prior to updating error estimates in C&D
covariance is estimated by adjusting the predicted value of tracking system algorithms.
the covariance matrix using the Kalman gain matrix.
7.2.3 Statistical Process Control
The Kalman gain matrix determines how much to
confidence is in the new measurement and how much Statistical Process Control (SPC) is the collection of
confidence is in the prior estimate. The confidence in a new methods for recognizing special causes of variance from the
measurement is based on the knowledge of the particular target value, bringing a process into a state of control, and
radar’s historical behavior using the baseline variance reducing variation about the target value. SPC is an area of
vectors containing AER values for each sensor. For many statistics focusing on the endeavor of constantly improving
C&D tracking systems, the base variance values are the quality of a measurement. The underlying concept of
expected to be those for beacon track at elevations above the SPC is based on a comparison of data from the present with
horizon, or at least 10 degrees. The algorithm then inflates data from the past, or historical data. In SPC, we take a
the base variance values by accounting for residual snapshot of how the process typically performs or build a
refraction in the elevation and range. These base radar model of how we think the process will perform and
variance values are expected to be derived from statistical calculate control limits for the expected measurements of the
analysis or from a specific number of samples using process output. Then we collect data from the process and
Equation (1). compare the data to the control limits. The majority of the
Before the C&D tracking system uses the radar measurements should fall within the control limits.
measurement, the algorithm checks to see is the Measurements that fall outside the control limits should be
measurement agrees with the predicted measurement. In examined to determine if they belong in the same population
doing this, the new measurement must be equal to a certain as the initial snapshot or model. In other words, we use
multiple of the combination of the base radar variance values historical data to compute the initial control limits, and then
and the covariance and the conversion matrices – or those compare the current data against these initial limits to
values developed from the Kalman gain matrix. If the determine if the process is in-control or out-of-control [11].
measurement does not agree with the predicted The main tool of SPC is control charts. Control charts
measurement, the measurement is then “smoothed” out or provide a graphical representation of the process allowing an
edited out. If the new measurements do not agree with the individual with or without the knowledge of statics to
predicted measurements after a configurable time, the immediately understand if the process is in-control or out-of-
measurement is thrown out and the radar data is considered control. Control charts are proven to effectively detect
erroneous and not used until a certain number of consecutive anomalous measurements and help keep the process in-
new measurements agree with the predicted measurements. control. The diagnostic information of control charts is
This is much like a “sampling” plan where inspections are significant as it allows for changes in the process by
tightened or relaxed depending on the number of defects experienced operators or engineers [12].
Gillespie, Page 15
The general form of the control chart, called a Shewhart multiple of the standard deviation. Usually, this multiple is
control chart, contains the measurement value versus the three, and, therefore, the limits are called the 3σ limits. This
sample number or versus time. The chart contains a center term is used whether the standard deviation is the universe
line that represents the mean value for the in-control process or population parameter, or some estimate thereof [11].
and two other horizontal lines, called the upper control limit Since a control chart compares the current performance
(UCL) and lower control limit (LCL). The UCL and LCL of the process characteristic to the past performance of this
are chosen so that almost all of the data points will fall characteristic, changing the control limits too frequently
within these limits as long as the process remains in-control could negate any usefulness. Control limits are only
[13]. Fig. 10 shows a typical Shewhart control chart. changed if there are valid and compelling reasons to do so,
including, but not limited to, at least 30 more additional data
points to be included in the calculation of the control limit,
or an out-of control signal [11].
7.2.4 Exponentially Weighted Moving Average
The Exponentially Weighted Moving Average
(EWMA) is a statistic for monitoring the process that
averages the data in such a way to give less weight to data
further removed in time. The EWMA technique is an
extension of the Shewhart technique. For the Shewhart
control technique, the decision regarding the state of control,
i.e., in-control or out-of-control, of the process at any time
depends solely on the most recent measurement from the
process and the degree of ‘trueness’ of the estimates of the
control limits from historical data. For the EWMA control
technique, the decision depends on the EWMA statistic,
which is the exponentially weighted moving average of all
historical data, including the most recent measurement. In
other words, the EWMA control technique allows trend
monitoring of the mean of the historical data, whereas, the
Shewhart control technique relies solely on the most recent
Figure 10 – Typical Shewhart Control Chart measurement to determine a shift in the mean of the
historical data. By the choice of the weighting factor, w, the
Many control charts set the UCL and LCL equal to the
EWMA control technique can be made sensitive to a small
‘0.001 probability limits,’ which means if the chance causes
or gradual drift in the process [14]. The Shewhart control
alone were influencing the chart, the probability of a point
technique is only reactive when the last data point is outside
falling above the UCL or below the LCL would be on out of
a control limit.
a thousand. How we calculate these limits determines how
The EWMA is arguably the most commonly used noise
likely we are to encounter a point outside of the control
filter algorithm in the process industry, however it is known
limits when there is no cause for the variability. Since one
by the name discrete “first-order, low-pass filter” [15]. The
out of a thousand is a very small risk, the ‘0.001 probability
EWMA statistics is calculated using Equation (4).
limits’ may be used to ensure that an investigation is
warranted to determine an assignable cause when a point EWMAt wYt (1 w) EWMAt 1 (4)
falls outside the control limits [11]. The UCL and LCL
Where n is the number of observations to be monitored, t is
pictured in the Figure 10 are the 3σ limits. The 3σ limits in
1 to n, EWMA0 is the mean of the historical data, Yt is the
the figure are calculated in Equations (2) and (3).
observation at time t, including EWMA0, and w is a number
UCL 3 (2) between 0 and 1, inclusive of 1, that determines the depth of
memory of the EWMA.
LCL 3 (3)
The weighting factor, w, determines the rate at which
‘older’ data enter into the calculation of the EWMA statistic.
Where, µ is the mean and σ is the standard deviation of A value of w = 1 implies that only the most recent
the measurements. From normal tables we can verify that measurement influences the EWMA (equivalent to Shewhart
the 3σ in one direction is 0.00135. Therefore, for normal Control Chart). A large value of w gives more weight to
distributions, the ‘0.001 probability limits’ are very close to recent data and less weight to older data; likewise, a small
the 3σ limits. In the United States, whether the value of w gives more weight to older data. The value of w
measurements are normally distributed or not, it is an is usually set between 0.2 and 0.3, although this choice can
acceptable practice to bases the control limits upon a be arbitrary. However, Lucas and Saccucci [14] provide
tables that help one choose a proper value of w. For this
Gillespie, Page 16
tutorial, the value of w was chosen to be 0.3. can easily observe the out-of-control signal occurs faster in
The chosen value, w = 0.3, gives 30% weight to the Figure 11 than it does in Figure 10.
most recent measurement and 70% to the historical data. There are general rules for detecting an out-of-control
This choice of w = 0.3 also influences the Average Run signal. These rules were originally developed by the
Length (ARL) of the control charts. The ARL indicates how Western Electric Company (WECO), and are referred to as
long, on the average, consecutive control chart points can be the WECO Rules. These rules are as follows:
plotted before a point will be detected beyond the control 1. Any point above the +3σ , or UCL, line,
limits. By choosing a value of w = 0.3, a 2σ shift in the 2. Two out of the most recent three points above the
process mean can be detected with a minimum ARL of 4.29, +2σ line but below the +3σ line,
meaning we shouldn’t have to recalculate the UCL more 3. Four out of the most recent five points above the
often than once every four to five points plotted [14]. +1σ line but below the +2σ line,
The estimated variance of the EWMA statistic is 4. Eight consecutive most recent points above the
approximated in Equation (5). mean (or center) line but below the +1σ line,
w 5. Eight consecutive most recent points below the
2
sewma ( )s 2 (5) mean (or center) line but above the -1σ line,
2w 6. Four out of the most recent five points below the -
The center line for the control chart is the mean value of 1σ line but above the -2σ line,
the historical data, or EWMA0. Therefore, the control limits 7. Two out of the most recent three points below the -
in the EWMA control chart are shown in Equations (6) and 2σ line but above the -3σ line,
(7). 8. Any point below the -3σ , or LCL, line,
9. Any six consecutive data points trending up,
UCLEWMA EWMA0 3sewma (6)
10. Any six consecutive data points trending down, or
11. 14 consecutive data points alternating up and down,
LCLEWMA EWMA0 3sewma (7)
i.e., in a saw-tooth pattern
The WECO rules are based on probability. For a
Figure 11 shows the same data from Figure 10, but
normal distribution, the probability of plotting a point
plotted using the EWMA statistic.
outside of the ±3σ is 0.3%. Therefore, if a point outside the
control limits is observed, we can conclude that the process
has shifted and is unstable. Similarly, of the listed rules, the
probability that numbers 2 and 3 occur is 0.3%.
The WECO rules were designed to be used with any
control chart technique. Additionally, when using the
EWMA control chart technique, if the difference between
the current and the previous EWMA statistic is greater than
1σ, this is to be considered an ‘out-of-control’ signal. A
greater than 1σ shift indicates a significant shift in the
computed moving average [11]. Even with the rules listed
above, one should be cautious in declaring a signal ‘out-of-
control’ in terms of the RMS random radar errors because
these errors are, in fact, random.
7.2.5 Implementing the CBM Analysis Technique for
Radar and C&D Tracking System Predictive
Maintenance
Before implementing this SPC technique to perform
predictive maintenance, both on the radar hardware and
C&D tracking system software, a baseline of nominal
Figure 11 – Typical EWMA Control Chart
performance must be developed. When using SPC to
One can observe that second to last EWMA value determine this baseline, to develop variance values and trend
plotted above the UCLEWMA in Figure 11. In Figure 10, all random radar errors for predictive maintenance, a one-sided
the measurement values are plotted below the UCL. The control chart should be used, which only contains the UCL,
‘out-of-control’ signal in the EWMA chart (EWMA value or +3σ value, 2σ value, +1σ value, and mean value. This is
plotted above the UCL) indicates that an investigation because when describing random radar errors, there is not
should begin to identify the cause. Although the plotting of usually a minimum amount of random error, or noise that is
one value outside of the control limits is not the sole desirable. To bind these errors by a minimum amount is not
indicator of an out-of-control signal in the SPC charts, one necessary, therefore, only a one-sided control chart is
required. Additionally, using a one-sided control chart
Gillespie, Page 17
limits the amount of rules that are used to determine an ‘out- historical radar performance and cannot be used to predict
of-control’ signal. when maintenance should be performed to prevent a failure.
Ideally, more than 100 data points should be used to The C&D tracking system radar variance values are
calculate an UCL and determine if the statistical process is used in part to determine whether a new measurement is
in-control. However, many times, these 100 data points can added to the Kalman filter, edited before adding it to the
be costly to obtain if not supported by normal operations. Kalman filter, or excluded from the Kalman filter. The same
The choice of the 30 most recent random error values can be variance value vectors also influence the radar’s ranking
used to account for the 25 data points desired to have a when compared to other radars. The most accurate variance
statistical process, along with an additional five data points values will allow for an optimal amount of Kalman filter
desired to determine if the process is in-control. Once a editing. In other words, the radar variance values selected
control chart is established, in general, all future data points must not be too large so that too much error during radar
that are not ‘outliers,’ or those points less than (μ + 4σ), will track is allowed, and must not be too small so that too little
contribute to the calculation of the variance values and error during radar track is allowed. The use of these values
performance trending. to predict radar maintenance and predict C&D tracking
The radar random errors one is trying to trend are, in system software maintenance optimizes the radar Kalman
fact, random. By using the EWMA statistic, trends in the filters by not allowing a potential radar instrumentation
mean value of the random errors are easily observed and problem to be masked with the radar variance values
analysis becomes more intuitive than the simple trending of
7.2.6 Demonstrating CBM Functionality for Radar and
the random errors. An ‘out-of-control’ signal is also C&D Tracking System Predictive Maintenance
typically observed earlier in the EWMA control chart than in
the general trending of errors. However, before accepting Demonstrating the benefit of this analysis to perform
the ‘out-of-control’ signal as an indication of anomalous predictive maintenance can also be more complicated, and
instrumentation performance, one must be sure that the error usually is a result of the review of data after the unwanted
plotted is not an erroneous indication due to a ‘wild-point,’ failure has occurred. A failure of a radar or C&D tracking
or ‘outlier.’ This is difficult given the fact that the points system during a mission can be costly, either due to a scrub
that are being plotted, or trended, are RMS random errors. If or rescheduling of the mission, or due to data loss. There
two ‘wild-points’ are consecutively received, then the points can also be safety impacts to the public or mission
are no longer ‘outliers,’ rather indications of significant personnel. These failure impacts make management more
degradation of instrumentation performance. receptive to utilizing this CBM technique. Additionally, the
The same variance values that are used by the C&D maintenance technique does not create an added cost due to
tracking system in the Kalman gain equations can also be on-board sensor requirements or software development.
used to provide a baseline for radar performance. This will This CBM technique is merely an application of sound SPC
allow for indications of non-nominal radar performance and base lining system performance.
while allowing for optimal Kalman filter performance. To
provide optimal variance values, statistical analysis should
be performed on the RMS random errors calculated post-
mission for each radar. In terms of a SPC control chart, the
Kalman Filter radar variance value should be equal to the
UCL calculated from the post-mission radar RMS random
errors to provide for the optimal amount of Kalman filter
initializations. These same variance values can also be used
to provide a baseline for the evaluation of future radar
performance, so that an unnecessary amount of Kalman filter
re-initializations will most likely be an indication of
degraded radar performance, not degraded Kalman filter
performance.
The idea of defining of a radar’s historical performance
using one value from one MRMS is inherently flawed. The
MRMS is used to provide a ‘snapshot’ of the
instrumentation systems’ performance during a specific
period. In that period, many unique factors can affect the Figure 12 – SPC Chart for Radar Random Range Errors
radar MRMS random errors during track, including the
number of missions supported (from none to many) and Figure 12 shows the root mean squared (RMS) random
whether or not poor radar performance had indicated that range errors for a particular radar. As can be seen on the
maintenance action needed to be taken during the quarter. chart, an “out-of-control” signal was detected at a specific
Because of these factors, using the individual MRMS point in time. On the subsequent mission, the radar suffered
random error value from does not accurately represent
Gillespie, Page 18
a failure that caused it to lose track and not provide its data to prevent unnecessary maintenance activities and keep costs
to the C&D tracking system. As discussed, though, this down.
analysis was not completed until after the failure, therefore Table 1 – Comparison of Filter Performance Utilizing CBM
maintenance was not performed on the radar to prevent the Technique
failure. Had this CBM maintenance technique been
implemented, the failure may have been prevented, as an Operation 1
“out-of-control” signal would have been detected, causing Radar
Periods of Filter Periods of Filter
subsequent investigations as to the cause and maintenance “Coast” “Off-Track”
performed on the radar. Before After Before After
For the C&D tracking system, the functionality of the 1 7 8 6 6
CBM technique can be demonstrated by comparing the 2 6 6 7 7
performance of the Kalman filter before and after making 3 0 0 0 0
changes to the radar variance values in the software for the 4 0 0 0 0
same operations. This is more complex than demonstrating 5 2 1 1 1
CBM functionality post failure for radar maintenance, as this 6 24 7 17 1
demonstration would require additional testing resources to Operation 2
“play-back” the operation through the software algorithm Radar
Before After Before After
using the updated radar variance values. 1 3 4 2 1
For the C&D tracking system, when the Kalman filter 2 0 0 0 0
“coasts” the algorithm are not utilizing the radar tracking 3 9 3 5 2
data, rather the algorithm utilizes the best estimate of what 5 6 1 3 1
the radar tracking data should have been (using the radar 6 24 0 21 0
variance values and accounting for other tracking
7 12 11 5 2
performance issues). This technique makes for a smoother
8 16 17 3 2
tracking and prevents erroneous data from corrupting the
Operation 3
filter. If the C&D tracking system does not receive radar Radar
Before After Before After
data that it considers valid in a certain number of
consecutive occurrences, the Kalman filter then indicates 1 5 6 5 5
that the radar data is “off-track” and does not use the data to 2 0 0 0 0
determine the location of the target. Using those “coast” and 3 1 0 1 1
“off-track” indications, one can demonstrate the value of the 4 0 0 0 0
CBM technique in maintaining and optimizing the C&D 6 1 0 0 0
tracking system software. 9 8 3 8 3
Table 1 shows the comparison of the filter performance
utilizing the same CBM technique to update the C&D 8. CBM+ AND PBL CONTRACTS
tracking system software that is utilized to determine when
radar maintenance should be performed. It can be seen that, It has been discussed throughout this tutorial that CBM
in some cases, the filter was editing, or smoothing, data that strategies and activities can be costly, in resources, processes
once the algorithm was updated, the filter no longer development, and technology acquisition. However, if
identified that data as being “off-track” or not in agreement implemented correctly and on the appropriate systems and
with the predicted value. This presents a significant issue equipment, CBM can provide a significant ROI by
for operations and safety. The C&D tracking system improving performance, reliability, and availability.
Kalman filter is only as accurate as its radar error modeling. 8.1 CBM Return on Investment (ROI)
For the tracking system, its estimate of measurement
“goodness” in the Kalman equations is based on knowledge In some cases, savings of 93% have been realized when
of how a particular radar has performed in the past. The implementing CBM strategies even in fielded systems post-
algorithm first assumes that the error is normally-distributed, failure [16]. The University of South Carolina (USC), the
zero-mean noise. The baseline standard deviations are CBM team there “combines research and implementation
essentially table look up values, which are best derived from solutions to support the U.S. Army Aviation program
statistical analysis. through historical data analysis, component testing, and
In demonstrating this functionality of the single CBM advanced condition indicator development.
technique to perform radar and C&D tracking system Utilizing these CBM practices has led to extensions of
software maintenance, one also demonstrates the need to component life limits and multi-million dollar military cost
continuously revise and update the CBM implementation avoidance [17].” The greatest ROI to date has been at a rate
plan. As radars are maintained as a result of the CBM of $2.8 million per year due to key discoveries at the USC
technique, performance will change; therefore, continuously CBM research center [18].
monitoring the plan and revisiting the strategy is necessary Supermarket industry executives and their management
Gillespie, Page 19
teams are also observing 18-40% maintenance savings using performance-based arrangements that deliver user
CBM, as compared to PM or RTF, strategies for their
refrigeration systems [19]. HUMS
Prognostics
These are significant ROI numbers showing cost
• Technologies • PBL
savings in the millions of dollars. Implementation for new • CBM+
RCM • Processes
acquisition and fielded DoD systems can provide valuable SIM • Enablers • Continuous Process
benefits to the overall bottom line. Improvement (CPI) TLCSM
• Cause & Effect
IETMs
PMAs Predictive Modeling
and Simulation (M&S)
8.2 CBM Return on Investment (ROI)
In some cases, savings of 93% have been realized when Figure 13. CBM Input into TLCSM
implementing CBM strategies even in fielded systems post-
requirements an incentivize Product Support Providers
failure [16]. The University of South Carolina (USC), the
(PSP) to reduce costs through innovation. These
CBM team there “combines research and implementation
arrangements are contracts with industry or intra-
solutions to support the U.S. Army Aviation program
governmental agreements [20]. PBL contracts are tied to
through historical data analysis, component testing, and
user (or warfighter) outcomes and integrate the various
advanced condition indicator development. Utilizing these
supply chain product support activities, such as supply
CBM practices has led to extensions of component life limits
support, sustaining engineering, maintenance, etc, with
and multi-million dollar military cost avoidance [17].” The
incentives and metrics. PBL focuses on combining the best
greatest ROI to date has been at a rate of $2.8 million per
practices of both Government and industry. It has been a
year due to key discoveries at the USC CBM research center
DoD policy and a strategic priority to increase the use of
[18].
performance-based arrangements to deliver product support
Supermarket industry executives and their management
solutions that satisfy Warfighter requirements.
teams are also observing 18-40% maintenance savings using
PBL is a contractor based support strategy in which
CBM, as compared to PM or RTF, strategies for their
performance outcomes are specified and the PSP provides
refrigeration systems [19].
services to achieve that outcome. Incentives are provided
These are significant ROI numbers showing cost
for achieving outcomes above the baseline. The single
savings in the millions of dollars. Implementation for new
Product Support Provider (PSP) will ensure operation
acquisition and fielded DoD systems can provide valuable
readiness for a fixed price. This is seen as a way to reward
benefits to the overall bottom line.
PSPs for implementing cost saving strategies using new
8.3 CBM Relationship with DoD Efforts technologies, which will drive a contractor to a higher level
of support and save the government money. The risk of
The DoD has many key initiatives that impact or
failing to meet performance outcomes is shifted to the PSP.
complement the CBM strategies, specially, Total Life Cycle
In a survey documented and released in March of 2008
System Management (TLCSM), RCM, PBL, Systems
by Aviation Week for Systems Applications and Products
Engineering, Information Technology (IT) Portfolio
(SAP), it was found that 30% of the survey population,
Management, Focused Logistics, and Sense and Respond
including those in DoD and industry, reported that less than
Logistics.
10% of their contracts were based on PBL agreements.
Under TLCSM, The DoD has made efforts to conduct
However, it also showed that over 35% of the survey
support operations in a more effective and fiscally
respondents were executing 10-40% of their contracts with
responsible manner. In TLCSM, Program Managers (PMs)
PBL acquisition strategies. The survey population also
are responsible for the overall management of system life-
indicated that in the future, they anticipated that over 33% of
cycle support, and CBM contributes to a number of process
their contracts would be PBL acquisitions [21]. This is a
improvement initiatives to attain system effectiveness and
significant piece of the contracting environment, and DoD
affordability life-cycle support objectives. CBM capabilities
initiatives are beginning to push contracting further into the
that feed TLCSM include technologies, such as Health and
direction of PBL acquisitions.
Usage Management System (HUMS) and prognostics,
As with CBM initiatives, the DoD and industry have
processes, such as RCM and SIM, and enablers, such as
acknowledged successes in cost avoidance and improved
IETMs and PMAs. This relationship is shown in Figure 13.
reliability and availability when PBL contracting is
Many support strategies are available for TLCSM, but
implemented. The first PBL contract in a program or on a
the preferred strategy is PBL, as stated in the DoD TLCSM
system is often a cost-type contract to collect sufficient data
Plan of Action and Milestones document dated January 6,
to understand the risk. During contract execution the
2003.
government collects cost data for negotiating future
8.4 Performance Based Logistics (PBL) Contracting contracts or option years. Follow-on contracts tend to be
Firm-Fixed Price (FFP) with incentives, as discussed
PBL is synonymous with performance-based life cycle
previously, to provide optimal support [22].
product support, where outcomes are acquired through
Gillespie, Page 20
A typical PBL contract will be FFP with yearly contract system. However, again, the incentive for future cost
line items (CLINs) and FFP sub-CLINs for support, repairs, savings is much less for a PSP than it is for the government.
new designs/builds, diminishing manufacturing sources, and Additionally, cost savings of 93% have been realized for
cooperative logistics supply support arrangements [22]. fielded systems utilizing CBM strategies. However, for a
Typical award fees are 3% with 50% of that award fee fielded system operating within a PBL contracting
given for performance above the standard, reducing future environment, this will typically be FFP contract, with a
cost is one of eight criteria where performance may be small fraction of the award fee going towards incentivizing
exceeded. For example, for a contract worth $10 million, CBM strategies for cost savings.
the typical award for performance above the standard is For PSPs striving to reduce the cost associated with
$150,000. Only those performance standards above 90% are maintaining the fleet, there exist CBM algorithms that can
subject to awards, and those below 80%, will actually have a schedule maintenance tasks in such a way as to create an
“negative fee,” which decreases the profit for the contractor. attractive profit margin within a FFP contract. In some
The remaining 50% of the award fee is for configuration cases, maintenance costs were reduced by up to 30%. This
management and quality. In addition to award fee allows the realization of the significant benefits associated
decrements, price reduction can be more than 10% for with the abilities to schedule maintenance based on failure
failure to meet metrics [22]. risk through CBM solutions. However, from a PBL
There is little incentive offered for the PSP to reduce contracting standpoint, the added performance assurance and
future costs to the customer. On FFP contracts, any savings maintenance cost reduction achieved by the PSP is best
go to the contractor. So that if future cost savings rewarded for taking this CBM initiative through award fee
techniques are applied, such as CBM, this is only effective incentives such as contract extension or larger compensation
for the PSP on projects that can absorb the initial capital [23].
investment, and non-recurring cost, in the contract period In order for the government to achieve the PBL
while still providing an acceptable ROI [22]. contracting goals, CBM strategies should be incentivized in
a way to allow for ROI for PSPs for initial capital
8.5 CBM and PBL
investment. This would benefit the government significantly
CBM can help focus the maintenance processes to by decreasing life-cycle costs and improving competition for
maximize equipment readiness with optimum resource PBL contracts on already fielded systems and new designs.
allocation. This fully complements the PBL concept. CBM CBM strategies have been proven to achieve significant
can become essential in achieving maintenance cost avoidance for the life of programs and systems in both
performance-based objectives, such as MA, MR, OC, and the DoD and industry. Even if the government shares the
MDT, which are described in section 4.3. “risk” and cost of implementing CBM strategies, the
However, as discussed, initial investments in CBM government will achieve more significant cost savings than
strategies can be costly, and when achieving and exceeding realized before in PBL contracting. There are significant
performance requirements such as future cost savings (OC) gains to be had both in industry, in terms of profitability, and
is only one-eighth of one-half of a 3% award fee ($18,750 in in DoD in terms of life cycle cost reduction, improved
the $10 million contract example in section 7.3), PSPs technologies, and greater competition for PBL contracts,
realize little benefit, and for the large part, negative cost thus driving government cost down for PBL contracts.
benefit, for applying and integrating future cost savings
9. CONCLUSIONS
techniques like CBM.
The National Contract Management Association It is the goal of this tutorial that the audience has gained
(NCMA) is starting to take notice of this, and also suggested the knowledge of CBM, the strategies and tools available for
ways to incentivize PSPs to propose and incorporate cost implementing CBM, and the understanding of the
savings on future contracts [22]. Federal Acquisition maintenance, failure prevention, and cost savings realized
Regulation (FAR) Part 48 – Value Engineering - provisions from CBM implementation. It is also the goal to have the
offer a method for government and contractor sharing of audience understand the government PBL contracting
savings. Value Engineering has been used for 50 years on environment and the benefits that CBM strategies can
hardware changes, but there are few for services, such as provide to PBL. However, the lack of incentives for future
CBM [22]. Value Engineering cost savings can be harder to life cycle cost savings in PBL contracting, make CBM
calculate for CBM, as the estimations for total life-cycle cost strategies and implementation nearly impossible and may be
savings may be over-stated or under-stated. the reason that more CBM strategies are not being
Although CBM can add much needed value to the intent implemented in PBL. The author hopes this goal has been
and goals of PBL contracting, in many cases, CBM is not achieved.
implemented as it tends to be cost prohibitive in a FFP The author would like to acknowledge all of the
contract. Initial PBL contracts tend to be cost-type, and in Logistics and Supply Chain professions at SAIC for their
these cases, CBM can be implemented with shared contributions to and support of this tutorial, notably, Dale
investment by both government and the PSP for significant Winstead, Brian Koziol, and Ricky Laws.
cost savings outcomes for the life of the program and
Gillespie, Page 21
10. REFERENCES 18. AH-64 TRGB Output Seal Field Repair, CAD Validated
PIA #7, “CWR from Significant USC CBM Lab
1. The Plant Maintenance Resource Center, “Introduction
Breakthrough & Maintenance Improvement”,
to Reliability-centered Maintenance”, http://www.plant-
September 3, 2013.
maintenance.com/RCM-intro.shtml, August 3, 2009.
19. Emerson Climate Technologies, “Condition Based
2. Cadick, John, Traugott, Gabrielle, “Condition Based
Maintenance: Putting Predictive Maintenance
Maintenance”, A White Paper Review of CBM Analysis
Algorithms to Work to Optimize Asset Maintenance
Techniques, Cadick Corporation Tech Bulletin TB-017,
and Performance in Supermarket Refrigeration
2009.
Systems”, October 2013.
3. Wiseman, Murray, “A History of CBM (Condition
20. Assistant Secretary of Defense (ASD) for Logistics and
Based Maintenance)”, Optimal Maintenance Decisions,
Materials Readiness (L&MR), “Performance-Based
Inc. 2006.
Logistics Comprehensive Guidance”, Memorandum,
4. Lawler, Jason, Felker, Douglas, “Condition Based
November 22, 2013.
Maintenance Gap”, Reliability Web.Com,
21. Aviation Week PBL Survey Conducted for SAP
http://www.reliabilityweb.com/art07/cbm_gap_analysis.
Americas, “Study Reveals Performance Based Logistics
htm.
Contract Activity Continues to Grow, Despite Industry
5. Rausand, M, Vatn, J. “Reliability Centered
Challenges”, March 2008.
Maintenance”, Risk and Reliability in Marine
22. Vickers, J, Mandelbaum, J. “Incentives in Service
Technology, Balkema, Holland, 19A98.
Contracts”, National Contract Management Association
6. Cadick, John. “Condition Based Maintenance…How to
(NCMA) Performance-Based Acquisition Community
get started…”, The Cadick Corporation. 1996.
Virtual Conference, March 31, 2011.
7. Gillespie, A, Monaghan, M, Chen, Y. “Comparison
23. Reimann, J, Kacprzynski, G, Cabral, D, and Marini, R,
Modeling of System Reliability for Future NASA
“Using Condition Based Maintenance to Improve the
Projects”, Reliability, Availability, and Maintainability
Profitability of Performance Based Logistics Contracts,”
Symposium (RAMS) 2012 Conference, January 2012.
Annual Conference on Prognostics and Health
8. DeSoto, Clinton, “Radar Techniques – I – Primer
Management Society, 2009.
Principles”, QST, April 1945.
9. Simon, Dan, “Kalman Filtering”, Embedded Systems 10.1 Industry Standards
Programming, June 2001.
1. ISO 17359:2011, “Condition Monitoring and
10. Welch, G. and Bishop, G. “An Introduction to the
Diagnostics of Machines – General Guidelines.”
Kalman Filter”, UNC-Chapel Hill, TR 95-041, July 24,
2. ISO 13372:2012, “Condition Monitoring and
2006.
Diagnostics of Machines – Vocabulary.”
11. NIST/SEMATECH e-Handbook of Statistical Methods,
3. ISO 13374-1:2003, “Condition Monitoring and
http://www.itl.nist.gov/div898/handbook/, April 2012.
Diagnostics of Machines – Data processing,
12. “Statistical Process Control”,
communication, and presentation – Part 1: General
https://www.moresteam.com/toolbox/statistical-process-
Guidelines.”
control-spc.cfm, September 2014.
4. ISO 13374-2:2007, “Condition Monitoring and
13. “What is Quality Control and Quality Control Charts”,
Diagnostics of Machines – Data processing,
Statistics Textbook,
communication, and presentation – Part 2: Data
http://www.statsoft.com/Textbook/Quality-Control-
Processing.”
Charts, September 2014.
5. ISO 13374-3:2012, “Condition Monitoring and
14. Lucas, J., Saccucci, M., Exponentially Weighted
Diagnostics of Machines – Data processing,
Moving Average control Schemes: Properties and
communication, and presentation – Part 3:
Enhancements, Technometrics, Vol. 32, No. 1 (February
Communication.”
1990), pp. 1-12.
6. ISO 13379:2012, “Condition Monitoring and
15. Tham, M., “Dealing with Measurement Noise,
Diagnostics of Machines – Data interpretation and
Exponentially Weighted Moving Average Filter”,
diagnostics techniques – Part 1: General Guidelines.”
http://lorien.ncl.ac.uk/ming/filter/filewma.htm,
7. SAE JA1011, “Evaluation Criteria for Reliability-
Chemical Engineering and Advance Materials,
Centered Maintenance (RCM) Processes referencing
University of Newcastle Upon Tyne, 1996 – 1998.
Message Specification”, August 2009.
16. Out of the Blue, Marine Insurance Newsletter by JLT
8. SAE JA1012, “A Guide to the Reliability-Centered
Specialty Limited, “Condition-based maintenance – a
Maintenance (RCM) Standard”, August 2011.
smart investment?”, February 2014,
9. MIMOSA/OSA-CBMv3.3.1, “Open System
http://www.jltgroup.com.
Architecture for Condition-Based Maintenance V3.3.1
17. University of South Carolina (USC), Condition Based
Product Specification”.
Maintenance (CBM) Team website:
http://cbm.me.sc.edu
Gillespie, Page 22
10.2 DoD Standards, Guidebooks, Instructions, and
Directives
1. “Condition Based Maintenance Plus, DoD Guidebook”,
May 2008.
2. “Performance Based Logistics Guidebook: A Guide to
Developing Performance Based Arrangements”, DoD,
2014.
3. DoD Instruction 4151.22, “Condition Based
Maintenance Plus for Material Maintenance”, December
2, 2007.
4. DoD Directive 4151.18, “Maintenance of Military
Material”, March 31, 2004.
5. DoD Instruction 5000.2, “Operation of the DoD
Acquisition Process”, May 2003.
6. Assistant Secretary of the Army (ALT) Memorandum,
“Condition Based Maintenance Plus -CBM+”, 20
March 2008.
7. Army Regulation 750-01, “Army Materiel Maintenance
Policy”, 20 September 2007.
8. Chief of Naval Operations (OPNAV) Instruction
4790.16, “Condition Based Maintenance (CBM)
Policy,” 17 December 2007.
9. Assistant Secretary of the Navy (Research,
Development & Acquisition) Memorandum, “Condition
Based Maintenance Plus Policy,” January 27, 2003.
10. Air Force Instruction 63-107, “Integrated Product
Support Planning and Assessment,” November 2004.
11. Commandant of the Marine Corps, Headquarters
Marine Corps Order 4000.57, “Marine Corps Total Life
Cycle Management,” September 2005.
Gillespie, Page 23