Professional Documents
Culture Documents
SOA Testing
SOA Testing
Service providers: A service provider is a component or New applications are being built using the five tier
set of components that execute a business function in architecture to enable them with SOA and EDA.
a stateless fashion, accepting predefined inputs and
outputs. Composite Applications
Service consumers: A service consumer is a set of Enterprises have an inventory of applications (SOA
components interested in using one or more of the services applications and non-SOA applications) to support their
provided by service providers. business processes. Extending the business processes to
collaborate with their value chain partners (suppliers and
Service repository: A service repository contains the customers) and other stakeholders (investors, employees
descriptions of the services. Service providers register their and regulatory bodies) to achieve real time agility and
services in this repository and service consumers access regulatory compliance is resulting in an environment
the repository to discover the services being provided.
AppLabs.com
App_WhitePaper_Approach_to_SOA_1v04 Page © 2007 AppLabs
of composite applications within an enterprise. Testing Ñ Backward compatibility;
SOA applications encompasses testing the functionality,
Ñ Interoperability;
interoperability, security, performance and other non-
functional aspects of the composite applications. Ñ Compliance;
Ñ Security.
Benefits, Challenges and Risks
Understanding business objectives, testing challenges
and analysis of testing risks is important to determine an
Business Benefits
adequate and accurate test approach.
Services as an architectural thinking has been around for
a while, with the introduction of CORBA???, COM/DCOM,
and RMI. Due to the inherent limitations of the technology Testing Approach
with respect to cross platform integration, the concepts
A successful test approach is one which ensures the
were not widely adopted. With the evolution of internet
business benefits of the software solution are achieved.
and its associated technologies (XML, SOAP, UDDI etc) the
This section highlights the testing challenges faced in the
implementation of SOA (methodology) has gained impetus
following two scenarios:
with web services (execution strategy). Major business
objectives envisioned with SOA applications are as follows: Ñ Existing applications are modified (service enabled)
so that application to application communication is
Ñ Reuse IT assets;
made possible; ISVs develop and release service packs
Ñ Integrate heterogeneous systems; or feature packs to make their applications service
oriented. Enterprises research opportunities to address
Ñ Real time agility; the disparities in their current applications;
Ñ Collaboration. Ñ New applications will be “designed” ground up with
service orientation as a goal rather than as an after
Challenges thought.
The new paradigm adopted by the development community
Both the scenarios have similar concerns but differ when it
has introduced more challenges in terms of testing the
comes to the amount of testing effort and test techniques
software built under the SOA philosophy. Following are
employed.
some of the top challenges testing SOA applications:
Ñ Functionality; Ñ Inter-application;
Ñ Performance; Ñ Intra-application.
AppLabs.com
App_WhitePaper_Approach_to_SOA_1v04 Page © 2007 AppLabs
These steps illustrate the order promising business flow.
To utilize the funnel approach, business experts create test
scenarios which would encompass the end to end path. The
required test data and expected results are also documented
at each step. Once this step is accomplished the test
Let us take an example to illustrate the funnel approach in scenarios are functionally decomposed to understand the
the perspective of an extended enterprise which helps to inter application entities, the next step is to further break
integrate customers and suppliers. This typically involves down the inter application entities to intra application
the following steps: entities finally boiling down to the units. In conjunction with
the funnel technique, AppLabs recommends exploratory
Ñ Customer uses the order enquiry web service of the
testing techniques (by bringing business experts) as they
enterprise sales application with an enquiry of his
are very effective instruments to unearth defects which
requirements;
cannot be detected by other techniques.
Ñ Order enquiry web service of sales application checks
Most of the commercial SOA test tools focus heavily on
with the Available to Promise (ATP) web service of order
unit testing. Some of the vendors offer business process
fulfillment application;
testing using the same unit tests (basically weaving
ü If there is enough material in stock that meets the together the units to form intra application bound tests).
delivery requirements of the customer, enterprise’s Generally, integration tests could be carried out using the
ability to fulfill the order is communicated back; normal functional tests if they possess the same or a similar
ü If there is not enough material in stock that meets technology stack.
the delivery requirements of the customer, Capable
to Promise (CTP) web service of the order fulfillment
Performance
application is triggered; The greatest challenge when it comes to performance
Ñ CTP web service in turn triggers capacity CTP web testing and service oriented applications is to design the
service from the order fulfillment application and an end user workload and model the end user environment
external component CTP web service from supplier (applications involved, software, hardware etc). The
application; predominant practice is to define and design usage modeling
for the entire suite of applications utilising Markov chains.
Ñ Supplier confirms back the commitment dates for the
quantities mentioned;
AppLabs.com
App_WhitePaper_Approach_to_SOA_1v04 Page © 2007 AppLabs
As shown in the picture below, stake holders and product challenge to test and make sure that old service consumers
managers are quizzed on the usage of the application. This are still supported. A solution for this challenge is to possess
would be used as a subjective assessment to support the test assets which are automated, representing the universe
analytical approach of Markov chains. The data of usage of consumers who are expected to be supported. These
probability is unearthed from raw data in the form of web test assets would be run against the new service to ensure
server logs and application server logs. This will help to backward compatibility.
design a realistic model that closely replicates end user
usage. Interoperability
Once the appropriate performance scenarios have been SOA became successful when the integration infrastructure
defined, multiple test tools/techniques are required because started adopting standards such as SOAP, XML etc. This
of the presence of different platforms and technologies. helped different platforms in an enterprise to talk to each
During test execution, monitoring application performance other (e.g. J2EE and .NET; different operating systems).
and collating data would be a challenge since there is no Though this is a boon for the CIOs, it is a challenge on the
“one stop shop” tool which gives insight into the overall application development groups to ensure interoperability.
big picture. In application to application communication, data transfer
is the key and data is stored, transmitted and manipulated
For applications which have been service enabled as an in a proprietary method (e.g. Big Decimal data type in
afterthought, the obvious challenge is to retain the same MS and Apache World are not compatible, the SOAP
quality of service measures and service level agreements headers are not identical on all the platforms). To simulate
when compared to the previous applications (since new interoperability tests, building a test environment which is
layers have been enabled, deterioration of performance is close to a real life scenario is a great challenge. Taking
predicted). advantage of software virtualization is one good approach.
For applications built with service orientation from the Crafting reusable test assets that work on many different
ground up, performance tests have to be carried out at platform combinations would also be of added value.
the unit level.
Compliance
Backward Compatibility SOAs (especially web services) are currently using specific
Services operate under contract (an interface enables standards to ensure that they are interoperable. Due
functionality to publish the contract); and service to the proliferation of productivity tools (development
consumption is based on this contract. The advantage of toolkits); there is room for heterogeneity being added to
SOA is the ease of changing the implementation without the integration infrastructure. This, however, might lead to
breaking the contract. As businesses evolve, software non compliance of the various standards adopted in the
infrastructure needs to adapt to the change. Some of context of W3C, WS-I, XML, and SOAP etc. As soon as the
these changes may require that the contract be broken. components are available, compliance tools can be run to
The challenge of infinite service consumers poses a serious ensure compliance with the different standards.
Security
SOAs implemented via web services have very similar
challenges to those which web applications face (for
example, cross site scripting, buffer overflows, SQL injection
etc). The greatest security challenge is in web services
which are open to public access where the universe of
users is unlimited (contrary to the enterprise scenario
where users are limited and can be authenticated).
Development phases
The following table shows the development phases and
the applicable test types in each phase:
AppLabs.com
App_WhitePaper_Approach_to_SOA_1v04 Page © 2007 AppLabs
Backward Inter-
Phase Functionality Performance Compliance Security
Compatibility operability
Inception
Review Requirements
¸ ¸ ¸ ¸ ¸ ¸
Elaboration
Functional/ Technical
¸ ¸ ¸ ¸ ¸ ¸
Design
Construction
Review Code ¸ ¸ ¸ ¸ ¸ ¸
Unit Test ¸ ¸ ¸ ¸ ¸ ¸
Integration Test (Inter
¸ - - - - -
app)
Integration Test (Intra
¸ - - - - -
app)
System Test ¸ ¸ - - - -
Verification Test - - ¸ ¸ - -
AppLabs.com
App_WhitePaper_Approach_to_SOA_1v04 Page © 2007 AppLabs