Security Testing:

Security testing evaluates system characteristics that relate to the availability, integrity,
and confidentially of system data and services.
• Users/clients should make sure their security needs are clearly known at requirements time, so
that security issues can be addressed by designers and testers.
Computer software and data can be compromised by:
(i) criminals intent on doing damage, stealing data and information, causing denial of service,
invading privacy;
(ii) errors on the part of honest developers/maintainers who modify, destroy, or compromise data
because of misinformation, misunderstandings, and/or lack of knowledge.
Sources of Damages:
(i) viruses;
(ii) trojan horses;
(iii) trap doors;
(iv) illicit channels
35
Effects of security breaches could be extensive and can cause:
• loss of information;
• corruption of information;
• misinformation;
• privacy violations;
• denial of service.
Developers try to ensure the security of their systems through use of protection
mechanisms such as passwords, encryption, virus checkers, and the detection and
elimination of trap doors.
Queries related to password
• What is the minimum and maximum allowed length for the password?
• Can it be pure alphabetical or must it be a mixture of alphabetical and other characters?
• Can it be a dictionary word?
• Is the password permanent, or does it expire periodically?
36
Areas to focus on during security testing
• Password Checking
• Legal and Illegal Entry with Passwords
• Password Expiration
• Encryption
• Browsing
• Trap Doors
• Viruses
The best approach to ensure security if resources permit, is to hire a so-called ―tiger
team‖ which is an outside group of penetration experts who attempt to breach the system
security.
Although a testing group in the organization can be involved in testing for security
breaches, the tiger team can attack the problem from a different point of view. Before the
tiger team starts its work the system should be thoroughly tested at all levels.
37
Recovery Testing:
Recovery testing subjects a system to losses of resources in order to determine if it can recover
properly from these losses.
• This type of testing is especially important for transaction systems, for example, on-line
banking Software
A test scenario might be to emulate loss of a device during a transaction. Tests would determine if
the system could return to a well known state, and that no transactions have been compromised.
Systems with automated recovery are designed for this purpose. They usually have multiple
CPUs and/or multiple instances of devices, and mechanisms to detect the failure of a device. They
also have a so-called ―checkpoint‖ system that meticulously records transactions and system
states periodically so that these are preserved in case of failure. This information allows the
system to return to a known state after the failure.
• The recovery testers must ensure that the device monitoring system and the checkpoint
software are working properly.
Focusing Areas during Recovery Test
• Restart
• Switch over

38
In these testing situations all transactions and processes must be carefully examined to detect:
• loss of transactions;
• merging of transactions;
• incorrect transactions;
• an unnecessary duplication of a transaction.

Regression Testing:
Regression testing is not a level of testing, but it is the retesting of software that
occurs when changes are made to ensure that the new version of the software has retained
the capabilities of the old version and that no new defects have been introduced due to the
changes.
• Regression tests are especially important when multiple software releases are
developed. Users want new capabilities in the latest releases, but still expect the older
capabilities to remain in place

39
Internationalization testing and Localization testing :
It is a type of non-functional testing.
• Internationalization is a process of designing a software application so that it can be
adapted to various languages and regions without any changes.
• Localization is a process of adapting internationalized software for a specific region or
language by adding local specific components and translating text.
Purpose:
The main purpose of internationalization is to check if the code can handle all
international support without breaking functionality that might cause data loss or data
integrity issues.
Internationalization Checklists:
1. Testing to check if the product works across settings.
2. Verifying the installation using various settings.
3. Verify if the product works across language settings and currency settings.
40
Adhoc Testing :
When a software testing performed without proper planning and documentation, it
is said to be Adhoc Testing. Such kind of tests are executed only once unless testers
uncover the defects.

• Adhoc Tests are done after formal testing is performed on the application.
• Adhoc methods are the least formal type of testing as it is NOT a structured approach.
• The success of Adhoc testing depends upon the capability of the tester, who carries out
the test. The tester has to find defects without any proper planning and documentation,
solely based on his intuition

Adhoc testing can be performed when there is limited time to do exhaustive testing
and usually performed after the formal test execution. Adhoc testing will be effective only
if the tester has in-depth understanding about the System Under Test.

41
Forms of Adhoc Testing :

Buddy Testing: Two buddies, one from development team and one from test team
mutually work on identifying defects in the same module. Buddy
testing helps the testers develop better test cases while development
team can also make design changes early. This kind of testing happens
usually after completing the unit testing.

Pair Testing: Two testers are assigned the same modules and they share ideas and
work on the same systems to find defects. One tester executes the tests
while another tester records the notes on their findings.

Monkey Testing: Testing is performed randomly without any test cases in order to break
the system.

42
Adhoc Testing can be made more effective by

 Preparation
 Creating a Rough Idea
 Divide and Rule
 Targeting Critical Functionalities
 Using Tools: Documenting the findings

43
Alpha, Beta and Acceptance Test :
The software is being developed to satisfy the users requirements, and no matter
how elegant its design it will not be accepted by the users unless it helps them to achieve
their goals as specified in the requirements.
Alpha, beta, and acceptance tests allow users to evaluate the software in terms of their
expectations and goals.
When software is being developed for a specific client, acceptance tests are carried out
after system testing. The acceptance tests must be planned carefully with input from the
client/users. The software must run under real-world conditions on operational hardware
and software. The software-under-test should be stressed.
• For continuous systems the software should be run at least through a 25-hour test cycle.
Conditions should be typical for a working day.
• Typical inputs and illegal inputs should be used and all major functions should be exercised.
• If the entire suite of tests cannot be run for any reason, then the full set of tests needs
to be rerun from the start.

44
 Acceptance tests are a very important milestone for the developers. At this time
the clients will determine if the software meets their requirements. Contractual obligations
can be satisfied if the client is satisfied with the software. Development organizations will
often receive their final payment when acceptance tests have been passed.
Acceptance tests must be rehearsed by the developers/testers. There should be no
signs of unprofessional behavior or lack of preparation. Clients do not appreciate
surprises. Clients should be received in the development organization as respected guests.
They should be provided with documents and other material to help them participate in
the acceptance testing process, and to evaluate the results. After acceptance testing the
client will point out to the developers which are not been satisfied. Some requirements
may be deleted, modified, or added due to changing needs.
If the client is satisfied that the software is usable and reliable, and they give their
approval, then the next step is to install the system at the client’s site. If the client’s site
conditions are different from that of the developers, the developers must set up the system
so that it can interface with client software and hardware. Retesting may have to be done
to insure that the software works as required in the client’s environment. This is called
installation test.
45
 If the software has been developed for the mass market (shrinkwrapped software),
then testing it for individual clients/users is not practical or even possible in most cases.
Very often this type of software undergoes two stages of acceptance test.
Stages of Acceptance Test:
1. Alpha test
2. Beta test
Alpha test:
This test takes place at the developer’s site. A cross-section of potential users and
members of the developer’s organization are invited to use the software. Developers
observe the users and note problems.
Beta test:
The software is sent to a cross-section of users who install it and use it under real
world working conditions. The users send records of problems with the software to the
development organization where the defects are repaired sometimes in time for the current
release. In many cases the repairs are delayed until the next release.
46
Usability Testing :
Usability is how appropriate, functional, and effective that interaction between the user
and the software.
User Interface Testing
The means that allows the users to interact with a software program is called its user
interface, or UI
List of seven important traits common to a good UI
1. Follows standards and guidelines
2. Intuitive
3. Consistent
4. Flexible
5. Comfortable
6. Correct
7. Useful
47
Follows Standards or Guidelines
 In testing software that runs on a specific platform, the tester need to treat the standards and
guidelines for that platform as an addendum to the product’s specification.
 Create test cases based on it
Intuitive
 Is the user interface clean, unobtrusive, not busy?
 Is the UI organized and laid out well? Does it allow users to easily get from one function to
another? Is what to do next obvious? At any point can user decide to do nothing or even back
up or back out? Are users inputs acknowledged? Do the menus or windows go too deep?
 Is there excessive functionality? Does the software attempt to do too much, either as a whole or
in part? Do too many features complicate users work? Do users feel like they’re getting
information overload?
 If all else fails, does the help system really help the user?

48
Consistent
Consistency within the software and with other software is a key attribute. Users develop
habits and expect that if they do something a certain way in one program, another will do
the same operation the same way
Ex.
 Shortcut keys and menu selections
 Terminology and naming
 Audience
 Placement and keyboard equivalents for buttons
Flexible
Users like choices—not too many, but enough to allow them to select what they want to
do and how they want to do it. The Windows Calculator has two views: Standard and
Scientific. Users can decide which one they need for their task or the one they’re most
comfortable using.

49
Comfortable
Software should be comfortable to use. It shouldn’t get in the way or make it difficult for
a user to do his work. Software comfort is a pretty touchy-feely concept
Features in identifying good and bad software comfort
 Appropriateness
 Error handling
 Performance
Correct
When testing for correctness, tester are testing whether the UI does what it’s supposed to
do.
To ensure correctness make sure to pay attention to
 Marketing differences
 Language and spelling
 Bad media
 WYSIWYG
50
Useful
The final trait of a good user interface is whether it’s useful. The tester is not concerned
with whether the software itself is useful, just whether the particular feature is
When testers are reviewing the product specification, preparing to test, or actually
performing testing,
• Ask if the features see actually contribute to the software’s value.
• Do they help users do what the software is intended to do?
• If testers don’t think they’re necessary, do some research to find out why they’re in the
software
Accessibility Testing :
• Testing for the disabled
Developing software with a user interface that can be used by the disabled isn’t just a
good idea, a guideline, or a standard—it’s the law

51
Accessibility Features in Software :
Software can be made accessible in one of two ways
1. to take advantage of support built into its platform or operating system
2. to have its own accessibility features specified, programmed, and tested
In testing usability for a product, be sure to create test cases specifically for accessibility
Capabilities provided by Windows for applications to be accessibility enable are
• StickyKeys
• FilterKeys
• ToggleKeys
• SoundSentry
• ShowSounds
• High Contrast
• MouseKeys
• SerialKey
52
Configuration Testing:
• Configuration testing is the process of checking the operation of the software under testing
with all the various types of hardware.
The different configuration possibilities for a standard Windows-based PC
• The PC - Compaq, Dell, Gateway, Hewlett Packard, IBM
• Components - system boards, component cards, and other internal devices such as disk drives,
CD-ROM drives, video, sound, modem, and network cards
• Peripherals - printers, scanners, mice, keyboards, monitors, cameras, joysticks
• Interfaces - ISA, PCI, USB, PS/2, RS/232, and Firewire
• Options and memory - hardware options and memory sizes
• Device Drivers
All components and peripherals communicate with the operating system and the software
applications through low-level software called device drivers. These drivers are often provided by
the hardware device manufacturer and are installed when you set up the hardware. Although
technically they are software, for testing purposes they are considered part of the hardware
configuration.
53
 To start configuration testing on a piece of software, the tester needs to consider
which of these configuration areas would be most closely tied to the program.
Examples:
• A highly graphical computer game will require lots of attention to the video and sound
areas.
• A greeting card program will be especially vulnerable to printer issues.
• A fax or communications program will need to be tested with numerous modems and
network configurations.
Finding Configuration Bugs:
The sure way to tell if a bug is a configuration problem and not just an ordinary bug
is to perform the exact same operation that caused the problem, step by step, on another
computer with a completely different configuration.
 If the bug doesn’t occur, it’s very likely a configuration problem.
 If the bug happens on more than one configuration, it’s probably just a regular bug.

54
The general process that the tester should use when planning the configuration testing are:
• Decide the types of hardware needed
• Decide what hardware brands, models, and device drivers are available
• Decide which hardware features, modes, and options are possible
• Pare down the identified hardware configurations to a manageable set
• Identify the software’s unique features that work with the hardware configurations
• Design the test cases to run on each configuration
• Execute the tests on each configuration
• Rerun the tests until the results satisfy the test team

55
Compatibility Testing :
• Software compatibility testing means checking that your software interacts with and
shares information correctly with other software.
• This interaction could occur between two programs simultaneously running on the same
computer or even on different computers connected through the Internet thousands of
miles apart
Examples of compatible software
Cutting text from a Web page and pasting it into a document opened in your word processor
In performing software compatibility testing on a new piece of software, the tester needs
to concentrate on
 Platform and Application Versions (Backward and Forward Compatibility, The
Impact of Testing Multiple Versions)
 Standards and Guidelines (High-Level Standards and Guidelines, Low-Level
Standards and Guidelines
 Data Sharing Compatibility (File save and file load, File export and file import )
56
Compatibility Testing:

57
Platform and Application Versions
Selecting the target platforms or the compatible applications is really a program management or a
marketing task. A Person who’s very familiar with the customer base will decide whether the
software is to be designed for a specific operating system, Web browser, or some other platform.
They’ll also identify the version or versions that the software needs to be compatible with.
Backward compatible:
If something is backward compatible, it will work with previous versions of the software.
Forward compatible :
If something is forward compatible, it will work with future versions of the software.

58
In compatibility testing a new platform, tester must check that existing software
applications work correctly with it

To begin the task of compatibility testing, tester needs to equivalence partition all the possible
software combinations into the smallest, effective set that verifies that the software interacts
properly with other software.
Factors to be considered in partitioning
• Popularity
• Age
• Type
• Manufacturer
59
In Compatibility testing a new application tester may require to test it on multiple
platforms and with multiple applications

60
Standards and Guidelines
There are two types of standards:
• High level
• Low level
High-level standards are the ones that guide the product’s general compliance, its look and feel, its
supported features, and so on.
Low-level standards are the nitty-gritty details, such as the file formats and the network
communications protocols.
Data Sharing Compatibility
The sharing of data among applications is what really gives software its power. A well-
written program that supports and adheres to published standards must allow users to easily
transfer data to and from other software to be a great compatible product.
Familiar means of transferring data :
1. File save and file load
2. File export and file import
3. Cut, copy, and paste

61
Testing the documentation:
If the software’s documentation consists of nothing but a simple readme file, testing it would not
be a big deal. The tester should make sure that it included all the material that it was supposed to,
that everything was technically accurate, and to run a spell check and a virus scan on the disk
Types of Documentation: ( components classified as documentation)
• Packaging text and graphics
• Marketing material, ads, and other inserts
• Warranty/registration
• EULA
• Labels and stickers
• Installation and setup instructions
• User’s manual
• Online help
• Tutorials, wizards, and CBT
• Samples, examples, and templates
• Error messages
62
Documentation on a disk label

63
The Importance of Documentation Testing
Good software documentation contributes to the product’s overall quality in three ways
1. It improves usability
2. It improves reliability
3. It lowers support costs

Effective approach to documentation testing :

64
Documentation Testing Checklist
1. General Areas

65
2. Correctness

66
Website Testing :
Web site testing encompasses many areas, including
• configuration testing,
• compatibility testing,
• usability testing,
• documentation testing,
• localization testing
Web page features
• Text of different sizes, fonts, and colors
• Graphics and photos
• Hyperlinked text and graphics
• Varying advertisements
• Drop-down selection boxes
• Fields in which the users can enter data
67
Features that make the Web site much more complex :
 Customizable layout that allows users to change where information is positioned on screen
 Customizable content that allows users to select what news and information they want to see
 Dynamic drop-down selection boxes
 Dynamically changing text
 Dynamic layout and optional information based on screen resolution
 Compatibility with different Web browsers, browser versions, and hardware and software
platforms
 Lots of hidden formatting, tagging, and embedded information that enhances the Web page’s
usability

68
Black-Box Testing
• Treat the Web page or the entire Web site as a black box
• Take some time and explore
 Think about how to approach testing a website
 What would the tester test?
 What would the equivalence partitions be?
 What would the tester choose not to test?

When testing a Web site, the tester first creates a state table, treating each page as a
different state with the hyperlinks as the lines connecting them. A completed state map
will give a better view of the overall task.

69
The tester should look for the following
• Text
Web page text should be treated just like documentation and tested accordingly. Check the
audience level, the terminology, the content and subject matter, the accuracy—especially of
information that can become outdated—and always, always check spelling
• Hyperlinks
Links can be tied to text or graphics. Each link should be checked to make sure that it jumps to the
correct destination and opens in the correct window .Make sure that hyperlinks are obvious. Text
links are usually underlined, and the mouse pointer should change to a hand pointer when it’s over
any kind of hyperlink—text or graphic. If the link opens up an email message, fill out the
message, send it, and make sure in getting a response.
• Graphics
Do all graphics load and display properly? If a graphic is missing or is incorrectly named, it won’t
load and the Web page will display an error where the graphic was to be placed. If text and
graphics are intermixed on the page, make sure that the text wraps properly around the graphics.
Try resizing the browser’s window to see if strange wrapping occurs around the graphic. How’s
the performance of loading the page? Are there so many graphics on the page, resulting in a large
amount of data to be transferred and displayed, that the Web site’s performance is too slow? What
if it’s displayed over a slow dial-up modem connection on a poor-quality phone line?
70
• Forms
Test forms just as you would if they were fields in a regular software program. Are the fields the
correct size? Do they accept the correct data and reject the wrong data? Is there proper
confirmation when you finally press Enter? Are optional fields truly optional and the required
ones truly required?

• Objects and Other functionality

71
Gray-Box Testing :
Graybox testing, is a mixture of black-box and white-box testing
Test the software as a black-box, but supplement the work by taking a peek (not a full look, as in
white-box testing) at what makes the software work.
HTML and Web pages can be tested as a gray box

White-Box Testing :
Features of a website tested with a white-box approach are
 Dynamic Content
 Database-Driven Web Pages
 Programmatically Created Web Pages
 Server Performance and Loading
 Security

72
Configuration and Compatibility Testing :
 Configuration testing is the process of checking the operation of the software with
various types of hardware and software platforms and their different settings.
 Compatibility testing is checking the software’s operation with other software
The possible hardware and software configurations might be that could affect the operation or
appearance of a web site are :
• Hardware Platform
• Browser Software and Version
• Browser Plug-Ins
• Browser Options
• Video Resolution and Color Depth
• Text Size
• Modem Speeds

73
Usability Testing :
Following and testing a few basic rules can help make Web sites more usable.
Jakob Nielsen, a respected expert on Web site design and usability, has performed
extensive research on Web site usability.
The Top Ten Mistakes in Web Design
1. Gratuitous Use of Bleeding-Edge Technology
2. Scrolling Text, Marquees, and Constantly Running Animations
3. Long Scrolling Pages
4. Non-Standard Link Colors
5. Outdated Information
6. Overly Long Download Times
7. Lack of Navigation Support
8. Orphan Pages
9. Complex Web Site Addresses
10.Using Frames
74