Big Data Capacity Planning
Big Data Capacity Planning
Abstract
Businesses are considering more opportunities to
leverage data for different purposes, impacting
resources and resulting in poor loading and
response times. Hadoop is increasingly being
adopted across industry verticals for information
management and analytics. In addition to new
business related capabilities, it offers a host of
options for IT simplification and cost reduction.
Initiatives such as offloads are at the heart of this
type of optimization. As a result, Hadoop capacity
planning should be carried out as the first step in
both IT-driven and business-driven use cases
whenever Big Data projects are considered.
WHITE PAPER
How is the data ingested, and at what The architecture needs to be planned based
frequency? on the ingestion type (in streams, batches
or from an RDBMS system) and supported
with capacity planning.
Does the Hadoop system need to be If the Hadoop system to be developed is
read- or write-intensive? write intensive, resources necessary to
quickly complete the writes need to be
planned. A few distributions can be
leveraged to write one copy and confirm
that it is done, while the others write all
three copies (replication factor three) and
confirm that the replication is done.
If it is to be read intensive, necessary
memory (perhaps in-memory) and network
resources should be increased.
How many concurrent users will there If the number of users is large, it is
be access the system? advisable to increase the nodes and their
resources (RAM).
Legend
R: Replication factor
i: Intermediate factor
N = 600/8 = 75
Conclusion
With new digital technologies gaining greater prominence, it is
imminent that Big Data with its quantitative abilities will lay the
foundation for improved qualitative analysis. The expected
increase in data implies an ever increasing focus on capacity
planning, a critical requirement for all production systems.
Capacity planning is an exercise and a continuous practice to
arrive at the right infrastructure that caters to the current, near
future, and future needs of a business.
References
This can include citations and references
WHITE PAPER
Contact
Email: [email protected]
All content / information present here is the exclusive property of Tata Consultancy Services Limited (TCS). The content / information contained here is
correct at the time of publishing. No material from here may be copied, modified, reproduced, republished, uploaded, transmitted, posted or distributed
in any form without prior written permission from TCS. Unauthorized use of the content / information appearing here may violate copyright, trademark
and other applicable laws, and could result in criminal or civil penalties. Copyright © 2016 Tata Consultancy Services Limited