Lecture 1 - Getting To Know Scalability
Lecture 1 - Getting To Know Scalability
Scalability
BITS Pilani Dr. Shreyas Rao
Associate Prof. (Off Campus), CSIS, BITS-Pilani
Pilani Campus
Instructor Profile
Dr. Shreyas Rao
• 18+ Years of Experience in IT, Teaching and Research
• B.E from VTU, M.S in Software Systems from BITS (WILP) and PhD from MAHE
• Worked as Business Analyst and Team Lead at SLK Software Services for 7+
years
• COE member in AI&ML and COE member in Data Science (Govt. Sponsored for
1.2 Cr)
No Course Objective
CO1 Build competence to design, develop, implement and manage scalable information
systems
CO2 Gain understanding of different techniques & tools for building and managing
scalable services
CO3 Gain understanding of challenges and best practices in creating and managing
scalable services
• Launched on 6-Oct-2014
• Opened at 8am
• Big discount in 70+ categories, flash sales, lucky draw
• Sold large units of Nokia Lumia 525, Samsung Galaxy Tabs
at throw-away prices
• Three Lakh order in 6 hours!
Negatives:
• Website crashed due to huge traffic and footfalls
• Already selected products vanished from cart after
recovery or appeared as sold out!
• Money got deducted from account, but order not executed
• Big complaints from customers
• Reviews were hidden; no refund and no cancellation of
orders
What steps did the Hotstar Cloud Architects take to handle 25.3 Million
concurrent users?
• Scalability
• Performance
• Availability
• Reliability
• Interoperability
• Testability
• Usability
• Modifiability
• Security
• Portability
• Maintainability
1. What is Scalability
2. Need for Scalable Architectures
3. Principles of Scalability
4. Scale Cube
5. CAP Theorem
6. Guidelines for Building Highly Scalable Systems
7. Architecture’s Scalability Requirements
8. Challenges for Scalability
9. Case Study – Uber / Netflix
The foundations of scale need to be built in from the beginning, with the
recognition that the components will evolve over time.
Ex: We have a Web-based (e.g. web server and database) system that can
service a load of 100 concurrent requests with a mean response time of 1
second. As the request load increases, we see the mean response time
steadily grow to 10 seconds with the projected load.
Requirement: Scale up this system to handle 1000 concurrent requests with the
same response time??
Solution:
Scale Up - running the Web server on a more powerful machine (Vertical)
Scale Out - run multiple instances of the Web server to increase capacity
(Horizontal, Cloning)
Is it that easy to scale?
So, In simple words, CAP theorem means if there is network partition and if you want
your system to keep functioning you can provide either Availability or Consistency and
not both.
https://bikas-katwal.medium.com/mongodb-vs-cassandra-vs-rdbms-where-do-they-stand-in-the-cap-theorem-1bae779a7a15
Strong Consistency - simply means that all the nodes across the world should
contain the same value for an entity at any point in time.
Eventual Consistency - at some point the system will let all users read the most
recently made updates, but all will not do so immediately. This greatly reduces
synchronism in our application and allows the system to remain available to all
users, even in the face of network partitions - during the partition, users may see
inconsistent data, but as the partition heals the consistency of the system will
eventually be restored. [Decide level of staleness that is acceptable]
1. Centralized Approach
2. Synchronous Communication
3. Cost
Source - https://dzone.com/articles/microservice-architecture-learn-build-and-deploy-a
Source - https://dzone.com/articles/microservice-architecture-learn-build-and-deploy-a
1. All the features had to be re-built, deployed and tested again and again to
update a single feature.
2. Fixing bugs became extremely difficult in a single repository as developers had
to change the code again and again.
3. Scaling the features simultaneously with the introduction of new features
worldwide was quite tough to be handled together.
Source - https://dzone.com/articles/microservice-architecture-learn-build-and-deploy-a
1. The units are individual separate deployable units performing separate functionalities.
For Example: If you want to change anything in the billing microservices, then you just have to deploy only
billing microservices and don’t have to deploy the others.
2. All the features were now scaled individually i.e. The interdependency between each and every
feature was removed.
For Example, we all know that the number of people searching for cabs is more comparatively more than
the people actually booking a cab and making payments. This gets us an inference that the number of
processes working on the passenger management microservice is more than the number of processes
working on payments.
Source - https://dzone.com/articles/microservice-architecture-learn-build-and-deploy-a
Source - https://www.geeksforgeeks.org/system-design-netflix-a-complete-architecture/
Source - https://www.geeksforgeeks.org/system-design-netflix-a-complete-architecture/
Other components
• ZUUL – Gateway Service that provides dynamic routing based on the input
request
• Inbound filter – Authentication, Routing
• Endpoint filter – Return static response or forward to backend service
• Outbound filter – Zip content and calculate metrics. Send response to Netty Web
server and to the client
Other components
Microservices
• Critical services – searching video, navigating to video, play video
• Make it independent from other services
Database
• MYSQL (RDBMS) – Data for billing information, user information and transaction
information. Deployed on ec2 instances.
• Cassandra (NOSQL) – Viewing history data (over 50 Cassandra Clusters with
500 nodes)
1. What is Scalability
2. Need for Scalable Architectures
3. Principles of Scalability
4. Scale Cube
5. CAP Theorem
6. Guidelines for Building Highly Scalable Systems
7. Architecture’s Scalability Requirements
8. Challenges for Scalability
9. Case Study – Uber / Netflix