The Azure Cloud Native Architecture Mapbook: Explore Microsoft Cloud's infrastructure, application, data, and security architecture

BIRMINGHAM—MUMBAI

The Azure Cloud Native Architecture Mapbook

Copyright © 2021 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author(s), nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Aaron Lazar

Publishing Product Manager: Denim Pinto

Senior Editor: Storm Mann

Content Development Editor: Nithya Sadanandan

Technical Editor: Gaurav Gala

Copy Editor: Safis Editing

Project Coordinator: Deeksha Thakkar

Proofreader: Safis Editing

Indexer: Rekha Nair

Production Designer: Nilesh Mohite

First published: February 2021

Production reference: 1160221

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-80056-232-5

www.packt.com

To my wife, Diana Lucic, who designed all the maps of this book.

– Stephane Eyskens

To my sons, Ben and Yoav, for showing me how talent and creativity evolve. To Tsippi and Shlomo Bobbe, for their love, support, and inspiration.

– Ed Price

Contributors

About the authors

Stephane Eyskens is an Azure solutions architect and a digital transformation advocate, helping organizations to get better results out of their cloud investments. As an MVP, he is an active contributor to the Microsoft Tech Community and has worked on multiple open source projects available on GitHub. Stephane is also a Pluralsight assessment author, as well as the author of multiple books and online recordings.

Ed Price is a senior program manager in engineering at Microsoft, with an MBA in technology management. He leads Microsoft's efforts to publish reference architectures on the Azure Architecture Center (http://aka.ms/Architectures). Previously, he drove data center deployment and customer feedback, and he ran Microsoft's customer feedback programs for Azure development, Service Fabric, IoT, Functions, and Visual Studio. He was also a technical writer at Microsoft for 6 years and helped lead TechNet Wiki. He is the co-author of five books, including Learn to Program with Small Basic and ASP.NET Core 5 for Beginners, available from Packt.

About the reviewers

Giorgos-Chrysovalantis Grammatikos is an Azure solutions architect with Tisski Ltd. He is an IT pro with over 10 years of experience in the industry, has achieved various Azure and other Microsoft certifications, and has been an Azure MVP since 2018. His specialization is in various Microsoft technologies including the Azure cloud, SQL Server, Power BI, and Hyper-V. He is an active member of the Microsoft community, blogging on his website cloudopszone.com and posting technical articles on the Microsoft wiki and dev blogs. He is a frequent public speaker at meetups and various Microsoft Azure events.

Sjoukje Zaal is a Microsoft chief technical officer at Capgemini, a Microsoft regional director, and a Microsoft Azure MVP with over 20 years of experience providing architecture, development, consultancy, and design expertise. She mainly focuses on cloud, security, productivity, and IoT.

She loves to share her knowledge and is active in the Microsoft community as a co-founder of the user groups Tech Daily Chronicle, Global XR Community, and the Mixed Reality User Group. She is also a board member of Azure Thursdays and Global Azure. Sjoukje is an international speaker, is involved in organizing many events, and has written several books and blogs. Sjoukje is also part of the MVP Diversity and Inclusion Advisory Board.

Table of Contents

Preface

Section 1: Solution and Infrastructure

Chapter 1: Getting Started as an Azure Architect

Technical requirements

Getting to know architectural duties

Enterprise architects

Domain architects

Solution architects

Data architects

Technical architects

Security architects

Infrastructure architects

Application architects

Azure architects

Architects versus engineers

Getting started with the essential cloud vocabulary

Cloud service models map

IaaS (Infrastructure as a Service)

PaaS (Platform as a Service)

FaaS (Function as a Service)

CaaS (Containers as a Service)

DBaaS (Database as a Service)

XaaS or *aaS (Anything as a Service)

Introducing Azure architecture maps

How to read a map

Understanding the key factors of a successful cloud journey

Defining the vision with the right stakeholders

Defining the strategy with the right stakeholders

Starting implementation with the right stakeholders

Practical scenario

Summary

Chapter 2: Solution Architecture

Technical requirements

The solution architecture map

Zooming in on the different workload types

Understanding systems of engagement

Understanding systems of record

Understanding systems of insight

Understanding systems of interaction (IPaaS)

Looking at cross-cutting concerns and non-functional requirements

Looking at cross-cutting concerns and the cloud journey

Zooming in on containerization

Solution architecture use case

Looking at a business scenario

Using keywords

Using the solution architecture map against the requirements

Building the target reference architecture

Code view of our workflow-based reference architecture

Looking at the code in action

Understanding the gaps in our reference architecture

Summary

Chapter 3: Infrastructure Design

Technical requirements

The Azure infrastructure architecture map

Zooming in on networking

The most common architecture

Data center connectivity options

Zoning

Routing and firewalling

Zooming in on monitoring

Zooming in on high availability and disaster recovery

Zooming in on backup and restore

Zooming in on HPC

AKS infrastructure

Exploring networking options with AKS

Exploring deployment options with AKS

Monitoring AKS

Exploring AKS storage options

Scaling AKS

Exploring miscellaneous aspects

AKS and service meshes for microservices versus Azure native services

AKS reference architecture for microservices – cluster boundaries

AKS reference architecture for microservices – cluster internals

Summary

Chapter 4: Infrastructure Deployment

Technical requirements

Introducing Continuous Integration and Continuous Deployment (CI/CD)

Introducing the CI/CD process

Introducing the IaC CI/CD process

The Azure deployment map

Getting started with the Azure CLI, PowerShell, and Azure Cloud Shell

Playing with the Azure CLI from within Azure Cloud Shell

Using PowerShell from within Azure Cloud Shell

Combining PowerShell and the Azure CLI from within Azure Cloud Shell

Understanding the one that rules them all

Diving into ARM templates

Getting started with ARM

Understanding the ARM template deployment methods

Understanding the ARM template deployment scopes

Understanding the ARM template deployment modes

Understanding the anatomy of an ARM template

Building a concrete example using linked templates

Getting started with Azure Bicep

Getting started with Terraform

Zooming in on a reference architecture with Azure DevOps

Using a simple approach to an IaC factory

Using an advanced approach to an IaC factory

Summary

Section 2: Application Development, Data, and Security

Chapter 5: Application Architecture

Technical requirements

Understanding cloud and cloud-native development

Exploring the Azure Application Architecture Map

Zooming in on data

Zooming in on cloud design patterns

Dealing with cloud-native patterns

Understanding the COMMODITIES top-level group

Exploring EDAs

Inspecting the Azure Service Bus configuration

Adding the other components to the mix

Developing microservices

Using Dapr for microservices

Understanding Dapr components

Getting started with Dapr SDKs

Looking at our scenario

Developing our solution

Testing our solution

Combining Dapr and the API gateway of Azure APIM

Summary

Chapter 6: Data Architecture

Technical requirements

Looking at the data architecture map

Analyzing traditional data practices

Introducing the OLAP and OLTP practices

Introducing the ETL practice

Introducing the RDBMS practice

Delving into modern data services and practices

Introducing the ELT practice

Exploring NoSQL services

Learning about object stores

Diving into big data services

Ingesting big data

Exploring big data analytics

Azure-integrated open source big data solutions

Introducing AI solutions

Understanding machine learning and deep learning

Integrating AI solutions

Dealing with other data concerns

Introducing Azure Cognitive Search

Sharing data with partners and customers (B2B)

Migrating data

Governing data

Getting our hands dirty with a near real-time data streaming use case

Setting up the Power BI workspace

Setting up the Azure Event Hubs instance

Setting up Stream Analytics (SA)

Testing the code

Summary

Chapter 7: Security Architecture

Technical requirements

Introducing cloud-native security

Reviewing the security architecture map

Exploring the recurrent services security features

Exploring the recurrent data services security features

Zooming in on encryption

Managing your security posture

Zooming in on identity

Delving into the most recurrent Azure security topics

Exploring Azure managed identities in depth

Demystifying SAS

Understanding APL and its impact on network flows

Understanding Azure resource firewalls

Adding the security bits to our Contoso use case

Summary

Section 3: Summary

Chapter 8: Summary and Industry Scenarios

Revisiting our architectures

Sample architecture

Solution architecture

Infrastructure architecture

Azure deployment

Application architecture

Data architecture

Security architecture

Visiting the verticals

Automotive and transportation scenarios

Predictive insights with vehicle telematics

Predictive aircraft engine monitoring

IoT analytics for autonomous driving

Banking and financial services scenarios

Banking system cloud transformation

Decentralized trust using blockchain

Additional financial services architectures

Gaming scenarios

Low-latency multiplayer gaming

Gaming using MySQL or Cosmos DB

Healthcare scenarios

Building a telehealth system on Azure

Medical data storage architectures

AI healthcare solutions

Predicting length of stay using SQL Server R Services

Producing and consuming IoT healthcare data

Confidential computing on a healthcare platform

Manufacturing scenarios

Supply chain track and trace

Industrial IoT analytics

AI and analytics manufacturing architectures

Oil and gas scenarios

Run reservoir simulation software on Azure

Oil and gas tank level forecasting

IoT monitor and manage loops

Retail scenarios

Retail and e-commerce Azure database architectures

Demand forecasting with Spark on HDInsight

Demand forecasting with machine learning

AI retail scenarios

Architecture for buy online, pick up in store

The unique values of this book

Summary

Why subscribe?

Other Books You May Enjoy

Preface

Have you ever visited a large city on your own? Sometimes you get lost, and sometimes you lose time. However, you can make your trip more valuable by taking an expert-guided tour. That's what this book is: an expert-guided tour of Azure. Our different maps will be your compass to sail the broad Azure landscape. The Microsoft cloud platform offers a wide range of services, providing a million ways to architect your solutions. This book uses original maps and expert analysis to help you explore Azure and choose the best solutions for your unique requirements. Beyond maps, the book is inspired by real-world situations and challenges. We will share typical and cross-industry concerns to help you become a better Azure architect. Our real-world-inspired architecture diagrams and use cases should put you in a better position to tackle your own challenges. Although an architecture role may be high-level, we also wanted to dive deeper into some topics and make you work harder on some use cases. In this respect, you will have some hands-on work to do too. However, our primary objective is to make you stronger in the various architecture disciplines that are scoped to Azure and that every Azure architect should be comfortable with.

Who this book is for

This book is intended for aspiring and confirmed Azure architects. This book is broad and encompasses multiple architecture disciplines and concepts, so you should ideally have a broad skillset to enjoy the book. Nevertheless, IT engineers and developers will also ramp up their knowledge and find value in this book.

What this book covers

Chapter 1, Getting Started as an Azure Architect, starts by sharing a view of the different architecture disciplines. We define the roles and responsibilities of the various architects (enterprise, solution, infrastructure, data, and security). The rationale of going through these definitions lies in the fact that, from our experience, we have noticed some knowledge gaps in what the different stakeholders are doing. This often leads to turf wars, which can be avoided simply by understanding the broader picture. We then introduce our maps, and we help you understand how to properly conduct a cloud strategy and what the key aspects are that will make your cloud journey successful. In a nutshell, we give you a glimpse into what it feels like to be an Azure architect who has to deal with all these different disciplines, and who sometimes must report to top management on strategic aspects.

Chapter 2, Solution Architecture, covers key aspects to consider when building a cloud solution. A solution architect is responsible for the end-to-end aspects of a solution, from its development to its monitoring. A solution architect knows what Agile methodologies are, as well as what ITIL, TOGAF, and COBIT are. They are the cornerstone of a solution, its main pillar. The primary role of a solution architect is to assemble all the building blocks to make a consistent and coherent design, as well as to talk to various stakeholders. Their stakeholders are other, more specialized architects, developers, and IT engineers, as well as enterprise architects and management. This chapter remains high-level from a technical perspective because we will still envision Azure as a whole. We share the solution architecture map, which encompasses many Azure services, and we explore multiple dimensions around the non-functional requirements. We also zoom in on Azure's container platform offering, which has been booming and expanding greatly over the last few years. Lastly, we will walk you through a concrete use case and a glimpse into what comes next, including a deeper dive into the technical and technological aspects.

Chapter 3, Infrastructure Design, delves deeper into technical matters. We will review the typical infrastructure topologies and we will zoom into infrastructure-specific concerns such as networking, monitoring, backup and restore, high availability, and disaster recovery (for which we'll see a sample use case). Because containerization has become mainstream, we will also dive into Azure Kubernetes Services (AKS) and unveil a dedicated AKS architecture map. You will learn that AKS is not really a service like the others, and we will walk you through a reference architecture to host a service mesh (for microservices) in AKS.

Chapter 4, Infrastructure Deployment, is almost entirely hands-on! You will learn about the different Infrastructure as Code (IaC) tools and frameworks. You will provision some Azure services using Azure Resource Manager templates, Bicep, and Terraform. Nevertheless, we won't forget our architecture glasses, so we will also look at the machinery of a Continuous Integration and Continuous Delivery/Deployment (CI/CD) factory.

Chapter 5, Application Architecture, looks at what the development architecture would look like for building an app on the Microsoft cloud. You may ask 10 different people what cloud-native means, and you might receive 10 different answers. So, we will start by explaining what we mean when we refer to the cloud and cloud-native solutions. Next, we will review some modern design patterns, such as CQRS, Event Sourcing, and so on. In the process, we will map them to the Azure services to help you identify how to bundle the services together in order to build solutions based on these patterns. Lastly, we will go through a microservices use case, using Dapr (Distributed Application Runtime), which is a very recent and promising framework for developing distributed applications. Throughout this chapter, our motto will be to not reinvent the wheel. Instead, leverage the ecosystem to design and build your solutions.

Chapter 6, Data Architecture, explores how data is processed and stored. Data is the new gold, and Azure contains many gold mines! In this chapter, we will consider traditional and modern data practices in opposition to each other, and see how to use both in Azure. We will also explore big data and artificial intelligence and analytics. At last, our hands-on use case is based on a data-streaming scenario. We are going to build a real-time dashboard, which consolidates aggregates of metrics from a fake speed detector (which we have developed for you). A separate real-time tile will show all the vehicles that should receive a fine (for breaking the law).

Chapter 7, Security Architecture, emphasizes and explains the importance of security in the cloud. Security is everywhere, and it's even more important with the cloud. This tends to awaken age-old fears and trepidations. This topic certainly deserves an entire book, so (to avoid writing a second book) we decided to be very pragmatic and to focus on the essential parts only. We start by giving you a glimpse into cloud-native security, to see beyond the technology and what the required mindset is. We will then explain why there is a paradigm shift in identity with the public cloud, by simply … proving it! Lastly, we will focus on the most recurrent security services and topics in Azure, which you must absolutely master as an Azure architect. Throughout the chapter, our motto will be to not simply stack network layers. Instead, think further and modernize your security practices.

Chapter 8, Summary and Industry Scenarios, revisits the topics covered in the book and consolidates our key ideas from each previous chapter. In other words, we'll identify what the most important aspects to remember are. In addition, we'll look at several key industry verticals through the lens of the previous chapters, to guide you through some existing architectures that you can continue exploring after you complete the book. We'll finish with some notes on the unique key values of this book, and a brief summary.

To get the most out of this book

To enjoy the book and practice our hands-on exercises, you will of course need an Azure subscription, and for the bravest readers (those who are implementing our use cases), you'll also need Docker, Visual Studio 2019, and/or Visual Studio Code. All our code samples are built in .NET Core. From a higher-level perspective, you'll be able to quickly grasp the concepts in this book if you're already an architect or a senior developer/IT pro. Don't worry if you need to use Google from time to time (to look up names and terms); it's perfectly normal, as we explore the main architectural dimensions. (We do explain all the basic concepts, but to keep the content focused for senior developers, senior IT pros, and new architects, the book is written with a certain expectation of technical knowledge.)

If you are using the digital version of this book, we advise you to type the code yourself or access the code via the GitHub repository (the link is available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.

Download the example code files

You can download the maps, diagrams, and sample code for this book from GitHub at https://github.com/PacktPublishing/The-Azure-Cloud-Native-Architecture-Mapbook. In case there's an update to the code, it will be updated on the existing GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Code in Action

Code in Action videos for this book can be viewed at http://bit.ly/3pp9vIH.

Download the color images

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://static.packt-cdn.com/downloads/9781800562325_ColorImages.pdf.

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: Locate and open the appsettings.json file, in the netcoreapp3.1 folder.

A block of code is set as follows:

public class DataObject{

    private string[] sensorNames = new string[] { Brussels,         Genval };

    public string sensorName { get; private set; }

    public double speed { get; private set; }

    public string plateNumber { get; private set; }

    public DataObject()

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

public class DataObject{

    private string[] sensorNames = new string[] {         Brussels, Genval };

    public string sensorName { get; private set; }

    public double speed { get; private set; }

    public string plateNumber { get; private set; }

    public DataObject()

Any command-line input or output is written as follows:

$ az storage account list

Bold: Indicates a new term, an important word, or words that you see onscreen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: Choose the Custom Streaming data tile type.

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, mention the book title in the subject of your message and email us at [email protected].

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata, select your book, click the Errata Submission Form link, and enter the details.

Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Reviews

Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!

For more information about Packt, please visit packt.com.

Section 1: Solution and Infrastructure

In this section, we will first revise the different architecture practices and dimensions, because the word architect is sometimes misunderstood or abused. We will share our views on the different architecture dimensions that we cover throughout the book. The initial chapter will set the scene for your journey. Then, we will get you started with Azure and acquainted with the services that you can assemble to design and build solutions. We will also focus on an essential foundation – the infrastructure – and you will get to know how to leverage Infrastructure as Code and automation, to get an optimal return on investment out of your cloud expenditures.

In this section, we will cover the following topics:

Chapter 1, Getting Started as an Azure Architect

Chapter 2, Solution Architecture

Chapter 3, Infrastructure Design

Chapter 4, Infrastructure Deployment

Chapter 1: Getting Started as an Azure Architect

In this chapter, we will focus on what an architect's role entails and explain the various cloud service models that are made available by the Microsoft Azure platform. We will describe how the numerous maps in this book are built, what they intend to demonstrate, and how to make sense of them.

More specifically, in this chapter, we will cover the following topics:

Getting to know architectural duties

Getting started with the essential cloud vocabulary

Introducing Azure architecture maps

Understanding the key factors of a successful cloud journey

Our purpose is to help you learn the required vocabulary that is used across the book. You will also understand the duties of an Azure architect. We will explain the most frequently used service models and their typical associated use cases, which every Azure architect should know. We start smoothly, but beware that the level of complexity will increase as we go. Let's start by getting acquainted with the definition of an architect.

Technical requirements

The Maps provided in this chapter are available at https://github.com/PacktPublishing/The-Azure-Cloud-Native-Architecture-Mapbook/tree/master/Chapter01.

Getting to know architectural duties

Before we define what an Azure architect is, let's first define what an architect's role is and how our maps specialize to reflect these different profiles. The word architect is used everywhere on the IT planet. Many organizations have their own expectations when it comes to defining the tasks and duties of an architect. Let's share our own definitions as well as some illustrative diagrams.

Enterprise architects

Enterprise architects oversee the IT and business strategies, and they make sure that every IT initiative is in line with the enterprise business goals. They are directly reporting to the IT leadership and are sometimes scattered across business lines. They are also the guardians of building coherent and consistent overall IT landscapes for their respective companies. Given their broad role, enterprise architects have a helicopter view of the IT landscape, and they are not directly dealing with deep-dive technical topics, nor are they looking in detail at specific solutions or platforms, such as Azure, unless a company would put all its assets in Azure. In terms of modeling, they often rely on the TOGAF (short for The Open Group Architecture Framework) modeling framework and ArchiMate. The typical type of diagrams they deal with looks like the following:

Figure 1.1 – Capability viewpoint: ArchiMate

As you can see, this is very high level and not directly related to any technology or