Practical Convolutional Neural Networks: Implement advanced deep learning models using Python

Practical Convolutional Neural Networks 

Implement advanced deep learning models using Python

Mohit Sewak

Md. Rezaul Karim

Pradeep Pujari

BIRMINGHAM - MUMBAI

Practical Convolutional Neural Networks

Copyright © 2018 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author(s), nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Commissioning Editor: Sunith Shetty

Acquisition Editor: Vinay Argekar

Content Development Editor: Cheryl Dsa

Technical Editor: Sagar Sawant

Copy Editor: Vikrant Phadke, Safis Editing

Project Coordinator: Nidhi Joshi

Proofreader: Safis Editing

Indexer: Tejal Daruwale Soni

Graphics: Tania Dutta

Production Coordinator: Arvindkumar Gupta

First published: February 2018

Production reference: 1230218

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-78839-230-3

www.packtpub.com

mapt.io

Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.

Why subscribe?

Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals

Improve your learning with Skill Plans built especially for you

Get a free eBook or video every month

Mapt is fully searchable

Copy and paste, print, and bookmark content

PacktPub.com

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.

Contributors

About the authors

Mohit Sewak is a senior cognitive data scientist with IBM, and a PhD scholar in AI and CS at BITS Pilani. He holds several patents and publications in AI, deep learning, and machine learning. He has been the lead data scientist for some very successful global AI/ ML software and industry solutions and was earlier engaged in solutioning and research for the Watson Cognitive Commerce product line. He has 14 years of rich experience in architecting and solutioning with TensorFlow, Torch, Caffe, Theano, Keras, Watson, and more.

Md. Rezaul Karim is a research scientist at Fraunhofer FIT, Germany. He is also a PhD candidate at RWTH Aachen University, Germany. Before joining FIT, he worked as a researcher at the Insight Center for Data Analytics, Ireland. He was a lead engineer at Samsung Electronics, Korea.

He has 9 years of R&D experience with C++, Java, R, Scala, and Python. He has published research papers on bioinformatics, big data, and deep learning. He has practical working experience with Spark, Zeppelin, Hadoop, Keras, Scikit-Learn, TensorFlow, Deeplearning4j, MXNet, and H2O.

Pradeep Pujari is machine learning engineer at Walmart Labs and a distinguished member of ACM. His core domain expertise is in information retrieval, machine learning, and natural language processing. In his free time, he loves exploring AI technologies, reading, and mentoring.

About the reviewer

Sumit Pal is a published author with Apress. He has more than 22 years of experience in software, from start-ups to enterprises, and is an independent consultant working with big data, data visualization, and data science. He builds end-to-end data-driven analytic systems.

He has worked for Microsoft (SQLServer), Oracle (OLAP Kernel), and Verizon. He advises clients on their data architectures and build solutions in Spark and Scala. He has spoken at many conferences in North America and Europe and has developed a big data analyst training for Experfy. He has an MS and BS in computer science.

Packt is searching for authors like you

If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.

Table of Contents

Title Page

Copyright and Credits

Practical Convolutional Neural Networks

Packt Upsell

Why subscribe?

PacktPub.com

Contributors

About the authors

About the reviewer

Packt is searching for authors like you

Preface

Who this book is for

What this book covers

To get the most out of this book

Download the example code files

Download the color images

Conventions used

Get in touch

Reviews

Deep Neural Networks – Overview

Building blocks of a neural network

Introduction to TensorFlow

Installing TensorFlow

For macOS X/Linux variants

TensorFlow basics

Basic math with TensorFlow

Softmax in TensorFlow

Introduction to the MNIST dataset 

The simplest artificial neural network

Building a single-layer neural network with TensorFlow

Keras deep learning library overview

Layers in the Keras model

Handwritten number recognition with Keras and MNIST

Retrieving training and test data

Flattened data

Visualizing the training data

Building the network

Training the network

Testing

Understanding backpropagation 

Summary

Introduction to Convolutional Neural Networks

History of CNNs

Convolutional neural networks

How do computers interpret images?

Code for visualizing an image 

Dropout

Input layer

Convolutional layer

Convolutional layers in Keras

Pooling layer

Practical example – image classification

Image augmentation

Summary

Build Your First CNN and Performance Optimization

CNN architectures and drawbacks of DNNs

Convolutional operations

Pooling, stride, and padding operations

Fully connected layer

Convolution and pooling operations in TensorFlow

Applying pooling operations in TensorFlow

Convolution operations in TensorFlow

Training a CNN

Weight and bias initialization

Regularization

Activation functions

Using sigmoid

Using tanh

Using ReLU

Building, training, and evaluating our first CNN

Dataset description

Step 1 – Loading the required packages

Step 2 – Loading the training/test images to generate train/test set

Step 3- Defining CNN hyperparameters

Step 4 – Constructing the CNN layers

Step 5 – Preparing the TensorFlow graph

Step 6 – Creating a CNN model

Step 7 – Running the TensorFlow graph to train the CNN model

Step 8 – Model evaluation

Model performance optimization

Number of hidden layers

Number of neurons per hidden layer

Batch normalization

Advanced regularization and avoiding overfitting

Applying dropout operations with TensorFlow

Which optimizer to use?

Memory tuning

Appropriate layer placement

Building the second CNN by putting everything together

Dataset description and preprocessing

Creating the CNN model

Training and evaluating the network

Summary

Popular CNN Model Architectures

Introduction to ImageNet

LeNet

AlexNet architecture

Traffic sign classifiers using AlexNet

VGGNet architecture

VGG16 image classification code example

GoogLeNet architecture

Architecture insights

Inception module

ResNet architecture

Summary

Transfer Learning

Feature extraction approach

Target dataset is small and is similar to the original training dataset

Target dataset is small but different from the original training dataset

Target dataset is large and similar to the original training dataset

Target dataset is large and different from the original training dataset

Transfer learning example

Multi-task learning

Summary

Autoencoders for CNN

Introducing to autoencoders

Convolutional autoencoder

Applications

An example of compression

Summary

Object Detection and Instance Segmentation with CNN

The differences between object detection and image classification

Why is object detection much more challenging than image classification?

Traditional, nonCNN approaches to object detection

Haar features, cascading classifiers, and the Viola-Jones algorithm

Haar Features

Cascading classifiers

The Viola-Jones algorithm

R-CNN – Regions with CNN features

Fast R-CNN – fast region-based CNN

Faster R-CNN – faster region proposal network-based CNN

Mask R-CNN – Instance segmentation with CNN

Instance segmentation in code

Creating the environment

Installing Python dependencies (Python2 environment)

Downloading and installing the COCO API and detectron library (OS shell commands)

Preparing the COCO dataset folder structure

Running the pre-trained model on the COCO dataset

References

Summary

GAN: Generating New Images with CNN

Pix2pix - Image-to-Image translation GAN

CycleGAN 

Training a GAN model

GAN – code example

Calculating loss 

Adding the optimizer

Semi-supervised learning and GAN

Feature matching

Semi-supervised classification using a GAN example

Deep convolutional GAN

Batch normalization

Summary

Attention Mechanism for CNN and Visual Models

Attention mechanism for image captioning

Types of Attention

Hard Attention

Soft Attention

Using attention to improve visual models

Reasons for sub-optimal performance of visual CNN models

Recurrent models of visual attention

Applying the RAM on a noisy MNIST sample

Glimpse Sensor in code

References

Summary

Other Books You May Enjoy

Leave a review - let other readers know what you think

Preface

CNNs are revolutionizing several application domains, such as visual recognition systems, self-driving cars, medical discoveries, innovative e-commerce, and many more. This book gets you started with the building blocks of CNNs, while also guiding you through the best practices for implementing real-life CNN models and solutions. You will learn to create innovative solutions for image and video analytics to solve complex machine learning and computer vision problems.

This book starts with an overview of deep neural networks, with an example of image classification, and walks you through building your first CNN model. You will learn concepts such as transfer learning and autoencoders with CNN that will enable you to build very powerful models, even with limited supervised (labeled image) training data.

Later we build upon these learnings to achieve advanced vision-related algorithms and solutions for object detection, instance segmentation, generative (adversarial) networks, image captioning, attention mechanisms, and recurrent attention models for vision.

Besides giving you hands-on experience with the most intriguing vision models and architectures, this book explores cutting-edge and very recent researches in the areas of CNN and computer vision. This enable the user to foresee the future in this field and quick-start their innovation journey using advanced CNN solutions.

By the end of this book, you should be ready to implement advanced, effective, and efficient CNN models in your professional projects or personal initiatives while working on complex images and video datasets.

Who this book is for

This book is for data scientists, machine learning, and deep learning practitioners, and cognitive and artificial intelligence enthusiasts who want to move one step further in building CNNs. Get hands-on experience with extreme datasets and different CNN architectures to build efficient and smart ConvNet models. Basic knowledge of deep learning concepts and Python programming language is expected.

What this book covers

Chapter 1, Deep Neural Networks - Overview, it gives a quick refresher of the science of deep neural networks and different frameworks that can be used to implement such networks, with the mathematics behind them.

Chapter 2, Introduction to Convolutional Neural Networks, it introduces the readers to convolutional neural networks and shows how deep learning can be used to extract insights from images.

Chapter 3, Build Your First CNN and Performance Optimization, constructs a simple CNN model for image classification from scratch, and explains how to tune hyperparameters and optimize training time and performance of CNNs for improved efficiency and accuracy respectively.

Chapter 4, Popular CNN Model Architectures, shows the advantages and working of different popular (and award winning) CNN architectures, how they differ from each other, and how to use them.

Chapter 5, Transfer Learning, teaches you to take an existing pretrained network and adapt it to a new and different dataset. There is also a custom classification problem for a real-life application using a technique called transfer learning.

Chapter 6, Autoencoders for CNN, introduces an unsupervised learning technique called autoencoders. We walk through different applications of autoencoders for CNN, such as image compression.

Chapter 7, Object Detection and Instance Segmentation with CNN, teaches the difference between object detection, instance segmentation, and image classification. We then learn multiple techniques for object detection and instance segmentation with CNNs.

Chapter 8, GAN—Generating New Images with CNN, explores generative CNN Networks, and then we combine them with our learned discriminative CNN networks to create new images with CNN/GAN.

Chapter 9, Attention Mechanism for CNN and Visual Models, teaches the intuition behind attention in deep learning and learn how attention-based models are used to implement some advanced solutions (image captioning and RAM). We also understand the different types of attention and the role of reinforcement learning with respect to the hard attention mechanism. 

To get the most out of this book

This book is focused on building CNNs with Python programming language. We have used Python version 2.7 (2x) to build various applications and the open source and enterprise-ready professional software using Python, Spyder, Anaconda, and PyCharm. Many of the examples are also compatible with Python 3x. As a good practice, we encourage users to use Python virtual environments for implementing these codes.

We focus on how to utilize various Python and deep learning libraries (Keras, TensorFlow, and Caffe) in the best possible way to build real-world applications. In that spirit, we have tried to keep all of the code as friendly and readable as possible. We feel that this will enable our readers to easily understand the code and readily use it in different scenarios.

Download the example code files

You can download the example code files for this book from your account at www.packtpub.com. If you purchased this book elsewhere, you can visit www.packtpub.com/support and register to have the files emailed directly to you.

You can download the code files by following these steps:

Log in or register at www.packtpub.com.

Select the SUPPORT tab.

Click on Code Downloads & Errata.

Enter the name of the book in the Search box and follow the onscreen instructions.

Once