Analysis and Visualization of Discrete Data Using Neural Networks Koji Koyamada Full Chapter PDF
Analysis and Visualization of Discrete Data Using Neural Networks Koji Koyamada Full Chapter PDF
https://ebookmass.com/product/building-computer-vision-
applications-using-artificial-neural-networks-2nd-edition-
shamshad-ansari/
https://ebookmass.com/product/hydrogeological-conceptual-site-
models-data-analysis-and-visualization-1st-edition-ebook-pdf/
https://ebookmass.com/product/data-science-with-rust-a-
comprehensive-guide-data-analysis-machine-learning-data-
visualization-more-van-der-post/
https://ebookmass.com/product/accelerators-for-convolutional-
neural-networks-arslan-munir/
Spatial analysis using big data: methods and urban
applications Yamagata
https://ebookmass.com/product/spatial-analysis-using-big-data-
methods-and-urban-applications-yamagata/
https://ebookmass.com/product/visual-data-insights-using-sas-ods-
graphics-a-guide-to-communication-effective-data-
visualization-1st-edition-leroy-bessler-2/
https://ebookmass.com/product/visual-data-insights-using-sas-ods-
graphics-a-guide-to-communication-effective-data-
visualization-1st-edition-leroy-bessler/
https://ebookmass.com/product/biostatistics-and-computer-based-
analysis-of-health-data-using-r-1st-edition-christophe-lalanne/
https://ebookmass.com/product/data-visualization-exploring-and-
explaining-with-data-1st-edition-jeffrey-d-camm/
Analysis and
Visualization of
Discrete Data Using
Neural Networks
This page intentionally left blank
Analysis and
Visualization of
Discrete Data Using
Neural Networks
Koji Koyamada
Kyoto University, Japan
World Scientific
NEW JERSEY • LONDON • SINGAPORE • BEIJING • SHANGHAI • HONG KONG • TAIPEI • CHENNAI • TOKYO
Published by
World Scientific Publishing Co. Pte. Ltd.
5 Toh Tuck Link, Singapore 596224
USA office: 27 Warren Street, Suite 401-402, Hackensack, NJ 07601
UK office: 57 Shelton Street, Covent Garden, London WC2H 9HE
For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance
Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to photocopy
is not required from the publisher.
Printed in Singapore
Contents
1. Introduction 1
1.1 Basic operations of Excel . . . . . . . . . . . . . . . . . . . . 2
1.1.1 Table components . . . . . . . . . . . . . . . . . . . . 3
1.1.2 Name box and formula bar . . . . . . . . . . . . . . . 4
1.1.3 Ribbon . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.1.4 File tab and Backstage view . . . . . . . . . . . . . . 5
1.1.5 Autofill . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.1.6 Relative reference . . . . . . . . . . . . . . . . . . . . 7
1.1.7 Absolute reference . . . . . . . . . . . . . . . . . . . . 8
1.1.8 Introduction of visualization with Excel . . . . . . . . 10
1.1.9 Introduction of PDE derivation with Excel . . . . . . 12
1.2 Basic operations of Google Colab (Colab) . . . . . . . . . . . 15
1.2.1 Code cell . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.2.2 Text cell . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.2.3 Introduction to visualization with Colab . . . . . . . . 18
1.2.4 Introduction of deep learning with Colab . . . . . . . 19
1.3 Organization of this document . . . . . . . . . . . . . . . . . 24
2. Basic 25
2.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.1.1 Data analysis using NNs . . . . . . . . . . . . . . . . 25
2.1.2 Format of physical data . . . . . . . . . . . . . . . . . 30
2.1.3 Physical data visualization . . . . . . . . . . . . . . . 30
2.2 Statistical Analysis in Excel . . . . . . . . . . . . . . . . . . 36
2.2.1 Correlation analysis . . . . . . . . . . . . . . . . . . . 37
2.2.2 F-test . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
v
vi Analysis and Visualization of Discrete Data
2.2.3 t-Test . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.2.4 Z-test . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.3 Regression analysis . . . . . . . . . . . . . . . . . . . . . . . 41
2.3.1 Model characteristics . . . . . . . . . . . . . . . . . . 42
2.3.2 Regression analysis assumptions . . . . . . . . . . . . 43
2.3.3 Regression analysis using the data analysis tools
in Excel . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2.3.4 Excel macro . . . . . . . . . . . . . . . . . . . . . . . 55
2.3.5 Implementation of the variable reduction method
using Excel VBA . . . . . . . . . . . . . . . . . . . . . 59
2.4 What-if analysis . . . . . . . . . . . . . . . . . . . . . . . . . 63
2.5 Solver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
2.5.1 Optimization . . . . . . . . . . . . . . . . . . . . . . . 68
2.5.2 Implementation of regression analysis . . . . . . . . . 69
2.6 Colab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
2.6.1 Correlation analysis . . . . . . . . . . . . . . . . . . . 74
2.6.2 F-test . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
2.6.3 t-Test . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
2.6.4 Z-test . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
2.6.5 Regression analysis . . . . . . . . . . . . . . . . . . . 77
2.6.6 Optimization problem . . . . . . . . . . . . . . . . . . 80
2.7 NNs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
2.7.1 Universal approximation theorem . . . . . . . . . . . 83
2.7.2 Regression analysis using NNs . . . . . . . . . . . . . 85
2.7.3 NN implementation using Excel VBA . . . . . . . . . 87
2.7.4 Function approximation using NNs . . . . . . . . . . . 95
2.7.5 Point cloud data analysis using NNs: Application
example . . . . . . . . . . . . . . . . . . . . . . . . . . 98
2.7.6 Line group data analysis using NN application
examples . . . . . . . . . . . . . . . . . . . . . . . . . 101
References 217
Index 221
This page intentionally left blank
Chapter 1
Introduction
In this chapter, you will learn practical methods for analyzing physical data
spatiotemporally defined by discrete points. Physical data refer to data
used to describe physical or other phenomena. The physical phenomena
refer to various phenomena that exist in nature. They include, for example,
weather, temperature, earthquakes, and floods. The physical data are often
defined as spatiotemporal discrete data. The spatiotemporal discrete data
refer to data that are not spatiotemporally continuous. When we handle
spatiotemporal discrete data, we usually divide the spatiotemporal axis into
discrete segments so that we can handle data in each segment.
Spatiotemporal discrete data can be used in a variety of fields. For
example, in the social and natural sciences, spatiotemporal discrete data
are sometimes used to analyze various phenomena and their continuous
changes. In biology and geology, spatiotemporal discrete data are some-
times used to analyze phenomena that occur in time.
Physical data can be measured in a variety of ways as follows:
1
2 Analysis and Visualization of Discrete Data
widely used by individuals and in businesses, and can handle large amounts
of data, making it suitable for aggregation and analysis. Excel can be used
to not only create a table but to also create a graph or chart. Additionally,
Excel has a variety of functions to organize data, allowing you to analyze
data and calculate formulas. Excel is commonly pre-installed on Windows
and Mac OS and sold as part of Microsoft Office. It is also available as a
cloud service, so you can use it from any Internet-connected environment.
The data to be analyzed are often provided in Excel format. There are
quite a few tools capable of reading and analyzing Excel data, but Excel
itself also contains functions to perform analyses. Excel provides tools for
determining correlation between data, testing for differences in the mean
and variance of data, and regression analysis. It also serves as an optimiza-
tion tool. These tools can be used to implement deep learning. Excel can
be regarded as a tool for preliminary analysis prior to utilizing dedicated
analysis tools, and it is advisable to become familiar with its functionalities.
This section describes the basic terminology in Excel and explains how to
perform what-if analysis using sample data.
• Name box: In Excel, you can name a specified cell or range. This is
called naming. Naming allows you to specify a cell or a range of cells. A
named cell or range is called a “name” in Excel. You can manage your
data more efficiently by specifying a named cell or range. Excel provides
a function called “name box” that allows you to specify a named cell or
range by entering its name.
• Formula bar: In Excel, you can calculate data by entering a formula.
You can enter a formula in a place called “formula bar”. The formula
bar is located in the Excel status bar and used to enter a formula into a
cell. Entering a formula allows you to calculate data efficiently.
Introduction 5
1.1.3. Ribbon
The Excel ribbon refers to the tabbed menus that appear in the main Excel
window. The ribbon has various functions and commands grouped together
into a tab according to the category. There are various tabs, such as Home
and Data. Users can use commands in the ribbon to manipulate Excel to
create or edit a table.
The tabs and the area containing the buttons which can be switched
by clicking a tab are collectively called the “ribbon” (Figure 1.3). You can
switch between buttons by clicking the tab placed above them.
1.1.5. Autofill
The Autofill function in Excel helps you enter data efficiently. It automat-
ically suggests data to be entered next based on the data that have already
been entered (Figure 1.5).
For example, suppose that columns contain month and day data. The
data “January”, “February”, and “March” are entered in the “Month”
column, and the data “1st”, “2nd”, and “3rd” are entered in the “Day”
column. If you enter “4” in the April cell of the “Month” column, “4” will
also be automatically entered in the 4th cell of the “Day” column.
The Autofill function allows you to enter data more efficiently, so you
can perform Excel work more efficiently. The Autofill function also reduces
input errors, thereby increasing data reliability.
Introduction 7
1. If you select the D2 address that contains a formula, a small square will
appear at the bottom right of that cell.
2. Drag it downward to automatically fill other cells.
3. Select the B2 address. The contents of that cell will be displayed in the
formula bar.
To learn how to use relative and absolute references, display the calcu-
lation results of numbers in column A × numbers in row 1.
Before visualizing data in Excel, you may have to organize and process
them. Excel also has a visualization add-in that allows you to do more
detailed visualizations.
Displaying a chart using Excel:
1. Creating a chart: Excel has a variety of chart types that allow you to
enter data and create a chart.
• Select data, select Chart from the Insert tab, and select a chart type.
• After a chart is created, the data appear in the chart area.
Introduction 11
• Autofill is used to fill variable values downward from the cell of row 2 of
column A.
• Autofill is used to fill variable values rightward from the cell of row 1 of
column B.
ut , ux , uy uxx , uyy
for the physical data (u) at each of the multiple coordinate points in each
Excel row. The regression analysis tool allows you to analyze the degree of
impact of each partial derivative on u [1]. As a result, a PDE
ut = a0 ux + a1 uy + a2 uxx + a3 uyy
is derived that describes the physical data.
As a concrete example, consider an equation of motion (Figure 1.12).
An equation of motion is an ordinary differential equation representing
the laws of motion of an object in physics. A schematic diagram is shown
here where the mass of an object is m kg, a spring constant is k, and the
proportional coefficient of air resistance is c.
This ordinary differential equation is solved to calculate the position
coordinate of the object while changing the time data. This equation
becomes a second-order ordinary differential equation at the object’s posi-
tion coordinate x, which can be solved by giving an initial condition. Enter
the coordinate data (x) and the differential value xt , xtt , uy uxx , uyy at each
of the multiple times in each Excel row. xtt , xt , x regression analysis tools
allow you to analyze the degree of impact of position and speed on xtt . As
a result, a PDE
xtt = a0 xt + a1 x + a2 (1.1)
is derived that describes the physical data.
The advection equation is a PDE that describes the phenomena in which
physical quantities, such as a substance and momentum, are carried along
14 Analysis and Visualization of Discrete Data
u t + a0 u x = 0 (1.2)
is assumed for partial differential terms. Even for nonlinear terms where a
coefficient is a function of physical data, they can be reduced to a regression
analysis problem by integrating the nonlinear parts with partial differential
terms.
For the partial differential terms, if an exact solution is given, an exact
differential value can be obtained. If not, the calculation of a differential
value becomes an issue. One possible strategy is to obtain and differenti-
ate an approximate function that distributes physical data continuously in
some manner. One such approximate function that attracts attention is an
approximate function based on neural network (NN). This will be discussed
in Chapter 2.
• To create a text cell, click “+ Cell” at the top of a Colab page and select
“Text”.
• To enter text, select a cell and enter a sentence in it.
You can use the Markdown notation to format headings, lists, and more.
Colab text cells are very useful for writing descriptions and documents.
In particular, they can be used to describe programs and interpret results.
18 Analysis and Visualization of Discrete Data
The Colab text cells also allow you to better describe the output of your
program.
For example, you can write the following code to draw a line chart with
Matplotlib:
The code above stores the X axis and Y axis values in x and y,
respectively, and draws a line chart using the plt.plot () function. The
plt.title (), plt.xlabel (), and plt.yllabel () functions are used to set
the title of the chart, the X-axis label, and the Y-axis label, respectively.
Finally, the plt.show () function is used to display the chart. As described
above, Matplotlib allows you to draw a line chart easily (Figure 1.15).
Colab has pre-installed libraries required for deep learning, such as Ten-
sorFlow and Keras, so deep learning can commence immediately just after
importing the libraries. Colab also offers GPUs for free so that high-speed
training using large amounts of data can be conducted. It is worth noting
20 Analysis and Visualization of Discrete Data
The above code uses Tensorflow’s keras APIs to read MNIST data and
build, train, and evaluate an NN.
Introduction 21
Stacking these layers on the Sequential model defines the NN from the
input layer to the hidden layer to the output layer. The model receives a
handwritten digit image of 28 × 28 pixels as input and is trained to classify
it into one of 10 classes.
22 Analysis and Visualization of Discrete Data
These settings define how the model learns and determines an optimiza-
tion algorithm, the loss function, and evaluation metrics.
In model training, Keras is used to train the NN model and each instruc-
tion in the above coding has the following meaning:
The model.fit () method trains the model using the given training
dataset. More specifically, the training data are split into minibatches and
the gradient descent method is used for each minibatch to update model
parameters. Then, the loss function and evaluation metrics for the model
are calculated to report the progress of training. Splitting training data
into minibatches means that a large dataset is split into several smaller
batches to update model parameters for each minibatch. This allows the
Introduction 23
Basic
2.1. Background
2.1.1. Data analysis using NNs
This document describes the analysis of physical data primarily using NNs.
An NN is a combination of mathematical models of neurons in the human
cranial nervous system (Figure 2.1). The concept of NN is inspired by the
mechanics of the human brain (all the interconnections between neurons).
NN refers to a mathematical model created to represent some of the fea-
tures of brain function on a computer. NN is one of the modeling methods
in machine learning (ML) that mimics the workings of the human brain
without the aim to model the billions of neurons in the brain accurately.
It simply attempts to simplify it and make it operational. In NN, when one
inputs data (numerical values), the numerical values are propagated with
weights to the next layer. This is similar to the sequence of processes in the
human brain, whereby synapses are connected with weights and neurons
produce outputs, which are then connected to the next level.
Areas of use for NNs include pattern recognition and data mining
(e.g., image recognition and recommendations).
NNs are widely used in video websites to analyze comments posted on
videos. Specifically, NNs combine natural language processing techniques
to perform sentiment analysis, topic classification, spam filtering, etc., of
comments. The objective of sentiment analysis is to determine whether
a comment falls into the positive, negative, or neutral category. NNs are
useful for sentiment analysis because they can capture the meaning of words
and sentences. Topic classification determines to which topic a comment
relates. For example, to classify comments about a particular product or
service, NNs can create a classifier using keywords related to that product
25
26 Analysis and Visualization of Discrete Data
other information. In general, NNs use deep learning models such as MLP
(Multilayer Perception) and LSTM (Long Short-Term Memory). There are
two main methods of stock price forecasting. One is to forecast time series
data and the other is to predict future trends, such as the rise or fall of a
stock price.
The objective of forecasting time series data is to use past stock price
data and other information to train NNs to predict future stock prices.
For example, LSTM can be used to predict future stock prices by learning
patterns of past stock price fluctuations.
Conversely, NN is used primarily as a classification problem in predict-
ing trends such as rising or falling stock prices. The objective is to predict
whether stock prices will rise or fall, based on past stock price fluctua-
tions and other economic indicators. In this case, classifiers such as MLP
(Multilayer Perception) and CNN (Convolutional Neural Network)
are used.
However, since many factors affect stock price forecasts, it is difficult
to make forecasts using only a single NN model. For this reason, ensemble
learning, which combines multiple models to make forecasts, is often used.
Image recognition has become an indispensable technology in automated
driving development. Image recognition systems using NNs are already in
use to recognize surrounding objects faster and more accurately on behalf
of the driver. In automated driving, NNs are used to recognize surrounding
objects. A typical approach is to collect data from cameras, radar, lidar,
and sensors and feed that information as input data to the NN. The NN
then extracts feature values from the input data and uses them to recog-
nize objects and extract information such as position, speed, and direction.
In general, CNN (Convolutional Neural Network) is widely used for object
detection and segmentation. Deep learning-based object detection models
such as YOLO (You Only Look Once), SSD (Single Shot Detection), and
Faster R-CNN are also used. These models can provide fast and accurate
object detection. NNs are also used in other aspects of automation, such
as vehicle tracking, lane recognition, and traffic sign recognition.
NNs can detect gastric cancer from endoscopic images, which allows
them to be used in practice. NNs have achieved excellent results in image
analysis and are used in various fields.
To detect gastric cancer, endoscopic images are input into NNs to iden-
tify regions where cancer is most likely to be present. Specifically, NNs can
now automatically identify regions on a gastric endoscopic image where can-
cer is most likely to be present and detect gastric cancers 6 mm or larger
28 Analysis and Visualization of Discrete Data
first introduced in 2017 by Qi CR, et al. [10] and is widely used in areas such
as 3D object recognition, autonomous vehicles, and robot vision. A unique
feature of PointNet is its ability to process point clouds directly. Conven-
tional methods typically represent point clouds by converting them to voxel
grids or 3D meshes and then applying a CNN. However, such methods can
lead to compression and loss of information. PointNet avoids these prob-
lems by processing point clouds directly. PointNet can be used for tasks
such as classification, segmentation, and object detection of point clouds.
PointNet learns features such as point locations, colors, and normals from
an input point cloud and combines these features to create a feature repre-
sentation of the entire point cloud (Figure 2.2).
Using this global feature representation, PointNet can generate classifi-
cation output data for a given point cloud data (Figure 2.3).
pixel values are calculated for each pixel. This basic processing step is
repeated several times. The pixel values are added to the frame buffer and
then divided by the number of iterations to obtain the final pixel values by
averaging.
In PBVR, the projected images from the generated particles are added
to the image plane to calculate the luminance values at the corresponding
pixels. PBVR makes particles completely opaque so that when viewed by
the eye, light, ci (i = 1, . . . , n), from the particles at the back is blocked by
the particles in the front. This effect can be achieved using the Z-buffer
algorithm. The pre-projection sorting and alpha compositing processes typ-
ically required in volume rendering are unnecessary. The translucency effect
is achieved in this averaging process.
In the projection process of the generated particles, the Z-buffer algo-
rithm is used to keep the projected images of the particles closest to the eye
position, and pixel values are calculated using these images. As particles are
opaque, they do not require alpha compositing or reordering according to
their distance from the eye position. For the last remaining particles, color
mapping and shading calculations are performed. Color mapping uses a
color transfer function to convert scalar data calculated by interpolation
at the particle position into color data. The shading is determined by cal-
culating the brightness based on the interpolated gradient vector and the
light source vector, both of which are computed at the particle’s position,
and then multiplying this value by the color data. To calculate the final
pixel values, we discuss ensemble averaging (Figure 2.4).
An ensemble is a set of results from multiple iterations of conceptually
equivalent trial experiments. Each set of ensembles has the same attribute
values for particle radius and particle density. In the ensemble averaging
method, a trial experiment is assumed to be a single image creation process
consisting of particle generation and projection with the same attribute
values. The seed for random number generation is changed for each trial
experiment. In ensemble averaging, the final pixel value is the average of
the pixel values over all iterations. Therefore, if the i luminance value is
Bi , the final pixel value is calculated as follows:
LR
Bi
B total (LR ) = B i = (2.5)
i=1
LR
The variance of the final luminance value, with the luminance value Bi
as the random variable, is expanded as follows:
L
R
Bi
total
BVar (LR ) = Var
L
i=1 R
⎧ ⎫
1 ⎨ LR ⎬
= 2 Var B i + 2 Cov B i , B j (2.6)
LR ⎩ i=1 i,j,i<j
⎭
The luminance values Bi are independent of each other. Thus, the value of
the covariance Cov(Bi , Bj ) is 0. The variance of Bi can be determined as
Bvar regardless of the number of iterations. Thus, the standard deviation
of the final luminance value is as follows:
total BVar
BVar (LR ) = (2.7)
LR
Note that the more the number of iterations used, the better the image
quality becomes. This method is often used to improve the image quality
of noisy images such as satellite images. The ensemble averaging method
can be used to control the level of detail in the display of volume rendering
results. Creating an average image with a small number of iterations is
36 Analysis and Visualization of Discrete Data
Figure 2.5. Relationship between the number of iterations and the quality of the gener-
ated images.
Updated editions will replace the previous one—the old editions will
be renamed.
1.D. The copyright laws of the place where you are located also
govern what you can do with this work. Copyright laws in most
countries are in a constant state of change. If you are outside the
United States, check the laws of your country in addition to the terms
of this agreement before downloading, copying, displaying,
performing, distributing or creating derivative works based on this
work or any other Project Gutenberg™ work. The Foundation makes
no representations concerning the copyright status of any work in
any country other than the United States.
• You pay a royalty fee of 20% of the gross profits you derive from
the use of Project Gutenberg™ works calculated using the
method you already use to calculate your applicable taxes. The
fee is owed to the owner of the Project Gutenberg™ trademark,
but he has agreed to donate royalties under this paragraph to
the Project Gutenberg Literary Archive Foundation. Royalty
payments must be paid within 60 days following each date on
which you prepare (or are legally required to prepare) your
periodic tax returns. Royalty payments should be clearly marked
as such and sent to the Project Gutenberg Literary Archive
Foundation at the address specified in Section 4, “Information
about donations to the Project Gutenberg Literary Archive
Foundation.”
• You comply with all other terms of this agreement for free
distribution of Project Gutenberg™ works.
1.F.
1.F.4. Except for the limited right of replacement or refund set forth in
paragraph 1.F.3, this work is provided to you ‘AS-IS’, WITH NO
OTHER WARRANTIES OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR ANY PURPOSE.
Please check the Project Gutenberg web pages for current donation
methods and addresses. Donations are accepted in a number of
other ways including checks, online payments and credit card
donations. To donate, please visit: www.gutenberg.org/donate.
Most people start at our website which has the main PG search
facility: www.gutenberg.org.