RMM Unit-I Introdution To Data Mining
RMM Unit-I Introdution To Data Mining
TECH CSE)
2019-2020
L T P C
4 0 0 4
Unit-III: Classification
Problem Definition, General Approaches to Solving a Classification
Problem, Evaluation of Classifiers, Classification Techniques, Decision
Trees-Decision tree Construction, Methods for Expressing attribute
test conditions, Measures for selecting the Best Split, Algorithm for
Decision tree induction; Naïve-Bayes Classifier, Bayesian Belief
Networks; K-Nearest Neighbor classification-Algorithm and
characteristics
Unit-IV: Clustering
Problem Definition, Clustering Overview, Evaluation of Clustering
Algorithms, Partitioning Clustering: K-Means Algorithm, K-Means
Additional Issues, PAM algorithm; Hierarchical clustering:
Agglomerative methods and divisive methods, Basic Agglomerative
Hierarchical clustering Algorithm, Specific Techniques, Key Issues in
Hierarchical Clustering, Strengths and weakness; Outlier Detection
1. Data Mining- Concepts and Techniques- Jiawei Han, Micheline Kamber, Morgan Kaufmann
Publishers, Elsevier, 2 Edition, 2006.
2. Introduction to Data Mining, Pang-Ning Tan, Vipin Kumar, Michael Steinbanch, Pearson
Education.
REFERENCE BOOKS:
2. Data Mining Principles & Applications – T.V Sveresh Kumar, B.Esware Reddy, Jagadish S
Kalimani, Elsevier.
UNIT-I
INTRODUCTION TO DATA MINING
Missing Data
Dimensionality Reduction
Feature Subset Selection
Discretization and Binaryzation
Data Transformation
Measures of Similarity and Dissimilarity –Basics
Definition-1:
Data mining (knowledge discovery from data)
Extraction of interesting (non-trivial, implicit, previously unknown
and potentially useful) patterns or knowledge from huge amount of
data
Data mining: a misnomer?
Alternative names
Knowledge discovery (mining) in databases (KDD), knowledge
extraction, data/pattern analysis, data archeology, data dredging,
information harvesting, business intelligence, etc.
Watch out: Is everything “data mining”?
Simple search and query processing
(Deductive) expert systems
Definition-2:
Data mining is defined as finding hidden information in a database.
Alternatively it has been called exploratory data analysis, data
driven discovery and deductive learning.
Definition-3:
Data mining refers to using a variety of techniques to identify
nuggets of information or decision-making knowledge in the
database and extracting these in such a way that they can be put
to use in areas such as decision support, prediction, forecasting
and estimation . The data is often voluminous, but it ha slow value
and no direct use can be made of it. It is the hidden information in
the data that is useful.
KDD-Knowledge Discovery in Databases: KDD is the non-trivial process of identifying valid, novel, potentially useful, and
ultimately understandable patterns in data”.
Data mining is a step in the KDD process consisting of particular data mining algorithms that, under some acceptable
computational efficiency limitations, produces a particular enumeration of patterns Ej over database F
Task-relevant Data
Data Selection
Warehouse
Data Cleaning
Data Integration
Databases
October 13, 2021 Data Mining 21
Knowledge Discovery (KDD) Process
KDD as a process is shown in the above figure and consists of an iterative sequence of the following steps:
1. Data Cleaning: To remove noise and inconsistent data.
2. Data Integration: Where multiple data sources may be combined.
3. Data Selection: Where data relevant to the analysis task are retrieved from the database.
4. Data Transformation: Where data transformed or consolidated into forms appropriate for mining by performing summary or aggregation operations, for instance.
5. Data Mining: An essential process where intelligent methods are applied in order to extract data patterns.
6. Pattern Evaluation: To identify the truly interesting patterns representing knowledge based on some interestingness measures.
7. Knowledge Presentation: Where visualization and knowledge representation techniques are used to represent the mined knowledge to the user.
Increasing potential
to support
business decisions End User
Decision
Making
Data Exploration
Statistical Summary, Querying, and Reporting
Database
Technology Statistics
Machine Visualization
Learning Data Mining
Pattern
Recognition Other
Algorithm Disciplines
General functionality
Descriptive data mining
Predictive data mining
Different views lead to different classifications
Data view: Kinds of data to be mined
Knowledge view: Kinds of knowledge to be discovered
Method view: Kinds of techniques utilized
Application view: Kinds of applications adapted
Outlier analysis
Outlier: Data object that does not comply with the general behavior
of the data
Noise or exception? Useful in fraud detection, rare events analysis
Periodicity analysis
Similarity-based analysis
Task-relevant data
Database or data warehouse name
Database tables or data warehouse cubes
Condition for data selection
Relevant attributes or dimensions
Data grouping criteria
Type of knowledge to be mined
Characterization, discrimination, association, classification,
prediction, clustering, outlier analysis, other data mining tasks
Background knowledge
Pattern interestingness measurements
Visualization/presentation of discovered patterns
October 13, 2021 Data Mining 42
Primitive 3: Background Knowledge
Simplicity
e.g., (association) rule length, (decision) tree size
Certainty
e.g., confidence, P(A|B) = #(A and B)/ #(B), classification
reliability or accuracy, certainty factor, rule strength, rule quality,
discriminating weight, etc.
Utility
potential usefulness, e.g., support (association), noise threshold
(description)
Novelty
not previously known, surprising (used to remove redundant
rules, e.g., Illinois vs. Champaign rule implication support ratio)
Motivation
A DMQL can provide the ability to support ad-hoc and
interactive data mining
By providing a standardized language like SQL
Hope to achieve a similar effect like that SQL has on
relational database
Foundation for system development and evolution
Facilitate information exchange, technology transfer,
commercialization and wide acceptance
Design
DMQL is designed with the primitives described earlier
Pattern Evaluation
Knowle
Data Mining Engine dge-
Base
Database or Data
Warehouse Server
Completeness
Consistency
Timeliness
Believability
Value added
Interpretability
Accessibility
Broad categories:
Intrinsic, contextual, representational, and accessibility
Data cleaning
Fill in missing values, smooth noisy data, identify or remove
outliers, and resolve inconsistencies
Data integration
Integration of multiple databases, data cubes, or files
Data transformation
Normalization and aggregation
Data reduction
Obtains reduced representation in volume but produces the same
or similar analytical results
Data discretization
Part of data reduction but with particular importance, especially
for numerical data
Motivation
To better understand the data: central tendency, variation
and spread
Data dispersion characteristics
median, max, min, quantiles, outliers, variance, etc.
Numerical dimensions correspond to sorted intervals
Data dispersion: analyzed with multiple granularities of
precision
Boxplot or quantile analysis on sorted intervals
Dispersion analysis on computed measures
Folding measures into numerical dimensions
Boxplot or quantile analysis on the transformed cube
October 13, 2021 Data Mining 58
Measuring the Central Tendency
1 n x
Mean (algebraic measure) (sample vs. population): x xi
n i 1 N
n
Weighted arithmetic mean: w x i i
x i 1
Trimmed mean: chopping extreme values n
w
i 1
i
N
i 1
( xi
2
)
N
xi 2
i 1
2
warehousing”—DCI survey
Data cleaning tasks
Fill in missing values
Identify outliers and smooth out noisy data
Correct inconsistent data
Resolve redundancy caused by data integration
October 13, 2021 Data Mining 73
Missing Data
technology limitation
incomplete data
inconsistent data
Clustering
detect and remove outliers
Y1
Y1’ y=x+1
X1 x
store
Schema integration: e.g., A.cust-id B.cust-#
Integrate metadata from different sources
rA, B
( A A)( B B ) ( AB) n A B
(n 1)AB (n 1)AB
Χ2 (chi-square) test
(Observed Expected ) 2
2
Expected
The larger the Χ2 value, the more likely the variables are
related
The cells that contribute the most to the Χ2 value are
those whose actual count is very different from the
expected count
Correlation does not imply causality
# of hospitals and # of car-theft in a city are correlated
Both are causally linked to the third variable: population
73,600 54,000
Ex. Let μ = 54,000, σ = 16,000. Then 1.225
16,000
Normalization by decimal scaling
v
v' j Where j is the smallest integer such that Max(|ν’|) < 1
10
October 13, 2021 Data Mining 89
Data Reduction Strategies
Why data reduction?
A database/data warehouse may store terabytes of data
smaller in volume but yet produce the same (or almost the
same) analytical results
Data reduction strategies
Data cube aggregation:
Data Compression
understand
Heuristic methods (due to exponential # of choices):
Step-wise forward selection
Decision-tree induction
A4 ?
A1? A6?
Reduced
> attribute set: {A1, A4, A6}
October 13, 2021 Data Mining 93
Heuristic Feature Selection Methods
Typically lossless
expansion
Audio/video compression
Typically lossy compression, with progressive
refinement
Sometimes small fragments of signal can be
s sy
lo
Original Data
Approximated
component vectors
The principal components are sorted in order of decreasing
“significance” or strength
Since the components are sorted, the size of the data can be
X2
Y1
Y
2
X1
above
Log-linear models:
The multi-way table of joint probabilities is
Partition data set into clusters based on similarity, and store cluster
representation (e.g., centroid and diameter) only
Can be very effective if data is clustered but not if data is “smeared”
Can have hierarchical clustering and be stored in multi-dimensional
index tree structures
There are many choices of clustering definitions and clustering
algorithms
Cluster analysis will be studied in depth in Unit-5
W O R
SRS le random
im p h o u t
(s e w it
m p l t)
s a me n
l ac e
rep
SRSW
R
Raw Data
October 13, 2021 Data Mining 107
Sampling: Cluster or Stratified Sampling
Discretization
Reduce the number of values for a given continuous attribute by
dividing the range of the attribute into intervals
Interval labels can then be used to replace actual data values
Supervised vs. unsupervised
Split (top-down) vs. merge (bottom-up)
Discretization can be performed recursively on an attribute
Concept hierarchy formation
Recursively reduce the data by collecting and replacing low level
concepts (such as numeric values for age) by higher level concepts
(such as young, middle-aged, or senior)
(-$1,000 - $2,000)
Step 3:
(-$400 -$5,000)
Step 4:
Similarity Measure
Numerical measure of how alike two data objects are.
Often falls between 0 (no similarity) and 1 (complete
similarity).
Dissimilarity Measure
Numerical measure of how different two data objects are.
Range from 0 (objects are alike) to ∞ (objects are
different).
Proximity refers to a similarity or dissimilarity.
Here, p and q are the attribute values for two data objects.