Predictive Modelling Sweta Kumari
Predictive Modelling Sweta Kumari
Modelling
0
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Table of Contents
Table Content 1
List of Figure
Data Dictionary 5
Problem 1: Linear Regression 6
1.1. Read the data and do exploratory data analysis. Describe the data briefly. (Check the null
values, Data types, shape, EDA). Perform Univariate and Bivariate Analysis. 3
Checking the top 5 records:.............................................................................................................6
Checking the Bottom 5 records:.......................................................................................................6
Checking the shape of the data frame:............................................................................................6
Checking the information of the data frame:..................................................................................7
Checking for Missing Values:...........................................................................................................7
Checking the summary of the data frame:......................................................................................7
Check for Duplicates:.......................................................................................................................7
Unique values for categorical variables...........................................................................................7
UNIVARIENT ANALYSIS:....................................................................................................................8
Skewness:........................................................................................................................................8
Bivariate analysis:..........................................................................................................................10
correlation matrix:.........................................................................................................................11
1.2 Impute null values if present, also check for the values which are equal to zero. Do they have
any meaning or do we need to change them or drop them? Do you think scaling is necessary in this
case? 15
Impute Null Values.........................................................................................................................11
StandardScaler Scaling...................................................................................................................12
BOX Plot Before Scaling.................................................................................................................14
Box Plot on Scaled Data.................................................................................................................14
Correlation Matrix on Scaled Data.................................................................................................15
1.3. Encode the data (having string values) for Modelling. Data Split: Split the data into train
and test (70:30). Apply Linear regression. Performance Metrics: Check the performance of
Predictions on Train and Test sets using Rsquare, RMSE. 18
Split X and y into training and test set in 70:30 ratio.....................................................................15
Variation influence.........................................................................................................................17
STATSMODEL.................................................................................................................................18
ITERATION 2...................................................................................................................................18
1.4 Inference: Basis on these predictions, what are the business insights and recommendations 19
Problem 2: Logistic Regression and LDA 19
1
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
2.1 Data Ingestion: Read the dataset. Do the descriptive statistics and do null value condition
check, write an inference on it. Perform Univariate and Bivariate Analysis. Do exploratory data
analysis..........................................................................................................................................19
Checking the top 5 records:...........................................................................................................19
Checking the Bottom 5 records:.....................................................................................................20
Checking the shape of the data frame:..........................................................................................20
Checking the information of the data frame:................................................................................20
Checking for Missing Values:.........................................................................................................21
Check for Duplicates:.....................................................................................................................21
unique values for categorical variables..........................................................................................23
UNIVARIENT ANALYSIS:..................................................................................................................23
Categorical Variables.....................................................................................................................23
Skewness:......................................................................................................................................24
Multivariate analysis:.....................................................................................................................25
correlation matrix:.........................................................................................................................26
Outliers..........................................................................................................................................26
2.2 Do not scale the data. Encode the data (having string values) for Modelling. Data Split: Split the
data into train and test (70:30). Apply Logistic Regression and LDA (linear discriminant analysis).40
Splitting data into training and test set..........................................................................................27
Model 1 - Applying GridSearchCV for Logistic Regression.............................................................27
Prediction on the training set........................................................................................................27
Getting the probabilities on the test set........................................................................................27
2.3 Performance Metrics: Check the performance of Predictions on Train and Test sets using
Accuracy, Confusion Matrix, Plot ROC curve and get ROC_AUC score for each model Final Model:
Compare Both the models and write inference which model is best/optimized. 28
Classification Report /Confusion matrix / Accuracy on the training data......................................28
AUC and ROC for the training data................................................................................................28
Classification Report /Confusion matrix / Accuracy on the Test data............................................29
AUC and ROC for the Test data......................................................................................................29
LDA MODEL....................................................................................................................................30
Training Data and Test Data Confusion Matrix Comparison..........................................................31
Training Data and Test Data Classification Report Comparison.....................................................31
Training Data Probability Predication & Score for Train and Test model.......................................31
AUC and ROC for the training & test data......................................................................................32
How to change the cut-off values for maximum accuracy?...........................................................32
Predicting the classes on the test data..........................................................................................33
2
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Classification Report of the default cut-off test data:..........................................................................33
Classification Report of the custom cut-off test data:.........................................................................34
2.4 Inference: Basis on these predictions, what are the insights and recommendations.Please
explain and summarise the various steps performed in this project. There should be proper
business interpretation and actionable insights present. 34
3
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
List of Figure:
4
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Data Dictionary :
5
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Problem 1: Problem Statement: Linear Regression
You are a part of an investment firm and your work is to do research about these 759 firms. You are
provided with the dataset containing the sales and other attributes of these 759 firms. Predict the
sales of these firms on the bases of the details given in the dataset so as to help your company in
investing consciously. Also, provide them with 5 attributes that are most important.
Ques 1: Read the data and do exploratory data analysis. Describe the data briefly. (Check the null
values, data types, shape, EDA). Perform Univariate and Bivariate Analysis
Ans: - We have successfully imported all necessary libraries and also checked latest versions of it.
We will read the file and make analysis out of it.
We find one Columns as Unnamed: 0 which is not adding any value to analysis so we drop this
records and again check top 5 records.
6
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Data type of dataset:
Exploratory Data Analysis: EDA is an approach to analyze data using both non visual and visual
techniques. It has a structured approach which involves thorough analysis of data to understand the
current business situation
7
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Univariate and Bivariate Analysis
1) Dist. plot and box plot of sales
8
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
5) Dist. Plot and box plot of employment
Fig-1
9
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Checking skewness of the dataset:
Fig -2
10
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Correlation Heat map of the data
Fig-3
Question.1.2) Impute null values if present? Do you think scaling is necessary in this case?
Ans: We will check the null value in dataset in order to check if there is any value missing in the
dataset .
From above table we found , total 21 values are missing in tobniq attribute.
We will impute missing values by using median so that no missing value is left is dataset for analysis.
11
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
After imputing missing value ,we found no values are missing in the dataset .
Scaling : When scaling of the data is done, all variables come at comparable level, scaling is
important when data has different in volume and magnitude. Data normalization is the method used
to standardize the range of features of data. Since the range of values of data may vary widely, it
becomes a necessary step in data preprocessing while using machine learning algorithms. Here in
current case where although data is in better state, however , Standard Scaler as well as Zscore used
to scale the data.
Let us plot the graph of the dataset with outliers before scaling the data.
12
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Fig-4
Now from above graph we found lots of outliers are present in the dataset .
Fig5
13
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Now let’s plot the graph of Correlation Heat map of the data after treating the outliers.
Fig-6
Observations:
1) We found we need to impute the dataset as missing values are present in tobniq dataset .
2) We have imputed missing values by using median of tobniq .
3) Scaling of dataset is necessary in order to bring all the dataset at one level.
4) We have scaled the dataset using StandardScaler
5) We also found lots of outliers are present in the dataset and it got treated .
Question 1.3) 1.3) Encode the data (having string values) for Modelling. Data Split: Split the data into
test and train (70:30). Apply Linear regression. Performance Metrics: Check the performance of
Predictions on Train and Test sets using R-square, RMSE.
Ans: We have split the given data set into train and test data for model building we have followed
the steps as given below:
Step -1 : Separate X & Y
Step -2 : Train test split – X_train , X_test, Y_train, Y_test.
Step -3 : Model is introduced.
Step -4: model.fit(X_train, Y_train)
Step -5: model.predict(X_train) & (X_test)
Step -6: Validation
14
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
We have applied Linear Regression on the given dataset.
15
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Variation Influence before scaling the dataset:
capital ---> 13.210120175427297
patents ---> 10.923471116505148
randd ---> 13.859560752076456
employment ---> 8.639388123955953
tobinq ---> 1.5045082851876506
value ---> 8.057752748316684
institutions ---> 1.2012687061404628
16
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Fig7
It is always a good practice to scale all the dimensions using z scores or some other method to
address the problem of different scales
Coefficients of different attributes of dataset after scaling the dataset using Zscore is given below:
The coefficient for capital is 0.25546427187105647
The coefficient for patents is -0.02692349037369152
The coefficient for randd is 0.053592321134503934
The coefficient for employment is 0.4344708559702377
The coefficient for tobinq is -0.04559176069854309
The coefficient for value is 0.29673076835504136
The coefficient for institutions is 0.008683246589433539
MSE : 0.26746436362040027
17
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Fig 8
18
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Notes:[1] Standard Errors assume that the covariance matrix of the
errors is correctly specified
Ques 1.4) Inference: Based on these predictions, what are the business insights and
recommendations.
Ans:
Business Insight:
2)With one unit increase in employment ,sales increases by 0.4 units, keeping all the other
predictors constant.
3)When capital increases by 1 unit ,sales increases by 0.3 units, keeping all other predictors
constant.
4)There are also some negative co-efficient values for instance. This implies ,when negative co-
efficient attributes are reduced the sales decreases ,keeping all the predictor constant.
Recommendations:
1)The investment criteria for any new investor are mainly based on the capital invested in the
company by the promoters and investors are vying on the firms where the capital investment is
good .
19
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
2)Investment criteria also depend on factor that company has good sales which is shown by
scatter plot.
3)To generate capital the company should have the combination of the following attributes such
as value, employment, sales, and patents.
6) Company should provide more employment as increase in sales can only be done by hiring
more no of people .
7) As sales increase , definitely any new firm will be highly interested to invest
Ques 2.1) Data Ingestion: Read the dataset. Do the descriptive statistics and do null value
condition check, write an inference on it? Perform Univariate and Bivariate Analysis. Do
exploratory data analysis.
Ans: We have successfully imported all necessary libraries and also checked latest versions of it.
We will read the file and make analysis out of it.
Bottom 5 records:
We will drop the unwanted attribute Unnamed :0 as its not adding any value to the dataset .
Let’s check the top 5 records after dropping unnamed attribute from the dataset .
20
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Bottom 5 records after dropping unnamed attribute from the dataset .
21
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Checking unique value in the dataset:
74:58:1 6
73:110:1 6
22
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
78:2:1 6
74:74:2 6
73:100:2 7
Name: caseid, Length: 6488, dtype: int64
We will impute the missing data and check if no values are left in the dataset by imputing it with
median value .
23
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Univariant Analysis and Bivariant Analysis :
Dist. and Box plot of different attributes of dataset:
Fig-9 -Problem 2
24
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Fig 10-Problem 2 -Pair plot
Let’s plot graph for the car crash dataset with outliers before scaling it .
Fig 11
Let’s plot the graph for Car Crash dataset without outliers after scaling it .
25
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Fig 12
Inferences :
1) Shape of the dataset is 11217, 15
2) Unnamed :0 attribute of the dataset needs to be dropped as its not adding value to the
dataset.
3) There are some missing values in the dataset which is imputed using median of it
4) There are 3 attributes of data type float64, 4 attribute of data type int64 and 8 attributes of
data type object
5) There is no duplicate record in the data set .
6) Outliers are present in the data set which need to be treated .
7) There are some unique values in the dataset which needs to be treated for further analysis .
Ques: 2.2) Encode the data (having string values) for Modelling. Data Split: Split the data into train
and test (70:30). Apply Logistic Regression and LDA (linear discriminant analysis).
Ques:2.3) Performance Metrics: Check the performance of Predictions on Train and Test sets using
Accuracy, Confusion Matrix, Plot ROC curve and get ROC_AUC score for each model. Compare both
the models and write inferences, which model is best/optimized.
Ans: We have s We have split ted the data into train and test (70:30). the data into train and test
(70:30). We have applied Logistic Regression and LDA (linear discriminant analysis) on the given
dataset
taking survived as target variable.
26
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Logistic Regression Model:
Logistic regression is a statistical model that in its basic form uses a logistic function to model a
binary dependent variable, although many more complex extensions exist. In regression analysis,
logistic regression (or logit regression) is estimating the parameters of a logistic model (a form of
binary regression). It is used in statistical software to understand the relationship between the
dependent variable and one or more independent variables by estimating probabilities using a
logistic regression equation. This type of analysis can help you predict the likelihood of an event
happening or a choice being made.
27
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Fig 13
Accuracy - Training Data 0.9698127627053879
Fig 14
Confusion matrix on the test data
28
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Fig 15
Accuracy - Test Data 0.9705882352941176
Fig 16
29
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
LDA Model:
Linear Discriminant Analysis or LDA is a dimensionality reduction technique. It is used as a pre-
processing step in Machine Learning and applications of pattern classification
Fig 17
30
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
AUC and ROC graph for test and train dataset:
AUC for the Training Data: 1.000
AUC for the Test Data: 0.904
Fig 18
Comparison between LDA and Logistic Regression Training and testing dataset:
31
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
1)Accuracy Score 0.9954
F1 Score 0.9974
32
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
3) Accuracy Score 0.9955
F1 Score 0.9975
Fig 19
33
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Predicting the classes on the test data
Fig 20
Ques 2.4) Inference: Based on these predictions, what are the insights and recommendations.
Ans:
Insights:
The model accuracy of logistic regression on both training data as well as testing data is
almost same i.e., 97%.
Similarly, AUC in logistic regression for training data and testing data is also similar.
The other parameters of confusion matrix in logistic regression is also similar, therefore we
can presume in this that our model is overfitted.
We have therefore applied Grid Search CV to hyper tune our model and as per which F1
score in both training and test data was 98%.
In case of LDA, the AUC for testing and training data is also same and it was 97%, besides this
the other parameters of confusion matrix of LDA model was also similar and it clearly shows
that model is overfitted here too.
Score of Both Train and Test Data are coming near-by.
Linear Discriminant Analysis Model Giving Better Recall and Precision in comparison to
Logistic Regression.
Hence, LDA Model cab be considered further upgrading the model.
Recommendations: Overall we can conclude that logistic regression model is best suited for this
data set given the level of accuracy in spite of the Linear Discriminant Analysis that the model is
overfitted.
34
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited