Discussion between Toros91 and KK2491

Feeds

07:21

1

A: Evaluation metrics for Decision Tree regressor and KNN regressor

Generally when ever we are trying to compare between models and to choose the best one, we go for other metrics like AIC, BIC, AUC(this is not applicable as it is used for classification algorithm) etc along with $R^2$. Now why are they important criteria because AIC tries to select the model th...

KK2491

Thank you so much for the detailed explanation. I got some real useful insights now. I will try to derive the AIC and BIC, will update you soon.

Toros91

sure, I've appended the links with respect to formulas on how to derive etc.

KK2491

Thank you one more time. AIC and BIC should be calculated only on Test data set right ?

Toros91

yes, in this Link you can see how can you derive them

KK2491

I have derived the AIC and BIC values and just updated the question with the output. Could you please have a look. I have calculated all the values for Test data, so n = number of samples in test data

Toros91

07:21

Yes you did it well and it is right. So you can choose the model: Decision Trees, as it has the least AIC and BIC values along with $R^2$.

But if the model is over fitting then I would suggest you to use, Cross Validation, K-folds. So, you can use all such techniques to generalize the model better or to make a robust model

KK2491

Thank you again. You are right. It does seem like decision tree model is over fitting very badly R square 99 %.!. I will try to implement Cross Validation, K-folds and update the output.

Toros91

Once you update the question, then you can ping here. I will check and update you.

my interpretation

KK2491

Hi Toros91

Toros91

hey

How are you doing?

KK2491

I am doing good. How about you?

I am kind of new to this site.

Toros91

07:30

I'm good too. Thanks for asking

KK2491

In case of decision tree, i think i need to prune the tree to avoid over fitting

Toros91

hmmmm no problem, you are being active and doing well.
So there is nothing to worry about.

yeah you need to prune tree, you need to add some miscellaneous cases(noise) to see how the model hadles

*handles

KK2491

oh k. I will try that out. and update if I end up in confusion.

Toros91

sure sure

KK2491

I really appreciate your help and thank you for the guidelines

Toros91

07:32

will try to help you as much as I can.

KK2491

Thank you again.

Toros91

No problem, you need not say thanks those many times. :)

KK2491

Just one doubt, in case of classification problem how the AIC, BIC and AUC is calculated ? If you have any link it would be really helpful

Toros91

sure would share in sometime

KK2491

yep.

Toros91

07:37

I think AIC and BIC are not used in Classification as they work based on the error values but I'm not sure. Will check and update you.

AUC is used used for classification to explain how much area is covered in the graph

KK2491

oh k. got it. The area under the ROC curve correct ?

Toros91

yeah

https://stats.stackexchange.com/questions/132777/what-does-auc-stand-for-and-what-is-it

Explains you very well about the AUC and explains with an example

I've see couple of sites where AIC and BIC values for Logistic Regression, so give me sometime will do research and update you back

KK2491

yea. ok. I will have a look. :-)

I am preparing a document for better understanding of the ML algorithms. Getting to know new terminologies and metrics now .

Toros91

AIC and BIC are used in model selection, For Example: you have 4 features then you can build a model with 3 features, 2 features and 1 feature and classify. Based on the AIC and BIC values would suggest you to choose the best one

https://pdfs.semanticscholar.org/presentation/6158/8549a553bfe1b480a4104e9a57757740a3a8.pdf

you can go through this PDF for better understandin g

*understanding

Do let me know if I'm confusing you or something, would try to explain you better

KK2491

07:57

AIC and BIC are used in model selection -- only for classification ? or this applies for regression as well ?

Toros91

they are applicable for both

KK2491

oh k. Irrespective of the model we are using correct ?

Toros91

yeah most of the cases.

in the link which I've share with PDF, in the you can see how the prof explained.
The links I provided in the answer explains you about how it is applicable to the regression models

KK2491

yep. ok. I will go through that thoroughly.

Toros91

yeah

good

you can understand clearly

In DS nothing is certain, you might have seen this. A Model might do for well with your data but the model might not be performing well to other similar data. So, you need to perform all the basic checks and choose the correct algorithm

So, you need to be patient and take it slowly by understanding the things which you come across like materials and knowledge bases etc.

KK2491

08:11

yea. You are exactly right. Appreciate your suggestions :-)

Toros91

:)

All the best and let me know if you have any questions

KK2491

yea Sure.

How about this chat ? this will be active always ?

Toros91

nope, It is inactive for 10-15 days it would be archived and deleted.

*if it

KK2491

oh k. that is sufficient .. anyways i ll be in constant touch with you. asking more and more doubts.

Toros91

sure, I'm here to help you by sharing my knowledge

:)

If you don't mind me asking, why din't you accept the answer for this question?
https://datascience.stackexchange.com/questions/30453/linear-regression-in-python/30454#30454

you din't like either of our answers?

KK2491

08:37

sorry ... i missed it ..

Toros91

the reason why I brought it up is, wanted to complete the answer if you need any additional information.

KK2491

i apologise .. It helped me a lot .. with the help of those answers i implemented some of the models ..

Toros91

oh is it, then it is fine.
:)

you don't need to apologize man

it is fine

KK2491

:-)

KK2491

09:05

sadanand-singh.github.io/posts/svmpython

Can you please have a look at that ..

I am confused whether he is doing binary classification or 3 class .!

Toros91

ohk will update you

KK2491

yep.

Toros91

He is classifying into 3 classes

SVM for Classification Problems
The iris dataset is a simple dataset of contains 3 classes of 50 instances each, where each class refers to a type of iris plant.

this explains it

KK2491

oh k

But by default SVM kernel will separate the samples based on the number of kernels ?

Toros91

The scores are computed on the full evaluation set.

precision recall f1-score support

0 1.00 1.00 1.00 12
1 0.73 0.92 0.81 12
2 0.91 0.71 0.80 14

this output explains how it is performing and from that we can derive the accuracy of the model

KK2491

09:12

oh k

but in the same article he mentioned if we have to go for multi-class then we need to do one vs all .!

Toros91

(1+.73+.91) / 3 = 0.88

and so on

@KK2491 where is it?

KK2491

09:28

SVM by definition is well suited for binary classification. In order to perform multi-class classification, the problem needs to be transformed into a set of binary classification problems.

Toros91

yeah

but he implemented this for classifying them into 3 classes

so this is multi-class classification

if you see in the graph you have 3 color dots
1. Red
2. Orange
3. Grey

each color represents a specific class

KK2491

10:04

yea.. exactly .. that's what confused me ..

In the script where it is mentioned .? do we need to mention it explicitly ? or will be taken care by the SVM ? As I understand SVM does binary classification

Toros91

10:26

yeah you need just say what you are using SVM for, like for classification or prediction

rest it takes care

KK2491

oh k.. I wll try to implement a problem and update you ..

KK2491

11:10

hi Toros91

Toros91

12:03

hey

what's up!

KK2491

13:29

I have just implemented one binary classification using Logistic, KNN, NB, SVM and Decision tree.. but none of them giving good accuracy..

Can you give me some tips to improve accuracy .

Logistic Regression - could not find a function to change the threshold to classify in python

tried normalizing the values .. but no improvement

Sam

13:43

Room for a little one

KK2491

14:17

Hi Sam.

Sam

14:55

@KK2491 Hey how's it going

Transcript for

Discussion between Toros91 and KK2491