@badp It looks like a good approach. However, neither the question, the tables, nor the reply can be fully understood except by those who play the game (that rules out me and probably the vast majority of participants here). Incidentally, confidence intervals answer the wrong question. You probably want to estimate the probability distributions of the costs that will be incurred (e.g., use tolerance intervals or prediction intervals).

@whuber I should've been clearer about the tables, then. I apologize

@badp No problem; I'm not complaining: the site is for videogamers, not statisticians! :-)

Hello

Anyone uses SPSS here?

Hi @Tom, what is your question?

@AndyW well I used bootstrapping to bump my model's accuracy from 20% to 65%. I have about 200 cases for 120 variables.

Do you think this model is really as accurate as it claims?

I used SPSS.

@Tom, what is accuracy? I encourage you to formulate a question on the main page. It does not appear SPSS has anything to do with your question, and so potentially many more people on the forum would be able to provide advice.

@AndyW it's just one of my many questions

I was hoping to have a chat with someone who is experienced with SPSS

(backspace button made me leave this chat as input was not focused, back now)

Why does the person need to have experience with SPSS? That would only help with programming related questions.

@AndyW well, for example, I'm using SPSS's model linear generation function and while its producing accurate models it does not seem to remove insignificant variables

I was wondering why not and how I can make it remove them

This is getting a bit out of my ken, so I will unlikely be able to help much. I encourage you to post a question to the main site.

Alright, thanks anyway

I'm not sure what the "model linear generation function" is, but what is the general requirement for a variable to either be included (or excluded) from the model?

@AndyW well it should have a significance of less than 0,2

The function is available at Analyze ==> Regression ==> Automatic Linear Modeling

I don't have such a function in the version at my desk right now (17). Are you using forward/backward step-wise regression? You could always just make the significance threshold arbitrarily lower.

@AndyW I'm using version 20. It uses "forward stepwise". I'm afraid there is no option to set a treshold

In the "normal" regression procedure there is an option to select the inclusion/exclusion criteria for variables in the model. See the CRITERIA subcommand for the REGRESSION function.

@AndyW alright. By the way do you know why it removes constants from the equation? I'm trying to insert dummy variables but it just removes them

In the question you just posted it would likely help if you post the syntax used to generate the model. Why would you want to include a constant in the equation, it won't help in predicting anything?

@AndyW of course. I have to multiply the dummy variable by another variable. I'm being a bit stupid. Thanks

You don't happen to know if SPSS can automatically make these interaction variables?

I cannot do them manually because I have so many variables

and I do not know which variables are important

I don't know about "automatically" in the regression equations, but you can make a series of interaction variables from the dummy variables pretty easily using DO REPEAT commands and LOOPS if necessary.

That would be a good question for StackOverflow, and such programming questions are more on topic over there than on the stats site.

Alright. I'm not sure if there is a max to the amount of variables I can have though. Eg. if I have 50 variables and I want to test for interaction between all of them that's another 50 * 50 = 2500 variables.

Well, the number of cases you have is typically considered the "max" number. Although I believe there are techniques for when the number of variables is larger than the number of cases.

Perhaps only focusing on variables whose main effects are significant would be a way to limit the number of interaction variables in the model. Although, again this is out of my ken and likely you would get much better advice from formulating a question on the main site.

I see, thanks

@AndyW do you happen to know when one should typically worry about overfitting? I'm wondering if I should do some tests to make sure that my model isn't accurate because of overfitting

If the goal is prediction I believe you should always be worried about over-fitting. And when the number of cases is low and the number of variables is high it makes it more difficult as well I believe. I don't how such tests would be done in your situation, but frequently one has a sample in which they estimate the models with a subset of the data, and then apply those models to a hold out sample not used in creating the initial model.

@AndyW that's smart, thanks a lot