EngrStudent

There is a new user question that likely needs to be migrated to stackoverflow. stats.stackexchange.com/questions/639127/… I don't think they know how to ask for help in this chat.

If someone has a request for an administrator to move a question from CV to SO, would they post it here?

@gung - fwiw, I made my best attempt at a set of models that span the space of important predictors by one-at-a-time excluding columns, then using glmulti and aicc to make a predictor. I am now trying to determine how well they handle small perturbations to the "invisible physics" of those indicator columns.

Boruta uses ranger, so its pretty fast. It can also handle very very big arrays using the extra-trees, but it requires extratrees package and abotu 5x multiplier on number of trees.

If the "field index" is too unique then it becomes a falsely informative variable. If you put row index in, and there was a clean hash to convert rownumber to label then a RF could memorize it. This is why I like Boruta, because it can tell you importances, and then the person with domain expertise can determine whether the values are reasonable and can detect falsely informative variables.

So you can augment the input so that it is [numsamples, numfeatures+2] in size, and the 2 augment columns are GPS coordinates.

best of luck.

You could also put an independent index (factor, not numeric) showing they have the same field

Sure, put the xy coords of the pixels into the inputs.

You can nearly always win by starting simple and working toward greater complexity. You can understand easier. They run faster. They provide decent checks for higher-complexity systems. There is broader literature. The software is more mature, more global, and more likely able to be understood by someone you need to communicate to.

:)

Sounds like you are having my kind of fun! I hope it is a blast.

all pixels are average counts of viable photons incident on the detector in the camera. Often the hardware uses weighted values in a neighborhood of a few pixels (~5) to smooth the pictures and make it less noisy.

convolution can allow subpixel accuracy

h2o has randomforest but I don't know if it is on gpu

which RF package are you using?

why do you need gpu with that small of data??

I want things to be perfectly perfectly accurate, and I think you might too. The more samples you put into train means you get that. But I do not want to fool myself into thinking I have a decent or great model when it is junk on new data.

yes.

Not the same. There are models in the RF ensemble that have seen the training data.

Here is a more recent reference: link.springer.com/chapter/10.1007/978-3-030-56485-8_3

If I have a 2-term linear model then I want to have something between 10 and 70 samples to fit. Personally I like to start around 300 but there are thresholds of uncertainty when you go from above 35 to below 35 and then when you go from above 5 to below 5.

I don't know your model. Some folks like OOB for RF but there are nuances (aka error to be had) there. In my personal opinion the split-then-fit makes it clearer that there is no bleed of test into training to corrupt your fit summary statistics.

In the theory there is "in the limit of infinite samples" but the real world imposes constraints that are more aggressive. I would look at leave-k-out cross-validation, but I would also be very very cautious of overfitting. I like somewhere between 5 and 35 times as many samples as I do parameters in the fit as a first rule of thumb. I would randomly split into train/test at 90/10, fit the model using train, test using test, store test value, and then repeat something like 100 times. I would look at the variation in the score across replicates as well as its mean.

I like the kappa because it is about agreement between raters. Personally I like to use the Boruta package because I can say "permutation importance says these are important". You need to make sure to have a train-test split where the model never sees "test" at all during training, and it is used to estimate performance in the presence of new data. You can repeat 100 times for random splits in the training then first look at variation in non-trained fold, then second test against the, pre-separated and never-touched, test to see how it fits in the distribution.

what do you mean by accuracy? How do you compute it?

Gotta run. I can talk with you another time if that is okay.

I'm sad you have to deal with that. I've worked for narcissists. It's hell.

@anongoodnurse - (mental illness) I don't know what NPD is but I grew up around a few breeds of mental illness (PTSD, Manic-Depressive, …)

@anongoodnurse - (rules) sounds like a version of Godel's incompleteness theorem. The rules are incapable of being sufficient, because humans qualify as fundamentally complex enough. Godel proved that if the system is above a water-line of complexity then the rules can never actually do what they are intended to do.

@anongoodnurse - There is a context around the ear-ring. Respect. Defiance. Sanity. Maybe your sister didn't have the resources to maintain homeostasis and engage, and was in her view a life/death battle. Maybe your sister was mentally ill, and needed professional help. I can't know, having not been there, and not being a party, or a professional.

@anongoodnurse - I like decorum, and I consider it very important to engage as much as possible in a one-on-one manner first. There is something to be said in that somethings can harm the community if it is initiated in public and decided in private. The needs of the many outweigh the needs of the few or the one, sometimes.

@SomeShinyObject - I had to say that to my 14 year old little sister when I was 24, her guardian, and she was addicted to Crystal Meth. It sucked. It was a last-line statement. It came just before the realization that my support of her in any way was enabling her suicide and the only way to not be complicit was actually to kick her out, which I hated, and I had to do.

Is star equivalent to up-vote?

I have no issue. The folks I asked answered my question by engaging genuinely.

I don't understand chats. I don't know what value comes from changing "rooms".

goodreads.com/quotes/…

@anongoodnurse - Humans judge themselves by their actions and others by their intentions. I guess it's because we have limited objective access to the intentions of others.

@anongoodnurse - I felt previously like parenting wasn't safe for me. I haven't engaged or contributed for a while. My last answer was in 2015.

@anongoodnurse - I am not trying to pressure anyone. I think that by engaging and answering they are .. engaging and answering, which is what I would like to see in a moderator.

@Stephie - I think you just gave me the biggest incentive I’ve ever heard to not become a moderator. :)

@Stephie - I don’t know that the two are separable. It’s a western idea that we are connected compartments. The eastern ideas that we are holistic beings, and every part touches every other part; every big thing shows itself everywhere in little and big ways. That’s just me disagreeing with the philosophy. I like that you were engaging on this and you are thinking on it.

@RoryAlsop - Fraught. I suspect every human wrestles for their entire life with “successful person “. It is big, and complicated. It is probably a bigger thing than Any individual is. Around 10,000 years ago we were still fighting this battle, and there is no “winner” yet. I personally think that victory is not the Destiination, but the process.

As parents we are given the incredibly vague and incredibly important task of “raising our children to be contributing members of society” and “raising our children To be successful persons”. Aren’t those fraught goals?

I was thinking of this as an analog to “who would I let babysit my kids?”. There are county requirements for the care of my foster to adopt daughter. They want an adult, but not just any adult. Safety, wisdom, and engagement are all things they want there.

I think that this is like a "weighted coin" where when it is near 50/50 it takes fewer samples to be sure about the weighting, but when it is near 100/0 it takes more samples to get to the same level of confidence. In our case that may be a measure of contiguous area, or pixels.

one sec.. my kiddo got a cut from the cat.

You could look at eye color and hair color