« first day (929 days earlier)      last day (4208 days later) » 

12:01 AM
@ChrisS Look up a local Motorcycle Safety Foundation (MSF) course. Many states offer a road test waiver for completion of the course. I took the course here in NY, so after nearly a decade of riding, I finally earned my "M" endorsement.
@jscott The local community college has a two weekend class. For $200 you get to use one of their almost-new motorcycles, go through all the requirements, and the road test. As long as you're not a total idiot you've got a license.
@ChrisS Yeah! That sounds pretty much exactly like the MSF course here, except for the "almost-new" part. The same local CC also does a vocational mechanics program which maintained the bikes for MSF. They ran well, but they weren't anywhere near new :) -- Sign up with a friend or two, it's really easy and dare I say "fun".
@jscott There are a couple cycle dealerships in the area that partnered with the program, including a "graduate" discount on a new cycle.... I'm not normally a proponent of public-private partnerships, but sometimes they workout really well
@Cole Are the wolf and the polar bear in Nebraska?
12:15 AM
Whoa, WTH? 343 firsts posts in the review queue all of a sudden.
@HopelessN00b Probably a change of algorithm
Yup, time to break out the clue-by-four again uh, help some noobs learn how to use the site.
12:54 AM
argh. Too many fucking tests in there too
if the queue is large, leave out the timewasting crap...
1:05 AM
At what point do you use memory limits in VMware?
@ewwhite Are memory limits really necessary? You just allocate the RAM you need, if you're overcommitted by more than X% then you buy more hardware
Exactly...
so I have a VM... 18GB provisioned... 4GB limit. Client calls and asks our NOC staff to reserve all 18GB...
I look and say, "hell naw"... because there's a 4GB limit.
terrible
remove the limit and all should be okay
@ewwhite Ha! Yeah, that's terrible.
@ewwhite wtfux, why don't they just pay for an 18GB limit
Turning off dedup and balloon is a bad idea.
1:09 AM
This is a 14-host cluster.
I'm very much tired of people deploying VMs the way they would for a physical box.
This vCloud shit is encouraging it.
@ewwhite How so
@JoelESalas it abstracts things too much. The clients using it think, "oh, I'll make this 16GB RAM and 4 CPU..." and don't realize that the underlying hosts are 96GB and 12-core.
then sometimes they'll hot-add RAM to make it 32GB... or add another four CPUs.
because the vCloud makes it all seem like an elastic resource...
when in reality, very few apps need more than 2 CPU and the appropriate RAM.
@ewwhite vCloud (and VMware as a whole, really) are tailor-suited to terrible enterprisey apps that used to live on physical hardware
of course virtualizing these is a much better option, but I've found that most of the "good" vcenter features don't benefit a properly-authored application
So it's basically, "Can't reimplement your application? Can't refactor into smaller pieces? HAVE WE GOT A FEATURE FOR YOU"
@ewwhite VMware encourages it, too. The "recommended" (default) RAM alloc when you're making a windows 2008r2 vm is 4GB.
@ShaneMadden Need more? JUST HOT ADD!
1:15 AM
@ewwhite Don't you charge them per the resources they allocate to machines?
@JoelESalas Yup.
@ChrisS Ideally... but I'm still doing that, too! (been watching webinars all day on how to build the metering tools for vCenter Chargeback Manager)
@JoelESalas hot-add does some weird stuff
@ewwhite Oh my yes. It's inadvisable in all but the most emergency of unplanned, poorly-executed deployments
That's when hot-add works at all...
@MichaelHampton I really want to type hot-ass
1:21 AM
Yeah, hot-ass me 16 more drums of lube!
Does anyone elses chat keep raising @ notifications from hours ago?
@jscott Yeah, mostly it's deferred maintenance that I hadn't had the time, space, or cash to do. Spark plugs, air filter. (Have to take half the bike apart to go to the air filter.) The DL's been ridiculously reliable. Probably moreso than most of the other Suzuki models that I've seen.
I keep getting alerted to some random comment every few minutes
@ewwhite So are crappy third party vendors. Kronos is recomending 4 cores now for the has-to-be-dedicated-because-their-coders-suck-so-much database VM. And it looks like we're going to mark right off that cliff and follow their crappy advice, too.
It's annoying the shit out of me, trying to get real work done :p
1:26 AM
@MarkHenderson Because it knows you're, like, so random. Dude.
@MarkHenderson so randum xD holds up spork
@HopelessN00b that's a case for a CPU limit
give 4 cpu, but limit the GHz allocated
@JoelESalas Hah spork
@ewwhite I've never seen an ESXi host break the 50% CPU utilization level, it's always RAM that comes first
what are you guys running?? jeez
@ewwhite Nah, we have plenty of spare cycles, don't need to throttle the host CPU. It's just sad how awful their best practices are.
1:29 AM
@HopelessN00b We work with a certain vendor (who was acquired by a certain search engine company) whose software requires 32-bit RHEL 5
@JoelESalas Jesus, what a racket.
Saying any more is probably NDA territory but rest assured it could really benefit from 64bits.
Well, that and RHEL 6 has been out since 2010, should have been plenty of time to start supporting the current version...
@HopelessN00b YEAH NO SHIT
Like I said, what a racket. I clearly got into the wrong business, when there's that much money to be made writing shitty code.
Speaking of crappy code, is it true that Ubuntu's network manager only supports user/pass for 802.1x authentication?

That sounds especially heinous.
1:40 AM
how does that look?
@HopelessN00b If it is, that's an Ubuntu thing. NetworkManager is perfectly capable of doing WPA2-Enterprise/802.1x with certificates.
@ewwhite 10GB balloon at like 30% usage??
The VM had a 4GB limit on it.
I removed the limit and moved it to another host.
@MichaelHampton Well, it what some noob is saying a question, so I'm not gonna take it as truth without validation.
@HopelessN00b Probably the idiot doesn't know where or how to place the certificate.
1:43 AM
And then the client just sent me this from VMware...
@ewwhite The best practice for running Java is not to, in point of fact. Tell them that.
3
@ewwhite I really like the undefined undefined part.
@ewwhite Reserve everything that's allocated, AND making sure ballooning's working correctly? Yeah.. no.
There's NO memory pressure.
Letting TPS and memory compression work is probably good.
And ballooning into the JVM's space to make the OS swap is good too.
But yeah.. reservations with no memory pressure are completely pointless
1:48 AM
@ShaneMadden any problems they may have had before were because the VM was artificially limited to 4GB
Well, do you guys charge extra for memory reservations? Because I'd see a point if that's the case.
@HopelessN00b nah, not here.
this client pays over $300k/mo.
@ewwhite Absolutely.
Yeah, then pointless. Only purpose is for over-commitment scenarios (right?), and in that case you should be dropping in more RAM or more hosts anyway...
14 hosts at 96GB RAM each
1:50 AM
It's good to have the memory management stuff working for if you lose a host and get accidentally over-committed that way too.
@ewwhite Niiiice.
I love HP Support. Emailed them saying "where the fuck are my VSA licenses?", then after double checking the PO, realised we didn't buy any, got an email back from HP "Here you go!" and I have another 4 VSA licenses in my entitlement portal
@ewwhite Do you want to laugh at me?
@MarkHenderson The upside of incompetent tech support.
@ewwhite I've got something that might give you a chuckle, oh illustrious hourly contractor.
1:52 AM
I'm not that good at reading these specs...
@WesleyDavid k
@ewwhite Looks good to me... Worst case allocations being at ~80 of total, you could lose 1 host and be fine, maybe even 2.
F-ing VirtualBox.... RDP connections accepted on Port 3389.... It's VNC.
@ewwhite Just a sec - preparing screenshot
0
Q: Onboard RAID with Windows 2012

JeffI have SSDs in an onboard RAID array. I am running the Windows 2012 Server install, and it says "We couldn't find any drivers. To get a storage driver, click Load driver." I tried to do some research, but I couldn't find anyone talking about that error message with Windows 2012, only Windows XP,...

-_-
@HopelessN00b with no overcommitment
1:59 AM
@ewwhite So after working off some prepaid time from a client, this is what March is looking like for me. =P
@Cole Really, what the hell, people. You're staring at an error message about not having drivers, and a god-damn button saying "load drivers" 50 pixels below it... so you go to another computer, open up a browser and ask a a question on Server Fault?!?!?!
@HopelessN00b right
@HopelessN00b OT, not a professional capacity. VTC
@WesleyDavid Okay with that?
@WesleyDavid I dunno about OT. Going with too localized (FakeRAID + Server 2012).
2:02 AM
@ewwhite No.
Especially considering that it will be more than cut in half after tax estimate and monthly expenses for rack, phone, a virtual server, and a few other business knickknacks.
@HopelessN00b @ShaneMadden Client just replied again... Our app is still taking about 20sec to dump garbage collection and it is not FULL GC, this is not normal. Normally we have seen this happening if memory is not reserved. Can you please do reserve the memory. Application is still performing poorly this application does really need memory to be reserved!!!
@ewwhite I'd be tempted to reserve it shut them up, say "test now" and when it doesn't help, you have your excuse to pull the reservation back. Not like it matters much with your cluster's resources.
@ewwhite What's the memory layout for that VM look like now? Has the stuff that was ballooned switched over to granted?
@ShaneMadden Well, I think the memory layout is still messed up. They'd need a reboot of the virtual machine to correct, no?
@ewwhite The balloon will sometimes not let go even if the memory pressure (in this case, the limit) is gone.
Check and make sure on the performance tab -> advanced -> memory, but yeah a reboot will reset the balloon and let their jvm go nuts
2:08 AM
The layout looks like this...
@ewwhite That's not too bad.
But they still need a real VM power off/power on?
Well, the balloon's gotten way smaller. And a balloon shouldn't cause slow java GC, it'll just cause pressure and maybe more often GC. I wonder if their slowness complaints are from earlier
Well, I take that back.
The balloon would cause the OS to swap JVM memory, which could definitely slow down the GC. A restart of the JVM would fix that.
and now...
@ewwhite That'll work. The only thing slowing them down now would be if the OS still has some JVM in swap.
2:17 AM
Well, we'll see. I guess I'm a dick... but I hate the idea of reserving memory when it doesn't seem necessary.
the only good reason to reserve RAM is when something like PCI-passthrough requires it.
@ewwhite Yeah, it's completely unnecessary if you don't have regular contention. And an unnecessary reservation can really shoot you in the foot when you actually need the contention stuff to work properly, like when you lose have your hosts for some reason.
but this is a daily challenge... I have NO idea why there were RAM limits on the VMs.
@ewwhite I can't think of anything where they'd get put in place automatically
@ShaneMadden my coworkers.
@ewwhite That'd do it.
2:22 AM
I think they deploy from a template, and the template had a RAM limit.
@ewwhite Can you complain to someone about that kind of tomfoolery?
@ewwhite Well, that is the smart way to do it... except for the reservation part.
Allocate 16GB, limit to 4GB, reserve 4GB?
@WesleyDavid Our NOC staff handle a lot of the day-to-day, and they haven't had formal VMware training.
@ShaneMadden Yeah, that's pretty assbackwards. If that's in the template, it needs fixing.
2:24 AM
@ewwhite Fiddling with the sliders is fun.
Now the client is calling our NOC
"We want RAM reserved!!!"
Geezus, we're ordering some 4G Unidirectional antennas, and they're called "G-Spot" - my boss is on the phone at the moment to his wife saying, very loudly, "I need to buy four G-Spots, which credit card should I use? What? G-Spots! GEE SPOTS!!". The walls here are thin, I just know the neighbours are listening
@ewwhite Ugh.
They want me to call them...
"while he appreciates your suggestions, what they need done is part of best practice according to vmware knowledge base and his softwares best practice"
@ewwhite Well, why not give it to them? Either it will actually help (unlikely, but who knows?) in which case, you should be doing it for them, or it won't, at which point you can say see, it doesn't work, the problem is with something else. Like maybe Java sucking."
2:28 AM
@MarkHenderson "No, we can't find your G-Spot! You're going to have to use your old method... What?! You're breaking up - your G-Spot is fading! JUST PLUG IT IN THE WALL AND FORGET YOUR G-SPOT!"
Seriously, why would you name something like that G-Spot? That's just dumbheaded.
Because the logic is faulty.
@WesleyDavid "No, you hook the RED wire up to the G-spot, then plug it in!!"
@HopelessN00b they had a 4GB RAM limit and wanted a 16GB reservation...
@HopelessN00b "Your G-Spot fell off and we need to send you a replacement!"
@WesleyDavid I dunno, its sort of funny
2:30 AM
@MarkHenderson Can I see your G-Spot?
@ewwhite Sure. All the more reason to do it - you know if you configure it their dumbass way it'll only make things worse. Do it, let them see that with their own eyes, and they'll be more open to doing it right. (In my experience.)
@WesleyDavid Sure but it'll be $200
@HopelessN00b Yeah, but I don't give up that easily...
@WesleyDavid: hmm, got a moment?
(and god, if i EVER run into you, I need to get you MANY coffees)
@JourneymanGeek Just a sec - gotta reboot. OS X is pinwheeling itself to death.
2:33 AM
@ewwhite It's not giving up, it's building consensus, getting them on-board. Once their idea fails and they can't hold onto it anymore, they'll be more open to other ideas... like doing it the right way. Which you'll provide them.
I manage ~350 VMs for them. There are NO reservations anywhere else..
... OS X... pinwheel of death? You obviously need to make a sacrifice to Saint Steve of Copertino ;p
@ewwhite Right. Another reason their logic is faulty, but that doesn't necessarily help convince them they're wrong.
Does anyone else get a sense of satisfaction watching firewall logs and seeing hundreds after hundreds of dropped requests?
No, I get a desire to make reading the firewall logs someone else's job.
2:41 AM
@HopelessN00b Oh I don't read them, I just sometimes like to watch the red flow
@HopelessN00b Ugh... Error when I try to reserve.
@ewwhite ...oooof course. What's the error?
@ewwhite What's it whining about?
Insufficient resources to satisfy HA failover.
@ewwhite Hah. Turn down admission control on the cluster?
Well, hmm
2:43 AM
I'm checking
Slot size is based on the largest VM for worst case.
So you'll probably need to turn off admission control completely
The Admission Control is set to 1 host tolerance.
moving to % based.
Yeah, but right now the slot size is probably a few hundred MB, since it's the largest (reservation (0) plus memory overhead) total on a VM
With a 16GB reservation, the HA admission control starts figuring that each slot is 16GB. You'd need hundreds of hosts for admission control to be enabled; you'll have to just disable it.
I just set 25% of resources for admission control
and that worked.
defeat... @HopelessN00b
Interesting.. that changes the slot math?
2:48 AM
Yeah, it might be, having to reconfigure the whole cluster for their dumbassery. :(
Watch it burn!
Ahh, yeah, the slot size math is just for host failures mode. Got it.
Shrink the reservation to more like 8 or 10, at least.
I went to 20%... yeah, watch them not be able to turn anything on
You had transparent page sharing saving you 3.5GB of physical memory
@ShaneMadden But they NEEEEEEEEEEED all 16 GB reserved!!!!!!!
2:50 AM
I never fully understood admission control
So can you please reserve the memory for this one VM? And if you have any warning for this host can you please turn them off as no doubt you will see some warning if host doesn’t have enough memory to cater for whatever VM this host have. However if there are production VM was hosted on in host please DO NOT turn of any warning or alerts.
@HopelessN00b Waving around a "best practices!!!!!" banner with no understanding of why. Grumble grumble.
@ShaneMadden Yeah, well, we do that on an hourly basis, so I'm used to that.
@HopelessN00b Ouch.
So I turned off all warnings and such :)
Let it burn
@ShaneMadden <sob> It's actually one of the less painful bits of stupidity we engage in.
2:55 AM
Hmm. I'm enjoying doing interview coding examples. Perhaps there's a future for me in DevOps after all.
@Adrian I struggle... but I've done a couple of decent coding samples.
@ewwhite This one's easy, so I've spent the rest of the hours building argument handling and error checking into it. Trying to decide if I should send both and say the first one was whipped out in 15 minutes and the pretty one was about a half hour more.
Is your DevOps missing some Ops?
@ShaneMadden sad thing is that i'm deploying my new clusters with LACP enabled on VMware.
@ewwhite Well, it certainly doesn't hurt, just adds some extra switch config work.
Just not gonna help when all your traffic has the same source and destination.
3:06 AM
@ShaneMadden Well, I have LACP enabled on my dvSwitches now... since the feature became available in the fall.
@ewwhite Nice.
Can we push more than 1GbE? yes... to storage... vmotion streams...
But I'm just getting fancy now.
@ewwhite Gotta get me a license with dvswitches one of these days
@ewwhite Network IO control! Too fancy!
It makes sense with 2 x 10GbE per server.
the common spec at work is 6 x 1GbE per server...
3 x static bond + 3 x LACP
which is a little rough
and they don't want me turning NIOC on at work... so we wait...
3:17 AM
10gb gear needs to get reasonably priced.
@ShaneMadden I was ghetto... and a bit of an early-adopter... so I'm on CX4 10GbE.
$200/card, used cables (Infiniband cables) and I'm just on HP 5400zl switches.
@ewwhite Heh, fair enough
0
Q: Gparted not detecting controller after removing a drive

Stanislav PalatnikRan into a weird scenario. Installed 5 hotswap HDD drives on a HP SmartArray P410 RAID controller running RAID-5. Went into Gparted and created different partitions. Realized one of the drives was faulty. Removed it. Rebooted into Gparted. It now cannot detect the controller at all, saying 'can...

@MichaelHampton I truly don't understand how people have so much trouble dealing with RAID.
Raid seems pretty easy to me.
3:32 AM
@HopelessN00b I spent my entire career with HP gear, so i take it for granted...
Not the RAID thing, but same how-effing-hard-is-this,-really???? situation.
P.S. I need to get this project done ASAP! My emergency, off-hours consulting rate is $300/hr. Cash or charge? — HopelessN00b 1 min ago
but when i see people choosing between controllers... or purchasing 3ware... or anything that's not a purpose-built/prebuilt setup, I do suspect that they're taking a more difficult path than necessary.
@ewwhite Well, I've been mostly Dell. But any server gear uses RAID, and even workstations have been doing it for many years.
Because it has a red LED light and is making crigeworthy noises. — Stanislav Palatnik 2 mins ago
@MichaelHampton Yeah, he might be right about the drive being faulty.
3:36 AM
And a P410 controller means Nehalem or newer server... so 2009 and later.
So there's a good chance he's within warranty.
Really? You came to Server Fault for help with an Excel formula?!
Well, that wouldn't be so heinous if it wasn't such a simple one.
And now for the delete-that-shit vote.
@MichaelHampton Good. Now go upvote 181 of my answers so I can do that without waiting too. :p
I love when the answer to a school question is Meterpeter. >=}
4:19 AM
@HopelessN00b Answer more questions!
@MichaelHampton I'm not that motivated to get to 20k. Besides, it's getting harder and harder to find a question to answer that I both know the answer to... and also don't find contemptible.
It's Business, It's Business Time....
@HopelessN00b Ha. I have whole series of days like that.
4:57 AM
I'm almost proud of the fact that my 20 year old DOS game crashes VBox
 
4 hours later…
8:41 AM
Morning all!
any curated/bookmarked URIs on burn out/extreme exhaustion?
@Sudhi Just pick up your favorite type and brand of alcohol and get hammered. You know, like the rest of the world does.
Hater's Gonna Hate.
3
Q: How to Move a Question to Serverfault.com aka Close and Move Broken?

Chris MoschiniI've noticed many questions that belong on other sites are just closed as Off-Topic with no comment or explanation as to where in the StackExchange universe they belong. This lead me to assume there is no move (migrate) feature, but in fact there is. So I saw a question that belongs on ServerFau...

2
Q: Irregular internet disruption: certain images and JS not loading

Vincent Veldkampfirst time on ServerFault, and I've got a nice little conundrum. Since a few months now, we've been having issues with our internet connectivity. Environment: Servers: 2 Terminal Servers as an RDSFarm running Windows Server 2008 R2 Browser: Internet Explorer 9 Test/debug browser: Chrome AntiVi...

If anyone can give me any more insight in this matter, it would be greatly appreciated! Been at this problem for over half a year now.
@tombull89 Why does that link take me to Mark's "kill the migration path" question????
8:53 AM
@HopelessN00b marks question was the one that was closed as dupe of
@tombull89 Oh, lol.
@VincentVeldkamp What more are you looking for... you self-answered, doesn't that mean you figured it out?
Well, not really as I thought this was the answer, the problem came back after a while. And since the entire process was handled in the comments on that answer I didn't know if deleting that answer would be appropriate
Ah, crap. If you don't want to delete the answer, you should at least edit it to reflect the fact that it's not the answer.... will also bump your question to the front page and get it some attention, so that would be a plus.
Thanks :), I'll edit it right away

« first day (929 days earlier)      last day (4208 days later) »