« first day (1292 days earlier)      last day (3975 days later) » 

15:01
@RyanRies No, because I'd have to live in Texas
:D
Even though this job is stressful as hell, I think probably the only thing I'd leave for is Azure :)
But you never know, my mind can be changed
The biggest decision I have to make today! What to eat for lunch.
@RyanRies Interesting.
However, the only place I'd probably move to in TX (and have been kinda looking) is Austin.
@MDMarra can any of your sccm goons answer any of these? serverfault.com/…
Yeah I feel ya... other than a relatively inexpensive cost of living, we don't have much going for us down here :(
15:04
I hear Austin is pretty cool so
in particular this latest one serverfault.com/questions/583149/…
@BigHomie I just pretend to know SCCM. I mostly do SCSM and SCO work these days. I get my SCCM info from guys that work for me, as I QA all of their design docs before they get to the customer :)
As Patton Oswalt described it, Austin is the one little bubble of sanity in this state. :)
@BigHomie I took a whack it, got all frustrated and "fuck PowerShell"-y, and then decided to drink a bunch of scotch and take a "nap."
@MDMarra I kinda figured, that's why I mentioned your goons
15:05
Does Server 2012 still have shitty file server performance like 2008
@BigHomie this is true
My goons don't Server Fault (as far as I know)
I barely answer anything anymore. I get too frustrated
@MDMarra I'll be deploying scom on our desktops soon so expect more questions on main about that
@Travis Define "shitty" SMB3 is fast enough to host SQL Server databases and Hyper-V VMs
@BigHomie woah SCOM to the desktop. That's hardcore.
What's the driver for that? Just super-proactive maintanence?
@MDMarra fanboi
@MDMarra Since migrating our file shares from '03 to '08 it takes 45 seconds or more for explorer to browse to the UNC path
15:07
@MDMarra yep, need some proactive monitoring, and the terminology has a serious learning curve to me
@Travis DFS or anything?
@Travis That's not normal for any version of Windows. You broke something :)
@Travis that's a problem.
@MDMarra are they behind the curve or do you keep the treadmill on 10
@BigHomie 11
15:09
lol
@NathanC No DFS that I know of
@RyanRies Seems like an okay place for me to be.
@BigHomie Well, we've hired 5 guys in the last 8 months
@MDMarra Wish I knew what it was.
So we busy busy
15:10
Dayum, 5 guys?
and now when using the "search" box in Windows 7... it takes forever for stuff to find
File shares arent indexed by windows by design
@HopelessN00b It boils down to calling a wmi method from powershell, but I can't find the object that has the method, and/or don't know why the method isn't showing up on the object its supposed to
you can install the windows search service server-side or add the shares to a library client-side to enable indexing
@Travis is your search service running?
15:12
@MDMarra Right, but just typing "calc" or "compmgmt.msc" yields piss poor results
@cole SCCM, VMM/Hyper-V, Lync/Exchange, SCOM, and there's an AD guy in the works
@Travis oh, on the server?
Yeah dude, something is busted
@MDMarra No, workstation
@MDMarra Should be. Never turned it off.
@Travis It will still pick through the server files, which is what's slowing down your search for something as simple as 'calc'
@MDMarra Damn dude.
The only change that was made to my workstation was I mapped an R: drive to a network path...
15:13
Enough of this Windows talk! Where are the Linux people?
kill -9 @ewwhite
Hmm...a good two dozen people just walked by my desk ...people from a company we do a majority of business with lol
@Tanner So it has something to do with network shares being slow also...which leads me back to my original problem
All this crap stemmed from migrating a SQL database over to a VM from a physical... I just can't seem to find any bottlenecks... Nothing in performance monitoring stands out as bad.
Tried wiresharking things?
15:17
@Travis If you punch in `\\shittyfileserver\somefolder` , does it connect immediately and slowly populate the view, or eventually connect and instantly displays the contents?
@Travis nm
@Tanner Eventually connect and instantly displays. Then a few minutes later I can punch it in and immediately populates
annnnd....bag of popcorn devoured. stomach demon satisfied
@HopelessN00b Not yet. @BigHomie Yep
15:18
@Travis and I'm assuming you looked @ disk write times?
Yeah, I think @hopeless has it right, bust out wireshark
@Travis the database is on a vhd? ssd?
@BigHomie It's on a SAN, 600GB 15k SAS
ahh, phew
RAID 10
10 drives... So IOPS shouldn't be an issue
15:20
are the network shares on the same SAN?
@BigHomie Yep. All of the VM infrastructure is on the same SAM. Different LUNs
@Tanner What am I looking for in wireshark as an anomaly?
What kind of VM? Can you easily check what I/O latency it's getting?
@ShaneMadden What do you mean what kind of VM
@Travis I'd look at name resolution first. Is it looking for the server using DNS or NetBIOS? Is it getting a response in a timely manner? If it is, take a look at the SMB traffic and see where it's hanging up.
@Tanner ok
15:24
@Travis What hypervisor it's running on
@Travis This might be handy if you want a packet capture server side: blogs.msdn.com/b/canberrapfe/archive/2012/03/31/…
and don't want to get funny looks for installing wireshark on a server... not that I've done that... or gotten funny looks...
@ShaneMadden ESXi 5.0
@Travis UPGRADE!!!!
@Tanner I'm the only one that can funny look wireshark on a server
@ewwhite To windows 2012? ;)
@Travis It'll track I/O latency in the performance tab - check and make sure it's not too high.
15:26
@Travis ESXi. There's no excuse.
@ShaneMadden You know this is my new crusade.
@ewwhite Laziness is always an excuse!
@ewwhite ? Oh, upgrades?
@Tanner I've installed wireshark on a couple servers...I have the portable one though
Install winpcap on start, removes it when done. no trace on the server
@ShaneMadden People don't upgrade their ESXi.
@ewwhite They most certainly do not.
15:27
@NathanC handy
There's no excuse to be on ESXi 5.0
I have it sitting on my network drive just for that :p
@ewwhite I like vSphere ?
@ewwhite I've still got two 4.1 boxes, heh.
@Travis it's a larger issue.
15:27
@ewwhite No, in all seriousness I did download the latest version yesterday and haven't put it on yet
@ShaneMadden 4.1 is understandable
5.0 isn't
@ewwhite I tried, but my manager said "no." Moron. >:/
@ewwhite How do you figure 4.1 is cool, but 5.0 isn't?
@HopelessN00b because there are significant barriers to move to 5.x from 4.1... Logicworks had dozens of ESXi 4.1 hosts in production.
but once on 5.x, there's no excuse not to keep shit updated
@ewwhite What's wrong with 5.0?
@ewwhite If I give you a phone number, will you scream that at the douchebag on the other end of the line?
15:30
@ewwhite I don't think it's understandable in this case. All the other nodes are the same exact build of 5.1, these two are some cisco setup for the phone system. Not sure why they didn't just put the phone VMs on the main cluster.
@Travis Other than it's not compatible with like Server 2012, for example? :p
@HopelessN00b Pfft... We'll be on Server 2015 before we upgrade ;)
@HopelessN00b yes.
13
Q: I'm tired of answering questions with: "Upgrade the firmware" or "Run updates"

ewwhite...or sometimes both! ESXi 5.1 crashing (bug fixed by updates) Installing XenServer 6.2 on Proliant DL160G6 freezes during "Installing from base pack" (firmware) Disconnected Network Adapter on VMware - only power cycle solve this (firmware) Unable to view sensor data in ESXi 5.x on DL380 G7 ...

Fair enough. I think what I need help doing is creating PSODs and crippling bugs/crashes on our older ESXi hosts. "Told you we need to upgrade..."
@ewwhite So you're saying I should run updates and upgrade?
15:34
@ewwhite "But we have 500 machines all running the same firmware, why does only this one have trouble?"
Hmm. Would that be an acceptable SF question? "How can I create bugs/crashes on ESXi? [old version]"
@HopelessN00b yeh if enough people upvote it
problem with a program... look at code.... THE HORROR. It's littered with different ways of checking for errors, but they all just goto error with no description of what failed. Makes me want to sacrifice a developer or two.
An alternative method is to panic the ESXi server's kernel, forcing a Purple Screen of Death (PSOD) and a resulting hard failure. You can accomplish this within the ESXi console by using the following command:
vsish -e set /reliability/crashMe/Panic 1
Hmm. I'm very tempted to try that.
@HopelessN00b Yeah, you can create PSODs... Send an NMI to the host from the ILO
@MikeyB That's troubleshooting
15:40
@MikeyB "Because your mother. Who cares? Upgrade."
@HopelessN00b test the NMI approach... it'll crash your servers :)
@ewwhite Yeah, I'm tempted. Then again, I'm outta here as soon as I can get out, so I also... don't care anymore, at this point.
Seems like just pointlessly creating work for myself at this juncture, I suppose.
@ChrisS haha error in <program>. If this continues, contact technical support.
@MikeyB Ever use turbostat?
@BigHomie The worst are [error]: contact your system administrator. I AM MY SYSTEM ADMINISTRATOR, BITCH!!!!
10
15:44
@HopelessN00b yeah that one too.
Dan
Dan
@HopelessN00b More to the point "I AM THE SYSTEM ADMINISTRATOR AND I HAVE NO CLUE WHAT'S HAPPENING"
@Dan That would be why they're the worst. If I'm contacting anyone, it's gonna be the developer(s). With a blunt object.
Dan
Dan
@HopelessN00b I'd prefer to contact them with a rocket propelled grenade
@Dan Expensive, and hard to get through customs, though.
My steel toed boots, on the other hand, get through customs every time.
@BigHomie Almost... ERROR: Could not initialize FFTW. Bad wisdom?
15:50
yeah I don't get why I can't talk to developers at $ShoddySoftwareCorp for support, especially tech support can't friggin help. Does anybody know how the software works?
@HopelessN00b "I'm not a bastard, I'm the bastard. And that's Ms Bastard to you."
lol
I suspected as much, but have to try every now and then.
(Missing a free lunch with a vendor 'cause I'm on the phone with our current webfilter vendor's support team.)
@HopelessN00b BARRRRRRACUDA
15:55
FC1 server, SSH accessible from anywhere, no fail2ban, root ssh password allowed, root password of 'pizza1', what could go wrong?
@ChrisS Fast Fourier Transform... Wizard?
@ewwhite As far as webfilters go, I like them. At least it's not the cloud crap we have now, which I have to support, but have no control over. :(
@ewwhite no: git://github.com/ajaiantilal/i7z.git
Client just had multiple switches reboot overnight..
won't give me enable password (ProCurve) because it's the same as their domain password.
ESXi keeps telling me "Network uplink redundancy lost" but I have two NICs active....
@NathanC Right-click on host and "Reconfigure HA"
16:05
@ewwhite Grayed out
@NathanC is this a single server?
@ewwhite I have two hosts, but both are independent other than being managed together on vcenter
single server yes
I reset it to green and it hasn't come back...so..
@NathanC Oh, make sure both uplinks are active for the management port group
Override Switch Failover - BLASPHEMY!
:)
All kinds of policy override blasphemy going on there!
16:10
I received this yesterday on the SUSE security mail list lists.opensuse.org/opensuse-security-announce/2014-03/…
Tell me I am wrong. Did they have an outdated Xen affected by 47 known vulnerabilities for years in one of their Enterprise versions?
@ewwhite So... "fix it your damn selves, then?"
@MDMarra no it's not. You're supposed to pin opposing port groups to different adapters.
@ewwhite Only if you don't buy a server with enough NICs to bind management and vMotion to different physical NICs :)
@MDMarra Multi-NIC vMotion.
Whats that mean
16:13
@MDMarra No Bitchassness
2
A: Network configuration for VMware w/ vMotion and a single switch

ewwhiteThis is good as-is. You won't need a special VLAN for vSphere replication. For small installations I often go with 4-pNIC solutions. Below is a 2 x vSwitch setup with VM traffic on vSwitch0 and storage on vSwitch1. Public, management, private and vMotion networks are on their respective VLANs, ...

Right, I meant you should have a pair for management, a pair for vmotion, and then however many NICs you need for data
@ewwhite For active/standby?
@ShaneMadden Yeah.
@ewwhite Ahh. Why no active/active?
You're talking about having management and vMotion on the same vSwitch over the same physical nics, but pinning active/standby to opposite physical adapters for each task, right?
16:15
@ShaneMadden Determinism... If I have 4 NICs, two dedicated to storage, two to management and vMotion, I typically want to be able to predict what traffic is flowing where.
@ewwhite My point was that NICs are cheap! Get 2 more ports and have dedicated vMotion interfaces!
@MDMarra Colo space, PCIe cards and switch ports aren't necessarily cheap.
@MDMarra Ports may be cheap, but so are my employers.
But my own installs are 2 x 10GbE most of the time anyway.
Switchports aren't cheap in a colo, but if you can afford to license vSphere, you can afford a NIC with two more ports
16:17
@ewwhite For troubleshooting purposes?
@MDMarra Russian keygen is only USD $50.
@ShaneMadden depends on the environment. Most clusters don't have a tremendous amount of traffic coming out of them. A small installation with 2 to 3 hosts needs resiliency across port groups, but isn't passing too much traffic. vMotion is the only thing that will peg a link. So I use the same vSwitch as management and VM traffic for that.
@jscott torrent keygen is $0
I've done vMotions over the same link as data and it didn't appear to affect anything
but, I've only done that a couple times to bounce a couple servers around when we moved...
@ewwhite Ahh, gotcha. Yeah, I like to be able to have multiple VMs in the same port group using different physical NICs to spread the load. And to know that nobody's gone and screwed up the switch port config on the standby uplink.
That's what the LACP is for ;)
I'm guilty of using LACP as much as possible now
just can't do it on the smaller sites
16:23
Depends on the load. In a lot of cases, active/active with balancing on port id is nearly as good as LACP. Without the pain in the ass.
@ShaneMadden I don't see why people perceive LACP as a pain in the ass. They must be using shitty switches?
active/standby management and standby/active vmotion on the same port group is common practice
@MikeyB Well, yes. But when you're comparing load balancing that requires config on the switch against load balancing that doesn't care what the switch is doing.. pain in the ass is relative. :)
@ShaneMadden OK when you're doing VMware, yeah, may as well let vmware handle it.
@pauska exactly!
But I'm a fan of the distributed switches now.
16:26
2x10gig is bad practice unless you're doing CNA
you really really don't want vmotion traffic to hit the same port as storage traffic
@pauska NIOC
Oops... need to update it after changing vCenter.
@ewwhite vNICs with reservations?
@ewwhite meh..
If you already have 6x10gig then I don't see why you would put storage in the same group
it's so much safer to separate it from other traffic types
Oh, that's 2 per host in this 3-host cluster
16:36
oh
well ok, better than nothing I guess
monitor your switch port buffers! :)
Cool. I figured out how to lockdown all the doors in the building with our security software. :p
@pauska 5400zl
@NathanC now you just need to figure out how to do that remotely. "Sorry, boss... tried coming into work today, but all the doors are on lockdown. Let me know when you fix it so I can come back to work."
@HopelessN00b I can do it remotely :P
All the doors are controlled by software on one of the servers....just need to be connected via VPN lol
There you go... never have to come into work again. Hmm. Wonder if this why they don't let me admin the security systems...
16:48
lol
Unfortunately it logs everything, so it'd be stupid to attempt something like that anyway. The amount of power the software grants is pretty awesome though
I almost wonder if we could somehow patch the security system into our alarm system (separate vendors) so if the security alarm trips it'd lock all the doors. :P
This is sad
I googled an exam number to see what certification(s) it applies to and the first 10 pages of google results are braindumps
I applied hotfix KB2550581 and rebooted...it seems I have no delay in accessing file shares now... so .!. @ewwhite and your upgrades!
On a side note, @ewwhite, I'm burning esxi 5.5 now to install ;)
@NathanC IT people can lock doors. Smart IT people can lock doors then wipe the logs without any trace :)
@MDMarra What's a braindump?
test dumps
files with the actual exam Qs and As in it
16:56
@MDMarra How is that possible?
I'm always semi-aroused by @MDMarra's avatar
@Jacob What do you mean?
@Travis ;)
@MDMarra Do people try to memorize it to write it out later or something?
@Jacob Probably leaked tests just like leaked OS releases
@Jacob Sometimes. Other times, corrupt Prometric/Pearson employees sell the exams to scum sites like Pass4Sure and then people pay for them. Then after they pass they post the paid-for files on free sites and bam - 10 pages of google results full of cheats
16:58
There are even services where for $1000, someone will take an exam for you. Their friends (or someone on the take) will work the sign-in counter at the testing center so they dont check ID.
The Simposons wriiters must be on the inside
@MDMarra For $1,000 I will take any test you want!
haha
@MDMarra That's a shame.
16:59
Fairly sure the punishment for getting caught is losing all certs from that vendor and being banned for life
*no warranties or guarantees expressed or implied
So that would rule you out for ever working for a partner :)
@Travis Pretty sure that @MDMarra can handle it by himself. :)
@Jacob Pretty sure any test @MDMarra wants to take probably would not be one I could pass
Nah, I don't have impressive certs
No MCM/MCA or anything. Just a couple of MCSEs, which aren't super prestigious anymore (partly because of all the brain dumps) and a VCP 5
17:03
@MDMarra Um, easier approach is photoshopping up something to put over a proper government ID (like on a transparency, appropriately cut to size). Not hard, and it's not like the people who do check IDs do a thorough job.
Some require a photo - VMware and Cisco for example
@ewwhite Can NewRelic monitor individual services?
They keep a photo of you on file
@HopelessN00b Not to mention that people doing this for profit probably take dozens of exams per week under various names
@MDMarra Point being that a fake ID, especially one that's not validated by an actual professional, is almost trivial to create these days.
Right. But say I sit for a VMware exam. They take my picture at the test site and keep it on file.
17:06
@MDMarra Fuck VMware anyway... for what they charge for that rip-off class you need to get a VCP, even asking for a photo seems especially low.
So unless the same exact ringer takes the test for the cheater for all VMware exams for the rest of time, it wont work with a fake ID
@HopelessN00b I got my VCP without the class :D
So I wouldn't know
hahaha
@MDMarra How'd you pull that off? And more importantly, is it reproducible?
@HopelessN00b Seriously? As much effort as setting up that would take just learn the material. :)
Bob
Bob
17:08
@MDMarra Unfortunately requires Win8 clients, though.
Lucky enough to not be on XP anymore. Mostly.
@Bob How long have you been awake man?
Bob
Bob
@Jacob Uh. Afternoon-evening nap? :P
@Jacob Yeah, but with a proper silicone mask, you can do other things, too. Like get a second passport so you can get to a non-extradition country if needed. :)
@HopelessN00b VMware runs a promotion for partners of a certain status from time to time. There's a promo code that employees can use to be certified by passing only the test. The thought is that if you're a partner and have whatever the status is, you'll have had adequate on-the-job training by certified engineers from inside your organization
@Jacob Naw.
17:09
Ah, damn. Nothing that's likely to apply to me anytime soon, then.
Unless it's Postgres/programming framework, etc.
@ewwhite What do you use to monitor processes ?
Sad truth...
@ewwhite "Did you get that alert?" "I'm monit!" ...wait..
Bob
Bob
17:12
So. Dell replaced one of the RDX drives. Here's hoping it works.
@MarkHenderson openvpn is pgood
Bob
Bob
What kind of fucking backup software bluescreens the fucking server when fucking dismounting fucking drives? Gah.
@Bob Why's it dismounting drives?
Bob
Bob
@JoelESalas Limitations of backing up onto RDX => creates a virtual drive container thing => needs to mount pre-backup and dismount post-backup
That's all handled by the backup program. Supposedly.
| Section2 = | Section6 = | Section7 = }} RDX, an initialism for Research Department Explosive, is an explosive nitroamine widely used in military and industrial applications. It was developed as an explosive which was more powerful than TNT, and it saw wide use in World War II. RDX is also known as cyclonite, hexogen (particularly in German and German-influenced languages), and T4. Its chemical name is cyclotrimethylenetrinitramine; name variants include cyclotrimethylene-trinitramine and cyclotrimethylene trinitramine. In its pure, synthesized state RDX is a white, crystalline so...
17:14
@MikeyB fftw.org No idea what the "W" stands for "Fastest Fourier Transform in the West"
Bob
Bob
@JoelESalas Except when it's not. It clashes horribly with certain network drivers commonly found on laptops. OpenVPN running + sleep => bluescreen :(
@JoelESalas Heh. I wish I had some handy.
RDX, also known as QuikStor and RD1000.
@Bob Really? Never seen that happen.
Bob
Bob
@Jacob Had two HP laptops that did that.
@JoelESalas I like that idea... RDX drives. Better than FDE - unauthorized access scrambles the drive (and the access-er) across 10 square city blocks.
Bob
Bob
Quite a lot of info (but no fixes) available if you googled it. I just remembered to disconnect before suspending.
17:17
@HopelessN00b C4 is better. You won't explode from sneezing.
Bob
Bob
@JoelESalas Basically, some standard for ruggedised removable hard drive cartridges and caddy.
Worst decision ever.
@Bob Gotcha.
I'm going full commodity with our storage
Bob
Bob
Should've gone with LTO. Or even consumer USB HDDs, those were more reliable -__-
My company paid for a VCP class ...it was a waste of money.
@Bob don't knock what you can do with a large sum of USB drives..
17:19
@Jacob RDX comes as a plastic explosive, too. Stable as C4.
@HopelessN00b I'm thinking of T4 then.
Dammit @HopelessN00b I'm a Sysadmin, not a chemist!
2
@Jacob Also not a miracle worker?
@JoelESalas No because if I was I'd be the first person I helped.
@Jacob Probably not. T4 is a synonym for RDX. :)
@HopelessN00b See above.
Bob
Bob
17:21
@JoelESalas Probably also want to avoid literally knocking them.
I know more physics them chem
Bob
Bob
Those things
They don't even eject properly :\
11
Q: Are RDX removable disks a good replacement for LTO tape?

ewwhiteOver a period of three weeks, I experienced six complete failures of LTO-1 and LTO-2 tape drives at client sites. Some had failed mechanisms. Others lost the ability to write reliably. These were HP Ultrium 232, 448 and 460 drives. Most of these units were deployed between 2006 and 2008, so the t...

Bob
Bob
@ewwhite It's not on USB2, thankfully.
I haven't actually tested speeds, though.
@Jacob I'd think that sysadmining would leave you more inclined to arcane knowledge in tools that you can use to dispose of lusers permanently.
Bob
Bob
17:24
Oh, they're up to 1.5 TB now.
@HopelessN00b Oh I do... Firearms.
@Jacob yay guns!
We use rotating seagate passport drives for off-site backups
I have guns.. I like guns, I want more guns, but this cracks me up cause I know this guy IRL
a nice rugged case to go along with it...and indeed, I accidentally dropped one and it still works just fine without any problem
Bob
Bob
17:26
@ewwhite Did you end up using them?
@Jacob Yeah, but guns fall short as a luser disposal tool really quickly as the density or number of lusers rises. Hence, explosives. You should really be prepared to deal with hordes of lusers, as a future-proofing technique, if nothign else.
@HopelessN00b I don't dispose of anybody.
Bob
Bob
@Jacob Right. wink nudge
@ewwhite Alright, so what's the advantage of RDX drives, over, say, just using a bunch of external hard drives? (Or USB keys/SD cards, for that matter.)
Bob
Bob
@HopelessN00b Supposedly, they survive falls better? :P
Much bigger than USB flash/SD.
@Bob <shrug> I've picked up flash keys as big as the largest SSD model they offer.
Bob
Bob
@HopelessN00b We're using 1.5 TB RDX drives. For the same price, you'd get... 256 GB on flash?
It was difficult enough getting enough to cover a week's worth of daily rotations.
@HopelessN00b ruggedized.
I'm still dealing with asia-exotics.com and not being able to get email to my clients.
hiss
@MattBear Well... didn't see that one coming.
17:51
@ewwhite that domain could be a brazzers subsidiary or a produce company. Either way, I'm not clicking.
@DennisKaarsemaker It doesn't work.
@ewwhite Maybe the NXDOMAIN?
@DennisKaarsemaker Exactly... It's a produce company in Bangkok.
# nslookup asia-exotics.com 8.8.4.4
Server: 8.8.4.4
Address: 8.8.4.4#53

** server can't find asia-exotics.com: NXDOMAIN
no MX records either
not even registered, says whois
17:57
oops, no "s"
From: Mail Delivery Subsystem [mailto:[email protected]]
Sent: Tuesday, March 25, 2014 1:09 PM
To: [email protected]
Subject: Delivery Status Notification (Failure)
@ewwhite Nice website.
no A record for the domain, some mailservers don't like that
hm
ok, rerunning dig does give an a record

« first day (1292 days earlier)      last day (3975 days later) »