« first day (951 days earlier)      last day (4316 days later) » 

12:02
@JennyD it's this issue that only affects @DennisKaarsemaker :)
12
Q: HP DL380p Gen8 (p420i controller) I/O oddity on XFS partitions

Dennis KaarsemakerOn DL380p gen8 servers using XFS on top of LVM on top of raid 1+0 with 6 disks, an identical workload results in a ten-fold increase in disk writes on RHEL 6 compared to RHEL 5, making applications unusable. Note that I'm not looking at optimizing the co6 system as much as possible, but at under...

@ewwhite That's interestng... unfortunately a bit outside of my area of expertise. I just thought the wording was funny.
@JennyD XFS is being sleazy with the raid controller, touching it far too often, in a very inappropriate way
I think LVM doesn't agree.
@DennisKaarsemaker Creepy indeed. Predatory, one might say.
Still, could have been worse. If it were Reiserfs the raid controller would be dead by now.
Our school VLE/intranet runs on something called ROCKLinux (never heard about it until a few days ago). That server had never been off in about three years til we shut it down tuesday for RAM upgrades.
12:12
Dead, fragmented and buried.
@tombull89 let me guess, it didn't come back up?
@DennisKaarsemaker oh, no, came back up fine. Just not heard of the name before.
@tombull89 I'd not heard it either.
@tombull89 lousy story :)
@DennisKaarsemaker fine. it asploded on startup and took out the rest of the rack.
@tombull89 much better. How else to teach the noobs that regular reboots are actually essential!
12:17
@JennyD ooh, evil thought: an @reboot cronjob that does 'sleep 6m; rm -rf /'
6months, not 6minutes :)
@JennyD well, I'll hold my hand up there - we've not rebooted it here but it's a managed box by the company that runs the VLE, so it's fully possibly they've rebooted it. Thing is, it's so locked down we can barely get into it to find out the uptime.
@DennisKaarsemaker that sounds like something you'd set up on leaving your job...
(generic "you", of course)
@tombull89 Yes, sometimes there are good reasons. But absent those reasons, I would not want a system running for more than maybe 6 months without a reboot. That's about as far back as it's actually possible to remember the reason for any changes that haven't been done properly.
@JennyD neh, I'm more the, "on april fool do 'wall System going down for shutdown NOW'"
@DennisKaarsemaker Ejecting all the CD drives in the servers was another one. Or use the internal speaker to play a tune.
@DennisKaarsemaker I still half regret' that i never got around to buying a plastic skeleton and put under the raised floor of the server hall before quitting my last job
2
12:25
# last reboot
reboot   system boot  2.6.18-194.8.1.e Fri Apr 19 08:17          (00:08)
reboot   system boot  2.6.18-194.8.1.e Fri Apr 19 08:09          (00:16)
reboot   system boot  2.6.18-194.8.1.e Tue Jul  6 12:36         (1017+19:49)
reboot   system boot  2.6.18-194.3.1.e Wed Mar  3 10:55         (125+00:37)
reboot   system boot  2.6.18-194.3.1.e Wed Mar  3 09:43         (125+01:49)
shame
my uptime is bigger than yours!
@DennisKaarsemaker that's like saying "my speakers are bigger than yours". Used to be cool, isn't any more.
@DennisKaarsemaker So you're behind on patches are you?
@JennyD I know. Apparently sarcasm doesn't transfer well over chat :)
@ScottPack actually, PCI milestone 2 is making us update a lot of servers right now
@DennisKaarsemaker Good man.
12:28
And as much as I loathe PCI-DSS, this is a good sideeffect
You know what I love? How clear cut the PCI-DSS really is. So long as you can figure out your merchant level you're good. You know what I hate? How nitpicky the PCI-DSS can be.
"My uptime brings all the exploits to the yard"
2
@DennisKaarsemaker or even if it does, that won't stop me from dropping a one liner that I just thought of and found funny :-)
When in doubt....snide sarcasm.
@ScottPack I've got one of those e-cards by my desk at work. It says "I'm a real bitch before I get my morning coffee. And afterwards." And still people get surprised when I'm sarcastic.
Mar 4 at 15:43, by Dennis Kaarsemaker
spinners.mdmarra.local
meh, lacks context
@JennyD A couple of years ago we got bored and placed whiteboard magnets, about an inch tall, on the top doorframe of each of the guys in the security office. We used them as mood bars. Until last month mine said, "wistful, mildly cranky"
That's on the wall behind me
Needed a test account in our intranet once. Forgot to delete it befor the new nametags were printed. Nagios has a nametag too :)
Nagios also tends to getinvited to parties
@DennisKaarsemaker Do you get invited to those parties?
@DennisKaarsemaker I think it was @MDMarra has a test accout named James T. Kirk and people were asking who it was.
12:49
@ScottPack Yes :)
There's a pacman in our graphite :)
@tombull89 Not I
I think it might have been @ChrisS
My test account is always Steve Stevens because I never forget what I made it
Not Job Jobsons?
Or Fluffy McFlufferton?
I have to use these accounts in demo, dude!
Although one of our wireless guys uses BWayne and Batman to show off ISE device association
Our guys tend to use things like "Test Test" because they're boring.
Dan
Dan
12:57
@ScottPack That's me [Initials]Test or [CompanyName]Test or mebbe TeacherTest / StudentTest in education etc
@Dan See. Boring.
Dan
Dan
@ScottPack Not what your mum said
@Dan I doubt that.
@Dan I never need to make test accounts as such anymore. But at my previous job I had a very free hand when naming servers. We did one particular setup for a customer we intensely disliked, and I named the mailservers after rabies viruses. The nfs servers were "marburg" and "ebola"
and for a main mail system I had "hallucinogenic plants and mushrooms" as the naming scheme.
2
Our systems folk do silly things like name our systems based on their purpose. Again. Boring.
13:00
I tend to use Sam Vimes, Arthur Dent, Hari Seldon or Mitch Rapp
Dan
Dan
@JennyD Oh god, I can't stand nonsense server names
they tend to not get noticed as test accounts, getting the AD police out of my hair.
Dan
Dan
@ScottPack Shit
@ScottPack Here it's even worse, sequential numbering
We, at least, use TV/Movie cops and then cnames if we want to be service specific.
13:01
@Dan Server names can be anything. But they should have aliases that speak to what they actually do.
@Dan mc201bpmdb-02
encodes datacenter, role, partition and order.
Dan
Dan
@JennyD Not really relevant in Windows world
I set up a secondary MX for our domain customers to use, and called it "planb". One of the sales staff came up and asked me to change it because he knew that one shouldn't name server after where they were housed in case you need to move it and then the name isn't right any more...
Morning
Afternoon
13:03
@JennyD hmm, with thousands of servers, I prefer the name to indicate at least role and sequence number. Boring but effective
@Dan We try and keep it light hearted for client demos haha
test1 doesn't get a chuckle like batman
@MDMarra mount /dev/horse
@MDMarra Similarly Job Jobson is terrible, but Fluffy McFlufferton is awesome.
@DennisKaarsemaker Yes, with that number of servers I agree. And I wouldn't mind it if the server names here at least indicated what system is on them. But they're all just a numerical series, and it's really hard to remember what "server 134" is used for, as compared to what "imap3" is used for. Or even what "belladonna" is used for.
Chaos here today.
Not at work, but this part of MA in general
13:05
With a large number of servers you need some way of knowing what is what, so sequential numbering in one series for all is even worse than "everyone decides on their own names"
@JennyD ooh yes, that sucks. We killed that scheme when we hit ahsrv-{1..80}
"ActiveHotels server 80"
Dan
Dan
@JennyD Oh yeah, when I turn up to sites where it's just "server ##" I want to just scream
All names should be of the form $DCNUM-$RACKNUM-$UNUM. You devops virtual peopleare on your own.
@MDMarra That's freaking awesome.
13:08
theres a youtube video of it too with sound
and it's awesome
Dan
Dan
@ScottPack I like [SITEID]-[ROLE]##
here's [school inital][server role][number], so SCHDC01, SCHWDS01, SCHEX01.
Dan
Dan
@tombull89 Exactly as it should be, though I like a hyphen
@JennyD Because if there's one thing that crusty IETF neckbeards know it's style.
2
13:09
@ScottPack Oppa neckbeard style!
Heeeeeeeeeeeeeeeeeeey sexy server
there's a cat right there.
that's great for a friday morning.
I want to take a thread and "move it to chat." The site used to prompt you for this
but now it doesn't
how do I do this me self?
@DennisKaarsemaker No.
Back at Algonet we had this mail server that was called "angel.algonet.se". And one of our customers called support and was very upset at getting emails from a "demon" at "angel". (i.e. mailer-daemon responses to wrong addresses). The support tech tried to explain. Then suddenly the customer got very quiet, and after long pause suddenly yelled: "I KNOW WHO YOU ARE! YOU ARE ANTICHRIST!"
Then he hung up. And cancelled his account.
@JennyD There's a DailyWTF story about someone who got very upset about a process called "deamon" on a FreeBSD box, which had a sticker of the devil-logo-thingy on it as well.
@mbrownnyc I thought it was 8 messages, unbroken, between the OP and commenter.
@tombull89 Yeah, I remember that one.
13:14
Naming conventions... Should we do the @MDMarra approach?
@mbrownnyc You can't initiate it yourself. You can make a new room and link the OP to it, but it won't pull in the comments. The site still initiates it, but it takes a lot of back and forth
@ewwhite SITE-DC-01, SITE-MAIL-01
dc0001.stickandballs.local, dc0002.stickandballs.local, etc.
I don't like the number padding on low-count systems... dc1, dc2, dc3...
stickandballs.local ಠ_ಠ
@ewwhite Consistency. If any server in your org can get to double digits, make them all double digits. Otherwise you have to remember which ones are 01 and which ones are 1
Dan
Dan
@ewwhite A single pad is okay
location-company-role##
in our case, OSL-SYS-DC01 and so on
or for a customer, OSL-JZ-WEB01
Dan
Dan
13:17
@MDMarra I say do 01 anyway. It doesn't take much to go from a company that only needs single digit servers to double digits.
works fine..
Just for clarification, I recommend site-role-##, not #### and not .local :)
Dan
Dan
Same goes for switches and such. CAB##-SW##
@MDMarra stop copying me!
@MDMarra I swear you configured mail00001.ad.coastbanana.com
13:18
I didn't copy you. You have a .local AD!
I could be wrong...
You are
it was bananahammock.local
Dan
Dan
I still think .local and .internal are mostly fine
NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO
2
Well, I just recovered from that server that was up for 1017 days, needed a fsck on a 1.1TB ext3 partition and was missing entries from /etc/fstab
13:20
@Dan Tell me that when you try and get a UCC cert for Exchange from VeriSign in two years
morning
should reboot testing be part of regular maintenance?
@Cole, you staying home?
Dan
Dan
@MDMarra Why'd you need a cert for the internal domain?
I had to ask the client if they didn't have their 1.1TB partition mount automatically on purpose..
13:22
@ewwhite There's not an explicit mount in rc.local or something worse? I've seen that.
@ewwhite only if you don't regularly build new servers of that role
@jscott rc.local? Is that @MDMarra's new server? :)
3
@Dan because if you don't, no one internally can use outlook
outlook autodiscovers exchange internally though AD, so it uses the internal hostname
@DennisKaarsemaker Seems irresponsible to build a DB system and just not update it for 3 years.
Dan
Dan
@MDMarra I'm not an exchange person, but we've got Exchange on loads of .local and .internal domains
So you need UCC certs that cover the internal and external presence
Dan
Dan
13:23
Or is this literally just a Verisign thing, cos we don't buy certs from there
@DennisKaarsemaker It's just like a DC, but with an "R" instead.
@Dan Right, you can get a UCC now for .local or whatever. That won't be the case in a couple of years
It's called the "certificate cliff" haha
@ewwhite meh. Master databases are annoying to reboot if you can't have downtime
A major consortium of SSL vendors decided not to issue certs to made up TLDs after a specific date in 2015
@MDMarra I'm sure we'll be able to find them on the black market.
13:24
I feel slightly more sympathetic to not rebooting those. But 3 years is old for sucha system. We recycle masters after 12-18 months
@DennisKaarsemaker seems to be in a cluster... I should check on db0000002.someclient.com
thanks @tombull89 and @MDMarra
Damnit!
[root@mdmarra-pgdb00000002 ~]# uptime
 09:24:16 up 1021 days,  3:10,  4 users,  load average: 2.11, 2.09, 2.09
@ewwhite What asshole leaves boxes running that long?
And the time is wrong.
13:27
Yeah, ntpd fails if it's more than 5ish minutes off.
I recommend you ntpdate time.nist.gov that bitch.
And fire whatever kid is that neglectful.
@ScottPack remember, cloud..
So is it MY firm's responsibility to suggest reboots? Or should client?
I suppose that would depend on your contract.
I just don't know.
@ewwhite You'll be running your own PKI to issue those
@MDMarra probably.
13:30
@Basil nope, at work
@Cole how chaotic is Boston?
Edit please...
0
A: Is it possible to increase the max limit of 10 pc's to any number of times without making the user a domain admin?

tony rothIn order to not drive myself nuts I found the following, you should try to rtfw. 1.Click Start, click Run, type dsa.msc, and then click OK. 2.In the task pane, expand the domain node. 3.Locate and right-click the OU that you want to modify, and then click Delegate Control. 4.In the Delegation...

@DennisKaarsemaker everything is shut down
@Cole No fly zone, too.
Thank god the MBTA is, the commuter rail runs directly behind my apartment.
I'm about ~35-40 minutes out from Boston, but still too close.
13:31
InfoSec folks: might wanna rethink your self-identifications as "white hats". Just saying.
I hope the other suspects don't get killed by police. I want to see justice done. That said, does Mass have the death penalty?
I don't think so.
well life in prison would work too.
There currently is no state death penalty in Massachusetts, life without the possibilty of parole being the only punishment for first-degree murder. The federal government prosecutes capital cases within Massachusetts, however.

On November 7, 2007, House lawmakers again overwhelmingly rejected a bill to reinstate the death penalty by a vote of 46 - 110. Two years earlier in 2005, the House defeated Governor Romney's legislation 53 - 100. Continue reading for a full Massachusetts history.
@Basil no
13:32
@Basil no but this may be a Fed. case
I would think.
Gitmo would work
@Basil the first guy was killed because he had a bomb strapped to him and was shooting at cops
@MDMarra Not blaming them. I know that they want to see a trial too
13:33
I doubt the other brother is any different
Jenna's work is shutdown
they don't seem interested in being caught
She's about 15 minutes out of Boston
Hmm. So half of an IT class is playing Halo. Either the teacher is AWOL or they've got a substitiute
Interesting
13:38
@tombull89 Want to mess with them?
Take turns deactivating their ports
too obvious
throttle the<strike>m</strike>ir ports
the m ir ports ?
14:12
@Hennes Them/Their
@tombull89 - maybe they are learning what IT folk really do all day
sign on bonus day....yippee... :) <now get back to work C>
@Windows people: Is there a way to tell for certain how an update installation was triggered? (Manual vs. Scheduled?)
I don't think so
maybe in the event log
Nothing as far as I can see. I've got this in the WindowsUpdate.log:
`2013-04-19 08:55:31:849 896 b58 AU # Initiating manual install`
(Trying to figure out how an update happened that took out our main mailbox server for 20 minutes this morning.)
Ah, I'm just being dense...
All our server updates are manual (during weekend windows), so I was thinking I had nothing to compare it with. Then I remembered I had a fleet of 150+ desktops, too..
Similar entry for one of those is: ` # Initiating scheduled install`. So it looks like I Can bollock someone for updating outside the window.
14:32
nice
NTFS Permissions - Y U NO WORKING
Group has full control on a folder yet cannot copy or create any files in it.
PEBKAC
@Cole bc share permissions too
@SmallClanger - go get em!
14:36
@RyanRies have you played with orchestrator at all?
also WTF HUGE SPIDER ON MY CUBE WALL
@Cole Both of those things are portents of the end times.
Both what Ryan and Mark said. Also, inheritance.
@MDMarra Just a little bit - unfortunately my current company bought a clone of Orchestrator from another company that I don't like as much.
If you see a dog speaking in tongues on the way home, head for the hills.
14:37
@SmallClanger - find the person responsible, pull them aside and make them a deal...they buy your lunch today and you keep what you know in your pocket. :)
@RyanRies I just started. its the coolest
My brain is fried after the whole virus thing last night
was up till 10PM dealing with that shit
working on SM/ORCH integration now
@MDMarra Yeah, you can automate damn near anything
got home, manager called and asked me to do more work from home and had to be in for 8am
14:38
@MDMarra - epo?
@Cole you're a man now. Kill it.
@TheCleaner The scary thing is, we only got two phone calls during the outage.
@TheCleaner I'm talking about System Center Service Manager and System Center Orchestrator
ah...big difference. :)
Though, I have McAfee ePO experience, if that's what you were talking about
14:39
yeah...I'm not a big fan of ePO. It's great when you have time to constantly admin it, but let it lie for a month or two and getting it back in line is a PITA.
yeah
I hate EPO
@DennisKaarsemaker I usually do
Used it on a daily basis for about a year
14:40
seemed like the stupid repositories would always get out of sync, etc.
but I couldn't reach him without climbing on my desk
I ran 3 ePO servers for about 4 years
yuck
@Cole Here in Texas we have "halloween decoration" style spiders running around. Not fun!
@Kevin all my nopes
There are 2 things that should just "work". a/v and backups. They should not require constant care and feeding.
14:41
@Cole Huh?
@Kevin do not want
Somehow this gigantic black and yellow bug got into my house a while back, I was afraid to smack it with a shoe or something because I was afraid that I'd just make it mad.
I hate bugs
@Cole Imagine finding this in your front yard tiptoeing across your lawn: upload.wikimedia.org/wikipedia/commons/e/e9/…
@TheCleaner You know what the worst part of ePO was? The horrible AD integration
14:43
@Kevin So cute and fuzzy!
No matter what you do, dozens of computers always end up on Lost&Found or whatever it was called
@Kevin just did a really loud EW
@Kevin I'd shriek like a little girl and RUN
@DennisKaarsemaker me too, fuck that
@Ryan I still remember a holiday in France when a hornet the size of my thumb hoved in through the open caravan door. Luckily I was holding a badminton racket.
14:44
Not a fan of big hairy things.
2
Except @MDMarra
@SmallClanger KILL IT WITH FIRE
@MDMarra The worst about almost any enterprise antivirus, EPO included, is that they give you a false sense of security while not actually doing anything except slowing your servers down, and if anything goes wrong, AV always gets blamed first regardless of if it's actually at fault
@RyanRies oh like our virus outbreak yesterday and Symantec was like "lol no viruses anywhere!" but it was spreading like wildfire
@Cole Yep
@DennisKaarsemaker Thanks, man. Means a lot
14:45
@MDMarra - yeah...the whole AD syncing of the trees/etc. never really worked well for us. They make it seem like it will find every computer and auto install VSE etc...but it was always a pain.
I blame that fact that those servers are a.) on an NT Domain and b.) Have the guest account enabled
@TheCleaner I pushed the agent out via GPO and had an install job in the ePO schedule for the AV component. I didn't trust ePO to deploy the agent
@Cole We've had a couple of those "scareware" viruses on staff laptops, either THIS IS THE INTERNET POLICE or SuperFAKEAntiVirus2013+++XXUltra edition. Always get it when they're at home.
We got sexy.exe/porn.exe/passwords.exe
@Cole oh yeah how did that go?
14:47
@MDMarra hey, you have the beard of ultimate awesomeness. How can I possibly not be a fan of that!
@tombull89 seems to be stable now
We just had them include it in the image at deployment...we eventually just yanked ePO and made the clients get their updates from McAfee. Went from around 70% having the latest DAT files to about 95%. That's better than having the central management for us... :)
@DennisKaarsemaker I envy @MDMarra's beard
heh...learn something new every day....never even knew about MS Security Compliance Manager. <the more you know>
14:50
So a Computer account in AD keeps becoming disabled. We've removed the machine from the domain and readded it, but still no dice. Anyone seen this before?
@TheCleaner Heh. Part of your problem might be that the agent doesn't like being sysprepped. It doesn't generate a new UID, so if it's captured, you'll have a bunch of computers on the same network thinking they're the same computer in ePO
McAfee recommends installation post-deployment, but the agent is highly scriptable and plays nice as an MSI for deployment
Sorry...it wasn't in the image itself...it was part of the "software install list" after the initial image was ran by the Helpdesk. At least that's how they were supposed to be running it. (ah to be free from that job...so nice)
ah right
then that's just epo sucking
although I will say that this year (before I left) we went with McAfee SaaS and that actually ended up working quite well as a GPO and online central mgmt.
Here we run Trend Micro...meh.
14:56
@TheCleaner We do system center online protection
we get all kinds of free MS licensing, though
Oh, ESET NOD for us.
err, endpoint protection. same thing
Are we the only Windows shop in the world using Microsoft System Center (née Forefront) Endpoint Protection?
@jscott Including Microsoft? Probably.
I think I read somewhere that they use Clam.
@jscott We are
Intel runs Symantec, despite owning McAfee
At least that was the case a year or two ago
14:58
@jscott Fun Fact! We're rolling out Sophos in spite of having a Select license!
@ScottPack Sympathies.

« first day (951 days earlier)      last day (4316 days later) »