The Comms Room

Miles Erickson

12:14 AM

serverfault.com/questions/95873/…

ancient NARQ

Chris S

12:25 AM

I'm half tempted to blanket close anything from 2009. There's so much crap from that time.

Michael Hampton

It's crap now. Was it crap then?

Chris S

No, the site has changed dramatically since inception as the honeymoon phase wore off and an influx of such crap started drowning the site in "I can has teh codez" type questions. A defensive reaction to that influx was tightening up the rules on what is an acceptable question. Roughly a year or so ago we went through another round of such.

Michael Hampton

Aha. I wasn't around in 2009, so I missed all the fun.

Chris S

The problem with establishing formal norms like these rules is that there will be a range of implementation methods. Some people will be very gentle and helpful while others with lean to the USENET style. We want to keep that range as close as possible to the site's "target", hence the new rules, then the natural reaction to the USENET-effect ala "Hunting of the Snark" and "Summer of Love"

Michael Hampton

Joel E Salas

12:36 AM

@ChrisS Honestly, the quality of the site's content is the only thing that differentiates it from any other Q&A site. An accepted answer on SE is worth a lot more than Yahoo Answers.Why? We vet our own and we're uncompromising, and should continue to be uncompromising.

2

HopelessN00b

1:08 AM

Well, that and the intelligence of the average site's user isn't "brain dead" to differentiate us from Yahoo! answers.

Michael Hampton

1:23 AM

@HopelessN00b I sometimes wonder...

HopelessN00b

I used to wonder too... but I stopped when I figured out that no good comes of it.

ewwhite

2:23 AM

Looking back, the 2009 stuff does seem pretty off...

HopelessN00b

Maybe it's just me being new here, but almost all the stuff pre-2012 seems to be either crap, or would-be-closed-as-not-constructive-inside-15-minutes today.

Michael Hampton

2:39 AM

@HopelessN00b A user with 214 reputation that I just looked at was rated "top 21% this year". The "average", then, is depressingly low.

Make that 240 reputation.

HopelessN00b

Yeah, you really wanna get depressed, take a look at Rajat's percentile ranking. *shudder*

On the other hand, there's that *I R TEH BEST SADMIN EV0R!!!!!* effect that kinda offsets the depression a bit.

(Or you could take out all the 1 rep users from the equation... helps a bit.)

ewwhite

2:55 AM

not a fan of this...

2

Q: OS or Onboard software RAID?

I plan on installing Debian 6 on my server build with a E3-1270, 16GB DDR3, and 2x120GB SATA3 SSDs. But this has me wondering, should I use Intel's built-in, on-board software RAID or the OSs'? Which will perform more efficiently? Which is simpler? Please give me a solid answer. Thanks!

I don't think enterprise or even professional solutions should be about software RAID... (ZFS is an exception)

Michael Hampton

@ewwhite It's pretty obviously an enthusiast home server, but perhaps the answers will be instructive as to why you don't do that.

ewwhite

@MichaelHampton Well, maybe I'm old-school...

but I've pretty much only done the HP and Dell thing. Hardware controllers seem to make more sense...

HopelessN00b

@ewwhite On the other hand, Linux software RAID is decent enough that it does work in a professional environment, and I think we've both seen enough professional environments where they're not willing to drop the extra money on a proper hardware RAID controller.

May as well help steer those poor IT folk away from trying fake RAID, which can seem like a better choice than software RAID, at least the first time someone tries it.

ewwhite

@HopelessN00b My last boss had never used HW raid... and I had to inherit a number of SW-RAID linux systems.

No alerts, no monitoring...

that was probably a function of the boss rather than a deficiency with SW RAID.

disks would fail.. I'd have to send a junior admin to go look.

HopelessN00b

3:12 AM

Yeah. Boss problem, not technology problem.

Am I just getting ancient and crusty in my young age, or is there really not enough money in the world to be an FTE at a shop like that? I can't see doing it again anymore, unless it meant enough money to retire to my newly purchased private island in a couple years.

ewwhite

@HopelessN00b I was hired to bring things up to standard.

And had to leave once it all failed.

HopelessN00b

Ouch. Didn't get hired quite soon enough?

ewwhite

No, I couldn't work with the guy.

Day #1, HR told me "XXX may be a little hard to work with... Please let us know if you have any problems"

Everything else was right... the money, the hours, the contract... I thought I did all the right things.

(having come from an even crazier environment)

HopelessN00b

Ah. Been there, done that.

Once upon a time I was dumb enough to think "pfft, they don't know what they're talking about. I can work with anyone... especially for *that* much money."

Michael Hampton

@ewwhite Sure, Linux mdraid has plenty of monitoring. A Red Hat box will even have it on by default...

ewwhite

3:17 AM

@MichaelHampton Right, I realized that this guy was just "doing it wrong".

Michael Hampton

I'd still rather hit up eBay and pick up a PERC 6/i though.

ewwhite

@HopelessN00b It wore down on me... Dude stopped talking to me at a certain point.

I dug deeper and found no backups, no docs, no plan...

Secret servers...

HopelessN00b

Yeah, I had one of them bosses once. Should have been in jail, instead he's an exec these days. Never again.

ewwhite

this guy should be in jail...

HopelessN00b

Yeah, oh well. At least you know better for the next life, or something. Heh.

ewwhite

3:23 AM

My project was killed... and I was let go.

so I was like, "should I just leave?"

Uh, we don't know...

Can you give us a list of things that we're exposed to? I mean, how bad will it be not having an admin?

HopelessN00b

Lol. Love it.

"Now that we're not gonna pay you anymore, could you give us some free work??"

I always find it so hard to walk away from those spots with the vaguest semblance of professionalism, and/or not telling them how I really feel. :)

ewwhite

@HopelessN00b No, i gave them a real accounting of what was wrong. My boss had been hiding a lot.

Michael Hampton

@ewwhite That's when you pack up your personal effects, shut down all the servers, and walk out.

ewwhite

I went to my desk... called my lawyer... sat around for a couple of hours, actually... the boss didn't even come to work. They couldn't find him.

tried to work out a contracting arrangement.

HopelessN00b

@ewwhite Better man than I. I would have told them what my hourly rate was, and kept my mouth shut until I got a signed SoW.

ewwhite

3:29 AM

They were sad to find out how bad things were. Shame. Wasn't a big deal, though.

The junior engineer left a week or two later.

But I've learned that working for people is tough :)

Nobody's ever going to really watch out for you.

HopelessN00b

If only working with people wasn't so necessary to like eat and such, huh? :)

ewwhite

I've had consulting relationships for years... so I've found that dynamic to be easier than the boss/employee relationship

HopelessN00b

Does keep it a lot simpler, yeah. Call me when you need something, I do it, you pay me. None of that having to put up with each other for 1000+ hours a year.

It's like having a wife, without the sex. shudder

ewwhite

@HopelessN00b Well, people treat you better. Maybe because it costs more... But I've been treated like dog-s**t by employers... and rarely like that in a consulting arrangement.

HopelessN00b

No doubt. Very weird, but very true. Tell an employer something for free, they'll barely acknowledge your existence. Sell the same people the same answer for 10 grand, and they'll kiss your ass like you're doing them a favor.

:?

People...

ewwhite

3:36 AM

I think the boss-man loses respect for you the first time they see you under a computer desk fishing cables...

HopelessN00b

Right, that's why I make the help desk guys do that these days. :)

ewwhite

Ah, that works.

HopelessN00b

Well, sometimes. When they don't fuck it up, anyway. :/

Joel E Salas

4:12 AM

Argh, I don't like specifically being asked to act unprofessionally

HopelessN00b

Seems like an odd request, yeah.

Stack Exchange

4:28 AM

BUT WHAT IF I REASSURE MYSELF WITH A JOKE AND THEN DON'T WORRY ABOUT THE RASH AND IT TURNS OUT TO BE DEATH MITES AND I COULD HAVE CAUGHT IT

Joel E Salas

@ewwhite Would it be career suicide to transition from 100% Linux administration to 100% Windows administration?

ewwhite

4:49 AM

@JoelESalas why would you do that?

Joel E Salas

@ewwhite Really cool company. Though in retrospect if they're running a full Windows stack it might not be the best option for me...

ewwhite

It depends on what/where you want to end up. I have minimal Windows skills... Enough to get through installations, but I'd defer to SME's like @mdmarra on that.

The Linux skillset is more lucrative... but in shorter supply.

Joel E Salas

@ewwhite Meaning there's less positions for it? I'm definitely getting swamped with calls, but they place heavy emphasis (and value) on 5+ years of experience. Not sure how someone like me's supposed to have 5+ years of experience with AWS.

"I've been deploying cloud systems using automation tools that didn't exist the day Amazon launched EC2"

ewwhite

@JoelESalas Don't take those numbers seriously.

I'm not sure what else to say to that.

There are a few types of Linux roles... most center around e-commerce, web-facing systems... The exposure in the average small business is lower now. Cloud matters... and then there's the specialist stuff like HPC and trading systems...

@joel do you enjoy consulting? I mean, what are you looking for?

Michael Hampton

Clients.

ewwhite

5:00 AM

heh, I'm probably not good for career advice either...

Joel E Salas

@ewwhite Honestly I'd just like a stable place that values tech and engineering. Working with cool technologies is a nice bonus, and feeling a growing sense of mastery is really important.

ewwhite

@JoelESalas Okay... SO my career... CS degree, 6 years in the produce industry... 4 years in finance following that, while keeping 30% of those clients from produce.

Michael Hampton

@JoelESalas Convert a bedroom into a server room. Fill it with cool stuff. Enjoy. :)

ewwhite

Most of my hardware chops came from doing a lot of installations... thousands...

that was in the produce and warehousing world...

Michael Hampton

@ewwhite OK, I'm curious, how much of that CS degree are you actually using?

ewwhite

5:05 AM

@MichaelHampton I don't program... but I've supported developers in every job since... In finance, where I had $400k/year developers to work with, my job was to make their life easier and more predictable.

Michael Hampton

@ewwhite Yep, that's about what I expected. :) All I got out of CS was learning how to drink heavily.

ewwhite

So I had to know everything around the development process... Source control, library management, profiling, how to distinguish HW versus SW issues...

Joel E Salas

@ewwhite Algorithms? :)

Michael Hampton

Of course, when I started CS, the 101 class was FORTRAN.

ewwhite

and then, an education component... I could have a brilliant developer who knows trading strategy and how to optimize for realtime environments...

Michael Hampton

5:08 AM

And no joke, the school not only had a mainframe, it had several. One of them it used for class registration. To register for a class, you had to get a punch card for the class before someone else did. It was ... a zoo.

ewwhite

but who may not know how to daemonize a process or use screen

or rotate their logs...

@joel I'm just saying that you'll pick things up in any position... but you have to keep an open mind.

If I were looking today (which I guess I'm not anymore...)... I'd focus on a solvent business.

And then decide if you want big environment versus small.

My career has lacked a team feeling... I've always been the senior person or "expert"

and haven't had many peers... this is why Server Fault has been good;

But big environment plusses... budget, a chance to learn, scale...

Minuses... silos; Do you want to be the Sharepoint Administrator?

In a recent job interview, I was asked "You say you've deployed VMWare... Did you do ALL of it??"

Yes

"Really? Soup-to-nuts?"

And that came from them seeing a lot of people who were limited in their roles.

Joel E Salas

@ewwhite That's actually a really valid point. I would very likely get silo'd into something, though these days it seems "Linux person" or "scalability person" is a valid silo in some environments

That's my favorite question: How does it scale?

ewwhite

@JoelESalas I don't think silo == good.

And really, I think the technologies are fracturing the field. I used to be a real generalist who could do a bit of everything. Now, it's deep in a handful of areas and shallow in a lot more. Even with Linux, I look at some of the questions on Unix.SE, and think, "who cares... not important... I'll research it if I need it".

As I've gotten older, it's about the solutions/result and less about the detail...

So the standards rise... Hardware RAID, mainstream gear, CentOS/RHEL, config management...

whereas I would experiment more with more esoteric solutions when I was younger...

@joel Scalability... okay. Web applications, big data, finance and research labs... Those are good places to go.

Joel E Salas

5:23 AM

@ewwhite Your wisdom is enriching to me

ewwhite

finance and big data are converging in that they're leveraging the same technologies.

the research labs (I'm not sure which ones are in Cali) are great for technical growth.

Joel E Salas

@ewwhite HPC is a fairly small and tight-knit community, from what I can discern. It's literally a few degrees of separation between financials and CERN.

ewwhite

they're usually interested in rolling-their-own and definitely have some interesting resources.

the money was never where I wanted... the energy research lab in the Chicago area tops out at $10k/month

Finance pulls people from research. I just interviewed at a trading firm where the last interviewer was a guy who interviewed me 4 years ago at a research lab.

@joel I'd suggest getting involved in projects. Learning Puppet and Chef... pick up on the things other admins are weak at (cough, scripting)

Joel E Salas

@ewwhite Right now I'm having to discern between true engineering positions and fad-type devops positions, all because of the term "puppet" on my resume

Miles Erickson

@JoelESalas @ewwhite is right -- anything you can do with scripting and automation will make you valuable in a way that others aren't. Engineering wants to kill administration. Be an engineer.

ewwhite

5:35 AM

E.g. I suck at scripting. I've been able to get by because I know tools and have found alternatives.

Miles Erickson

@JoelESalas Also, especially if you don't mind working for people who need things to happen yesterday, there's nothing wrong with being interested in how things scale.

ewwhite

But the fad devops positions aren't all bad.

I've managed a lot of servers and environments at the same time, so it's hard to envision ONE role that would keep me busy enough.

so most of the startups I've talked to really didn't have much of a footprint.

Joel E Salas

@ewwhite Entrepreneur!

ewwhite

well, once all the servers are built... the puppet installed and the systems tuned...

I've been doing Exchange upgrades and Windows AD builds all summer.

(random, but I find a lot of companies need good direction)

phone systems, integrating broadband, internet load balancing, PC refreshes, virtualization, VDI...

not glamorous stuff, but things that apply to a lot of businesses and can be helpful.

Joel E Salas

@ewwhite Phone systems are definitely not glamorous, but having someone around who's familiar with them is such a huge perk

ewwhite

5:43 AM

Every trading firm I've worked at ended up doing some sort of phone system installation (usually Asterisk-based), a mail system change, and some level of Windows work. I had that experience from doing so many build-outs in the past.

@joel so that points to the small firms... do a bit of everything, keep your Linux chops up. Get a RHEL cert.

again, I'm not the best job resource.. I've had a tumultuous, but lucrative career...

(although, a company that pushed me out 4 years ago has gone through 6 people in trying to replace me... vindication)

Miles Erickson

@ewwhite Tumult is the new normal. If your career has been lucrative AND you've had fun, then you're winning.

ewwhite

@MilesErickson lucrative, but stressful and emotionally-taxing.

So, not much fun...

Miles Erickson

@ewwhite Heck, I got offered a job at Microsoft in 1996. If I'd taken it, I'd probably still be working there. Pretty amazing that I dodged that bullet.

(on the other hand, I'd probably be a fair bit wealthier than I actually am.)

ewwhite

right...

Joel E Salas

@ewwhite I'm not saying much, but I'm definitely digesting what you say carefully, so thank you for sharing your experience

ewwhite

5:54 AM

no prob

my hard lesson is that I can't trust employers fully...

Miles Erickson

@ewwhite Hm. I like to tell myself that if I'm not having fun in my daily life, then, I'm probably doing it (life/work) wrong. Doesn't always fix it right away, but... there's no sense in complacency.

ewwhite

@MilesErickson yeah, I left some bad situations... but have some searching to do on my own as well.

Miles Erickson

@ewwhite That's definitely a lesson worth sharing. Corollary: when you can afford a house, live in a studio. Save half of what you earn. Laid off? Time for another trip around the world.

[disclaimer: I have implemented this philosophy only partially.]

Michael Hampton

@MilesErickson I interviewed at MS for the IE for UNIX project. Bullet dodged.

ewwhite

Throughout my situation, there's never been a financial issue. In fact, I've made more every year of my of career... That's what's so twisted. It's really been an issue of finding purpose.

I tend to get hired by business leaders... who see the mix of business and tech knowledge... but my conflicts come with other tech staff.

Miles Erickson

5:58 AM

@MichaelHampton Wait, that does sound right... IE is the perfect browser for eunuchs!

Michael Hampton

@MilesErickson I didn't get it. They scrapped it not too long afterward.

ewwhite

One firm refused to adopt DNS... another insisted on Gentoo across 600+ servers. Another place was a battle between KVM and VMWare...

Michael Hampton

@ewwhite Gentoo..for servers? I never could figure out why anybody would do that.

Journeyman Geek

lol

Miles Erickson

@MichaelHampton There is no good reason to do that, but it still happens.

Journeyman Geek

6:00 AM

@ewwhite: I cannot imagine how messy the initial deployment of gentoo on those would be

ewwhite

@MichaelHampton it was terrible... and I had to own it. I switched to CentOS where I could... but it caused resentment.

So at that point, my boss stopped talking to me.

same with the KVM versus VMWare at the following firm

Miles Erickson

@ewwhite This may prove that culture is everything, and a key element to look for in a prospective employer's culture is sanity.

Michael Hampton

Eh, that's not so bad. I could deal with either.

Joel E Salas

@MilesErickson You mean sanity relative to your own, right? :)

Miles Erickson

@JoelESalas Right, of course.

ewwhite

6:02 AM

It was bad... it was a case of the companies being held-hostage by a senior technical person who hadn't documented...

Miles Erickson

@JoelESalas It could go both ways... maybe you would be bored to tears if all 600 of your servers used CentOS.

Joel E Salas

@MilesErickson I would be in tears. Of joy. And automation.

ewwhite

Maybe... that was 600 servers in 25+ data centers...

Joel E Salas

You know what. GMOs are bad and everything, but these grapes are fucking good.

ewwhite

so that made it a nice treat when a particular system was too old to emerge to the current release...

Miles Erickson

6:03 AM

@JoelESalas I have about 50 bunches of grapes growing on one vine on my back porch. I'd encourage you to come by and eat some of them when they're ripe.

ewwhite

eww

Miles Erickson

@JoelESalas You know what I loved most about being a college student in L.A.? It was the Grand Central Market at this time of year... they used to sell oranges and bananas, 10 pounds for a dollar.

Joel E Salas

@MilesErickson Hoooly crap. That was a perk of being at Cal Poly too, delicious orange juice. Too bad it was way too expensive for the average student.

Miles Erickson

@JoelESalas of course, that was 17 years ago... no idea what things cost now. I used to go in there with $3 and drag home enough bananas to feed an army of monkeys.

@JoelESalas No comment as to who trained the monkeys, nor what they were fighting for.

Joel E Salas

12 Monkeys

12 Monkeys is a 1995 American science fiction film directed by Terry Gilliam, inspired by Chris Marker's 1962 short film La jetée, and starring Bruce Willis, Madeleine Stowe, and Brad Pitt, with Christopher Plummer and David Morse in supporting roles. After Universal Studios acquired the rights to remake La Jetée as a full-length film, David and Janet Peoples were hired to write the script. Under Terry Gilliam's direction, Universal granted the filmmakers a $29.5 million budget, and filming lasted from February to May 1995. The film was shot mostly in Philadelphia and Baltimore, whe...

Iain

6:43 AM

G'day

Joel E Salas

►

I guess it doesn't like time tags

Michael Hampton

Debugging an SELinux policy is a great way to spend Monday morning, don't you think?

Iain

@MichaelHampton When is it ever not ?

Joel E Salas

7:04 AM

@MichaelHampton i can't think of a more "Monday" task

Michael Hampton

Aaargh.

Iain

@MichaelHampton you're good with php ?

Michael Hampton

@Iain Shh, don't tell anybody.

Got an practical, answerable question based on a real life problem?

Iain

@MichaelHampton this serverfault.com/questions/418952/setting-path-for-weberver-user

Michael Hampton

7:19 AM

Let me answer a question with a question. stackoverflow.com/q/3550449/1068283

Iain

@MichaelHampton that doesn't really do what the OP wants though :)

Michael Hampton

@Iain No, but it's good enough for DEVELOPERS :)

Iain

what I found was that I could use fastcgi_param PATH=... and it would set a PATH that could be read by getenv('PATH') but system('echo $PATH') sill retuned the same original unchanged PATH

Michael Hampton

That would have been my next guess. Blah.

Iain

so really my practial ... is why the difference between the two

Michael Hampton

7:30 AM

OK, so set the path in your php-fpm startup script.

Oh wait, that was the previously given answer.

Iain

bingo

in my case I hade php-fcgi but it's the same thing

Michael Hampton