The Comms Room

Chris S

20:01

@voretaq7 Oh yeah; perfecta. Or better yet, it's an option on your account webpage too... Put the most commonly abused ports on a list.

Zoredache

@ChrisS I bet I could avoid any of that silly anti-sock protections...

voretaq7

@ChrisS yeah, Cablevision/Lightpath does that I believe

lovesh

anybody wants to discuss about servers under high traffic?

i mean A VPS under high traffic

MDMarra

What about them?

Chris S

For the icing on the cake, send you an email any time those blocks are "violated", with something like "We noticed some unusualy activity from your computers. Traffic destined to port XYZ, which is blocked by default (you can unblock it in your account settings: ${link}). This is commonly due to one of your computers being infected with a virus. If you don't have virus protection and updates installed on your computer, you should consider doing so"

voretaq7

20:04

@lovesh not unless you define "high traffic"

Chris S

@voretaq7 8 requests per second. Really high!

lovesh

our VPS just got about 10k hits and the CPU usage went to 100%

voretaq7

@lovesh (I'm a little jaded because of all the "HALP! HALP! I'M BEING DDOS'd! 15 REQUESTS PER MINUTE!" questions we see)

MDMarra

10k hits in how long?

voretaq7

@lovesh 10k hits... since the dawn of time? In a second?

MDMarra

20:05

1 second, 1 minute, 5 minutes?

voretaq7

:)

lovesh

i think more requests mean more RAM usage because we run apache as prefork

voretaq7

10k in a second (or even a few minutes) would qualify as high load

lovesh

10 k hits in 2 hours

Chris S

1.4 requests per second

lovesh

20:06

or maybe less

MDMarra

Thats only like 1.5/s

Chris S

Someone find TomTom's cell phone!

voretaq7

@lovesh so about 1.4/sec... that's not terrible

MDMarra

thats really not high load

voretaq7

what's your monitoring system saying to you? :)

Chris S

20:06

@MDMarra I don't know if that even qualifies as "load"

lovesh

maybe it was in less time but the tech support said that this is because of high traffic

MDMarra

I mean, that's a steady load I suppose

but certainly not high load by any stretch

lovesh

but we have a VPS of 768 mb ram and single core processor

can u say why a high CPU usage?

Zoredache

@lovesh if you allow Apache mpm/prefork to create more processes then what you have memory for, then your server will start acting very badly. Either decrease your maxclients, get a faster server, or offload some of your traffic via proxies/cache/cdns/etc.

lovesh

@voretaq7 we dont have a monitoring system

MDMarra

20:08

@lovesh No, can you?

It's your app. How can any of us, who haven't seen your setup at all, tell you why your app is behaving badly?

lovesh

@Zoredache thats wat i am saying wats needed is more ram not more CPU and ram usage was abt 70%

@MDMarra i was asking in general if u can say why high CPU usage occurs because we have a site running Joomla

Chris S

My VPS, a $10/month plan (so really cheap VPS), can handle just over 1000 requests per second. For static content, I'd say that the minimum threshold for calling it "high" load.

Zoredache

start top/iotop and start collecting data. There is no way we could possibly know.

MDMarra

@lovesh Get monitoring. Monitor. Review monitor data. React accordingly

lovesh

@MDMarra ok

voretaq7

20:11

@lovesh Your lack of monitoring makes me a sad unicode panda! 🐼

Chris S

@voretaq7 nobody can see that except you

lovesh

and one more unusual thing i noticed there was a parameter quotaugidlimit that had a hard limit of 600 but would sometimes go to 88 million

voretaq7

(or basically what @MDMarra said -- Your site obviously has a problem, but without monitoring to tell you what's going wrong you don't know if you need more CPU, more RAM, you're getting bogged in a database somewhere...

Aaron

Also the pile of poo falls in the same category

MDMarra

@ChrisS It renders on my iphone, jsut not on my Windows 7 PC in firefox.

What's wrong with that

voretaq7

20:12

@ChrisS Well as I've said before you all need to install a broader set of unicode fonts!

Adrian

@voretaq7 +1. And I know because I'm in the same situation.

voretaq7

@MDMarra Apple has embraced God's Own Unicode!

Install Nagios.
Then you'll have 2 problems (No monitoring AND a Nagios installation to maintain!)

Chris S

@MDMarra Quiet with all your facts and heresy.

lovesh

@MDMarra can you tell me wat quotaugidlimit exactly is

Zoredache

@lovesh Speak with TomTom about your performance issues, he will offer to host your site on his smartphone.

4

MDMarra

20:13

@lovesh sure

Jacob

Hello Everyone

lovesh

i asked tech support and they say that it is the number of cpanel instances

Jacob

@Zoredache The SI unit of snark, the TomTom

MDMarra

@voretaq7 I'm telling you dude. Opsview > Nagios (even though they're pretty much the same)

voretaq7

@MDMarra I'm telling you (a squiggly line drawn on the wall in crayon) > Nagios -- Nagios by itself blows! Opsview makes it tolerable and functional :)

MDMarra

20:17

The whole nagios layout isn't bad

Peter Grace

nagios forever

you guys suck

;)

MDMarra

and if you have 100 of the same box it's great

it's when you have an infrastructure like me where you have 50 servers and the only two that are the same are DCs when nagios makes you want to facepunch yourself with a hammer

Jacob

@PeterGrace What do you use to monitor, that screenshot makes me want it :)

Peter Grace

Orion, but that's a custom interface.

Orion is wicked expensive too

(I also hate it)

lovesh

quotaugidlimit is Number of user/group IDs allowed for the Container internal disk quota

thats cryptic

can somebody decrypt it

MDMarra

20:19

What's hard to understand?

lovesh

i dont even know which salt they used for encryption

Jacob

Dear HardAdmin you are POINTLESS. blocking everything I to go!

lovesh

and can somebody recoomend a VPS. We are done with hostgator

MDMarra

@lovesh If you have a container (probably an OpenVZ container), it's the max number of UID/GIDs that you can have in that container

lovesh

i dont understand wat thes uid and gid are for?

MDMarra

20:21

@lovesh not to be insulting or anything, but it doesn't sound like hostgator is the problem...

Jacob

@lovesh Bigbrainhost.com

MDMarra

@lovesh It sounds like you need someone that knows what they are doing on staff

2

lovesh

@MDMarra no but today i had a chat with the tech support and the chat system was too messed up and even the guy responding didnt seem good

Adrian

@voretaq7 The boss is really liking the idea of Spiceworks right now. We may end up with that.

Jacob

@MDMarra + one MEEELION

lovesh

20:23

@MDMarra doing on staff? ddnt get u

Peter Grace

"Someone who knows what they are doing, on staff"

MDMarra

@lovesh What does "didn't seem good" mean? Just from this chat transcript, it doesn't seem like you have even a basic understanding of Linux. How can you expect anyone to ghelp you?

Honestly, it's not their tech support's job to make sure that you don't kill your own VPS from not having a clue

@lovesh It means "hire someone that knows what they are doing"

Iain

@MDMarra I think I might make that an auto comment

MDMarra

@Iain I aim to please

Jacob

@Iain It's becoming more necessary daily

@Adrian I'm looking at spice works also.

@Adrian I like the agent so I can place it offsite

Iain

20:26

is today the beginning of the (almost) end for the US elections ?

MDMarra

@Iain It's basically the end of the Republican Primaries

voretaq7

@Iain it's SUPER TUESDAY - YAAAAAAAAY!

BANG

Jacob

@Iain end of Primaries

MDMarra

Not the complete end, but they usually know who the nominee is at the end of today

Although, that might not be the case today, the way everything is split

lovesh

@MDMarra i thought i understood Linux basics. If my post i dont understand wat thes uid and gid are for? made u think that then i know linux has users which belong to groups and every users has permission to do and not do stuff. But wat i meant in that sentence was i dont understand why they are there for the container and in that number(86). I think i am sounding cryptic :). May be because i am bad at expressing mysqlf. ....

voretaq7

20:28

@MDMarra yeah. I'm also impressed with how many votes Brokeback Romney is getting

well

impressed maybe not the right word. Frightened?

Iain

@MDMarra which means we'll get more of it till the end of the week and then it'll go quiet for a while :)

Jacob

@lovesh Please spell check that.

lovesh

@Jacob sorry... i am being too lazy. i apologize

Jacob

@Iain You hear about? I imagine they headline goes a bit: Those bloody Yanks are attempting to elect another stupid president again.

MDMarra

@lovesh Honestly, UIDs and GIDs are very basic. You also do no monitoring at all, but then expect us to be able to help you. And you also consider 1.4 hits per second high load. You really just need to learn the basics of systems administration before you place the blame on someone else.

It's not fair to blame your host for things that are clearly your problem

2

lovesh

20:30

ok

MDMarra

That's the last I'm going to say on the topic

Jacob

@MDMarra This really deserves an auto comment

Iain

@Jacob we get some American politics on the news. The BBC has correspondents following the circus atm

voretaq7

@MDMarra @lovesh to be fair we don't know if it's your problem or not -- this is why you need some monitoring implemented :)

lovesh

can u recommend some monitoring system which has worked well in your experience

voretaq7

20:31

@Iain oh you poor misguided brit... This is not the circus.

This is the guy standing outside the tent shouting "HURRY! HURRY! HURRY! SEE THE AMERICAN POLITICAL SYSTEM WALLOW IN ITS OWN FILTH!"

The circus comes around May/June.

Jacob

@Iain I like the BBC world news channel. I feel it gives me less BullShit(tm) than US outlets

Iain

@voretaq7 looks like a circus, smells (very) like a circus ...

voretaq7

@Iain you're smelling the filth. And the stench of failure.

That's normal around our politicians.

At best this is the freak show

the main event isn't for months

lovesh

@voretaq7 can you recommend some monitoring system which has worked well in your experience?

Iain

isn't this really a race to see who looses to Obama ?

voretaq7

20:34

@lovesh I can recommend about a dozen - you can also search server fault for "monitoring system" :)

lovesh

ok

voretaq7

@Iain well if Gingrich wins the primary he might win the general

MDMarra

Not a chance

voretaq7

He's crazy. But he's not Batshit-Brand Crazy

MDMarra

I don't think any of them can win

voretaq7

20:34

@MDMarra The best chance out of all the republican candidates

(Unless Palin steps up and runs. Then we're doomed.)

MDMarra

she wont

too late now

voretaq7

I don't think it matters if any of them win. The government here is Proper Fucked, and without radical change (like voting out the entire congress and outlawing private campaign contributions) it's not gonna get better :-/

MDMarra

but it can still get worse

voretaq7

It's really just a race to see who will screw up for the next 4 years.

@MDMarra No, it will get worse. the only question is the pace.

I'm jaded though. Remember my congressional delegation includes Schumer and King...

appleinsider.com/articles/12/03/06/… FLY RUMORS! FLYYYYY

MDMarra

New Apple TV

a week after I bought one

motherfuckers

voretaq7

20:41

@MDMarra 30 day buyer's remorse window!

MDMarra

@voretaq7 No receipt :(

The sales drone didnt email me the receipt for the first time ever

voretaq7

bought it on a credit card? They'll probably take it back.

MDMarra

and i tossed the paper one

yeah i did buy it on a credit card

ewwhite

So much anger!

voretaq7

or call 1-800-my-apple and tell them you need a copy of the receipt, they can probably magic one up for you :-)

ewwhite

20:43

Should I get @Ethabelle in here?

WesleyDavid

@ewwhite Why, she hasn't been harassed enough in person?

voretaq7

@ewwhite will she disparage the candidates and kvetch about Apple rumors?

ewwhite

Harassed?

lovesh

@voretaq7 @MDMarra we have some monitoring in Parallel Power Panel. In that i can see disk usage,CPU usage,memory and things like diskspace,diskinodes,etc. Is it not good enough?

ewwhite

No, but the tone of the discussion here. Angry.

MDMarra

20:44

@lovesh Is it historical?

Does it show you more than just now?

WesleyDavid

@ewwhite I'm thinking of a certain time when vague references to her were getting starred in very, very creepy ways.

lovesh

really

MDMarra

Can it alert you?

Can it give you the state that the system is in when an alert is sent?

lovesh

@MDMarra sorry didnt get this one

WesleyDavid

I have no idea how @RebeccaChernoff manages to keep her cool - but I suspect it has something to do with having the ability to drop records in the user database.

voretaq7

20:45

@WesleyDavid BTW to answer your question from before, no :-)

MDMarra

@lovesh Does it give historical data

WesleyDavid

@voretaq7 sad face

voretaq7

@WesleyDavid voretaq';---\nROLLBACK; drop table users;

lovesh

@MDMarra no

voretaq7

because transactions are cool.

Rebecca Chernoff

20:46

@WesleyDavid shh, don't give away my secrets!

ewwhite

:)

voretaq7

@lovesh not particularly useful

MDMarra

@lovesh Then it's not monitoring

WesleyDavid

@voretaq7 EYE SEA WUT EWE DID THEIR

voretaq7

@RebeccaChernoff @Rebecca';DELETE FROM votes WHERE user='wesleydavid';'Chernoff'!

WesleyDavid

20:47

You know you're pedantic when even your misspellings are spelled correctly.

@voretaq7 hiss spit

voretaq7

update food_dish set tuna=0 where owner=@WesleyDavid;

WesleyDavid

@voretaq7 Oddly, that was one of my lunches today. A can of tuna and some peasant loaf.

voretaq7

@WesleyDavid cat.

lovesh

@MDMarra but can you please explain one thing what quotaugidlimit does? i am getting mad over it. The tech support says that it shows the number of cpanel instances that are active right now and what i understand from its definition is that it is the number of accounts you have for cpanel. I have been googleing this for abt 2 hours now but cant get this straight

i found somewhere that it increases by 2 every time you add a new user because 1 for user and one for group but you can have more than one user in a group

MDMarra

@lovesh Sorry, I've really had enough of this.

This isn't a live support room and UIDs/GIDs are really quite basic

as is reading documentation

lovesh

20:53

@MDMarra ok i ll try harder to understand.

Iain

click avatar. click penultimate item on list

MDMarra

Um

So I'm reading an article on WindowsITPro about improvements to Hyper-V in Windows Server 8 and the big thing that they're touting is being able to use a SMB share for shared storage of VMs.

Does that scare the living shit out of anyone else?

Jacob

@MDMarra Link?

MDMarra

windowsitpro.com/content1/topic/windows-server-8-hyperv-141821/…

It only works with SMB 2.2, so you'll need a WS8 file server as well

Iain

what does smb 2.2 bring to the party ?

MDMarra

21:00

It seems like it would be great for poor-man's HA/Live Migration

@Iain I don't know specifically, but I suppose it has special magic to do with file locking and concurrent access needed to do live migration of HA/FT of Hyper-V VMs

WesleyDavid

@Iain How many are you up to now? =)

Pauska Sock

@ChrisS What's wrong with sock accounts? :*(

Lucas Kauffman

mmmm can anyone recall that story about that sys admin in some US city that shut down (I think) a subway because he found his co-sysadmins to be too incompetent ?

he refused to give them root access on the servers, in the end he gave the key's to the city's mayor

MikeyB

@LucasKauffman Mmm yeah, San Fran I think… Terry Childs.

Lucas Kauffman

but I cant seem to find the story

Chris S

21:10

@PauskaSock I've been in a bad mood lately and I've got an itchy DESTROY trigger finger... >:[

WesleyDavid

@PauskaSock Don't make @ChrisS angry. You wouldn't like him when he's angry.

Pauska Sock

@ChrisS Awww... I hug you!

Oops, I should have taken a bath first.

MikeyB

Terry Childs

Terry Childs is a former network administrator, living in Pittsburg, California. In June 2008, he was arrested on computer crime charges for refusing to divulge the passwords to San Francisco's FiberWAN system to his supervisors. After being arrested he was held on $5 million bail. He is also accused of tampering with the network and subversively avoiding auditing checks. He was found guilty of one count of felony network tampering on April 27, 2010 by a jury. On July 30, 2010, Childs' attorneys filed motions requesting a new trial and for an arrested judgment. Both motions were denie...

Lucas Kauffman

@MikeyB tx !

Jacob

@MikeyB Who the hell is that?

MikeyB

21:15

@Jacob That's the guy @LucasKauffman was asking about.

Lucas Kauffman

@Jacob he didnt want to hand over passwords because he considered his co-admins and other people that he had to give the pw to, incompetent :p

voretaq7

@LucasKauffman which is when you give it to them in a sealed envelope and tell them if they open the envelope you're no longer responsible for the system.

3

Lucas Kauffman

I had to do that once, because someone thought he could do a better job, he ended up crashing the service and deleting a website

Zoredache

@voretaq7 There suppodely was a policy. He claims he was following policy, by saying he would only give them over to the mayor.

voretaq7

@Zoredache Not knowing the policies he operates under I won't say that's entirely impossible.

Policy here requires our CEO to sign off on every major procedure change

Lucas Kauffman

21:23

They tried to make him look bad by saying he had priors, which were basically some youth mistakes that were utterly unrelated to his case

Chris S

It is just me or does the chat search function blow the big one?

Zoredache

@voretaq7 He worked for the government. The whole case sounded like a combination of extremely screwed up management combined with a slightly over-paranoid security guy.

Iain

@ChrisS it's always worked ok for me but I've never tried anything complicated

tombull89

Blog - Stack Exchange: The New Stack Exchange Beta Theme

Robert Cartaino on March 06, 2012

I wanted to give you a quick look at the new Stack Exchange Beta theme. Yes, we are retiring the familiar “Sketchy” theme and rolling out a more-polished and finished design for the beta sites.

Raise the curtain, cue the trumpet fanfare…

Alas, poor Sketchy…

Over the course of the next few weeks, we will be rolling out the new Stack Exchange Beta theme to all the sites still in beta.

At first glance, the new beta theme looks like an unembellished version of any graduated site: Finished, but without any particular “beta” theme, per se. But that’s sort of the point. …

Chris S

@Iain I'm trying to find that Question where I'm the 5 votes to close, and I know someone mentioned it in chat... but I'll be bent if I can find it using the search function

tombull89

21:25

No more sketchy for Beta site...

Lucas Kauffman

I liked sketchy

Iain

@ChrisS I think it was me who mentioned it

voretaq7

I vaguely recall thinking he should have just given them the password in a sealed envelope back when this whole thing happened.
I've bever refused to turn over passwords to an authorized agent of my employer - to do so would be wholly unprofessional.

MikeyB

@voretaq7 Yeah, that's where he fell down. The entire case against him was sketchy, but he didn't do the best thing for himself.

@ChrisS Pretty sure I mentioned it.

Chris S

I thought it was Voretaq, or Iain... But I didn't try to put that in the search...

Iain

21:26

@ChrisS serverfault.com/questions/264539/…

voretaq7

I have however done so only with their express written consent to extortionate consulting fees should they use the password and then destroy the universe

Chris S

@Iain damn... my own answer too.... this just isn't my day

voretaq7

@ChrisS self-loathing: MUST DELETE ANSWER MOAR!

Iain

@ChrisS chat.stackexchange.com/… the search I used

Chris S

@voretaq7 yes, well the answer deserved it... reminds me why I now read the whole question before answering

voretaq7

21:33

@ChrisS I read the question and sometimes still answer the wrong thing. My english isn't so good.

Aaron

That terry guy was in San Francisco right?

I think the one that got me was that his level of paranoia was so high that he never wrote the configs on the routers, on the grounds that it was a security risk if it was powercycled, someone could crack it and get the configuration

Lucas Kauffman

@Aaron he's right

But normally it's so small, the risk overcomes the inconvenience

or is it the other way around ?

Aaron

I'm not saying the theory isn't sound, but if you have a larger risk of people breaking into your colos, powercycling your routers, and stealing your configs than you do of accepting the loss of connectivity during a power loss that you have to drive to the site and re-enter the configuration by hand, then your threat model needs examining

Zoredache

@Aaron That doesn't seem entirely unreasonable. At least in some of the posts it sounds like the equipment was being left in public areas where people could fairly easily get physical access.

Though one does wonder why he wouldn't just have all his configurations in a repository somewhere.

Mark Henderson

@Aaron Im fairly sure thats how RSA had all their keys stolen

Lucas Kauffman

21:39

It's like in Belgium where they had insanely expensive cisco swtiches and routers in closets you could break the lock with a screwdrive. It regulated the motorway signs

voretaq7

0

A: need not user but ?role?

What you appear to want (a section of the disk where only you can see the contents and/or make changes) is best accomplished with Unix File Permissions. Create a group for people who should be able to read files / execute programs in a specific directory, and set permissions appropriately - no n...

sigh

Mark Henderson

One of their firewalls didn't invoke its own rules until too late in the process, and that was enough time during a maintenance window for hackers to grab the config from the firewall

And then it was an open day on the RSA networks

Lucas Kauffman

@voretaq7 aaaaw I was hoping you asked a question :(

Aaron

@MarkHenderson sounds like they didn't do enough penetration testing!

voretaq7

@LucasKauffman If I ever ask a question like that, you will kill me. That's what friends do. Right?

@MarkHenderson WHEEEEEEEEEEE

Lucas Kauffman

21:41

@voretaq7 because you ask so nicely

voretaq7

@LucasKauffman thank you. I'll remember to leave you out of my will so you don't inherit my debt :)

ugh, slashdot

"Warner Bros: New Program To Digitize Your DVDs"

And what does the first D stand for? DUNGBEETLE???

Lucas Kauffman

@voretaq7 mmmm how about those apple shares ?

voretaq7

@LucasKauffman I'm hoping it crashes below 500 so I can buy more :P

Lucas Kauffman

colon :p ?

WesleyDavid

@LucasKauffman -oscopy?

Lucas Kauffman

21:45

@WesleyDavid you wish!

WesleyDavid

@LucasKauffman claws curtains

Lucas Kauffman

@WesleyDavid takes the scope here kitty kitty

voretaq7

@WesleyDavid tuna--;

@LucasKauffman Ripples.... RIPPLES!

Ripples! Get out of the avionics bay!

Lucas Kauffman

mmmm I was thinking more of : There will be blood

voretaq7

@WesleyDavid::

Aaron Copley

21:48

@voretaq7 Big media aside, my mind is still blown. SMH

voretaq7

^^ Bad cats get fed to the engines ^^

@AaronCopley WhatNow?

we're talkin' big media son!

they don't do nothin' by hand. That's why they have INTERNS.

Aaron Copley

@voretaq7 Big media aside: Mind still blown.

Lucas Kauffman

wow, a car with a BMW 64 liter V12

Mark Henderson

@LucasKauffman This that one they had on Top Gear?

From 1946 or something :p

Lucas Kauffman

@MarkHenderson aye, :p watching the epi now

too bad he can't put down the power to the wheels :(

Mark Henderson

21:53

I suspect that if he could the handling would be rubbish. The engine weighs something like 2000kg

Next up they have a Bently with a spitfire engine :P

Adrian

W.T.F. my agency is co-hosting a national conference in a couple weeks. They just realized that it's $350/day to rent projectors and they need 6 more. And they don't want to buy projectors, so our boss is going to be going around to 6 of our buildings the day before he flies out to borrow their projectors and 6 laptops for the 5 days of the conference and check them all in a 2nd suitcase.

Mark Henderson

@Adrian I hope they all come home at the end :P

Lucas Kauffman

@Adrian cheapo's :p

Mark Henderson

But $350/day for a projector IS absurd

Aaron Copley

That's a big suitcase.

Mark Henderson

21:55

But they're probably higher quality than the ones hes borrowing

Lucas Kauffman

@MarkHenderson I just came when that spit's engine started

Chris S

5pm; time to take a couple deep breaths and plan out the next 2.5 years of my life.

Transcript for