English Language & Usage: Multi-Layered Discourse Room

tchrist

00:01

@MrHen Context switching is unnecessary provided you stick to threads.

MετάEd

I seem to have run out of queues. My work here is done.

KitFox

00:42

Oh.

skullpatrol

Oh?

MετάEd

Oh!

KitFox

Yes.

Exactly.

Wait, sim's getting married?

points suspiciously at star board

skullpatrol

01:16

who is sim?

Mitch

@Cerberus words are overrated. Banana. Slip. Fall. Laughter ensues.

@MrHen that's number play, which of course are just special kinds of words.

KitFox

@skullpatrol One of our mods.

Cerberus

@Mitch Slapstick, my favourite kind of comedy.

Mitch

@KitFox that's an interesting inference

Cerberus

Where does the name come from anyway?

skullpatrol

01:20

@KitFox icic

Mitch

@Cerberus it's one of the cultural universals of humor.

KitFox

@Mitch Is it?

Mitch

everybody laughs at somebody falling down.

@KitFox Yes. it is. THere was a study done.

Cerberus

@Mitch It's stupid, and I usually don't find it funny at all. Sorry.

KitFox

You will never guess what I am doing right now.

Cerberus

01:21

I like social comedy, not physical comedy.

Mitch

@Cerberus It applies to humanity, not to robots.

Cerberus

Pah.

Mitch

@KitFox I know I know. I'm not going to tell though.

I give up! You're typing?

KitFox

In between typing.

Mitch

Resting?

your fingers?

KitFox

01:23

I'll give you a hint. It involves 14*4 + 23

Mitch

_calculating... _

another regex thing broken

Cerberus

How much is that, what, 69?

Mitch

what? -79-

Cerberus

Close.

I was thinking she must have admirable typing skills.

Mitch

the guy who got 1538

Cerberus

01:30

Heh.

One does not need typing skills for that.

Mitch

Guess what I'm doing right now.

Cerberus

Threes!

Other things!

Is either option correct?

Mitch

Dammit! How did you know?

but not threes.

cornbread ninja 麵包忍者

wonders what sim looks like

KitSox

@cornbreadninja麵包忍者 It's been 20 years. You can't just waltz back into my life like this.

cornbread ninja 麵包忍者

01:42

@KitSox Baby, I tangoed.

skullpatrol

:D

KitFox

@cornbreadninja麵包忍者 Pretty much like you expect.

skullpatrol

a perfect 10?

KitFox

snorts

I mean, yes.

skullpatrol

Bo Derick was not a 10, imo

!!wiki Bo Derick

KitSox

01:49

@skullpatrol The Wikipedia contains no knowledge of such a thing

KitFox

!!wiki Bo Derrick

KitSox

@KitFox The Wikipedia contains no knowledge of such a thing

KitFox

Hmm.

skullpatrol

see :D

KitFox

!!wiki Bo Derek

KitSox

01:49

Bo Derek

Bo Derek (born Mary Cathleen Collins; November 20, 1956) is an American film and television actress, movie producer, and model perhaps best known for her role in the 1979 film 10. The film also launched a bestselling poster for Derek in a swimsuit, and subsequently she became one of the most popular sex symbols in the 1980s. Her later films were not well-received, either critically or at the box office. She makes occasional film, television and documentary appearances. Early life Bo Derek was born Mary Cathleen Collins in Long Beach, California. According to Internet Movie Database she i...

skullpatrol

icic

cornbread ninja 麵包忍者

@KitFox um . . .

You think so?

KitFox

I don't know. Have you seen her picture?

cornbread ninja 麵包忍者

No.

In my head, she has dark bobbed hair, glasses, and wears doc marten mary janes.

KitFox

Do you know anything about her?

Because you just kind of described Anna Lear.

cornbread ninja 麵包忍者

01:57

Solamente un poco.

I see your Anna Lear and raise you Alison Bechdel.

And I meant hair in a bob.

With bangs.

Maybe a barrette or a scarf or a headband with something attached to it at a jaunty angle.

Wait a minute.

Cerberus

@Mitch Ahh I knew it!

cornbread ninja 麵包忍者

This is how I should try to make characters.

skullpatrol

!!wiki Anna Lear

KitSox

@skullpatrol The Gods of Wikipedia did not bless us

skullpatrol

Wiki has Gods?

KitFox

02:01

Anna Lear ♦, New York, NY

24.7k 6 71 115

.net c# java programming-languages self-improvement unit-testing web-development

skullpatrol

:-O

KitFox

She's a mod and a developer for SE. I thought everyone knew who she was.

tchrist

I don’t know what she’s about.

Rather, I don’t know what her edit was about.

@skullpatrol Yes, and does SE.

KitFox

@tchrist Pardon?

tchrist

Oh, she popped in and fixed AN ALL-CAPS QUESTION THAT WASN’T REALLY A QUESTION WORTH HAVING, BUT MAYBE SOMEONE WILL FIX IT. I KINDA DOUBT IT.

-1

Q: Please help me correct this sentence

Also, the diversification of tourism centers in Istanbul project carried out throughout the year of 2013, called Istanbul within Istanbul and sponsored by the Turkish Development Ministry's Istanbul Development Agency has been completed, introduced and evaluated as model project.

grammar

Very Constantinopolitan.

KitFox

02:15

Mostly it probably hurt her head.

tchrist

That’s understandable.

Sounds like a spamvert for some renewal project for Istanbul, the Constantineopolitan Remake.

No hope for it. It’ll just get lost in the traditional Byzantine bureaucracy there.

Somebody needs to send the Turks a few spare articles. Don’t ask the Russians, though.

Cerberus

@terdon: Wow, voting to close your own question? You're a great sport, I'm sure there is a badge for that...at least I'll vote you up. — Cerberus 22 mins ago

tchrist

This is really good.

10

A: What's the meaning of "pence" in this context?

I contacted the author of the quoted example and this is the response I got (emphasis mine): In this context it's supposed to be analogous to "guide" or "direct". The point of my argument being that antidepressants increase motivation for severely depressed. Unfortunately that motivation can ...

KitFox

Mr.Hen is like that.

He was around quite a lot before your time, then gone for years and he's back off and on now.

tchrist

He’s user 6006. I’m user 2085.

Just sayin’.

KitFox

02:29

Yep. And I knew him before I knew you.

tchrist

We haunted differently.

KitFox

Then it should be no surprise to you.

How are you feeling, @tchrist?

tchrist

You know, the word pace can be a preposition when used in sentences like “Nor, pace Mr. Smith, was I for one moment defending immorality in the journalist.” It originated as the ablative singular of Latin pax meaning peace, as in pāce tuā. But I feel like fuck could move into that sort of slot pretty slickly. It would make a nice antonym to the preposition pace. — tchrist 1 min ago

Very little surprises me.

Well, that’s not true.

I’m easy to surprise.

But surpassingly difficult to astonish.

KitFox

I'm cleaning my tarot cards. How does that strike you?

tchrist

I have a boss who thinks that the reason our code base sucks donkey dick is because in business, you just take what you can get for programmers.

I call bullshit.

The problem is the Obamacare problem.

KitFox

02:36

?

tchrist

The place that pays me has so many layers of idiotic bureaucracy and redtape and bullshit in the procurement process that nobody who is anything but shitty would ever bother getting a job there now. The only good programmers there are the old ones. All the young ones are complete idiots.

Because when the procurement process is that difficult, the real talent says no fucking way am I putting up with that.

See Obamacare website debacle: same problem, same cause, same result.

KitFox

I see.

tchrist

The problem is they virtually only ever hire offshore overseas idiots.

Because they cost nothing.

They think it’s a bargain.

But they have created a fricking disaster of a code base.

And we have no hire-fire power over them.

So we’re fucked, badly.

They don’t get fired for writing bad code.

They get promoted for writing code as fast as possible, not as good as possible.

So it is shitass code from stem to stern, as bad as it gets.

Today I found a disastrous bug in a mission-critical application within our suite wherein the idiots weren’t checking the the status of a completed child process, even if it had aborted and dumped core.

Then I found that this same shitty code HAD BEEN COPIED TO TEN DIFFERENT PROGRAMS verbatim.

So you can’t even fix it once in one place.

You have to fix it ten times.

KitFox

Wow, that's infuriating.

tchrist

I told my boss I was on the verge of tears.

I took most of the afternoon off, it is so frustrating, and they are so dumb.

He’s wrong that we just have to take what we can get because we’re business not academia.

KitFox

02:42

Academia is poorer than business.

tchrist

He thinks I’m smarter than those kids are because I’m older or was more academic.

It is not true.

I was a far better programmer when I was half my age — which in fact, is their age — than they shall ever, ever be.

Yes, I’ve been in academia. They pay next to nothing.

KitFox

They wanted to put CS undergrads on my development "team" at the last place.

tchrist

But even the worst ones there were infinitely better than the best of these.

KitFox

I saw the code they'd written for a website project, holy crap, what garbage.

tchrist

I was a “Senior Professional Research Assistant”, basically some research prof’s gopher.

Yes, and then there’s that.

They’re reading in UTF-8 and emitting it as ISO-8859-1 on our website.

And don’t understand what they’re doing wrong.

That’s the least of our woes, really.

KitFox

02:44

Plus, I needed to log in to the website! Oh noes! Luckily, the username was admin and the password was admin.

tchrist

It still aggravates me.

You know, I just had that very experience. Different word, but same-old, same-old.

Americans get fired for the least security-related blunder.

But offshore we cannot fire, and they keep doing this shit.

KitFox

I should pick your brain about encodings sometime. I have a feeling that I should know more about what I need to think about with this current project.

tchrist

It’s a lot easier than people make it. Interacting with databases can be a bit prickly sometimes though.

The bottom line is there is no such thing as a text file.

That’s the brain-bug.

All files have an encoding.

If you do not know the encoding, you might as well have encrypted data for all the good it will do you.

Vs lbh qb abg xabj gur rapbqvat, lbh zvtug nf jryy unir rapelcgrq qngn sbe nyy gur tbbq vg jvyy qb lbh.

Quoth Caesar.

Cerberus

@tchrist Why not?

KitFox

I think, but I'm not positive, that the majority of web content is stored in a database and barfed out by the CMS.

Cerberus

02:49

Are these people in India?

tchrist

@KitFox Yep.

KitFox

I think knowing something about encoding is probably important.

tchrist

@Cerberus Yep.

Or Istanbul.

Cerberus

I see.

tchrist

For all the Byzantine code.

KitFox

02:49

But I don't know what.

Cerberus

Which is worse?

Oh haha.

tchrist

I’m joking you.

Cerberus

Yeah I got it.

Byzantines speak very poor English, comparatively.

tchrist

@KitFox Well, ok. There are two issues: the encoding of the data in the database and the encoding you need to use in the presentation layer. These may well be different.

It’s really much better to unify on UTF-8, but this is not always an option.

KitFox

So what kind of problems will I see if the encodings are not harmonious?

tchrist

02:51

If you know the legacy encoding, you can upgrade safely.

You will see mojibake.

KitFox

Squares?

With hex numbers in them?

Cerberus

What I hate most are the diamonds with question marks in them.

tchrist

> Mojibake (文字化け?) (IPA: [mod͡ʑibake]; lit. "character transformation"), from the Japanese 文字 (moji) "character" + 化け (bake) "transform", is the presentation of incorrect, unreadable characters when software fails to render text correctly according to its associated character encoding.

Cerberus

Ugh.

Yeah those suck too.

KitFox

OK. So will it always be that obvious?

Cerberus

02:53

What causes this?

tchrist

You will see “Prêt-à-Manger” turn into “PrÃªt-Ã -Manger”.

Cerberus

I see this all too often, on the websites of respectable institutions.

tchrist

@Cerberus Encoding error.

Cerberus

Oh...

tchrist

The content is not in the declared encoding.

It’s lying.

Cerberus

02:54

Ah, I see.

Most unfortunate.

KitFox

How will I know? Do I have to check the encoding of every page on the site?

tchrist

> Mojibake is often caused when a character encoding is not correctly tagged in a document, or when a document is moved to a system with a different default encoding. Such incorrect display occurs when writing systems or character encodings are mistagged or "foreign" to the user's computer system; if a computer does not have the software required to process a foreign language's characters, it will attempt to process them in its default language encoding, usually resulting in gibberish.
Transferring messages between different encodings of the same language can also result in mojibake. Japane…

@KitFox Unlikely.

Have you seen an actual problem?

This is not true:

> Mojibake in English texts generally occurs in punctuation, such as em dashes (—), en dashes (–), and curly quotes (“, ”), but rarely in character text, since most encodings agree with ASCII on the encoding of the English alphabet. For example, the pound sign "£" will appear as "Â£" if it was encoded by the sender as UTF-8 but interpreted by the recipient as CP1252 or ISO 8859-1. If iterated, this can lead to "Ã‚Â£", "Ãƒâ€šÃ‚Â£", etc.

2 mins ago, by tchrist

You will see “Prêt-à-Manger” turn into “PrÃªt-Ã -Manger”.

KitFox

@tchrist No, I don't think so, but there are 14,000 static pages.

tchrist

Checking them all is very, very, very easy, of course.

The thing is, looking at their own content declaration may not suffice.

The webserver might override it.

KitFox

sighs

tchrist

02:57

Or they may read in data from elsewhere and then misrepresent it.

You would have to also check your webserver config.

The worst is that some browsers will honor a webpage overriding the HTTP header, and some will not.

So you can never know if you are screwing people.

Best to have both agree.

The problem is that this:

<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

Or something like it may appear in the HTML, and it is supposed to override the HTTP header.

But some browsers ignore it.

The default encoding on the web is ISO-8859-1. It’s a problem when you can’t override it.

Here, if you do nothing else in your life, never name a file foo.txt ever ever again.

Name it foo.ascii or foo.cp1252 or foo.macroman or foo.latin1 or foo.utf8 or foo.utf16

But not foo.txt

That tells you nothing, because there is no longer any such thing as a “plain” text file.

You have to know its encoding. No ifs, ands, or butts.

And yes, I’ve written programs that work from a statistical model to “guess” the encoding.

They get 99.97% correct — on the set they were trained on, of course.

But on other stuff, well.

The problem in the West is all the 8-bit encodings.

The problem in the East is all the 16-bit encodings.

KitFox

If I didn't know, is there a best guess?

tchrist

If everyone would just unify to UTF-8, much pain would be avoided.

@KitFox Not without a statistical model to work from.

Do you know that it is English, first of all?

If not, all bets are off.

If so, and you have a reasonable training set, you can do well.

KitFox

OK, so if we assume that the content is generated through a CMS, what kind of issues should I anticipate?

tchrist

It probably thinks it knows the encoding of the data it is working with.

It certainly will tell you that the stuff it's giving you is in such and such an encoding.

Where it is or not, well, that’s elsewhere.

The thing is, things that are not UTF-8 that claim to be always invalidate nearly instantly.

The problem is that going the other way around, you cannot detect the error.

That is, if you send something is UTF-8 through a system that says it’s something else like cp1252, you’ll see stuff like my mangled French.

But if you go the other way, it will instantly invalidate as illegal strings in UTF-8.

KitFox

Could other encodings sneak in if the content is created using the CMS?

tchrist

03:09

What I’m seeing is that we have different point-of-sales data entry drones in different countries feeding in data in whatever they have handy. Then they lie to us about what it is. In order of most frequent lies, it is cp1252, then iso-8859-1, then MacRoman.

If the content system says it is giving you ASCII, this is trivial to check: no bytes valued 128–255.

KitFox

I see, so the input points could pick up differently-encoded data and then that's pulled out of the database...

tchrist

My problem is that 1) I have feeds that claim to be ASCII that have high-bit bytes, and 2) I have feeds that claim to be and truly are UTF-8 but our CMS is shoving them out as ISO-8859-1.

KitFox

Is there a way to handle that?

tchrist

@KitFox Exactly right.

@KitFox I recommend Guido.

And broken kneecaps.

KitFox

Is he a fixer?

tchrist

03:11

Get a formal agreement.

Then hold their feet to the fire.

Make them contractually swear to the encoding they have agreed to provide to you.

KitFox

Can the inputs be converted or at least checked somehow?

tchrist

When you can show them they are breaking that contract, you can force them to fix it.

They can be checked, yes.

But not with 100.00% surety on arbitrary data.

At least, not if it is 8-bit data.

If it is multibyte UTF-8, as I said, it is very hard to sneak 8-bit data through that is still legal in UTF-8.

The problem is the unibyte data of myriad encodings.

You can check for certain things that “can’t” happen, say in cp1252.

But it all comes down to counting up likelihoods.

It is um, not something you want to tackle.

I’ve done it. It is um, verging on serious programming.

KitFox

Well, I'm not a programmer anymore.

tchrist

Do not try it at home, or anywhere.

KitFox

So I wouldn't be the one doing it.

But I would be the one asking lots of questions about it.

tchrist

03:15

Get everybody to agree to tell you what encoding their stuff is in.

Do not accept “text” for an answer. Ever.

Because then you have a chance. Even if different sources use different encodings, so long as they are not lying, you can deal with it.

I once had to process a 9 GB text file, plain text database thing, which had allowed anyone to enter stuff in it.

It randomly contained at least 4 different encodings in it. It was a nightmare.

Because people just have no clue about what their system is entering.

After a while, you develop a knack for looking at a piece of mojibake and being able to almost immediately see just what they did wrong.

But to reach that point, you have to have looked at a whole lot of fuckups, so you can recognize them.

It is not something anyone ever sets out to do.

But it happens, and you develop a sense for it.

KitFox

The development of fu.

tchrist

Yes.

KitFox

I was integrated with my code on the last project. Did I tell you that they are discontinuing it?

tchrist

Rob thinks it’s all hopeless because of the fucked-up database problem I just told you about.

I don’t know what that means, that you were integrated with your code.

Rob thinks there are too many databases with randomly mixed up encodings in the same thing that people will never unravel it. I think he’s almost wrong.

But mostly right in practical terms for many situations.

KitFox

@tchrist I could see the breaks in it, knew where it was failing. Almost Zen.

tchrist

03:22

I can survive because the very bureaucracy that I abhor requires formal, binding agreements on all upstream and downstream feeds. So when somebody screws up, we can apply the thumbscrews until they fix it.

Because they promised, and we have it in blood.

KitFox

It's almost a shame that we'll never work together.

tchrist

Rats on a sinking ship, eh?

KitFox

Pardon?

tchrist

Of course, when we screw up, then we get our thumbs screwed.

I mean, did you see that it was failing and flee?

KitFox

Fair's fair.

@tchrist Well, no. I'd fix it.

tchrist

03:24

Thing is, we get blamed when somebody upstream of us feeds us garbage and we pass it along.

KitFox

I'm being romantic about my code.

tchrist

And when it’s $140 million bucks for a five-day screw-up, there is extremely strong motivation to make it right.

Maybe that was an 11-day one. I forget.

In any event, the Guidos they send for you went you’re talking ten figures is enough to terrify anyone.

We saved the day because we have an internal data retention policy and could rerun the billing data and get all the money back.

But it really is a lot of money.

If employees got even a 1% bounty on the money they rescued for the company, many of us could retire forever, and not even our grandchildren would ever need to work again.

But it never works that way.

If anyone ever cites this, I was hallucinating, and I work for a cobbler who lives in a shoe.

@KitFox That’s not necessarily a bad thing.

KitFox

I don't even know what to say about that.

I'm hungry. What you got in your pocketses?

tchrist

Pride of workmanship is better than the reverse.

Mojibake, per usual.

Mojibake

(; lit. "character transformation"), from the Japanese 文字 (moji) "character" + 化け (bake) "transform", is the presentation of incorrect, unreadable characters when software fails to render text correctly according to its associated character encoding. Causes Mojibake is often caused when a character encoding is not correctly tagged in a document, or when a document is moved to a system with a different default encoding. Such incorrect display occurs when writing systems or character encodings are mistagged or "foreign" to the user's computer system; if a computer does not have the soft...

I have cake.

KitFox

noses in tchrist's pockets

tchrist

03:31

Mithrandir was there first.

Fortunately, it wasn’t fishcake, so he left it be.

cornbread ninja 麵包忍者

I got some gummy bears. They're real soft.

KitFox

pricks up ears, licks crumbs off whiskers

cornbread ninja 麵包忍者

Oh, and a few tater tots.

KitFox

tongue lolls

I could make potato treats, but I should be in bed already.

cornbread ninja 麵包忍者

stuffs bear into tot, tosses tot into the air

Have you looked into Scrolls?

KitFox

03:35

snaps tot out of air

@cornbreadninja麵包忍者 Mrpf.

What should I ask the project manager candidate who I'm interviewing tomorrow?

tchrist

hm

I assume they’ve managed projects comparable in size to the one they’re being interviewed for.

But what about comparable in deadlines, complexity, external requirements, etc?

KitFox

Well, there's the rub. I don't know what project he'd be on.

tchrist

What was their worst disaster they confronted as a PM?

Oh hm.

We have both a PM and a BA.

They take care of a lot of the paperwork nonsense, and the sweet-talking to externals.

KitFox

I might be able to find that out before the interview. I don't have any of his application materials either.

@tchrist I'm a BA! That's what I do now!

tchrist

How can you interview him then? This sounds odd.

Ah.

KitFox

03:39

I was asked to do it because my manager is out of town.

tchrist

You wanna know something funny?

KitFox

She was much more excited and pleased about it than I thought she'd be.

@tchrist Always.

tchrist

My previous BA, who is a delightful and frankly marvelously cute young woman from Kiev, is a better programmer than some of the offshore staff. She looked at their code that was fouling up and saw the bug immediately. I was quite impressed.

cornbread ninja 麵包忍者

!!afk scrolls

tchrist

It was a simple logic error.

KitSox

03:41

@cornbreadninja麵包忍者 Just go already!

cornbread ninja 麵包忍者

._.

KitFox

@tchrist Sounds delish.

tchrist

Oh she is, she is. That’s when I said to my boss that we need better programmers, if even our BAs can beat them.

KitFox

I think my background as a developer has been extremely useful in my job so far. I can see why the technical side gets frustrated with the others.

They don't think right.

tchrist

She has lots of spunk. I don’t know if it’s cultural. It’s so very opposite of the subcontinental culture.

KitFox

03:43

Also, I think this is a job that is suited to my talents. I feel like I'm actually good at this.

tchrist

The think about our Russian and Ukrainian folks, who are all onshore and in the local office here, is that they work incredibly hard and are really smarter than you would believe.

KitFox

Well, you know how I feel about Russians.

tchrist

They’re US citizens now. Our ironman tester is a former KGB intelligence agent! Nothing perturbs him. He endures pressure as though it were nothing. We have two Russian testers, actually, but he’s amazingly diligent.

The techies should not be allowed to talk to non-techies.

KitFox

I agree.

tchrist

We specifically have our BA be our externals contact, with everything going through her. She finesses well.

If she is on vacation, our PM takes that position. In no circumstances do you want a developer doing it.

KitFox

03:47

The developer I work with apparently rather notoriously told the stakeholders group to be quiet and raise their hands if they wanted to speak when he was presenting to them the first time.

tchrist

Even though both those women are former developers themselves.

Oh my.

KitFox

He brags about it.

I use it to support my position.

Which is that he shouldn't attend stakeholder meetings.

tchrist

They never let us get anywhere close to the enduser customers. Although other people in a “cousin” group have a bit of customer facing stuff. Very scary.

I’m delirious. Was up at 4:30, probably same actual time as you. Must sleep. I seem to live in the Atlantic timezone.

KitFox

I think this is the guy I'm interviewing. Anything jump out at you?

Oh. Never mind.

tchrist

Are you doing health are now?

KitFox

03:50

And we're just very east in the Eastern, damn it.

tchrist

I ddi that at the previous job, kinda.

Maine should be on Atlantic time, everyone knows that.

KitFox

Biomedical research, so it's related.

tchrist

Nope.

KitFox

Non-profit.

tchrist

Yes.

I trained my encoding-guesser on all of PubMed, and most of Elsevier.

Well, PMC.

KitFox

03:52

Hmm.

tchrist

It’s very good at guessing the encoding of biomedical research papers written in English.

Ok, talk later. Sleep now.

KitFox

Good night. Thank you.

tchrist

Sure. Hasta.

Jasper Loy

09:53

@KitFox I was referring to someone you don't know at all, lol, just to clarify.

Transcript for

English Language & Usage: Multi-Layer…