English Language & Usage: Multi-Layered Discourse Room

GnomeSlice

3:00 PM

So I got a job at a security company

wtf I can't be a security guard

KitFox

Congrats.

terdon

@KitFox Case?

GnomeSlice

@KitFox All this grepping is getting me all hot and bothered

KitFox

@terdon I copypasted the things I wanted to match.

terdon

I might be able to help if you show me the input and regex,

KitFox

3:03 PM

I can in a minute.

Mr. Shiny and New 安宇

@KitFox what are you using to grep? cmd-line grep tool?

George Pompidou

ugh… another black person got shot by a white person and it's the end of the world again in America.

terdon

I think she uses some kind of strange version, not the standard GNU grep.

George Pompidou

it's Treyvon 2.0

GnomeSlice

!wiki grep

George Pompidou

3:09 PM

who uses non-gnu grep?

O.o

KitFox

@Mr.ShinyandNew安宇 Yes, and also AstroGrep.

GnomeSlice

!!wiki grep

What the hell is grepping.

terdon

@GeorgePompidou Cygwin, UNIX, BSD, OSX,

Mr. Shiny and New 安宇

@KitFox because if you're using a command line grep you might have an issue with escaping/quoting the pattern so that the shell doesn't screw it up

terdon

Grep

Grep is a command-line utility for searching plain-text data sets for lines matching a regular expression. Grep was originally developed for the Unix operating system, but is available today for all Unix-like systems. Its name comes from the ed command g/re/p (globally search a regular expression and print), which has the same effect: doing a global search with the regular expression and printing all matching lines. == History == Grep was created by Ken Thompson as a standalone application adapted from the regular expression parser he had written for ed (which he also created). In ed, the command...

KitFox

3:10 PM

@Mr.ShinyandNew安宇 Theoretically, putting it in a text file fixed that.

George Pompidou

@terdon nobody on earth uses original unix grep, and osx is bsd grep

which functions the same as gnu grep.

I also was just being sarcastic.

and what the heck is cygwin?

Matt Эллен

@GnomeSlice like gripping, but with fewer hands

George Pompidou

is that like smoking cigarettes when you're using windows?

GnomeSlice

??

Mr. Shiny and New 安宇

@KitFox ... so you're sourcing your pattern from the text file?

GnomeSlice

3:12 PM

Oh it's a computer thing

Mr. Shiny and New 安宇

@GeorgePompidou cygwin is a unix environment for windows.

Matt Эллен

you can search through text for a pattern

George Pompidou

@Mr.ShinyandNew安宇 I know, I was also joking about that.

KitFox

@Mr.ShinyandNew安宇 Yes.

Mr. Shiny and New 安宇

@KitFox then hm.

George Pompidou

3:14 PM

this is genius. I've been trying to get linkedin to stop spamming me for a few days, and just now read: "changes will take effect in a few days"

what's so hard about not emailing me starting today

Matt Эллен

yeah. I've never understood why companies do that

terdon

@GeorgePompidou No it doesn't unfortunately. It lacks all sorts of options. There are in fact a surprising variety of grep implementations, most of which are not GNU grep. Believe me, we get many questions about that on U&L.

George Pompidou

hm. I've been using OS X and gnu grep the exact same for years now.

Matt Эллен

Underwear & Lingerie

George Pompidou

I guess I just got lucky.

Mr. Shiny and New 安宇

3:15 PM

@GeorgePompidou It's probably 90% CYA text in case there are emails in transit already. 10% chance that they have so much volume of mail they actually queue it up to send it out in the future, and they can't (or can't be bothered to) stop it once it's queued.

KitFox

@Mr.ShinyandNew安宇 When I'm off the phone, Il post a sample.

Mr. Shiny and New 安宇

@KitFox k. Do you have cygwin installed?

George Pompidou

@Mr.ShinyandNew安宇 I thought of that, but I unsubscribed Tuesday from daily group update emails that get sent every morning at 10. and still got them up to today.

KitFox

@Mr.ShinyandNew安宇 No.

George Pompidou

@KitFox your problem is you need to uninstall Windows. it's a virus.

Mr. Shiny and New 安宇

3:17 PM

@GeorgePompidou It's also possible they have a distributed system that doesn't push changes like that through very quickly. Which would be dumb, or at least annoying.

terdon

@GeorgePompidou Off the top of my head, OSX grep does not support -o which is among the most useful grep options.

George Pompidou

it's just dumb. it's 100% possible to stop sending me emails within the day.

@terdon sure it does. I just did it.

terdon

@GeorgePompidou Huh, not according to its man page. How about -P?

George Pompidou

one sec, let me switch to my mac and show you.

terdon

ss64.com/osx/grep.html

Perhaps you've installed GNU grep on your mac?

George Pompidou

3:19 PM

> fi:~ george$ grep -V
grep (BSD grep) 2.5.1-FreeBSD
fi:~ george$ grep
usage: grep [-abcDEFGHhIiJLlmnOoPqRSsUVvwxZ] [-A num] [-B num] [-C[num]]
[-e pattern] [-f file] [--binary-files=value] [--color=when]
[--context[=num]] [--directories=action] [--label] [--line-buffered]
[--null] [pattern] [file ...]
fi:~ george$

I can also send you screenshot of manpage. it's nearly identical to gnu.

I may just do a diff on it.

but I have better things to do.

terdon

@GeorgePompidou Huh, OK. I've been reading the online manpages and they're different. I also answered a question where the OP was lacking some basic grep functionality in OSX. Anyway, you also have the other *nixes, busybox embedded systems etc, all of which have simpler grep implementations.

George Pompidou

sure.

like I said, I know that there are other greps. I was just making a sarcastic reply to a "probably has some non standard gnu grep" comment

because I myself use freebsd grep more often than not.

and it's the same.

Matt Эллен

nuh uh. they have different version information

George Pompidou

here's a unix question for you @terdon. how can I redirect a manpage to a file without it coming out all screwy?

GnomeSlice

watch me mace myself on the first day

Matt Эллен

3:24 PM

first day of what?

George Pompidou

of Christmas, duh.

terdon

@GeorgePompidou screwy? Just man ls > file should work.

GnomeSlice

25 mins ago, by GnomeSlice

So I got a job at a security company

George Pompidou

it doesn't. comes out all messed up. maybe it's because manpages don't get sent straight to output

Matt Эллен

@GnomeSlice I see! Good luck

George Pompidou

3:25 PM

they work like more or less or something.

KitFox

This is the sample:

<head><a class="Mice" onclick="_gaq.push(['_trackEvent', 'button', 'searchMice', 'micedb']);" href="http://my.url/query/"></head><body id="top" class="js" style="font-size: 68.75%;"><a title="" target="_self" href="http://my.url/query"></body>

This is the regex: <body.*<[^<>]*href="http://my.url/query/*[^<>]*>

terdon

@GeorgePompidou No, you have the groff or whatever formatting stuff. You mean you see strange characters like [M or whatever?

George Pompidou

nope. just same character multiple times a lot.

KitFox

I'm still on the phone.

George Pompidou

I got it.

man grep | col -b > ~/Desktop/mangrep.txt

^.^

capote.s3.amazonaws.com/s/mangrep.txt

there we go

terdon

3:27 PM

@GeorgePompidou Weird. man grep > foo works perfectly for me.

Mr. Shiny and New 安宇

@KitFox you have links in <a in the head??!

terdon

@KitFox looks fine to me. It matches <body id="top" class="js" style="font-size: 68.75%;"><a title="" target="_self" href="http://my.url/query"> from your file.

KitFox

Shit.

That's not at all what I meant to do.

George Pompidou

where's your grepping prowess now Kit

chuckles

KitFox

Well, it still should have matched!

terdon

3:33 PM

What is it you actually want to match?

Mr. Shiny and New 安宇

It matches for me in jEdit. I'm going to try cmd line now. But you have query/*, is that intentional?

KitFox

I just got off the phone. Let me spend a moment orienting.

GnomeSlice

The hone?

George Pompidou

what's a hone?

terdon

She was honing here regex skills.

George Pompidou

3:34 PM

ooh! she meant a phone!

GnomeSlice

I like hone.

Mr. Shiny and New 安宇

$ grep -E '<body.*<[^<>]*href="http://my.url/query/*[^<>]*>' test.txt
<head><a class="Mice" onclick="_gaq.push(['_trackEvent', 'button', 'searchMice', 'micedb']);" href="http://my.url/query/"></head><body id="top" class="js" style="font-size: 68.75%;"><a title="" target="_self" href="http://my.url/query"></body>

It matches for me with cygwin grep

my test.txt has the whole text in one line, but still.

if I reformat the html like so:

<head>
  <a class="Mice" onclick="_gaq.push(['_trackEvent', 'button', 'searchMice', 'micedb']);" href="http://my.url/query/">
</head>
<body id="top" class="js" style="font-size: 68.75%;">
  <a title="" target="_self" href="http://my.url/query">
</body>

it no longer matches

KitFox

Hmm. That must be it then.

Something with whitespace or newlines?

Here is the revised test text:

<div id="header"><a class="Mice" onclick="_gaq.push(['_trackEvent', 'button', 'searchMice', 'micedb']);" href="http://my.url/query/"></div><div id="envelope"><a title="" target="_self" href="http://my.url/query/f?q"></div>

And revised pattern: <div id="env.*<[^<>]*href="http://my.url/query/*[^<>]*>

It should match the second url and not the first.

Mr. Shiny and New 安宇

grep doesn't match across newlines

George Pompidou

I hope nobody comes by my cubicle because I farted.

I said, as the CEO of the company pops up to talk to the guy next to me.

KitFox

3:40 PM

@Mr.ShinyandNew安宇 I don' t think I knew that.

terdon

All such tools are line-based.

KitFox

I wonder if I can count on their inconsistency.

Mr. Shiny and New 安宇

@KitFox There are variants that support it. or other tools. regexes aren't inherently restricted to single lines

terdon

If you have access to basic *nix tools, you can simply remove the newlines: tr $'\n' ' ' < file | grep PATTERN

Or perl -pe 's/\n/ /' file | grep PATTERN

KitFox

There's a link in the header section that is to be replaced. I want to find all the other pages where the link appears, but not in the header section (because that is on all pages).

Mr. Shiny and New 安宇

3:43 PM

can you count matches?

KitFox

Oh, yeah, I could do that. It's a lot of content though, and I don't want to scrape it again.

Mr. Shiny and New 安宇

are there pages where the link ISN'T in the header?

KitFox

@Mr.ShinyandNew安宇 I could, but it returns so large that the file barfs.

@Mr.ShinyandNew安宇 No.

Mr. Shiny and New 安宇

because you could just find all the pages where count >= 2

$ grep -E 'http://my.url/query' -c test.txt
2

KitFox

Oh god, it's worse than I thought.

Mr. Shiny and New 安宇

3:46 PM

$ cp test.txt test2.text
$ grep -E 'http://my.url/query' -c test.txt test2.text
test.txt:2
test2.text:2

George Pompidou

did you guys know that grep is a noun and verb in the Oxford dictionaries?

KitFox

It's matching "...query/" when I put ..."query" as the pattern. That's annoying.

This is an impossible frigging task.

Mr. Shiny and New 安宇

grep shows matching lines

so besides identifying files that need changing, what do you want to achieve?

GnomeSlice

All this grepping is making me frisky.

George Pompidou

frisk frisk

terdon

3:58 PM

Well, after all, frisking does involve some groping.

KitFox

OK, here's the new plan. Tell me if it's stupid.

Step 1. Map the old query url to something else.

Step 2. Put a permanent redirect to the old query's original url to point it to the new search url.

I love how talking it through makes it apparent that I have wasted a couple of hours trying to figure out something that it is not my job to figure out.

Mr. Shiny and New 安宇

for Step 1 do you mean "make a new search url" and Step 2 "rewrite the old search url to match the new one"?

That is a reasonable thing to do when changing the incoming urls is impractical.

KitFox

No, I just realized that the dev has to solve this problem, not me.

But the requirement is "replace the old search with the new search, but keep access to the old search".

Mr. Shiny and New 安宇

yeah you can just make a requirement that says "all the old pages must still work, somehow"

what does "keep access" mean?

KitFox

So if you have the old site bookmarked, you should redirect to the new search instead, but there is a link from the new search back to a place where you can use the old search if you prefer it.

Is the plan.

Mr. Shiny and New 安宇

4:05 PM

ah

does the new search use the same query syntax as the old?

KitFox

No.

Mr. Shiny and New 安宇

or rather, will the new search produce sensible results for old queries?

KitFox

The new search is completely different.

Mr. Shiny and New 安宇

hm.

I'd leave the old search alone and just update pages to point to the new search. no mapping of anything.

but I don't know how many pages you need to touch to make that happen.

KitFox

The old search used a cobbled together query for an Oracle database. The new one uses facets and things to search against a mongo db.

Updating the pages looks like it could be a massive undertaking.

George Pompidou

4:07 PM

yay! my boss said I can leave early today for my drive to Boston ^.^

is excited to see his best friend

Mr. Shiny and New 安宇

Can you translate the old queries into new ones?

George Pompidou

jiggles around in office chair excitedly

KitFox

@Mr.ShinyandNew安宇 I don't think so. It uses an Apex form or something. I can understand that it is sending parameters, but it makes no sense to me.

Mr. Shiny and New 安宇

I don't know what an Apex form is. But if there were some way to algorithmically take the input to one query and translate it into the input for the new query then you could replace the old search with the new, and on the results page put a link to the (moved) old search page.

Then the old pages don't need to change but they get the new search.

Though I have to wonder what the point is. Is the old search being retired? Deprecated? Is it just too slow to use? etc.

KitFox

No, it's not like that. It's "Want to search for mice? Click this link."

Embedded randomly into various pages.

The old search bites ass.

So we're replacing it with a new search application.

One with business logic!

GnomeSlice

4:11 PM

Bites ass?

KitFox

Anyway, we launched the beta version last night.

Mr. Shiny and New 安宇

Are the pages where the search links appear generated, or static?

KitFox

static

GnomeSlice

@KitFox Hey, is your internet working today?

user116848

Man, it's very depressing today. I just got my result and got failed in one of the papers :(

GnomeSlice

4:13 PM

You must check out today's music, it's so good. soundcloud.com/nudiscoyourdisco/holic-santiago-wmc-2014

user116848

Hi folks

KitFox

Hi @Arrowfar. Sorry to hear it.

@GnomeSlice I'll check it out. Thanks.

user116848

Yeah, me too. I hope I clear it next time.

Mr. Shiny and New 安宇

@Arrowfar boo! Well, don't give up.

KitFox

You only failed one, right? So that's many you did not fail.

user116848

4:15 PM

@KitFox, @Mr.ShinyandNew安宇 Yeah thanks guys.

user116848

Failing sucks :(

user116848

BTW it was one of the ACCA papers.

KitFox

caca papers? I think I would want to fail those.

GnomeSlice

^

user116848

Yeah. It is one of those accountancy qualifications (UK one)

GnomeSlice

4:18 PM

tata papers, on the other hand...

user116848

They have 14 papers in it and I have cleared 10 of them so far. 4 to go. It has been almost 6 years. I am that slow, huh?

user116848

Normal people usually take four to five years to complete it.

KitFox

Eh, so you have a life.

GnomeSlice

Heh

user116848

Yeah I do.

George Pompidou

4:24 PM

normal people take 6-7 hours to drive to Boston. my record is 5.5

KitFox

Oh ffs, astrogrep gives me the option of specifying a number of occurrence anyway.

George Pompidou

but I did get a speeding ticket when setting the record.

I'm going to try to beat it this time.

KitFox

I thought you lived in Chicago.

George Pompidou

nope. upstate NY for school.

KitFox

Oh.

skullpatrol

4:25 PM

Too much time spent on the internet @Arrowfar ;-)

George Pompidou

I need a more fuel efficient car :<

KitFox

Maybe you know Sara and ... the guy she married back in 2001 or 2 or so.

If you live in Troy anyway.

user116848

@skullpatrol Yeah I think so. I will have to watch my time carefully.

George Pompidou

haha. nope, Rochester.

I almost went to RPI in Troy.

KitFox

Oh, then you know Stephan and Marlene.

And Dayeol.

skullpatrol

4:26 PM

@Arrowfar time is money.

George Pompidou

my neighbor's name is Stephen.

Kit, I'm starting to think there are all of these connections because you just know someone everywhere in every situation.

KitFox

Is he French Canadian?

George Pompidou

he's Korean and Colombian.

close. but not quite.

GnomeSlice

Close enough.

George Pompidou

jincks

KitFox

4:27 PM

Then it's not him.

user116848

@skullpatrol You are right.

George Pompidou

know anyone at Tufts?

or in the greater Boston area?

in particular people from Newton or Chicago

KitFox

Yes.

Relations.

Friends and former friends.

George Pompidou

do you know any gay puerto rican black supremacists who are convinced that the "upper middle class white imperialist men" are out to get them?

KitFox

Oh, I forgot Stephan is at URMC.

@GeorgePompidou Just the one, but she's a woman.

George Pompidou

4:29 PM

sigh

KitFox

And she lives somewhere near Miami Beach.

George Pompidou

they're everywhere!

KitFox

Oh and looks like Daeyeol is at Yale now.

George Pompidou

that's a silly name.

and an appropriate name of a school for the name.

does he drive a Huayra too?

KitFox

It's Korean. Pretty common too, iirc.

George Pompidou

4:31 PM

that'd be funny. Daeyeol drives a Huayra at Yale.

all impossible things to say.

KitFox

I don't get why that's impossible to say.

George Pompidou

it's literally impossible.

user116848

Bye guys. I should go and study

KitFox

Good luck.

George Pompidou

literally 100% of humans are completely unable to say any of those three words.

KitFox

4:34 PM

This is so goddamned asinine that I can't find anything on this effing site even if I have an idea what the effing directory structure looks like.

George Pompidou

what is asinine, and what are effing sites and directory structures?

how does one eff?

KitFox

It's fucking without you in it.

George Pompidou

hm.

update on my position on votes and points: I now officially care about reputation points temporarily until further notice.

I recently found that at a certain reputation, I can edit posts without needing approval. I want this a lot.

goes to answer easy questions for extra rep

KitFox

4:53 PM

Here is a page that I know is in the research directory. I'm on the main page, and I figure, oh, must be this link that says "Research". Nope. That takes me to the research subdomain. Turns out, I wanted the link that said "tell me more".

Cerberus

Hmm America is bombing the Islamic crazies.

I guess that is good.

KitFox

My other favorite one is the "Cre Mice by site of expression" button that takes you to a page that tells you about the cre mouse repo with a hyperlink to lists of cre strains, which if you look there, gives you several more hyperlinks of the various kinds of cre tools, and then if you are very nice, you might click through to a list that actually contains the site of expression. But not always.

Transcript for

English Language & Usage: Multi-Layer…