English Language & Usage: Multi-Layered Discourse Room

Zairja

17:00

@tchrist I would edit this user's "edit" and "update" as they don't add anything to the question. I also find it ironic that the user states "I’m not asking for everyone’s opinion" but the top two answers are just that. :b

tchrist

@Zairja I quite agree.

@MετάEd Eh?

MετάEd

@tchrist It's not an unmatched (.

tchrist

Unclear. What is the pre-interpolated code?

MετάEd

I'm afraid I'm trying to construct a mnemonic memory circuit using stone knives and bearskins.

tchrist

Fear is the mind-killer.

MετάEd

17:04

@tchrist Quit quoting competing science fiction universes.

Star Trek is the only true Ghod.

tchrist

I'm sure it really is an unmatched left paren.

I just don't yet know for sure why. Perhaps you are accidentally self-hosing via the interplay of interpolation and metaquoting. But I cannot say without seeing what the perl compiler saw, not just what its regex compiler saw.

MετάEd

The pre-interpolated code? if ( $message =~ /$patterns[ $i ]/ )

tchrist

That's simple enough then.

MετάEd

Damn straight.

tchrist

Although I don't know what the slashes are for.

In any event, you have an unmatched paren in that array element.

What you are doing with \Q in there I have no idea, since it is not respected there.

MετάEd

17:08

I assume I am using it wrong.

I don't get no respect.

tchrist

The regex compiler does not know what \Q is.

Only the perl compiler does.

MετάEd

Dammit.

tchrist

You must arrange that the right compiler sees it.

\Q is a string-interpolation thing.

MετάEd

It's a double quoted string operator, isn't it.

David Wallace

@Noah I see "performing a survey" and "taking a survey" as quite different activities.

Taking a survey means I am one participant. I answer the questions (or do whatever the survey requires). Performing a survey means that I conduct it. In other words, I am the designer or organiser of the survey; I set everything up and collect the answers.

Do you want me to find examples of these two usages for you to show your teacher?

MετάEd

17:15

@tchrist Well, nuts. I can't very well let the user supply a pattern that I run through string interpolation.

tchrist

$a = "\Q$b" compiles into $a = quotemeta($b). If you hide the \Q inside $b, it will not be seen this side of a string eval. And using slashes doesn't change that.

$thing =~ quotemeta($pattern[$i]) but if that is all you are doing, then you don't need a regex, and can just use index/rindex.

MετάEd

Right, no, but you can see from the sample pattern that I'm trying to switch meta quoting on and off here and there.

tchrist

$thing =~ /your stuff \Q$literal[$i]\E more your stuff/ is more normal.

That way the perl compiler sees it before passing it to the regex compiler. It's an interpolation step.

If you had suppressed interpolation by using m'something', it would not see the variable to expand, nor the backslash to process.

Just as is in fact happening here where you hide the \Q in the variable.

MετάEd

What I'd really like to do is rewrite the regex compiler a little so that all metacharacters have a prefix escape.

So \| for alternation, etc.

tchrist

I know I have a serious section on this very issue in the regex chapter.

That would be ultra wrong.

MετάEd

17:19

It would be. I'm all about the wrong.

tchrist

One never backslashes a nonword character for its meta sense.

This is a central and pervasive principle.

One backslashes word characters to get a meta sense.

Nonword characters are already meta.

One backslashes them to dispell their metacity.

JSBձոգչ

@tchrist you speak the true and unadulterated gospel. alas, i sometimes have to use regex engines which violate these principles sob

MετάEd

Yes, that's true of PRE but inconvenient for the task. A regular expression language in which nothing is a meta character except when escaped would suit the purpose better.

tchrist

@JSBձոգչ Just say FMH^H^H^HNF^HW.

@MετάEd No.

MετάEd

I thought I'd get there with \Q\E but made a fool of myself.

tchrist

17:22

You just put them in the wrong place.

Don't hide it inside a variable.

Perl never does multiple levels of evaluation without you telling it to do so.

That would be wrong.

The shells are notorious for that nonsense.

MετάEd

No, I get that: I can easily arrange for string interpolation. But that creates a new problem: suddenly my user input not only can trigger \Q\E but also any kind of string interpolation. That's open to injection attack.

tchrist

It's like putting the literal strings '$a' inside a variable, and expecting it to auto expand twice.

@MετάEd No you aren't.

MετάEd

Sure I am. Any input which is interpolated will not only have \Q\E interpolated but whatever else the user sticks in the input. So the user can interpolate my internal variables or what have you.

JSBձոգչ

@tchrist isn't that possible with the //ee hack?

tchrist

You would have to explicitly beg Perl to let you shoot yourself. That sort of thing is absolutely forbidden by default, and for eminently defensible reasons.

JSBձոգչ

17:26

@tchrist is the //ee hack the begging of which you speak?

tchrist

@JSBձոգչ That is something else. You are talking about string interpolation on the RHS of an s///. The second element is not a pattern, but a string.

@MετάEd No, of course not.

@MετάEd This is exactly what perl does not do.

If you want a variable to expand in a pattern, you can do that via /blah $var blah/, but it is not as though the stuff that is in there again counts as code to be re-re-re-interpolated. I don't care what it holds, it will not happen unless you minimally take two further steps, one of which is highly non-obvious.

MετάEd

The intent is to let the user supply input such as: "foo \Q [bar] \E baz", which should be fed to the regex engine as "foo [bar] baz".

tchrist

So the user is not supplying a literal after all.

MετάEd

But if I make that possible, then that also lets the user supply input such as "foo $myvariable baz", which will pass the value of $myvariable, a value the user should not be able to dereference, to the regex engine.

The user is supplying a string. Right now I pass the string directly to the regex engine. If I do what you suggest, feed it first to string interpolation, then I get this undesirable side effect that he can basically look up my skirt.

tchrist

If you are going to do an eval, you of course have to be much more careful.

MετάEd

17:32

I don't see how to let the user take advantage of \Q and \E without using eval.

tchrist

You realize that \U \L \u \l \F all have the same issue as \Q \E have.

MετάEd

Right, those are quoted string metacharacters.

Interpolated string metacharacters.

tchrist

@MετάEd You either do it yourself via $var =~ s/\\Q(.*?)(?:\\E)?/quotemeta($1)/ges first, or else you use the $compartment->reval($dangerous) method from the Safe module.

I don't pretend that that s/// is optimal, but you get the idea.

Damn it.

I can't get the backslashing right due to markup idiocy.

Use two.

MετάEd

Instead what I'm going to do is either just make the user put up with PRE and have to do a lot of escaping, or else offer a simple RE that I devise (easier to use but less powerful).

Sure.

tchrist

There, that should do it.

MετάEd

17:36

That reminds me of the Telnet song.

tchrist

No, do not do your own RE. Trust me.

Although maybe you should use fileglob notation instead?

Robusto

Somebody pour me a beer. Please.

tchrist

@MετάEd Just make them use backslashes like God and Ken intended.

MετάEd

♫ control-uparrow-control-uparrow-control-uparrow Q ♫

@Robusto You're a beer.

Robusto

@MετάEd Not what I asked for.

MετάEd

17:38

@Robusto I used to be a consultant.

@tchrist I'm thinking along the lines of BRE but make the user escape all metas.

tchrist

♫ //GO.SYSIN DD *, DOODAH, DOODAH ♫

MετάEd

@tchrist That's the spirit!

Robusto

@MετάEd Ah. Makes sense.

still waiting for beer

tchrist

@MετάEd No, give them real Perl regexes. Just don't expect variable re-interpolation or double-quote processing for the love of God.

MετάEd

@tchrist I sense a religious argument.

tchrist

17:39

Go look at the atomic column of my metacharacter table in the regex chapter.

MετάEd

PRE is the Only True Regular Expression Language?

I love PRE. I also love fileglobbing. The right tool for the job.

tchrist

@MετάEd If you think the difference between ERE and BRE is one of religion rather than practicality, then you are not paying attention.

BRE sucks dd.

And I do not mean that in a good way.

Nor a sysin one.

It occurs to me that by PRE, you do not actually mean Posix ERE.

MετάEd

@tchrist Correct. I mean Perl RE.

Which I love, as I said.

Matt Эллен

blog talk

EL&U Blog

Discussion for the EL&U Blog. For more info see meta.english.s...

tchrist

I do not understand why you want to give the user quotemeta capabilities. Just let them use plain ole PREs, which in fact do not include \Q \L \U \F \l \u \E contrary to appearances.

Let them do their own backslashing. All other roads lead to madness.

MετάEd

17:48

@tchrist I don't want to give the user interpolation capabilities. I do want to make it easier for them to supply inputs which contain a lot of characters that will need to be escaped.

tchrist

hmms ponderously

Then perhaps consider the $their_input =~ s/\\Q(.*?)(?:\\E)?/quotemeta($1)/ges route first, but I don't know where the hobgoblins are in that particular selva oscura.

Oh, I see one.

Sorry.

The minimal match is wrong here.

You have to do the negative lookahead.

MετάEd

See that's just making it complex. I am all about the simplify.

tchrist

Simplify for whom? Them? Or youm?

MετάEd

Maybe I really am on the right track when I mention globbing.

For both. It should be simple for them, and not so complicated for me that I miss a corner case and create a security issue.

tchrist

It's your job to make their life easier, not theirs yours.

$their_input =~ s/ \\Q (?<Quote_MetaEd> (?: (?! \\E ) . )* ) (?: \\E )? / quotemeta $+{Quote_MetaEd} /gsex

facilissimo

Sorry about that gay sex bit.

However, I promise you there can be no security concern there if you just then do /blah $their_input blah/. Just don't be in the scope of a use re "eval" pragma is all.

!YANETUT

MετάEd

18:00

@tchrist Sure.

tchrist

You realize you've killed the chat.

MετάEd

I see what you are doing there, just a poor man's interpolation of just \Q\E. That gets me to where I was heading. I'm still not sure I like it. Because then when the user does need regex, it's bracketed by \E\Q basically. Which is a mess.

Sep 12 at 21:56, by ΜετάEd

I killed the room.

Oct 5 at 16:14, by MετάEd

Dammit. I killed the room again.

It happens.

tchrist

"Poor man"?

Read the RHS and re-assess your notion of poor.

MετάEd

@tchrist No, it's a great way to do just \Q\E interpolation. Which is a poor man's string interpolation because that's all it does. Which is what I was trying to do.

But now I've looked at what the user will have to put up with. It's just as bad.

tchrist

@MετάEd Whatever do you mean?

The user does not use that. You do.

So you should make it more legible and maintainable, viz.

$their_input =~ s{
    \\Q                         # first a literal and true \Q to start the metaquoted group
    (?<Quote_MetaEd>            # begin Quote_MetaEd capture variable
        (?:                     # begin repetition group
            (?! \\E ) .         # anything but a literal \E
        ) *                     # rep group repeated ad libitem
    )                           # end   Quote_MetaEd capture variable
    (?: \\E )?                  # followed by an option \E
}{
    quotemeta $+{Quote_MetaEd}  # now metaquote whatever you got

MετάEd

18:08

I mean \Qmetaed postfix/smtp[\E\d+\Q]: is no better than metaed postfix/smtp\[\d+\]:.

In some respects worse.

tchrist

What is wrong with making them quote their metas themselves?

MετάEd

@tchrist The lesson on readable RE is wasted on me:

5

A: How to check whether a string contains small letter, big letter, special character and number?

Because these will not appear in any particular order, you will need lookahead assertions for each required character class: (?=.*[A-Z])(?=.*[a-z])(?=.*[0-9])(?=.*[~!@#$%\^&*()\-_=+\|\[{\]};:'",<.>/?]) (NOTE: because backslash, caret, hyphen, and square brackets can be special inside...

tchrist

Notice I've used named captures, too.

MετάEd

@tchrist For this application that's fraught with error. The user will be supplying long strings of text, turning them into patterns by carefully reviewing them for RE metacharacters. String which contain a lot of vertical bars, brackets, parens, and dots. So easy to miss one or two.

tchrist

Here's a tip: if use use utf8, then you can even name your captures things like MετάEd. However, this is apt to get you Talked About.

MετάEd

18:14

But still the user will, in some places in the string, need to use metacharacters for wildcards, alternation, etc.

tchrist

@MετάEd Then give them fileglobs and be done with them.

MετάEd

Seriously considering it, now that I've been burned by \Q\E.

tchrist

You have not been burned.

You have been learned.

MετάEd

Fire hot.

Graag stay away from fire.

Schooled.

Nice rhyme though.

tchrist

Remember to include {foo,bar} fileglobbing as (foo|bar) regex alternation.

MετάEd

18:15

Memorable.

RegDwighт

0

A: Why are 'Closed' questions left up?

First of all, you forget that you don't see deleted questions. You don't know whether or not questions get deleted, and how many. Speaking for myself, I have personally deleted over a hundred questions last week alone. But you didn't see that. You can't. What you do see are questions that are st...

I really don't understand people.

MετάEd

@tchrist Alternation, character classes, and ? and *.

RegDwighт

I tried to keep the snark to a minimum, though.

tchrist

Sempiternal September.

@MετάEd Prolly just as well. Give them \b and they're guaranteed to screw it up.

MετάEd

@tchrist I may not give them any anchors at all.

Want to match to the end of the string? Put a * there.

Or put literals there.

Otherwise, no match.

tchrist

18:20

I don't understand.

Fileglobs have implicit ^$ anchors.

fileglob of foo*ba? means regex of ^foo.*ba.$.

Modulo mumbles about /s & /m.

MετάEd

I am not going to literally offer fileglob. Fileglob looks for files, except for expressions using nothing but braces.

So yes, implicit anchors, \[\] for classes, \{\} for alternation, \? for {1}, and \* for {0,}.

damn backslash.

tchrist

I am using the term fileglob for its syntax, not for its calling for the massively overloaded File::Glob module's operator.

You'll be sorry.

MετάEd

I'll be dead.

I tend to take the long view.

The whole notion that $ is a metacharacter but S is not, that \S is a metacharacter but \$ is not, is ridiculous.

JSBձոգչ

@MετάEd why is that ridiculous. it's patently obvious

MετάEd

My application is a case in point: It's ridiculous that you have to sift through each string to differentiate between the characters which are metacharacters without an escape and the characters which are metacharacters only with an escape.

When you are trying to match long strings which contain a lot of nonalphas, it's annoying.

I totally get that the typical RE syntax is useful when your literals are mostly alphanumeric. You aren't having to do much escaping. You still run into trouble with punctuation, though.

I just happen to have inputs that aren't Shakespeare.

tchrist

18:33

@MετάEd No. It is brilliant, in fact. You fail to appreciate why.

MετάEd

@tchrist You're quite welcome to defend it. I simply consider it a compromise between escaping all metacharacters and escaping all ordinary characters, a compromise which works best when your text is mostly alphanumeric.

Another solution would be to mark up the metacharacters. I would love to see that. All metacharacters in bold or something along those lines.

But we don't have rich text programming languages.

At best we have syntax highlighters.

tchrist

@MετάEd Or at worst.

MετάEd

18:55

@tchrist labs.oracle.com/projects/plrg/fortress.pdf (PDF)

The Fortress programming language. The Spruce Goose of rich text programming languages.

JSBձոգչ

weird. their pdf spends several pages discussing the fonts in which Fortress is rendered

tchrist

@JSBձոգչ No, not exactly. What they are saying is how to convert ASCII into code points.

That is not my best effort as a description, sorry.

JSBձոգչ

19:10

@tchrist they define that a[i] is supposed to be rendered as a with i subscript. which is a reasonable convention, but i've never seen any other language define this in their spec

tchrist

That isn’t specifying the font.

Well, not the font face.

It is something else.

JSBձոգչ

they also define the font face elsewhere

in that same section

anyway, we understand each other, and i hope we both agree that it's unusual

tchrist

Sure.

I didn't see the face.

a[i] => 𝑎ᵢ

Except that the subscript should also be in italic.

> A program consists of a finite sequence of Unicode 5.0 abstract characters. In order to more closely approximate mathematical notation, certain sequences of characters are rendered as subscripts or superscripts, italicized or boldface text, or text in special fonts, according to the rules in Appendix B. Although much of the program text in this specification is rendered as formatted Unicode, some text is presented unformatted to aid in exposition.

I’d not yet made it to Appendix B. It is, after all, on page 191.

Mr. Shiny and New 安宇

19:59

> The name “Fortress” is derived from the intent to produce a “secure Fortran”,

I think I've read enough already

Haha: "The venerable Motif graphical toolkit has been relicensed to LGPLv2.1; the code is now hosted on SourceForge. Most of the world is unlikely to care much, but, as they say, better 20 years too late than never."

> Back in ðe day, ðis could have prevented ðe KDE‐Qt licenſiŋ ſhenanigans & ðe need to create Gnome. Preſumably, it ƿould alſo avoided a ſeparate deſktop by Canonical. Noƿ, ¿is it ſtill relevant to anyone but Solaris uſers?

MετάEd

20:35

@Mr.ShinyandNew安宇 Did you see that Guy Steele was one of the ringleaders?

Mr. Shiny and New 安宇

@MετάEd No. But I don't know who that is anyway.

MετάEd

Really?

Guy L. Steele, Jr.

Guy Lewis Steele Jr. (), also known as "The Great Quux", and GLS (), is an American computer scientist who has played an important role in designing and documenting several computer programming languages. Biography Steele was born in Missouri and graduated from the Boston Latin School in 1972. He received a BA in applied mathematics from Harvard (1975) and an MS and Ph.D. from MIT in Computer Science (1977, 1980). He then worked as an assistant professor of computer science at Carnegie Mellon University and a compiler implementer at Tartan Laboratories. Next he joined the supercomputer c...

Mr. Shiny and New 安宇

yeah reading that now

Hm, not sure why I've never heard of him before

seems like a fairly important fellow

Zairja

20:50

Speaking of, I just learned John McCarthy passed away just over a year ago. I'm sad. :(

tchrist

It's nothing like Dennis.

tchrist

21:49

@MετάEd I think we call that emigrating when it’s to a whole nother country.

MετάEd

@tchrist I avoid confusing people with emigrate/immigrate. I just use migrate.

tchrist

What’s to confuse? The damned Texans most all have the pin–pen merger anyway.

tchrist

23:42

That being clearly established, let us return to Celani and Tullis, who, we assure the reader, have not been idle during our brief but necessary digression.
Indeed, during this time, Celani — whom the reader may have begun to suspect of playing a more passive than active part in the engagement —has gripped the bottom of Tullis's tunic and, as he raised his arms, has pulled it over his head and tossed it aside if it were of no more value than the leaf of a chublik tree upon the arrival of winter; and, indeed, to the two of them at this time it had no more value than this, because just as the…

TL;DB.

From here which is from here.

Cerberus

Hi.

tchrist

And well met.

Cerberus

My arthropod friend has brought his big sister.

I will take a picture.

But I'm slightly intoxicated.

tchrist

Crabs again, eh?

This is your wee mite?

Cerberus

Yes.

But less wee.

tchrist

23:46

Just as well: the smell will get to you after a spell.

Cerberus

It is (supposedly) uploading.

tchrist

ok

I’ve put in 42 hours this week already, and Friday is yet unbegun, let alone Saturday.

I tried to nap at a milestone point, but was too wired to be tired.

Then much to my surprise, I chanced upon an unopened bottle of mead in the fridge, and fell to it.

Or upon it, perhaps. I no longer know. I am not used to such things.

EL&U Blog

Transcript for

English Language & Usage: Multi-Layer…