The APL Orchard

Define "normalization", please?

why ¯3?

@rak1507 I'll answer that in a moment.

@JeffZeitlin For ASCII and most European languages, there's no difference between lowercasing and normalising case. However, some languages have multiple forms of a single letter. Normalising makes all those forms the same, so they can be compared easily.

2:37 PM

@Adám - So this would be applicable to Arabic and Hebrew, for example?

E.g. Greek has two lowercase forms of Σ: σ and ς

Even Latin script (like in English and German) used to use a medial form of S: ſ

@JeffZeitlin Interestingly, Unicode doesn't define those as variants of each other. I'll look into that.

Got it. Does it also "de-diacriticize", so that e.g., á and a compare as equal? Or is that too culture/language dependent?

@JeffZeitlin No it doesn't, and it also doesn't do decomposition or other length-changing normalisation. This explains the ¯3…

@rak1507 2 and ¯2 and ¯4 are reserved for length-changing mapping (upper/lower) and folding (normalisation).

what about 0

The normalisation is given a negative number because it mainly consists of lowercases.

2:42 PM

Plans for "Sentence case" and "Title Case"?

@rak1507 We couldn't use 0 because there'd be no obvious corresponding number for the length-changing equivalent. Also, we may extend 0 to mean a case-query.

I like 0 for query.

@JeffZeitlin No, since these are context-dependent. E.g. try to sentence case 'the ABC Co. has done well'.

Mmmm... yeah, I see that that can be an issue.

@JeffZeitlin You can often get an acceptable result using regex, and ⎕R/⎕S have indeed been extended with folding too (they already had mapping).

So, a simple exercise: Given a character vector, uppercase the first character.

E.g. 'hello, world!' → 'Hello, world!'

2:47 PM

{1↓⍵,⍨1 ⎕C 1↑⍵}

I was starting to look at it in train form and changed my mind... :)

That's nice, but any other bids?

⋄ {1↓⍵,⍨1 ⎕C 1↑⍵} 'hello, world!'

@Adám hello, world!

@JeffZeitlin Can you spot your mistake?

No... (don't have 18 here to check. Yet.)

@JeffZeitlin You have the bot now!

2:51 PM

Wait... I think I may see it...

⋄ 1 ⎕C@1 ⊢ 'hello, world!'

@Bubbler hEllo, world!

Damn ⎕IO...

⋄ 1 ⎕C@⎕IO ⊢ 'hello, world!'

@Bubbler Hello, world!

⋄ {(1↓⍵),⍨1 ⎕C 1↑⍵} 'hello, world!'

2:52 PM

{(1 ⎕C 1↑⍵), 1↓⍵} 'hello, world'

@JeffZeitlin Hello, world!

⋄ {(1 ⎕C 1↑⍵), 1↓⍵} 'hello, world'

@rak1507 Hello, world

Ah, these are all nice.

(@Moonchild Is ⎕IO←0?)

The @ solution is really nice

2:53 PM

Yeah, OK. Needed the parens, or it was dropping the uppercased H from preending H to the whole original string.

@rak1507 Indeed. That's the one I was thinking of.

Next up: A better (still not perfect) palindrome checker. Given a string without diacritics but may have spaces, determine if it is a palindrome.

Yeah, the @ solution is slick...

Ignoring spaces?

Yes, e.g. 'race car' → 1

And also 'Σοφος' → 1

'hello' → 0

OK, so that's normalization...

2:57 PM

'Νιψον ανομηματα μη μοναν οψιν' → 1

⋄ {⊢≡⌽1 ⎕C ⍵}'race car'

@rak1507 1

@rak1507 That's giving a 1 for the wrong reason.

ah

Mixture of train and dfn.

2:59 PM

⋄ ((⊢≡⌽)¯3⎕C~∘' ')¨ 'Νιψον ανομηματα μη μοναν οψιν' 'Hello world'

@Bubbler 1 0

oh forgot about the spaces oops

⋄{(⊢≡⌽)(1 ⎕C ⍵)~' '}¨ 'race car' 'Νιψον ανομηματα μη μοναν οψιν' 'Hello world'

@rak1507 1 1 0

@rak1507 Fails on 'ẞUß'

¯3 then instead of 1 I guess

3:01 PM

Yes. Important distinction there.

Here's a trick too: Monadic ⎕C is ¯3∘⎕C

clever

Can anyone find my 11-character solution? (Hint: a train ― of course.)

Maybe we should continue meanwhile…

Let's have a look at ⎕DT. It is the all-knowing wizard of date-time conversions. Basically, it allows you to convert any numeric representation of a date-time into any other representation. You can use it to glue together two 3rd-party systems that otherwise can't easily communicate.

⋄ 20 ⎕DT 44053.674 ⍝ to Unix time

@Adám 1597162233

The basic representation of a moment is the number of days since 1899-12-31.

What's the 44053.674? Excel?

3:12 PM

@JeffZeitlin No, Excel is off-by-one due to perpetuation of an issue in Lotus 1-2-3.

⋄ 40 ⎕DT 44053.674 ⍝ to Excel

You'd think this many years after 1-2-3 is gone...

@Adám 44054.674

is there a way to do {(f ⍺ g h ⍵) i ⍵} in a train? where f g h and i are all functions

The reason you want our system (which was actually the original one) is that you can then find the day-of-week with 7|⌊:

⋄ 7|⌊44053.674

@Adám 2

3:14 PM

0: Sunday, 1: Monday, etc.

@rak1507 I think (f g∘h)i⊢

ooh cool thanks!

@rak1507 Alternatively: ⊢i⍨∘f g∘h

Does anyone use some software that has its own date format?

Answer: Yes, you all do. APL does. It has the 7-element vector ⎕TS for the current Time Stamp.

⋄ ¯1 ⎕DT 44053.674 ⍝ to ⎕TS

@Adám Illegal code

Huh.

⋄ ¯1 ⎕DT 44053.674 ⍝ pretty please

@Adám Illegal code

3:18 PM

@DyalogAPL Really?

⋄ ¯1 ⎕DT 44053.674

@Adám Illegal code

xpqz

     ¯1 ⎕DT 44053.674
 2020 8 11 16 10 33 600

Oh well. Yeah, that ^

xpqz

My code isn't illegal

(@Moonchild Any idea what's going on here?)

@Razetime Welcome to. We're in the middle of an APL lesson, but feel free to hang around.

The left argument, as you've seen, tells ⎕DT what you want to convert to. The numbers are largely arbitrary, but not entirely so. Positive codes indicate a scalar format (one number per date-time) and negative numbers indicate a vector format (multiple numbers per date-time).

Also the number divided by 10 and floored indicates the family. So we had 2(0) for UNIX and 4(0) for applications (Excel).

The last element of ⎕TS is the milliseconds. We can get more precision in the ⎕TS-style result by using ¯2 for microseconds and ¯3 for nanoseconds:

           ¯2 ⎕DT 44053.674
┌─────────────────────────┐
│2020 8 11 16 10 33 600000│
└─────────────────────────┘
           ¯3 ⎕DT 44053.674
┌────────────────────────────┐
│2020 8 11 16 10 33 600000000│
└────────────────────────────┘

Notice also that vector formats are enclosed. This allows ⎕DT to handle arrays of dates:

      ¯1 ⎕DT 44053+⍳3
┌─────────────────┬─────────────────┬─────────────────┐
│2020 8 12 0 0 0 0│2020 8 13 0 0 0 0│2020 8 14 0 0 0 0│
└─────────────────┴─────────────────┴─────────────────┘

I won't bother you with all the allowed codes. They are readily available in the documentation.

What you do need to know is how to convert from one of these formats.

Until now, we've just used the Dyalog day number. That's the default for simple scalars in the right argument. The default for enclosed vectors is the ⎕TS format (¯1).

If your input is anything else, you need to give ⎕DT a two-element left argument. The first element is the input type, and the second is the output type.

E.g. This converts an ISO year, week of year, day of week to ⎕TS-style:

      ¯11 ¯1⎕DT⊂2020 40 3
┌─────────────────┐
│2020 9 30 0 0 0 0│
└─────────────────┘

3:31 PM

anyone have a link to solution answers from last year?

@rak1507 No official solutions have been published. You may be able to find individually published solutions.

AviFS

⋄ 'Testing ','Dyalog ','Bot '

@AviFS TestingDyalogBot

Ok thanks

How about handling custom string formats, e.g., '2020-08-11 1132'

AviFS

3:33 PM

@Adám Interesting. Never played around with SE bots, so I don't know what's possible. But wondering if when one edits a Dyalog Bot thing, if the bot can then edit its response?

Challenge: Given two style dates (as a 2 element vector of Y,M,D vectors), compute the inclusive number of days between them. E.g. (2020 6 25)(2020 08 10) should give 47.

ngn

@Marshall @dzaima ping (sorry for the interruption)

@JeffZeitlin We'll get to that next.

AviFS

Whoops, sorry. Didn't realize APL Cult. is still going on. Glad I didn't miss the whole thing! Didn't mean to interrupt

@AviFS I guess it is potentially possible. Ask Moonchild.

(2020 08 10)(2020 6 25) should also give 47. And (2020 08 10)(2020 08 10) should give 1.

If anyone wants more time, let me know now. Otherwise, I'll assume nobody is working on the challenge :-)

OK: {1+|-/1⎕DT⍵} will do.

@JeffZeitlin OK, now for the last 15 mins, let's talk about converting date-times to text.

The first step is to convert to a Dyalog day number.

Then you use 1200⌶ to convert that to text. It takes a left argument which is a pattern using a scheme we came up with, which I think is a really neat one. Much better than any of the existing ones out there.

3:46 PM

Wait, what if I have the string and want to convert it to a date/time?

      'YYYY DD MM hhmm'(1200⌶)1⎕DT⊂2020 08 11 11 32
┌───────────────┐
│2020 11 08 1132│
└───────────────┘

Hello

@JeffZeitlin We're considering adding that. It has the problem of some formats being ambiguous. E.g. what is 2020111 using the YYYYMD format?

@Quintec !

@Quintec Hi.

3:49 PM

@Adám - Agreed, but some formats are simply abominations - like that one. It should be YYYYMMDD, forcing the zero-padding.

@JeffZeitlin Sure, and we might add that.

My first inclination is to say "if the format and date given don't parse unambiguously, throw an error". So that particular example would do so.

The system in the pattern for 1200⌶ is that numeric parts of the date are uppercase, while parts of the time are lowercase. You can use a single character for a variable-width pattern, or multi-character for a 0-padded pattern. If instead you want space-padding, use an underscore as the first character:

      'YYYY-DD-MM@hh:mm' 'YYYY-D-M@h:m' 'YYYY-_D-_M@_h:_m'(1200⌶)¨1⎕DT⊂2020 8 11 1 3
┌──────────────────┬───────────────┬──────────────────┐
│┌────────────────┐│┌─────────────┐│┌────────────────┐│
││2020-11-08@01:03│││2020-11-8@1:3│││2020-11- 8@ 1: 3││
│└────────────────┘│└─────────────┘│└────────────────┘│
└──────────────────┴───────────────┴──────────────────┘

1:3 i sincerely hope no one writes times like this

Of course, but you may want to write:

      'YYYY MMM D "at" h:mm'(1200⌶)1⎕DT⊂2020 8 1 4 30
┌──────────────────┐
│2020 AUG 1 at 4:30│
└──────────────────┘

3:54 PM

"tt:mm" for 24-hour times?

Ah, MMM, neat.

@JeffZeitlin No, t is for 12-hour. h is for 24-hour.

@Quintec Yeah, and furthermore, our format allows casing:

      'YYYY Mmm D' 'YYYY mmm D'(1200⌶)¨1⎕DT⊂2020 8 1
┌────────────┬────────────┐
│┌──────────┐│┌──────────┐│
││2020 Aug 1│││2020 aug 1││
│└──────────┘│└──────────┘│
└────────────┴────────────┘

And foreign languages:

      '__fr__YYYY Mmmm D'(1200⌶)1⎕DT⊂2020 8 1
┌───────────┐
│2020 Août 1│
└───────────┘

@Adám - Huh. Seems to be backwards from most other dt formatters...

@JeffZeitlin That one was the most controversial of our choices. I guess you can remember it as "hour" of the day, and simply "time" (which isn't enough to specify the hour).

fwiw python also uses %H for 24 hour time

3:59 PM

@Quintec But a completly non-mnemonic %I for 12-hour time!

yep, and moment.js uses H for 12 hour time, and... k!? for 24 hour

Anyway, 1200⌶ has lots of options too, including custom languages. Have a look at the documentation.

Any questions before we conclude today's lesson?

Juust one..

Since Dyalog day number is just one number, I don't suppose there's support for timezones?

Why 1200 I-bar instead of a quad fn?

4:04 PM

@Quintec Yes and no. There's no mechanism for attaching a timezone to a value. However, we do provide 'Z' and 'J' to mean Zulu time and local time, allowing you to compute the current offset. (We also included the other military time zones for convenience.)

Cool, thanks.

@JeffZeitlin Because it was unclear if this formatting should be stand-alone, folded into ⎕FMT or ⍕, or be an extension of ⎕DT. Notice that the left arguments of ⎕DT and 1200⌶ don't conflict, so we could allow 20 '%ISO%' ⎕DT dates to format Unix times directly, without converting them first.

('%ISO%' is a built-in pre-defined pattern equivalent to 'YYYY-MM-DD"T"hh:mm:ss'. You can define more such patterns.)

I'd argue for folding it into ⎕DT

Me too :-)

OK, thank you all for participating today!

Razetime

4:50 PM

It was interesting to watch this chat

@Razetime :-) Interested in APL?

5:25 PM

Anyone up for discussing 2019's phase 1 q1 with me? my solution is 1∘=|∘⍳∘≢⊂⊢ but I'm wondering how that can be made better, there seems to be a lot of ∘s and idk if that's necessary.

@rak1507 Hang on, let me look up the problem.

dyalog.com/uploads/files/student_competition/… here's a link @Adám

Thanks.

@rak1507 That looks good for a tacit solution. If you find yourself using a lot of monadic functions, it is often better to go with a dfn: {1=⍵⊂⍨⍺|⍳≢⍵}

right, let me wrap my head around that

how does ⍨ work here?

X f⍨ Y is Y f X

5:38 PM

and f here is ⊂?

same as {1=(⍺|⍳≢⍵)⊂⍵}

⍨ is "commute"; it swaps arguments.

alright got it thanks

need to use it more

Wow problem 2 is very similar to what you mentioned in your webinar!

@rak1507 Try to write an alternative solution that constructs the Boolean mask directly, without going through "mod" |

Hmm, I'm not sure how I'd do that, can you give me a nudge?

@rak1507 I'll give you two hints: N↑Y will pad Y appropriately if it runs out of elements before reaching length N while N⍴Y will recycle elements from Y to reach length N.

RGS

5:55 PM

@Adám meanwhile I also found and fixed a bug with Sphinx; something tells me we're gonna be able to build an awesome book out of jupyter notebooks!

I got a dfn solution, {⍵⊂⍨(≢⍵)⍴⍺↑1} but I can't see how to turn it into a tacit solution

@RGS Looking forward to tomorrow.

((f ⍵) g (h ⍺)) ⍳ ⍵

it's basically that ^

@rak1507 Awesome, that was my exact dfn, character for character.

oh great!

5:56 PM

@rak1507 Are you on 18.0?

yep

⊢⊂⍨≢⍤⊢⍴⊣↑1⍨

At the right end, ⍨ takes an array and derives a constant function.

whew let me think about that for a sec

You know about the tools you have to analyse trains?

pen and paper?

that and seeing the boxes and stuff

5:59 PM

That too. I was thinking ]box on -trains=box and ]box on -trains=tree and ]box on -trains=parens

The tree structure is nice thanks

The constant function is a really nice feature, before I've used (⍴ A) ⍴ const

This is probably a dumb question but is there a function equivalent to [] for indexing?

RGS

@rak1507 definitely not a dumb question... you may want to look at ⌷ or ⊃

ok thanks :)

So ⌷ does what I want but if I wanted (a b c) ⌷ d to be a ⌷ d, b ⌷ d, c ⌷ d, how can I do that? It's like ¨ but in reverse

6:15 PM

@rak1507 Enclose a b c or use ⌷⍤0 99

@rak1507 Btw, there is no "¨ but in reverse". ¨ is symmetric.

Alright thanks!

@rak1507 You may want to watch Richard's webinar on Selecting from Arrays.

I will, thank you

An alternative rendering of "the Indexer": ⌷⍨∘⊃⍤99 0⍨

6:33 PM

P2←⌷∘'FDCBA'(⊂1+65 70 80 90∘⍸)
P2←{(⊂1+65 70 80 90⍸⍵)⌷'FDCBA'}
here are my tacit and DFN attempts at question 2

is there anything that can be improved?

@rak1507 I would personally have used ⍨ or ¨ or ⍤ there, but that's more a matter of style.

@rak1507 You can avoid the 1+ by including 0 in the look-up array.

Oh yeah true, thanks

I agree using ⍨ makes it look cleaner

For the dfn, I'd simply use bracket indexing. Reads better: {'FDCBA'[0 65 70 80 90⍸⍵]}

Right yeah makes sense

How would you redo the tacit version?

Not sure. Our indexing functions are just so awkward. Maybe 0 65 70 80 90∘⍸⊃¨∘⊂'FDCBA'⍨ or 0 65 70 80 90∘⍸(⌷⍤0 1)'FDCBA'⍨

'FDCBA'⌷⍤0 1⍨0 65 70 80 90∘⍸ isn't too bad.

Ideally, I'd define I←⌷⍨∘⊃⍤99 0⍨ and then use 0 65 70 80 90∘⍸I'FDCBA'⍨

6:44 PM

It's really great how even simple problems like these can have multiple different solutions and be useful for learning

Yeah. If you're looking for learning. Trying to solve each problem in multiple ways is great.

I think I'll go through all of last years Phase 1 and then start looking at some Phase 2s hopefully.

TessellatingHeckler

"LISP symbols and how useful they would be in a fight" might be an easier one to start with >_>

... now that I write that down; I remember xkcd.com/297

8:00 PM

Looking at question 3, how can I format things to 1 decimal place but if they're a whole number it doesn't end .0? 1⍕number doesn't seem to work.

@rak1507 You can either multiply by 10, floor, then divide back, or execute (⍎) after 1⍕.

Alright great, I'd always thought of ⍎ like eval/exec and automatically stayed away from it, but it seems to be the easiest solution here.

@rak1507 Indeed, you should be careful using ⍎, but if you know what the content is composed of, you're fine. 1⍕ will not format text that can be interpreted as code. The safe alternative to ⍎ (for making text into numbers) is ⎕VFI.

Alright cool thanks

Btw, do you know about aplcart.info ?

8:36 PM

I do now!

Seems very useful

Here's my first attempt at P3,
P3 ← {{⍺,(⊣,(⍎1⍕100×÷∘(≢w)))(1-⍨≢⍵)}⌸'ABCDF',(w←⍵)}

Please feel free to tell me how absolutely horrible it is, because it looks awful and I'm sure can be a million times better

@rak1507 Nah, that's not bad. I've got {(100÷⍨≢⍵)(⍎1⍕÷⍨)@3⍤1{⍺,2⍴¯1+≢⍵}⌸'ABCDF',⍵}

much nicer using a tacit function so you don't need to rename ⍵

Well at least there's no clever thing to specify default values

It isn't even a tacit function, just an expression for a constant factor.

@rak1507 No, that's something I often miss with ⌸

@rak1507 Maybe a clearer solution: 'ABCDF'∘{⍺,(⍪,∘⍎1⍕100×⊢÷∘≢⍵⍨)+/⍺∘.=⍵}

8:54 PM

that is nice

@rak1507 Another approach: 'ABCDF'{⍺,t,⍎1⍕100×(t←≢⍵∩⍺)÷≢⍵}⍤0 1⊢

nice

(Fixed)

10:56 PM

Hey, I'm trying to make a train that will filter out arrays of two number pairs to only give ones where the sum is a certain number, and I've tried (10=+/¨∘|)/⊣ but it doesn't seem to work, and I'm not quite sure why. Doing ((10=+/¨∘|) array) / array works, and that's the (f x) g x pattern I asked about earlier! So if anyone can help that'd be great

Marshall

@rak1507 It's this problem.

@rak1507 That's the exact problem with / family in trains. That doesn't work because the (10=+/¨∘|)/ part is interpreted as a reduce rather than a replicate.

ohhhhh! ok, is there an easy way to fix this?

If you have Dyalog 18.0, you can use ⊢⍤/ in place of /.

ok I do, great thanks!

it works thanks!

btw, is there any reason to use ⊢ or ⊣ in different scenarios?

11:08 PM

@rak1507 ⊢ can be used to split the array right operand and array right argument.

like in 1 ⎕C@⎕IO ⊢ 'hello, world!' to split between ⎕IO and 'hello, world!

⋄ ((3=+/¨∘|)⊢⍤/⊢),∘.,⍨¯2 ¯1 1 2

@rak1507 Illegal code

it's not

;(

It is needed because they will strand together without the separator

⋄ ((3=+/¨∘|)(/⍨⍨)⊢),∘.,⍨¯2 ¯1 1 2

dzaima

11:10 PM

@rak1507 if you're asking about ⊢⍤/ specifically - no, the function left of ⍤ is always called monadically so there's never any difference

@rak1507 Illegal code

Why am I getting illegal code when it runs fine for me

⋄ ((3=+/¨∘|)(/⍨⍨)⊢)(,∘.,⍨¯2 ¯1 1 2)

@rak1507 Illegal code

It works fine for me too.

@Moonchild ^^

Right anyway is there an easier way to do that

TessellatingHeckler

11:11 PM

@rak1507 The bot is still new, early days on polishing it up, I think

Ok cool

dzaima

⋄ 2/2 ⍝ is slash allowed?

@dzaima 2 2

⋄ ((3=+/¨∘|)(/⍨⍨)⊢) ⍝ testing for the function itself

@rak1507 Illegal code

dzaima

11:12 PM

⋄ ∘.,⍨¯2 ¯1 1 2

@dzaima Illegal code

hm

But yeah, is there a better way to do that? I feel like there might be something clever with the outer product or something

@rak1507 Check out other possible ways here.

ok thank you

P4←∩∘(,∘.,⍨⍳8)((((3=+/¨∘|)⊢⍤/⊢),∘.,⍨¯2 ¯1 1 2)∘+⊂
this is the tacit solution I came up with, anything noticeably bad?

11:32 PM

@rak1507 It has an unmatched paren.

oh yeah oops, copy paste fail

P4←∩∘(,∘.,⍨⍳8)((((3=+/¨∘|)⊢⍤/⊢),∘.,⍨¯2 ¯1 1 2)∘+⊂)

I think you're overusing ∘. Do you know A g h train?

P4←(,∘.,⍨⍳8)∩ moves+⊂ should work, where moves is (((3=+/¨∘|)⊢⍤/⊢),∘.,⍨¯2 ¯1 1 2)

Also you can simplify ∘.,⍨⍳8 to ⍳8 8.

@rak1507 If you want feedback for other solutions too, consider posting a question to codereview.SE with tag apl.

Alright I will thanks, didn't know that existed

Also ⍳ 8 8, I knew there was a much simpler solution but it just wasn't coming to me, :facepalm:

No problem, it takes time to master the features

Also I'm not sure why I did ∩∘, I think I tried without it and it didn't work for some unrelated reason so decided to use ∘.

11:43 PM

But one thing I can say for sure is that it's much easier to master all the built-ins in APL than in other languages, say Python or JS.

You're probably right, it's just the one thing being used for 17 different things depending on context that trips me up

I have years of experience with both Python and JS, and I still have to google every time I want to use a less used built-in

Well if you're including modules, I don't think anyone would say they know every single module in the python standard library