Lesson 48 - ⎕C ⎕DT 1200⌶ – Conversation in The APL Orchard

Adám

Aug 11, 2020 14:30

Welcome to APL Cultivation!

rak1507

that's brilliant!

xpqz

\o/

Adám

We've been going over core language additions made in 18.0.

Jeff Zeitlin

/me finally gets a chance to pay attention!

Adám

I suppose that ⎕C and ⎕DT and 1200⌶ are up next, unless I hear protests now.

rak1507

Aug 11, 2020 14:31

Another nooby question, is there a way to make a function that is the equivalent of {1=⍵}

Adám

@rak1507 1∘= or 1=⊢

rak1507

Oh right, I tried the first one but I think I must've had the wrong circle character. Thanks

Adám

So, since nobody protested, let's look at ⎕C.

It finally adds easy-to-use Case operations to APL.

And there's really not so much to it. The left (currently, we might extend that in the future) argument has to be a single simple scalar integer, 1 or ¯1 or ¯3.

1 does upper-casing

¯1 does lower-casing

¯3 does case normalisation

Jeff Zeitlin

Define "normalization", please?

rak1507

why ¯3?

Adám

Aug 11, 2020 14:35

@rak1507 I'll answer that in a moment.

@JeffZeitlin For ASCII and most European languages, there's no difference between lowercasing and normalising case. However, some languages have multiple forms of a single letter. Normalising makes all those forms the same, so they can be compared easily.

Jeff Zeitlin

@Adám - So this would be applicable to Arabic and Hebrew, for example?

Adám

E.g. Greek has two lowercase forms of Σ: σ and ς

Even Latin script (like in English and German) used to use a medial form of S: ſ

@JeffZeitlin Interestingly, Unicode doesn't define those as variants of each other. I'll look into that.

Jeff Zeitlin

Got it. Does it also "de-diacriticize", so that e.g., á and a compare as equal? Or is that too culture/language dependent?

Adám

@JeffZeitlin No it doesn't, and it also doesn't do decomposition or other length-changing normalisation. This explains the ¯3…

@rak1507 2 and ¯2 and ¯4 are reserved for length-changing mapping (upper/lower) and folding (normalisation).

rak1507

what about 0

Adám

Aug 11, 2020 14:41

The normalisation is given a negative number because it mainly consists of lowercases.

Jeff Zeitlin

Plans for "Sentence case" and "Title Case"?

Adám

@rak1507 We couldn't use 0 because there'd be no obvious corresponding number for the length-changing equivalent. Also, we may extend 0 to mean a case-query.

Jeff Zeitlin

I like 0 for query.

Adám

@JeffZeitlin No, since these are context-dependent. E.g. try to sentence case 'the ABC Co. has done well'.

Jeff Zeitlin

Mmmm... yeah, I see that that can be an issue.

Adám

Aug 11, 2020 14:44

@JeffZeitlin You can often get an acceptable result using regex, and ⎕R/⎕S have indeed been extended with folding too (they already had mapping).

So, a simple exercise: Given a character vector, uppercase the first character.

E.g. 'hello, world!' → 'Hello, world!'

Jeff Zeitlin

{1↓⍵,⍨1 ⎕C 1↑⍵}

I was starting to look at it in train form and changed my mind... :)

Adám

That's nice, but any other bids?

⋄ {1↓⍵,⍨1 ⎕C 1↑⍵} 'hello, world!'

Dyalog APL

@Adám hello, world!

Adám

@JeffZeitlin Can you spot your mistake?

Jeff Zeitlin

No... (don't have 18 here to check. Yet.)

Adám

Aug 11, 2020 14:50

@JeffZeitlin You have the bot now!

Jeff Zeitlin

Wait... I think I may see it...

Bubbler

⋄ 1 ⎕C@1 ⊢ 'hello, world!'

Dyalog APL

@Bubbler hEllo, world!

Bubbler

Damn ⎕IO...

⋄ 1 ⎕C@⎕IO ⊢ 'hello, world!'

Dyalog APL

@Bubbler Hello, world!

Jeff Zeitlin

Aug 11, 2020 14:52

⋄ {(1↓⍵),⍨1 ⎕C 1↑⍵} 'hello, world!'

rak1507

{(1 ⎕C 1↑⍵), 1↓⍵} 'hello, world'

Dyalog APL

@JeffZeitlin Hello, world!

rak1507

⋄ {(1 ⎕C 1↑⍵), 1↓⍵} 'hello, world'

Dyalog APL

@rak1507 Hello, world

Adám

Ah, these are all nice.

(@Moonchild Is ⎕IO←0?)

rak1507

Aug 11, 2020 14:53

The @ solution is really nice

Jeff Zeitlin

Yeah, OK. Needed the parens, or it was dropping the uppercased H from preending H to the whole original string.

Adám

@rak1507 Indeed. That's the one I was thinking of.

Next up: A better (still not perfect) palindrome checker. Given a string without diacritics but may have spaces, determine if it is a palindrome.

Jeff Zeitlin

Yeah, the @ solution is slick...

rak1507

Ignoring spaces?

Adám

Yes, e.g. 'race car' → 1

And also 'Σοφος' → 1

'hello' → 0

Jeff Zeitlin

Aug 11, 2020 14:56

OK, so that's normalization...

Adám

'Νιψον ανομηματα μη μοναν οψιν' → 1

rak1507

⋄ {⊢≡⌽1 ⎕C ⍵}'race car'

Dyalog APL

@rak1507 1

Adám

@rak1507 That's giving a 1 for the wrong reason.

rak1507

ah

Adám

Aug 11, 2020 14:59

Mixture of train and dfn.

Bubbler

⋄ ((⊢≡⌽)¯3⎕C~∘' ')¨ 'Νιψον ανομηματα μη μοναν οψιν' 'Hello world'

Dyalog APL

@Bubbler 1 0

rak1507

oh forgot about the spaces oops

⋄{(⊢≡⌽)(1 ⎕C ⍵)~' '}¨ 'race car' 'Νιψον ανομηματα μη μοναν οψιν' 'Hello world'

Dyalog APL

@rak1507 1 1 0

Adám

@rak1507 Fails on 'ẞUß'

rak1507

Aug 11, 2020 15:01

¯3 then instead of 1 I guess

Adám

Yes. Important distinction there.

Here's a trick too: Monadic ⎕C is ¯3∘⎕C

rak1507

clever

Adám

Can anyone find my 11-character solution? (Hint: a train ― of course.)

Maybe we should continue meanwhile…

Let's have a look at ⎕DT. It is the all-knowing wizard of date-time conversions. Basically, it allows you to convert any numeric representation of a date-time into any other representation. You can use it to glue together two 3rd-party systems that otherwise can't easily communicate.

⋄ 20 ⎕DT 44053.674 ⍝ to Unix time

Dyalog APL

@Adám 1597162233

Adám

The basic representation of a moment is the number of days since 1899-12-31.

Jeff Zeitlin

Aug 11, 2020 15:11

What's the 44053.674? Excel?

Adám

@JeffZeitlin No, Excel is off-by-one due to perpetuation of an issue in Lotus 1-2-3.

⋄ 40 ⎕DT 44053.674 ⍝ to Excel

Jeff Zeitlin

You'd think this many years after 1-2-3 is gone...

Dyalog APL

@Adám 44054.674

rak1507

is there a way to do {(f ⍺ g h ⍵) i ⍵} in a train? where f g h and i are all functions

Adám

The reason you want our system (which was actually the original one) is that you can then find the day-of-week with 7|⌊:

⋄ 7|⌊44053.674

Dyalog APL

Aug 11, 2020 15:13

@Adám 2

Adám

0: Sunday, 1: Monday, etc.

@rak1507 I think (f g∘h)i⊢

rak1507

ooh cool thanks!

Adám

@rak1507 Alternatively: ⊢i⍨∘f g∘h

Does anyone use some software that has its own date format?

Answer: Yes, you all do. APL does. It has the 7-element vector ⎕TS for the current Time Stamp.

⋄ ¯1 ⎕DT 44053.674 ⍝ to ⎕TS

Dyalog APL

@Adám Illegal code

Adám

Huh.

⋄ ¯1 ⎕DT 44053.674 ⍝ pretty please

Dyalog APL

Aug 11, 2020 15:17

@Adám Illegal code

Adám

@DyalogAPL Really?

⋄ ¯1 ⎕DT 44053.674

Dyalog APL

@Adám Illegal code

xpqz

     ¯1 ⎕DT 44053.674
 2020 8 11 16 10 33 600

Adám

Oh well. Yeah, that ^

xpqz

My code isn't illegal

Adám

Aug 11, 2020 15:19

(@Moonchild Any idea what's going on here?)

@Razetime Welcome to. We're in the middle of an APL lesson, but feel free to hang around.

The left argument, as you've seen, tells ⎕DT what you want to convert to. The numbers are largely arbitrary, but not entirely so. Positive codes indicate a scalar format (one number per date-time) and negative numbers indicate a vector format (multiple numbers per date-time).

Also the number divided by 10 and floored indicates the family. So we had 2(0) for UNIX and 4(0) for applications (Excel).

The last element of ⎕TS is the milliseconds. We can get more precision in the ⎕TS-style result by using ¯2 for microseconds and ¯3 for nanoseconds:

           ¯2 ⎕DT 44053.674
┌─────────────────────────┐
│2020 8 11 16 10 33 600000│
└─────────────────────────┘
           ¯3 ⎕DT 44053.674
┌────────────────────────────┐
│2020 8 11 16 10 33 600000000│
└────────────────────────────┘

Notice also that vector formats are enclosed. This allows ⎕DT to handle arrays of dates:

      ¯1 ⎕DT 44053+⍳3
┌─────────────────┬─────────────────┬─────────────────┐
│2020 8 12 0 0 0 0│2020 8 13 0 0 0 0│2020 8 14 0 0 0 0│
└─────────────────┴─────────────────┴─────────────────┘

I won't bother you with all the allowed codes. They are readily available in the documentation.

What you do need to know is how to convert from one of these formats.

Until now, we've just used the Dyalog day number. That's the default for simple scalars in the right argument. The default for enclosed vectors is the ⎕TS format (¯1).

If your input is anything else, you need to give ⎕DT a two-element left argument. The first element is the input type, and the second is the output type.

E.g. This converts an ISO year, week of year, day of week to ⎕TS-style:

      ¯11 ¯1⎕DT⊂2020 40 3
┌─────────────────┐
│2020 9 30 0 0 0 0│
└─────────────────┘

rak1507

anyone have a link to solution answers from last year?

Adám

@rak1507 No official solutions have been published. You may be able to find individually published solutions.

AviFS

⋄ 'Testing ','Dyalog ','Bot '

Dyalog APL

@AviFS TestingDyalogBot

rak1507

Ok thanks

Jeff Zeitlin

Aug 11, 2020 15:33

How about handling custom string formats, e.g., '2020-08-11 1132'

AviFS

@Adám Interesting. Never played around with SE bots, so I don't know what's possible. But wondering if when one edits a Dyalog Bot thing, if the bot can then edit its response?

Adám

Challenge: Given two style dates (as a 2 element vector of Y,M,D vectors), compute the inclusive number of days between them. E.g. (2020 6 25)(2020 08 10) should give 47.

ngn

@Marshall @dzaima ping (sorry for the interruption)

Adám

@JeffZeitlin We'll get to that next.

AviFS

Whoops, sorry. Didn't realize APL Cult. is still going on. Glad I didn't miss the whole thing! Didn't mean to interrupt

Adám

Aug 11, 2020 15:34

@AviFS I guess it is potentially possible. Ask Moonchild.

(2020 08 10)(2020 6 25) should also give 47. And (2020 08 10)(2020 08 10) should give 1.

If anyone wants more time, let me know now. Otherwise, I'll assume nobody is working on the challenge :-)

OK: {1+|-/1⎕DT⍵} will do.

@JeffZeitlin OK, now for the last 15 mins, let's talk about converting date-times to text.

The first step is to convert to a Dyalog day number.

Then you use 1200⌶ to convert that to text. It takes a left argument which is a pattern using a scheme we came up with, which I think is a really neat one. Much better than any of the existing ones out there.

Jeff Zeitlin

Wait, what if I have the string and want to convert it to a date/time?

Adám

      'YYYY DD MM hhmm'(1200⌶)1⎕DT⊂2020 08 11 11 32
┌───────────────┐
│2020 11 08 1132│
└───────────────┘

Quintec

Hello

Adám

@JeffZeitlin We're considering adding that. It has the problem of some formats being ambiguous. E.g. what is 2020111 using the YYYYMD format?

rak1507

@Quintec !

Adám

Aug 11, 2020 15:48

@Quintec Hi.

Jeff Zeitlin

@Adám - Agreed, but some formats are simply abominations - like that one. It should be YYYYMMDD, forcing the zero-padding.

Adám

@JeffZeitlin Sure, and we might add that.

Jeff Zeitlin

My first inclination is to say "if the format and date given don't parse unambiguously, throw an error". So that particular example would do so.

Adám

The system in the pattern for 1200⌶ is that numeric parts of the date are uppercase, while parts of the time are lowercase. You can use a single character for a variable-width pattern, or multi-character for a 0-padded pattern. If instead you want space-padding, use an underscore as the first character:

      'YYYY-DD-MM@hh:mm' 'YYYY-D-M@h:m' 'YYYY-_D-_M@_h:_m'(1200⌶)¨1⎕DT⊂2020 8 11 1 3
┌──────────────────┬───────────────┬──────────────────┐
│┌────────────────┐│┌─────────────┐│┌────────────────┐│
││2020-11-08@01:03│││2020-11-8@1:3│││2020-11- 8@ 1: 3││
│└────────────────┘│└─────────────┘│└────────────────┘│
└──────────────────┴───────────────┴──────────────────┘

Quintec

1:3 i sincerely hope no one writes times like this

Adám

Aug 11, 2020 15:54

Of course, but you may want to write:

      'YYYY MMM D "at" h:mm'(1200⌶)1⎕DT⊂2020 8 1 4 30
┌──────────────────┐
│2020 AUG 1 at 4:30│
└──────────────────┘

Jeff Zeitlin

"tt:mm" for 24-hour times?

Quintec

Ah, MMM, neat.

Adám

@JeffZeitlin No, t is for 12-hour. h is for 24-hour.

@Quintec Yeah, and furthermore, our format allows casing:

      'YYYY Mmm D' 'YYYY mmm D'(1200⌶)¨1⎕DT⊂2020 8 1
┌────────────┬────────────┐
│┌──────────┐│┌──────────┐│
││2020 Aug 1│││2020 aug 1││
│└──────────┘│└──────────┘│
└────────────┴────────────┘

And foreign languages:

      '__fr__YYYY Mmmm D'(1200⌶)1⎕DT⊂2020 8 1
┌───────────┐
│2020 Août 1│
└───────────┘

Jeff Zeitlin

@Adám - Huh. Seems to be backwards from most other dt formatters...

Adám

@JeffZeitlin That one was the most controversial of our choices. I guess you can remember it as "hour" of the day, and simply "time" (which isn't enough to specify the hour).

Quintec

Aug 11, 2020 15:58

fwiw python also uses %H for 24 hour time

Adám

@Quintec But a completly non-mnemonic %I for 12-hour time!

Quintec

yep, and moment.js uses H for 12 hour time, and... k!? for 24 hour

Adám

Anyway, 1200⌶ has lots of options too, including custom languages. Have a look at the documentation.

Any questions before we conclude today's lesson?

Jeff Zeitlin

Juust one..

Quintec

Since Dyalog day number is just one number, I don't suppose there's support for timezones?

Jeff Zeitlin

Aug 11, 2020 16:03

Why 1200 I-bar instead of a quad fn?

Adám

@Quintec Yes and no. There's no mechanism for attaching a timezone to a value. However, we do provide 'Z' and 'J' to mean Zulu time and local time, allowing you to compute the current offset. (We also included the other military time zones for convenience.)

Quintec

Cool, thanks.

Adám

@JeffZeitlin Because it was unclear if this formatting should be stand-alone, folded into ⎕FMT or ⍕, or be an extension of ⎕DT. Notice that the left arguments of ⎕DT and 1200⌶ don't conflict, so we could allow 20 '%ISO%' ⎕DT dates to format Unix times directly, without converting them first.

('%ISO%' is a built-in pre-defined pattern equivalent to 'YYYY-MM-DD"T"hh:mm:ss'. You can define more such patterns.)

Jeff Zeitlin

I'd argue for folding it into ⎕DT

Adám

Me too :-)

OK, thank you all for participating today!

Lesson 48 - ⎕C ⎕DT 1200⌶

The APL Orchard

Participants