@MartinJaniczek You need to do a Fourier transform on the samples to obtain a set of sine waves.

Tried answering an old string question: Try it online!

Any good golfs?

Perhaps I should add a special golf keyword to Kap? Let's call it G. Then the command Gnnn would be the solution to code golfing challenge nnn on SO.

Maybe create a new programming language for that. Then you don't even need a keyword. Just an efficient way of encoding the challenge ID in a string.

@EliasMårtenson haha, I'm sure there's some joke language which does that

Actually, now that I think about it. It should be easy to implement. The compiler would go to the SO page and download one of the solutions and use that.

Beautiful.

perfection

just select the one with the lowest bytecount

it transpiles to any language

It's so obvious in retrospect. I think this may be the solution to all future programming problems.

<klg> prior art: StackSort xkcd.com/1185

@EliasMårtenson there was even a family of languages that encode the program in the language name :)

@ngn Nice :-)

There are also tricks where you encode the program in the filename. Thus, every program is zero bytes in length.

0x7F and "Lenguage, 0 bytes" joke

@DyalogAPL @ klg sleepsort is missing :) ({⎕←⍵}⊣⎕dl)peach (⊢-⌊/)

@Russtopia Sorry for not responding earlier. I would definitely recommend not using the built-in editor. Even Jürgen explained in a recent message on the mailing list that it was only added for compatibility with older APL's.

I'd recommend using gnu-apl-mode for Emacs. But then again, I'm biased. :-) Since I wrote that mode

@EliasMårtenson And then it isn't even compatible ¯\_(⍨)_/¯

@Adám From the mailing list discussions, it seems he only implements features that some of the hardcore APL2 people ask for. :-)

There are some heavily traditionalist people on the list.

Apparently almost every language out there has its Emacs mode

And he really goes out of his way to make sure they are happy.

@EliasMårtenson Clearly, none of them ever did A/¨B

@Bubbler Yes, but some are better than others. My Emacs more is heavily integrated with the running APL instance in a way that is similar to how SLIME works for Lisp

@Razetime ∧/~ -> ⍱/ (as the argument is of length 2)

@Adám I've seen people being confused about arrays inside of arrays, and that's literally what the "2" stands for right? The first one didn't have that, if I understand things correctly.

@EliasMårtenson Oh, wow.

@Razetime or better: ∧/~x:'U'⋄' ' -> ' U'[⍱/x]

@Razetime 'AEIOU'∊⍨⍺⍵ -> ⍺⍵∊'AEIOU'

Mini TryAPL based on Razetime's work. (Note: You may have to reload the page the first time.)

@Razetime the whole {∧/x←⍺⍵∊'AEIOU':'N'⋄' U'[⍱/x]} could be simpler: {'U N'[+/⍺⍵∊'AEIOU']}

that is neat

There are a lot of optimisations (golfs?) that could be done too. Do you want another PR?

I don't really see the need to golf it

but document-idle is a good idea

I'll add that

@Razetime You mean remove?

I meant edit

@ngn @Razetime taking a step back to look at even wider context: {' ',2{∧/x←'AEIOU'∊⍨⍺⍵:'N'⋄∧/~x:'U'⋄' '}/⍵} could be {'U N'[1,2+/⍵∊'AEIOU']}

@Razetime can ⍵ ever be 'U'? no digit from the input is translated to 'U'

⍵ can't be U since the digits translate to OIREASGTBP

@Razetime best so far: 'OIREASGTBPU N'[11~⍨,∊(⊢,¨⍨10+1,2+/5∘>∧2∘≠)⍎¨⍕⎕]

er.. without the , of course: 'OIREASGTBPU N'[11~⍨∊(⊢,¨⍨10+1,2+/5∘>∧2∘≠)⍎¨⍕⍵]

@ngn original question

sometimes it's easier to infer the question from the answer than it is to read the question :)

this question is very oddly phrased lol

are we allowed to use ⍞ instead of ⍕⎕ when the input is an integer?

@Razetime new best: ∊'U'⍬'N'[1,2+/a∊'AEIO'],¨a←'OIREASGTBP'[⍎¨⍞]

@ngn Your APL has destructuring, like so (a b)←1 2

It looked neat, so I implemented the same

What are the rules for destructuring in your implementation?

@EliasMårtenson it requires ( ) and supports fewer cases than dyalog

@EliasMårtenson also, ngn/apl is now officially dead

@ngn Oh, I did not know Dyalog had it.

@ngn Oh really. Why?

@EliasMårtenson i haven't worked on it for a long time. for practical purposes (i.e. not golfing) i like k much more than apl.

@ngn Fair enough.

I have some personal issues with APL-like code written using only ASCII. The standard APL character set is already constrained.

@EliasMårtenson k is not an apl written using only ascii. it went through a long period of evolution on its own.

j is the one that remained close to its parent

@ngn Sure, that's not what I meant. I looked at K, and the language is difficult to get a handle on, and I think it would be easier to read with more symbols.

@EliasMårtenson "difficult" - really?? you find it more difficult than apl? i'd like to know why :)

@ngn Yes. But that doesn't mean it is more difficult. It just means I haven't spent much time with it.

i sympathize with the sentiment about symbols vs ascii, but you know, it's so much easier when you don't have to deal with fonts and keyboard settings. you can sit at any computer anywhere and just start typing in any editor.

ascii is universally supported. i don't like it, i think it's badly designed, with some unnecessary chars, and some important missing chars, and many wasted codepoints. it's too american-centric. but i can't help it. that's the way it is, it's the closest we have to a universal alphabet in computing.

@ngn Nothing you say is wrong. But for the same reason I don't bow down to popularity and install Windows, I don't want to work on/with things I dislike. :-)

Of course, things are differnet when I get paid for work. That's why I have an editor open with a Java webapplication open right now.

@EliasMårtenson windows may be popular among the general population, but for developers it's the norm. go for bsd! :P

@ngn I'll one-up you one better :-) I have a raspberry pi next to me with RISCOS running on it :-)

@EliasMårtenson ok, you win

Haha :-)

It's actually a pretty fun os

@Adám regarding your recent comment about people with cs degrees having difficulties with picking up apl: i think it's not so much cs itself (education definitely helps!) but the fact that cs is taught using loopy languages - that builds the wrong kind of habits for an array language

@Adam It looks as if TryAPL always renders a notebook markdown cell with backticks in it using the fixed-width (code) font. Is that a bug or a feature?

@ngn I didn't have that problem, but then I started my maths degree in 1966, first met APL in 1968 and did my cs masters in 1974 :)

@RomillyCocking what languages did they use for education back then?

I went to Birkbeck College, which was atypical in many ways. It offered part-time and evening courses, and had more of a vocational bias than most CS courses. We were taught to program in PDP8 Assembly, a little COBOL (ugh), Lisp (mmm, tasty) and Algol68 which I liked.

The course included some good theory as well: Graph theory, theory of Computation, and an outstanding Adaptive Systems course taught by my lifelong fried Tom Westerdale, who was a student of John Holland of genetic algorithms fame.

Of course by 1974 I'd already been programming for quite a while.

@EliasMårtenson Not sure if it counts but of the 10 Pis running on my desk atm, three are running Dyalog APL, all are running Python, one has a 170 GB Postgres database on a 1 TB SSD, a couple are running TensorFlow and one is running PyTorch. All under Raspberry Pi OS, though, which is a bit boring.

*fied -> friend :) Programming for six decades, still can't type!

I recently learned some COBOL. Why? Fun. I like weird things.

I gave up on COBOL when I learned that

@ngn wowowo

@RomillyCocking Those values aren't needed in modern COBOL. And even in the old ones as you pointed out they are basically just comments with a hardcoded format.

But comments that are required and then ignored. That's madness.

It may be that some compiler uses those values for something, but the only old compiler I have access to (IBM COBOL on MVS, running in an emulator on my machine) doesn't seem to care.

The simple COBOL program I had to write as coursework took about 70 lines of code. It would be two short lines (or one medium-length one, maybe 30 chars) in APL.

Sure sounds like old mainframe COBOL. In modern COBOL it surely have been no more than 40 lines.

@EliasMårtenson Lol

Also the 70 'lines' were on punched cards. Since I typed even worse then than I do now you can imagine how long it took to write/debug the program.

@RomillyCocking Oh you didn't use 3270 terminals?

I'm looking at a test program on my emulated mainframe right now: photos.app.goo.gl/7E3VJ55tgzXsTs9Z7

That was at Uni, where we submitted jobs remotely via a card reader connected to a CDC6600 via an IBM m/f running HASP.

Those were deffo not the days.

I used 3278s a lot later once I'd stared earning a living writing APL. But before that, we used daisy-wheel terminals with an APL character set.

columbia.edu/cu/computinghistory/diablo.html

@ngn small problem: we also need to detect and perform decoding

@Razetime why? does the challenge require that?

@ngn yep lol

@Razetime 'OIREASGTBP'{≡⍵:10⊥⍺⍺⍳⍵~'UN'⋄∊'U'⍬'N'[1,2+/a∊'0134'],¨⍺⍺[⍎¨a←⍕⍵]}

cool, post it

@Razetime i just wanted to give you a something to start golfing from

well, that's barely anything like what I originally made...

I'll definitely try understanding it

how could i have missed this: a∊'0134' -> a∊⍕1340

why not a∊⍕⍳5

@Razetime what about the '2'?

oh well

@ngn -3: ≡'OIREASGTBP'{⍺:10⊥⍺⍺⍳⍵~⍵⍵⋄∊⍵⍵[1,2+/⍵∊⍕1340],¨⍺⍺[⍎¨⍵]}'U'⍬'N'⍕

what about ~⍵∊⍕2

@Razetime but ⍵ could be among '56789' too

we want to detect only the digits corresponding to vowels

oh ok

Are assoc lists (dfns.dyalog.com/n_alists.htm) generally considered OK or is there a more idiomatic APL alternative for the same purpose?

@MartinJaniczek Traditionally, APLers would just implement it themselves using separate arrays for the keys and the values. This collection of utility functions are pretty light-weight, so I guess that's about the same. If you're going to do a lot of lookups, then you might want to look into hashed arrays.

Right, I was just wondering - since I have 3 arrays I need to keep track of "per key", then it's probably better to have them like X X X X, Y Y Y Y, Z Z Z Z, instead of inside one nested array (X Y Z) (X Y Z) (X Y Z) (X Y Z)?

Didn't know about hashed arrays, will read up help.dyalog.com/18.0/index.htm#Language/I%20Beam%20Functions/…

Data types are important too. If your arrays have mixed data types (e.g. 1st element a charvec, 2nd a float, and the third an integer matrix) then you may be better of "transposing" things into a character matrix, a float vector, and an integer 3D array.

This technique is known as inverted tables.

With the XXXX, YYYY, ZZZZ approach I wouldn't be mixing datatypes but I would be nesting arrays. The idea is, I have a concept of "release", and each release has preprocessed files on disk (fields = string vector, weightings = float vector, answers = bool matrix). I want to load multiple releases, and thus multiple those vectors/matrices.

In those resulting nested arrays, would the elements all have identical shape? E.g. each element is a 7 element vector?

no, those would be variable

I'm unsure if mixing those into a matrix and tracking "real length" elsewhere would be benefitial

I see. Well, once you go nested, varying types don't matter.

@MartinJaniczek That could work, depending on how much wasted space you have. Note that databases usually store string columns padded to a consistent length.

Releases vary quite a lot. There are a few "main" ones, having tens of thousands of elements, and a lot of small ones with only hundreds of elements.

So mixing stuff together into the maximum length would probably result in quite a lot of ~zeros~ padding

On the other hand, nested array have an overhead of like 40 bytes per element!

I'm wondering about (instead of nested array) keeping a flat array of eg. the weightings, but then I'd also probably have to keep a separate "index" array of which weighting belongs to which release... Might be worth it?

You have to decide what you optimise for. RAM is fairly cheap these days, but then again, so are CPU cycles. Developer time is costly.

@MartinJaniczek A traditional approach is the segmented array, where divisions are kept track of with a Boolean vector (remember that those only use 1 bit per element), but of course, if your Boolean would be very sparse, then indices are more efficient.

@Adám Is there any talk / paper / blogpost / doc about this?

@MartinJaniczek with tens of thousands per item, I don't think nested would have like any practical drawbacks

@MartinJaniczek There are various talks about why flat arrays are so efficient, but I don't personally know of any that help you decide. @Marshall can you recommend anything?

In any case I'm going to prototype it with nested arrays because I can wrap my head around how that would work, and then a future Martin can play with optimizing that to hashed / segmented / flat / whatever if needed :)

@Adám Isn't that why sqlite, postgres and every other database I've used has varchar? That is often the default for char if no length is specified in the column definition.

@Adám I think I misunderstood your message the first time I read it. I took it to mean "per scalar", now I'm thinking you meant "per top-level item"?

@MartinJaniczek getting hashed arrays to work is just as simple as doing a 1(1500⌶)array and mutating it carefully

@MartinJaniczek Right, per outer element.

@MartinJaniczek per every "object". An "object" in Dyalog can be a whole numerical vector, a generic vector, but also pure scalars if e.g. you're storing mixed type arrays.

@RomillyCocking Varchar are quite inefficient, no?

@Adám There was a lot written about this in the 70s (when APL did not have nested arrays, and people had to use component files) and the late 80s (when they did). I'll see if I can find anything useful.

@Marshall This claims 80-byte overhead of nested arrays on 64-bit architecture, but is that really so? ⋄ b←,¨a←⍳1000 ⋄ 1000÷⍨-/⎕SIZE⍪'ba'

@Adám 46

@Adám they are memory-efficient. but first of all, it may surprise you that they exist.

@Adám It depends. The database with which I am working at the moment has 189 M records, of which the main three are a title (which can be over 1000 chars, but is usually ~100), an abstract (which is anything between 300 chars and 10,000) and a json field which is potentially very large. For that sort of data fixed column widths are out of the question.

Right.

And it sounds like @MartinJaniczek is in a similar situation of wildly varying sizes.

Also, his data is relatively large, so a 40 byte loss per item is negligible.

But many real-word databases have similar srtuctures, and the overhead for varchar is to fixed width much as the overhead for nested APL arrays is to padded matrices.

But is storage (both temporary and permanent) isn't an issue, and the nested representation is easier to work with, then are there actually any downsides to nested?

(Of course, if your data is truly multi-dimensional, then it should be stored as such.)

@Adám In situations like @MartinJaniczek's nested wins hands down I think.

Yup.

@Adám interestingly enough

@dzaima some form of laziness?

@MartinJaniczek it definitely is actually storing each item as an array, but + somehow ruins some packing?

That's so weird

@dzaima That's space needed to evaluate, not to store.

ah, that.. makes sense

Still strange that it requires nearly double the memory

@dzaima (but it isn't reusing the memory?)

It has to copy the array for the addition (though I'm not sure why it doesn't detect 0+ as a no-op), so twice the memory.

Ah

@dzaima Yeah, why isn't it in-place‽

Oh, I know why.

@Adám I'm current'y working on APL code to read jupyter notebooks in order to establish the network of dfn references, to find out what primitives are used where, and in which markdown cells they are described. It's so much easier working with nested structures even for a 1st gen APL dinosaur like me.

I never liked ]SpaceNeeded as it's method is very dodgy. It doesn't ask (have a way to) the interpreter.

Of course the data volumes are trivial.

]SpaceNeeded probably ends up duplicating stuff and so the array is actually shared, and it doesn't in-place. Something like that.

wasn't there some non-GCing alternative to ⎕WA or am i imagining things?

I don't think so. There's a non-compacting ⎕DR though.

@RomillyCocking Can you give me an example of a cell content that is being handled wrong?

@Adám Yup. I have a cell and a screenshot. I'll email them to you.

Can you email tryapl@ ?

Sure.

Done

Unicode APL logo: 𝛢⅊

Facepalm. One surprising property of tacit functions vs dfns - in tacit fns, arrays used in the body use their values at the time of definition, but in dfns they have "later" binding - using their current value. That's a bit of departure from my expectations (which are that way probably due to Haskell's tacit functions behaviour)

ie. is_loaded←{releases∊⍨⊂⍵} vs is_loaded←releases∊⍨⊂

(But it's a little bit hard to compare those as Haskell is pure and there is no "redefinition" / "re-assignment")

@MartinJaniczek I actually came up with a trick to have true lazy tacit functions in Dyalog. Interested?

Sure I'm interested :)

You can use the built-in editor or just type in the session:

∇ is_loaded←is_loaded
  is_loaded←releases∊⍨⊂
∇

Hmm. So, tradfn that returns a tacit fn-as-a-value?

Exactly. I built a library that makes this easy for you. Let's you simply write:

It has to return a new releases∊⍨⊂ everytime it's called, right?

]lazy is_loaded←releases∊⍨⊂

@MartinJaniczek Yes. Using the current value of releases.

@Adám hah :) I didn't know these are user-definable

( ]whatever )

Yeah, )name are system commands (built-in), while ]name are user commands written in APL.

In fact, you can load the definition (so you can look at it, or even modify it) of any existing user command by typing ]uload name

There's also a wizard for creating your own: ]unew name

I'm trying to understand the concept of what's inside Lazy.dyalog. Are you generating code and then ⎕FX "evals" it?

@MartinJaniczek Yes, bu that happens down here.

@Adám That's discussing the overhead of one link in a list, which both places the element in its own array and uses length-2 pointer lists for the pairs. The pairs take 6 words and the elements take ~4 relative to elements in simple arrays.

I see. Thanks.

@Adám @MartinJaniczek I don't know any resources about how to choose a flat structure for not-obviously-flat data. The general rule if performance is the highest priority would just be to pick whatever uses the smallest number of arrays (keeping in mind that non-homogeneous arrays are stored as many arrays).

Can I somehow throw an error (say, INDEX ERROR) from within a dfn?

Happy New Year's Eve everyone!

I was very sorry to read (belatedly) of Keith Smillie's passing. I noticed that the sigapl page for his writings seems to be missing a text I had found on his page at U of Alberta: webdocs.cs.ualberta.ca/%7Esmillie/RevisedTopics/Topics02Rev.pdf

It was called Topics in Computing and I had been working on converting the J code to APL last summer... Ah! Why didn't I think of archive.org already?

Hmm, it seems that this second chapter was the only one that was ever archived :(

@MartinJaniczek Yes you can throw an index error: ⎕SIGNAL 3

Well, look at that. My first commit of 2021 was for Kap: github.com/lokedhs/array/commit/…

@EliasMårtenson That's a pretty impressive project. How long have you been working on KAP?

@ab5tract happy new year's eve to you too! (i still have 2.5h to go)

@ab5tract wrong link? 404

I think that was the point, he didn't archive it in time before it was removed

@ngn Yup, that's my problem :( .. the whole directory disappeared. I think I have a backup on an old hard drive, will have to take a look when I get back to Amsterdam.

@rak1507 ah.. stupid me. it's missing the whole text.

It's really weird as it isn't linked from his home page or anything. I honestly can't recall how I ever found it!

That is, from what it looked like in 2018: web.archive.org/web/20180828141903/https://… . But I remember having trouble finding it from anything other than browser history.

Ah! It was the 'Topics' link down on the bottom of the link index. Anyway, hopefully I can find all of the PDFs. It would be a real shame to lose a sizable chunk of writing from him.

The uni might have backups, you could contact them maybe as a last resort?

@rak1507 email sent. Fingers crossed

Hope they have a copy for you

There was a fair amount of musing on computer history on those pages. It seems like a real bummer to have it just drop off the web :/

I'd hope they would have some form of archive, it seems sad to delete a bunch of someone's work permanently after they die

Fully agree. I'll update here, of course.