the k tree

coltim

01:30

@chrispsn that test case is frustrating =|

I managed to get it down to 28 bytes though

kinda makes me wish for an expand primitive

coltim

01:51

(in that it feels like there's a more clever way to update the input =|)

I guess if null bytes are a valid fill character then maybe ` {c$x*y{c_x,c:y~*x}/x} would do the trick =P

24 bytes can be done with {y{c_x,("*";y)c:y~*x}/x}

and 20 bytes if the inputs can be flipped

oh it doesn't handle if the first character matches, shucks =(

chrispsn

02:06

@coltim would & on a dict count?

What would you want it to do?

coltim

@chrispsn hmm I think that's the equivalent of APL's compress

expand is sorta the inverse of that (see aplwiki.com/wiki/Expand)

I assume here the fill element would be " " so something like "*"^x\... would do the trick

coltim

02:34

ok, assuming null bytes aren't ok, I have 25 bytes with this. I guess so long as only the basic expand is considered (only 0's and 1's), it can be implemented as x@<>b

coltim

03:00

@coltim hmm it's a bit more nuanced. it fails if the first character to find is also present after the last character to find

chrispsn

11:39

@coltim this passes all tests - nice!

ngn

link with all tests (click "see full text" before clicking on the link):
https://ngn.bitbucket.io/k/#eJxVUtuK2zAQfc9XDOqbCJSyFEpSSl/7Dcu2yLr5IkfWJdbIIfvtlezNZYXxSGfOzJEOow4XQsnf/Pvnr3zh/3DPD/md4vUrXne7eLgs7+oVj/ntuouv5A8ICycbgdtT9NZAbCWESUoBLEJqO96CsU2I0gcQnSRHwq2ogd4Wt/TTEp+PVG6BvFW9KEPkLBSN6LuTLn0iC/VfKSzcajbyWZs2MT2Bm/MpLqMqxPMGPSN0g+gdW4udHxhHhJRUz6APuRdKMZiT0ljLHFfzGusTVlH1fOt5PW6tFkhL0HZuW+jtlKDTHGE8Yw84jl1AcYIIZmxKN83PPZrm5o9+bllVzrSn+IAMbTaJARX0elxkn6cGMiPHPXl2WX4yRkmwmAfIhSGx0koeH3kLyYMau6GkbPK3HU2ervuVNGm5RMh24dk5EB4jaD6zHloEAx0CyrEbR4uPu0iyf/nxhXyoDI6bARpoBx19W1lVZSD7by93TugYTJ57…

chrispsn

@coltim super clever. i've never seen something like this 'consume from front but append result to back' approach

ngn

@coltim pls post it so i can give it a bounty

(it will take more than a week, as there's a currently active bounty for apl and i can't add a new one)

chrispsn

11:59

@coltim i'll post this on kcodetweets (crediting you) if that's cool

ngn

12:10

@coltim @chrispsn c_x,c:y~*x -> y_x,y~:*x for 24

chrispsn

@ngn nice. would it be a little faster (no change in function) if using = instead of ~?

ngn

@chrispsn i'm in the middle of refactoring all arithmetic, i don't know how it will end up

btw, is there a term or a short phrase to describe "user sees single int type"? i.e. if an operation on smaller ints overflows, they are automatically converted to larger ints

chrispsn

size-independent?

autosize?

ngn

autosize (or autosized?) ints might work

i'm also considering "fluid ints"

chrispsn

v8 describes it as 'elements kind transitions' v8.dev/blog/elements-kinds

btw do you ever 'go smaller' eg if you know that a list of longs is just 1s and 0s?

ngn

12:26

@chrispsn for now i'd like to make it correct for 'go larger'

ktye

@ngn are 8 bit the smallest size?

ngn

it may be beneficial to "squeeze" in certain cases, but i think generally it's not as important as going larger

@ktye currently 16bit, soon 8bit, maybe some day 1bit too but i'm not sure it's worth it

ktye

what can shrink, besides logicals <=>.. maybe modulo?

ngn

@ktye does "where" (&x) count as shrinking?

ktye

yes, that could too. you can check the length in advance.

so could find.

@ngn is the int type invisible to the user (or added as a suffix)?

ngn

12:36

@ktye currently it's invisible in the output but you can still type 0h,0i,0l. eventually i'll remove that too.

there's still the question of what to do with nulls..

chrispsn

it might be useful to be able to 'lock' a type eg fp16 for machine learning training

ktye

@ngn what will @x return, always the same?

ngn

@chrispsn i was talking only about ints. if i ever add a 32bit or 16bit float type, they won't overflow, of course

@ktye probably just `i

the round-tripping axiom is not violated, as 123i~123l

ktye

@ngn arthur does 0. you could have a special type for atom null, and maybe a flag that a vector contains nulls, and a pointer to an index list. Then do the operation on all elements and update the index list as the combination of x and y for dyads.

ngn

@ktye these index lists sound complicated. i think i'll just use 64bit when there's at least one 0N. there are so many more important things to fix..

ktye

12:47

@ngn how does 64 bit help? you still lose nulls on arithmetics?

ngn

@ktye well, 64bit works like before

@ktye what do you mean by nulls on arithmetics?

ktye

@ngn 1+x if x contains nulls

ngn

1+0N is -9223372036854775807, it's not meaningful

is k9 going to have null-preserving arithmetic?

ktye

with the index list, you'll have null propagation just like floating point. you also would not have to test on each element of a vector, which makes it fast.

@ngn i think k has no 0N for ints. they are just plain 0.

ngn

that's an option too but then i'll have no moral right to say i've reimplemented k6 :)

chrispsn

13:21

Seeing @coltim's answer makes me wonder in which other situations '0N'-style is useful versus k9's

Probably the most controversial change

(Overall I prefer it so far but always good to see counterexamples)

ngn

13:49

@chrispsn my guess: the same situations in which 0n is useful - missing data

of course in golfing we'd abuse any feature present in the language, useful or not :)

coltim

14:21

@ngn how are you able to enter such a long URL? I would always get some "message too long" thing

ngn

@coltim insert a newline with shift-enter somewhere in the message (but not at the end)

SE chat renders multiline messages differently

coltim

@chrispsn I think of it as a hack to work within the constraints of over/scan (that being that you can't return out-of-band information, as that would then get fed into the next iteration). using ~ was intended to reduce the chance for conflicts (where the values you actually want to return overlap with the input) although in practice (particularly here) it doesn't make a difference

@ngn ooh good to know, thanks!

@chrispsn be my guest!

@ngn will do! and thanks for the byte saving =P

@ngn was k6 ever "finished"?

ngn

@coltim i don't know. it was never published and never officially pronounced dead, afaik. there's too much secrecy around k in general.

but the fact that shakti exists and is going in a different direction is evidence we are unlikely to see anything like k5 or k6 again :(

ktye

14:42

@ngn "different direction"? arent' you just steering in a similar direction with the integers?

coltim

@ngn I feel like there's some pun/play on words here with "reimplemented" (if it was never fully implemented to begin with)

ngn

@ktye i'm cherry-picking the good parts

ktye

@ngn what are the bad parts (besides expressions?)

ngn

@coltim well, right.. there was an implementation in progress, which i never saw, but i had seen k5, which judging by oK's docs must have been very similar. i wanted to make a simple fast vector language which people would use, so my best option was to piggyback on k5-6 syntax to ease adoption. but then shakti happened.

@ktye this may sound vague: i can't point to a single thing, but i think overall the primitives fit together slightly better before the k9 reshuffle

this is a matter of taste. i like some k9 things and dislike others.

but it doesn't matter what i like.. the availability of paying customers decides the fate of a language :|

coltim

ok, tried to explain my golf it as much as I could (honestly there are parts of it I don't really understand, ha)

juanez

17:50

@ngn if you make ngn/k more approachable (like q) you may find larger adoption

shakti, on grounds of it's k only mode, is unlikely to find mainstream adoption imo, especially with kx having a similar product with established market share...

kdb is 'good enough' and the secret sauce is no longer secret. many firms do not appreciate k as a language (because of the scarcity of developers)...

ngn

@juanez but q is so ugly compared to k..

juanez

beauty is in the eye of the beholder ;)

(and if you cant win, join them... )

ktye

what aspects of q do you mean? that monadic functions have names, or the table/sql interface?

ngn

@juanez it doesn't have to be mainstream to be successful. arthur's name alone can attract a lot of attention and money..

juanez

i dont doubt Arthur will find commercial success.. but i, dont think it'll be close to what he had with kx

ktye: both. i know it's unpopular in k-circles, but most people do prefer "count" over "#" especially since they are all overloaded - so if you're quickly scanning code someone else wrote, in my experience "count" is easier to find/parse than "#"

ngn

18:00

@ktye yes, the use of english words for monadic functions, mostly. i don't know what "the table/sql interface" is

juanez

ksql i assumed ?

ktye

@juanez yes.

ngn

ugh, right.. that exists

ktye

i don't see how shakti is different. it's only k mode, but it has builtins defined as words and ksql as well.

juanez

to be honest i've not looked at k9.. i didnt find it appealing to look at work-in-progress on something im unlikely to ever use

ngn

18:02

have you ever felt the urge to use a select statement in a k program? :) me - never

juanez

i dont write k-programs...

k is not what made kx successful

q is

(and kdb, ofcourse)

ngn

@ktye well at least it has monadic +, -, etc

coltim

@juanez I agree with this (in that the market value of q/k is mostly in the k/qsql and database aspects)

ktye

there is, e.g. count,first,last,countd,sum,min,max,avg

ngn

yeah, i know, and that sucks

ktye

18:10

@ngn whats the point? you can also just define them in k. with monadics, there's no difference.

coltim

I guess also in that to a certain degree it's a DSL for setting up e.g. ticker plants or whatnot (I feel that things like partitioned tables are a super convenient fit for e.g. daily security prices)

it's just tough to identify other niches where there's an equivalent opportunity for a more targeted language =|

ktye

what is a partitioned table? is it that the columns are stored in junks?

coltim

@ktye so a splayed table has one file per column. with a partitioned table you pick a column, and then it creates a subfolder for each distinct value of that column (each subfolder containing one file for each column). I'm explaining it poorly but kx's docs are probably helpful

ktye

i see, thanks

ngn

has there been any news from "the platform" since the initial announcement of coming "soon"?

i don't visit hn, reddit, etc often enough

ktye

18:22

i dont think so.

juanez

19:13

kdb's are most commonly partitioned by 'date'

the p-attr is also extremely useful for on-disk fast lookups..

i would think that kdb is suitable for quite general purpose applications, it's a great fit for finance, but i could see it easily extend into healthcare and a variety of other areas

i've not seen anything new about 'the platform' .. i was/am a little excited about it, but we shall see

juanez

19:52

l2021.01.29 7GB (c)shakti 2.0
" hello world "
" hello world "
`$" hello world "
` hello
`$'" hello world "
` `h`e`l`l`o` ` `w`o`r`l`d` ` `

spaces not allowed in names ?

ktye

@juanez they are, but limited to 8 bytes: `$"hello wo"

juanez

20:58

ah, i suppose that makes sense. will naturally limit the misuse of symbols/names

chrispsn

22:52

@coltim ah very good point

@juanez that will be extended to a larger limit afaik - just nyi

juanez

23:21

ah, i see.

ngn

@chrispsn but still a limit?

juanez

i think a limit makes sense, tho in the language it may be... too opinionated i suppose...

personally, i've set limits on a number of hdb's in my care, symbols cannot exceed a certain size...

the things some put in there can be entertaining? :)

ngn

my strategy is to represent symbols as 4byte ints. if the symbol consists of <4 chars, the int is the chars themselves with a '\0' at the end. for longer symbols the int is the negation of an index in a global list of strings.

Transcript for