@nathanrogers yeah, something like that

the mapping between the empty symbol (`) and the identity function (::) is there to prevent the dict's values from collapsing into a typed array. this way i can assume the namespace's values are pointers without checking its type.

so then I have access to the symbols of a namespace, and the functions, I can then define a map of test data for each symbol in another dict who's key is also the symbol of the function, call the function on inputs defined in that dictionary

I still don't like having the names mentioned twice, that's kind of annoying

I guess with my naming schema they're already ordered

(i should make a userscript to create a codeblock of selected text. it's so annoying to get it right)

so I could just define an array for it, but that means I can't define specific test cases, just the ones that get returned from the file read

@ngn can I \l as namespace?

@nathanrogers would it be simpler if you used a list of pairs like: ((s2015011;dataforday1);..)?

@ngn itd be ((f;input;expected) ;...)

@nathanrogers not in ngn/k. maybe one day if i get to implementing \d ..

@nathanrogers that makes sense. or you can define a tester function and call it with t[f;input;expected]

@ngn so then I'd probably want to define a wraper function which returns the exports of the file, and appends them to their respective namespace?

@ngn did you see my commit from last night?

github.com/ndrogers/adventofcode/blob/main/k/2015.k

line 19

dayNum maps to parsers from the file for each case has different formats of input

@nathanrogers those are triples of (f;input;expected)? great, so you've already done it :)

dayNum also maps to the file

yeah

but I want to create a generic test file

that will run tests for every year

@nathanrogers ok, i see what you mean

@nathanrogers i don't have a solution for that now

@ngn you mean no programmatic way to populate a namespace?

possible workarounds: include the year in the name, or run each problem in a separate process and collect its output and compare against the expected

@nathanrogers there is a way to populate: ns[`key]:value but when you \l a file its content will end up in the root namespace

if I return ns?

that wouldn't work?

@nathanrogers ns.f will try to look up that path from the root namespace. if ns is local, you should use ns[`f] instead (a problem inherited from the original k's design)

but this method won't clobber names in the root correct?

like if I have multiple different names of ns, the name f won't get clobbered ?

right?

@nathanrogers hang on, wouldn't it be simpler if you use ns.f:{..}; ns.g:{..} in the global scope (for all years) and rename ns immediately after \l?

I could name ns differently for each year

that's no problem

right?

\l 2015.k
a15:ns
\l 2016.k
a16:ns
..

I don't understand a15:ns

@nathanrogers namespaces are just dictionaries. : is assignment.

well I wouldn't need to manually reassign if I just name them differently in the first place

@nathanrogers i thought you were trying to avoid that

no I was trying to avoid a15s011

brecause with namespaces, it means I defined the function table once, and just pass in the year in question

or rather just the namespace

@ngn What does (:) dex mean? It doesn't seem to work in current master, fails with 'cpl

@nathanrogers do you insist on having the solutions for a given year in the same file?

@copy yeah, i realized that mistake later and corrected it. try f:(:); f[2;3]

@copy "dex" is latin for "right", sometimes apl's ⊢ and j's ] are called dex.

Ah (I don't know any of latin, j or apl)

@ngn This should be added as well

@copy many people don't. i'd better rename it to "right".

@ngn I don't insist, I'm just trying to get my head wrapped around any sort of project architecture

but i think a file for a single line is a bit overkill

especially when some functions are first class functions that depend on other definitions to be fully formed

@nathanrogers i was going to suggest running each solution in isolation, like i did here (i can explain everything there), but my runner doesn't support multiple years - it should be easy to modify

nevermind... that blasted trialing \n

@copy "dex" is now "right"

@ngn where are your source files?

@nathanrogers click on my avatar and see the link there

I found it. not used to this git page

@ngn By the way, I have some shorter aoc solutions if you're interested (better algorithm, not better golfing): copy.sh/kaoc20/15.k copy.sh/kaoc20/25.k

yeah I gotta say, I don't really follow the project structure

@copy you solved problem 15 in k?!

@nathanrogers a20/*.k are the solutions for aoc2020. a20/i/* are the inputs. a20/o/* are the expected outputs.

so you just execute the file to get an answer

@ngn Yes, it becomes trivial if you represent it as a mapping from number to round it was said rather than array of numbers

my computer can't handle arrays that big :)

sorry, I'm repeating myself... forgot I asked already :C

@ngn really? I think I'm running @copy's a20/15... takes 17 seconds and the k process caps out at ~2GB of ram

@nathanrogers yes, i use one of ngn/k's extensions for forking a process and collecting its stdout: `x(("program";"arg1";"arg2");"input")

in this case the standard ."\\program" could work too (no args or input are needed)

what i don't like about ."\\program" (in both k proper and ngn/k) is that it starts a shell to run the program, instead of fork()-ing it directly. waste of pids :)

dinner time. i'll be back.

so `x is a special symbol?

@nathanrogers there are many special symbols

how do I find their definitions?

search the code for the stuff here: git.sr.ht/~ngn/k/tree/master/item/k.c#L34

@nathanrogers this should be a chapter in \ docs

I agree :)

@nathanrogers i've just added some examples

nice

fork are arguments to ."\\"?

@nathanrogers no, ."\\" is the official way to do it. it starts a shell. `x@ is my own little extension for direct forking.

Isee

how do I talk to processes? @ngn

either to kill, read, write?

@nathanrogers you can give input when you start a process and get output when it finishes. there's no facility for getting file descriptors to a process's stdin and stdout, though i think that would be useful.

if you can do that, I can work with `x as a stopgap while native functionality isn't available

@nathanrogers but how to do that so it's in the spirit of the rest of k..

name it and pass it to one of the n: ?

monadic < with a symbol argument is "open", it can create file descriptors for files and for client sockets. maybe "fork process and give me it's stdin and stdout" should be its job?

process: `x@(...)
3: process

or something

i/o is so hard to design in a consistent way

Ja, I really wish I had something valuable to contribute either for design or extensions or conversation, but I still feel like a foreigner in these lands

That is aside from "HOW DO I....!?"

But why not something like proc n: datatowrite for writing and reading just n: proc

Like with other io?

@nathanrogers "proc" being the file descriptor?

that's what i imagined, 1:fd is read() and fd 1:data is write()

the hard part is opening the file descriptor

I figured proc:`x@...

How would you obtain the file descriptor of your process otherwise?

the current idea is for open (<) to be able to create file descriptors for various things

I see

<`:/path/to/file or <`:host:port or <`::port (implicit localhost)

Looking a little j aren't you?

@nathanrogers yeah, too many colons :)

i don't have a good idea what to do with server sockets and accept()

and fork()-ed processes

and i don't like the way the original k handles that

What's that

@nathanrogers it's an argument to the k interpreter, e.g. k -p 1234 opens port 1234, accepts connections, reads and executes k expressions, and writes results over the socket

and it can be changed at runtime with \p 1234

it doesn't have that low-level unixy feel of dealing directly with file descriptors

I see

You want to uniform io then

Just another obvious observation is that processes need to be killable programmatically

@nathanrogers that means exposing pids too

No destructuring bind? Or argument expansion?

@nathanrogers what are those?

[a,b]=[1,2]

And f(*args) in python, or f(...args) in js

@nathanrogers ^this?

@nathanrogers f . (x;y;z) ←→ f[x;y;z]

Oh nice

So is the problem with processes that theres just too much to keep track of?

@nathanrogers maybe not too much: one pid and two(or three) fds

What if the problem with io in array languages is trying to shove stateful behavior into arrays, and maybe it's ok to bend the design so long as the design bends the same way for operations outside your environment

io is inherently stateful, there's no way around it

That's what I'm saying

And state full, behavioral code doesn't fit neatly into arrays or functions either

So the "notation" of any should reflect that to the reader

Maybe a namespace or dictionary is in order

@nathanrogers what for?

For interaction with Unix files and processes

To represent that interaction and encapsulate the implementation

@nathanrogers you mean return a dict like `pid`stdin`stdout`stder!1234 5 6 7 instead of just 1234 5 6 7?

(those backquotes and backslashes in md.. they get me every time)

@nathanrogers "encapsulate the implementation" sounds like oop

Well there is state

And you want to make interacting with state the same across multiple types

And those types need the same collection of data

And share methods

@nathanrogers you know k dicts are values (not mutable like dyalog's namespaces)?

Yes

the state is already managed by the operating system

all you need for reading and writing is a file descriptor, an int

for me, the statefulness of io is not a problem. the problem is to organize all the primitives for interacting with the outside world in a tidy and consistent way, ideally fitting in with the rest of the language

we've got open/close as < and >, read/write lines/bytes as 0: and 1: - those look nice and symmetrical

fork(), accept(), wait4(), kill(), .. are odd ones

Just use unicode fork

:D

🍴

Fork for fork knife for kill

🔪

🙇

Accept

`fork@ `accept@ `wait@ `kill@ .. would at least be typable :)

You don't code on a mobile phone?

@nathanrogers i don't even use one, except in exceptional circumstances

☠ for kill

🤡 to spawn zombie process

@ngn so you only interact with the world through k expressions?

Sending and receiving SE messages from a headless Unix shell?

@nathanrogers and email :)

Wait email through k. Or k through email?

@nathanrogers that's an idea :) "someone@example.com"0:lines

Frills

delete' 0: "my@email.address"

would anybody miss this syntax for symbols if i removed it - `:/path/to/file.txt?

it can always be replaced with `"/path/to/file.txt". kona, oK, and (last time i looked) k9 don't support it.

@nathanrogers can i show you how client sockets work?

(requires git pull and make and an installed netcat)

yes please

building

@nathanrogers open two terminals side by side

in one of them: nc -l -p 1234

in the other start k as usual

nc -l -p 1234 means "netcat listening on port 1234"

yes

in the k repl type: f:<`":127.0.0.1:1234"

I'm familiar with netcat, but don't use it much

f should be 3 or another small number

@nathanrogers great

f is 3

at this point we have established a tcp connection between k and netcat

huzzaaah!

and now?

for something completely different?

you can read with 1:f

k will block and when you type something in netcat and press enter, it should appear in the repl

you can write with f 1:"hello\n"

oh neat

and obv f 0: "text"

@nathanrogers yeah, i haven't tested all combinations, but i think they should work

so if I wanted to read from a stream to file it ould be "file" 0: 1: f

also: f 0:("hello";"world")

slick

@nathanrogers regular files are special (sounds like a contradiction) in that they are read with mmap()

i assume you're familiar with mmap()?

no

my unix is skin deep

more of a user

it's a syscall like read(), write(), fork(), kill().. etc

it's used for reading files efficiently

ok

with read() you need to fill a buffer and then copy it to a larger buffer, and it might need to grow.. there's a lot of copying going on, and you have to go through the whole file before being able to use it

"file" 0: 1: f does work, but here's my question, will the socket read until the end of lines, or will it read until end of what is in the socket, or some kind of socket size?

in contrast, mmap() reserves a region in virtual memory and establishes fault handlers for it, so the content there materializes only when you try to work with it, i.e. read it or write it

I see

so mmap() is instant even for enormous files, because it's like lazy loading

that's excellent

:D

what happens if when you read

there's too much stuff in it?

btw, dyalog has ⎕map for that, in case you ever need it. it's better than ⎕nget but harder to use.

@nathanrogers "too much stuff", like not enough ram to contain the whole file?

i think the OS unmaps some of the old pages in that case

or if it's not properly configured, it might start swapping..

@ngn I mean more than the space allotted

does it deallocate as your read through, or it persists?

now send me your proxy IP and we'll switch to chatting through K

@nathanrogers "deallocate"? from the point of view of the process (k), it's just virtual memory, a contiguous range of addresses that you can read from and write to. it's the OS's concern to create this illusion by filling it in from disk when necessary and freeing up physical ram when necessary.

i see

(by "disk" i mean any persistent storage)

so you can write and read so long as you have disk space

oh hey, did you get file appending implemented?

I was doing it in kona, and thought it was ngn, so I tried and it overwrote my file :P

@nathanrogers of course. the point is, it's faster when reading is done lazily.

@nathanrogers not yet. what's kona's syntax for that?

(,"file")6: lines

let me double check

ah 3: and 5:

says \:

(,"file") 6: lines

worked just fine

interesting, they have separate verbs for reading "mmaped" vs "copied to memory"

so when a message is pushed, and I'm reading from a socket, the message is already locally caching and filling up prepared storage and is just hanging out until I read from it? or it only reads complete message?

like what happens if I read from the socket while a particularly large message is mid-flight

@nathanrogers both sides in a socket have buffers for both reading and writing. tcp has mechanisms to control the flow of data, e.g. when one side is too slow, it can signal the other one to slow down too.

oh so that's a protocol thing not a memory management thing

@nathanrogers it should block in that case, i think..

@nathanrogers tcp is a "stream" protocol, so there's no concept of a "message"

packet?

but then packet sizes are fixed

udp and ip use packets, but tcp creates the illusion of persistent connections and bidirectional streams of bytes

so then who exactly is managing the messages over the socket?

@nathanrogers there's no concept of a "message". if an application wants to process some sequences of bytes as a unit, it should determine their boundaries by looking at the data itself (e.g. use headers, like http does) and not rely on a particular pattern of read()-s from tcp

the tcp/ip stack is complicated. it can route your ip packets over different paths, it can split or recombine them en route, it can deliver them in a different order or not at all. tcp takes care that all that looks smooth to the user. but it doesn't guarantee that when the sender write()-s something in one chunk, the receiver will read() it in a single chunk.

so applications are on their own to determine the boundaries of "messages" that should be processed transactionally

I see, so these are raw sockets. I would say, read until newline, or whatever form of data I'm expecting

I could be reading lisp expressions, and I'd wait until I get balanced parens or some such

f 0: $!10000000

the server seems to know the format of the data though

@nathanrogers what if you're writing an online shooting game and it's more important for the bytes to arrive as soon as possible rather than only when a 0x0a byte happens to appear in the stream? one size can't fit all

@nathanrogers wdym? it's just ascii bytes