Pyth5

I've done most of the boilerplate for Pyth 5, and can really start working on implementing the operators - I just need to fix the above package structure and tests

isaacg

Awesome

orlp

I've done a couple of cute things

github.com/orlp/pyth5/blob/master/env.py#L21

that should make operator overloading quite a bit easier

isaacg

4:50 PM

My pypi acount is isaacg

That's a nice one

orlp

I also have cleanly seperated Pyth

there is now a lexer.py that creates tokens

parser.py turns tokens into an AST

codegen.py turns an AST into Python code

and env.py runs the code and defines the builtins

isaacg

Beautiful

orlp

everything is pretty clean, except the lexer

but you can't really make clean lexers =/

they're massive state machines

isaacg

Can it handle things like F?

orlp

that will be handled at the parser

in particular, I have made the lexer pretty interesting regarding comments, strings and UTF-8

the lexer operates on bytes, not on strings

isaacg

4:53 PM

I see.

orlp

if it encounters a ", it will read an UTF-8 string

if it encounters a ." it will read a binary string, and turn it into octects

isaacg

Ah. That's nifty.

Does the lexer handle numbers, too?

orlp

so ."test" becomes [116, 101, 115, 116] in the code

@isaacg yes

isaacg

I see. That's be creat for golfing komogorov problems

great*

orlp

it also correctly detects "dot operators", like .A

isaacg

4:55 PM

Excellent.

orlp

and the leading zeroes are seperate tokens

very importantly - it can handle newlines, indentation and comments unambiguously

isaacg

What about \A?

orlp

that becomes "A" in the code

don't think .\ would be useful

isaacg

Is that lexer level or parser level, I mean?

orlp

lexer

the lexer strips away any amount of even spaces at the start of a line

so you can indent by 2 or 4 spaces

isaacg

4:57 PM

Same as -m, basically?

orlp

yes, but I've made it unambiguous

isaacg

Seems good.

orlp

newlines are removed, unless the previous line ends in a number and the next starts with a number or .

isaacg

Hm

That I'm not sure about.

orlp

the only use for newlines is token seperation

isaacg

4:58 PM

One very useful property of newlines is that they can trigger printing in anything

orlp

care to elaborate?

isaacg

For instance, a $ expression

orlp

oh about that

I think about fully removing $

isaacg

or if you want to print the return value of a p.

orlp

and replacing it with a Pyth interpreter instead

isaacg

4:59 PM

Like an exec function?

orlp

like a run_code function

isaacg

Right.

orlp

$"+3 5" would return "8"

isaacg

That seems good. And running Python code would just be impossible?

orlp

yes

it's a security hazard anyway

isaacg

5:01 PM

True.

orlp

also it breaks the self-containment of Pyth

isaacg

Yeah.

orlp

without $ we could start writing, say, interpreters in C++

isaacg

That's a good point.

orlp

I'm purging tuples entirely, but I'm not entirely certain what to do with the number system

isaacg

5:02 PM

Here's an example of where newlines are useful. Say you want to pop a list and print he result. Pop is default non printing, so you put a newline in front.

What about the number system?

Removing floats?

orlp

what I was thinking about is to merge a int/decimal/fraction system into one 'number' type

using the decimal and fraction modules

isaacg

Seems good overall.

orlp

I also had a tiny idea not really relevant that could be imported into Pyth right now

isaacg

Yeah?

orlp

+Z) -> abs(Z), -Z) -> -abs(Z)

isaacg

5:05 PM

Ah.

Sure, why not?

orlp

there's a couple points about the unified number system

first: what should the precision be?

100 decimal digits?

30?

isaacg

What's the downside to more digits?

orlp

slow

isaacg

OK

orlp

this is where the optional gmpy2 dependency comes in, too

without it it could be slow

isaacg

5:07 PM

Right - faster when it's needed

I think 100 digits is a good idea.

It's a golfing language - it's OK to be a bit slower.

orlp

basically my idea is that when you say for example /100 30 that internally it stays a fraction

auto-print will print the decimal though

and we'll have functions to get the denominator and numerator

isaacg

And when you say ^2 .5it makes an extended decimal, right?

orlp

yes, it will compute the result as a 100 digit decimal

isaacg

Rationals seem good as the default.

And decimals as the backup.

orlp

that's exactly my idea

isaacg

5:09 PM

OK.

orlp

complex numbers would work the same, but as a pair of these 'numbers'

isaacg

Sure.

orlp

then the big question I have left

how much of the functions should we redesign/move around?

isaacg

As many as we need to.

orlp

before I start implementing stuff it could be useful if we'd just write the docs first

to see what everything should do

isaacg

5:11 PM

OK. There's 2 parts to that, I guess - what we want to keep, and what we want to add.

orlp

3. what we want to remove

isaacg

Right.

orlp

4. what we want to move around

there's some things that I'd prefer as a two-character operator

isaacg

E, for instance.

orlp

hmm maybe

how many digits should be autoprinted btw?

isaacg

5:13 PM

Hm.

orlp

the problem with 100 decimal digits is that if you do /1 3 you get 100x 0.33333333333333333

isaacg

I'm not really sure.

One thing I was wondering about is more ways to manipulate functions. = and F are great, and we should have more things like them. One source of more single letter commands for them is statement characters. #, I, E, D, W, L, M, A.

I'm going to start adding some of those to Pyth.

But I'm sure more could be done in Pyth 5.

orlp

the new lexer/parser/codegen model should be very flexible

about the arity in the docs

it never was clear what N and co means

isaacg

Ah.

N basically means it won't be autoprinted at the beginning of a line.

Wasn't very useful or important.

orlp

err

I mean

S and X

isaacg

5:23 PM

S is special - doesn't take arity.

X is unbounded.

orlp

oh I just spotted a mistake

S autoprints, but shows as N

wait

a lot of them don't make sense

isaacg

Sorry.

N means it is autoprinted

stuff like V, W, R, etc. have Y.

orlp

i think for now I'll omit it from the docs

isaacg

Seems good.

orlp

keeping the docs to 80 columns or something small seems nice

for split view editors pyth/docs

isaacg

5:28 PM

Sure, why not.

Speakin gof the docs, web-docs shouldn't be a separate file, it should be autogenerated.

orlp

yeah

I was thinking of that too

for now I'm not going to do the server at all by the way

in fact, now that Pyth will be a proper package, I think it should be fully standalone

you should be able to just require Pyth in your webserver and it should work

isaacg

Make it a separate repository, perhaps?

orlp

yeah, like that

you should be able to just do import pyth; pyth.run_code("Q", "13")

orlp

5:54 PM

@isaacg what's your opinion on dependencies?

most notably nose for testing

isaacg

6:09 PM

Dependencies are OK by me, if they make the code nicer.

orlp

7:09 PM

@isaacg check out the testing framework I made: github.com/orlp/pyth5/blob/master/test.py

isaacg

That's certaintly a novel approach.

Or is that just how unittest works generally?

Either way, looks nice.

orlp

nah, this is not how unittest usually works :P

@isaacg about the license - you use MIT, but I prefer the zlib license

do you want to continue using MIT, or switch to zlib?

zlib allows anything, except claiming a modified version is the original

isaacg

I'll read them over.

orlp

I don't like about MIT that programs have to produce / give the license if they use parts of the code

isaacg

7:48 PM

Switching to zlib seems fine to me. If someone wants to use Pyth for something, I don't really care how.

@orlp

orlp

ok cool

@isaacg gist.githubusercontent.com/orlp/cbf427c41448b0432979/raw/…

that would be the license

isaacg

Looks good.

orlp

8:21 PM

@isaacg looks like pyth is already taken on pypi

pyth-lang?

isaacg

Oh, right, I remember seeing that.

pyth-lang is fine.

orlp

@isaacg hmm, should I also make the package name (the thing you import) pyth_lang to prevent collision?

I don't want to change the command though, I want pyth as the command

isaacg

I guess so.

Yeah, pyth as the command seems good.

orlp

8:39 PM

@isaacg could you do this?

git clone https://github.com/orlp/pyth

python3 -m unittest

isaacg

Sure.

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK

orlp

sorry

did you cd into pyth first?

isaacg

Yes

orlp

python3 -m unittest discover

isaacg

Same output

orlp

8:41 PM

python3 -m unittest pyth.test

wait

fuckkkkkkkk

isaacg

AttributeError: 'module' object has no attribute 'test'

orlp

git clone https://github.com/orlp/pyth5

oh

hurr durr

Right, of course.

8:42 PM

I made you clone my fork ^^

so just clone the pyth5 followed by python3 -m unittest

isaacg

...............
----------------------------------------------------------------------
Ran 15 tests in 0.003s

OK

orlp

cool

here's the real magic

isaacg

I agree

orlp

python3 -m unittest pyth_lang.test.Add

isaacg

ImportError: No module named 'pyth'

orlp

8:44 PM

ofc

pyth_lang hurr durr

isaacg

....
----------------------------------------------------------------------
Ran 4 tests in 0.001s

OK

orlp

keep forgetting these name changes

isaacg

:Looks good.

orlp

ok, everything works then

isaacg

Sweet.

orlp

8:44 PM

pip3 install -e .

and then pyth

isaacg

I've got pyth hooked up to current pyth, so I'd rather not.

orlp

all right

you could edit setup.py

entry_points={"console_scripts": ["pyth=pyth_lang.pyth:cli"]},

isaacg

OK, I'll make it pyth5

orlp

to

entry_points={"console_scripts": ["pyth5=pyth_lang.pyth:cli"]},

(sorry, I keep running into stackexchange chat throttling)

isaacg

It didn't work

Oh, I forgot to sudo it

It works!

orlp

8:48 PM

pip3 uninstall pyth-lang

and then pyth5 should be gone again :)

isaacg

That didn't work

orlp

sudo?

isaacg

Can't uninstall 'pyth-lang'. No files were found to uninstall.

Even with sudo

orlp

that's odd

what if you try to uninstall pyth_lang?

isaacg

Same message, down to the -.

orlp

8:51 PM

pip3 uninstall .

otherwise I don't know

isaacg

No output, no effect on pyth5

Weird.

orlp

@isaacg I reproduced on my linux server, will try to debug

isaacg

Good luck.

orlp

9:57 PM

@isaacg well that was a weird trip

went to freenode.org#python

turns out you're not supposed to run pip as sudo

isaacg

10:14 PM

Oh

orlp

@isaacg it's going to have to be a manual deinstall - I've found a clean way to install/uninstall in the future

what OS do you run?

@isaacg you'll have to do something like this

sudo rm /usr/local/bin/pyth5

sudo rm /usr/local/lib/python3.2/dist-packages/pyth-lang.egg-link

sudo vim /usr/local/lib/python3.2/dist-packages/easy-install.pth

and remove the line with pyth-lang on it

in the future, use pip3 install --user -e . and add ~/.local/bin to your path

sorry, I wasn't aware of this

orlp

11:11 PM

@isaacg I'm changing the order in which the operators come

to !"#$%&'()*+,-/:;<=>?@[]^_`{|}~abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVW‌XYZ

(this is lexicographical order, except the symbols come first, and the lowercase letters come before uppercase)

isaacg

11:46 PM

I run ubuntu

No problem, I'm going to leave it installed anyways

We should just know how this works.

The operator order for the data.py stuff? or for the docs?

orlp

@isaacg pretty much everywhere

I do suggest you uninstall and reinstall properly

isaacg

OK, seems fine. Having a standard order seems like a good idea.

orlp

I will forget how sometime in the future

isaacg

OK

orlp

and then it'll be painful

isaacg

11:54 PM

For the reinstall, I just use pip without sudo?

orlp

yes

github.com/orlp/pyth5#installing-pyth

those are proper installation instructions

isaacg

For the third uninstall step, with easy-install.pth, none of the lines mention pyth-lang

import sys; sys.__plen = len(sys.path)
/home/isaac/prog/pyth5
import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p=getattr(sys,'__egginsert',0); sys.path[p:p]=new; sys.__egginsert = p+len(new)

orlp

the pyth5 one

should be removed

isaacg

OK]

Transcript for