Axtell

« first day (265 days earlier) ← previous day next day → last day (369 days later) »

Downgoat

00:04

@Mego does SQLAlchemy batch together queries? E.g. if I did [model.do_query_and_get_result() for model in lots_of_models] would that make like 100s of DB queries

1 hour later…

Downgoat

01:26

Async iterators are so beautiful :')

    async *[Symbol.iterator]() {
        let data;

        do {
            data = await this.nextPage();
            yield* data;
        } while(this.areMore);
    }

quartata

not to brag or anything but python has had those since 3.5

@Downgoat I don't think there'd really be any way for it to know that it was part of a batch like that, so yeah you're probably going to get n+1 there

Downgoat

@quartata opls python async-ing is yucky

quartata

no really it's actually good now

Downgoat

@quartata it could queue requests and when the first one is lazily obtained it'll obtain all values

quartata

it sucked in 3.4 I'll give you that

@Downgoat yeah but it'd have to be pretty clever to be able to leverage the fact that it's the same table

I know AR isn't that clever at least. It's dumb as bricks.

Downgoat

01:34

@quartata idk you still need to talk to asyncio API directly quite a bit not enough syntax sugar imo

@quartata rip ok that is bad

I guess I'll just ask mego to see if he knows how to make better

quartata

Ah, but that means you can write your own event loops

Downgoat

why do I want to do that

quartata

Coroutines and the event loop being separate is a very good thing

@Downgoat so that you can, say, use libuv

Or use curio

or do whatever you want

Downgoat

@quartata why do I want to use explicitly get libuv to back my event loop when language can automatically do

quartata

Conceptually coroutines really aren't tied to the event loop so why should it be designed that way. The event loop is just something that wakes up certain coroutines

@Downgoat depends on your usecase

Downgoat

01:37

programming languages should make decisions on what they do— not leave it so open ended to the extent that I need to manually plug in libuv to event loop to coroutine

well at least in a high-level language like python

quartata

Well Python did make those decisions. But you can change them.

One of the reasons I appreciate the asyncio API being a bit more explicit is that it make it very easy to throw threads into the mix

Event loops become objects, and you can have different ones running in different threads

And it's good it doesn't hide that from you because you can't mix them

When working with another library, you provide the loop

And the nature of the loop allows for an almost declarative syntax: I set up all the tasks, say what's going to happen, before the loop starts

I like that a lot

Downgoat

is the async/await syntax python 3.6 or something else

quartata

Once you're in the event loop, in a coroutine, then it's hidden from you more. Pretty much the only three functions I call are ensure_future, gather and wait. You're right that ensure_future could be a keyword

@Downgoat yes

Downgoat

because i remember there was a reason we went with celery over asyncio i forgot what it was

quartata

Celery is very different....

Downgoat

01:44

i mean what we use it for we could mimic with event loop + smaller thread pool

quartata

Celery is for running synchronous tasks in a separate process, maybe even on a different server

Since Flask isn't really async it's tough to do these things in the web server process. You could have a separate thread, but then the GIL might be a problem

Downgoat

@quartata different other the hood but still solving similar problem

quartata

There are situations where sync and async can mix, but this is not one of them

Flask doesn't make it easy

thats why I like Tornado or Japronto better

Even though they're lower level

Downgoat

Should I say "Anyone" or "Anybody" in axtell. I mean I don't want to discriminate against protists

quartata

Anyone

It's unfair of me to lay this entirely on the door of Flask when really it's the fault of WSGI but

1 hour later…

Anonymous

03:04

@Downgoat no

Anonymous

We use Celery instead of async because of GIL and because of scaling. We won't need more than one celery server even when we grow large, and Celery supports multiple clients over internet very easily

quartata

so yeah pretty much what I figured, then

Downgoat

ok so just learning about GIL: Why even have threads over event loop if they wont execute at same time

quartata

Couple reasons. Working with synchronous APIs is the main one

Another thing to keep in mind is that the GIL is an implementation detail. Not all Python interpreters have it

There's always a usecase for separate processes and threads, for situations where you need preemption. Really then it comes down to what platform (multiprocessing is a nightmare on Windows) and how fast you need to spin them up (threads are faster than processes to create obviously)

Downgoat

03:39

>_< ok i have no idea how to write the controller for the feed

@Mego ready to merge: github.com/Axtell/Axtell/pull/205

14 hours later…

Anonymous

17:33

Installing MySQL 8.0 on dev server for testing - if all goes well, I'll do the upgrade tonight on prod

Anonymous

19:40

Pro tip: don't lose mysql root password when doing upgrades

Anonymous