@MarkyMarkov Yeah, I'm thinking this needs more work in Beep Boop Maggot before being released in here. He's not going to learn anything good from Marky anyway.
I wonder if you'll get any better than this. You seem to be flat-lining between 10 and 12 perplexity, and your learning rate is dropping pretty low now.