« first day (317 days earlier)      last day (317 days later) » 

Anonymous
3:14 AM
I figured out the problem with periodic tasks and now I feel foolish
 
3:31 AM
@Mego I don't think deploy failed but uwsgi is shut down rn O_o
 
Anonymous
That's odd. Also deploy didn't start?
 
I just re-ran deploy
 
Anonymous
Odd
 
Anonymous
I don't think I broke anything
 
Sep 18 03:37:44 ppcg-v2 uwsgi[23247]: [uWSGI] getting INI configuration from ppcg-v2.ini
Sep 18 03:37:44 ppcg-v2 uwsgi[23247]: *** Starting uWSGI 2.0.17 (64bit) on [Tue Sep 18 03:37:44 2018] ***
Sep 18 03:37:44 ppcg-v2 uwsgi[23247]: compiled with version: 7.2.0 on 24 March 2018 21:10:56
Sep 18 03:37:44 ppcg-v2 uwsgi[23247]: os: Linux-4.13.0-46-generic #51-Ubuntu SMP Tue Jun 12 12:36:29 UTC 2018
Sep 18 03:37:44 ppcg-v2 uwsgi[23247]: nodename: ppcg-v2
Sep 18 03:37:44 ppcg-v2 uwsgi[23247]: machine: x86_64
Sep 18 03:37:44 ppcg-v2 uwsgi[23247]: *** Operational MODE: preforking ***
oh stuck on this again
 
Anonymous
3:48 AM
I think I fixed it
 
Anonymous
I forgot the difference between kill and pkill :P
 
Anonymous
We should be good once the deploy finishes
 
Anonymous
4:06 AM
I don't know wtf is going on anymore
 
Anonymous
Oh thank goat, it's alive
 
Anonymous
And search probably works too
 
Anonymous
I'm the greatest, even when I'm panicking and have no idea what I'm doing
 
Anonymous
mic drop
 
Oh I pray to goats celery doesn’t time out
 
Anonymous
4:42 AM
We're not dead yet so I count this as a success
 
i have no idea why tf the task has been running for 30 minutes
 
Anonymous
Which one?
 
(venv) travis-deploy@ppcg-v2:/var/www/ppcg-v2$ celery inspect active -A celery_server
-> w1@ppcg-v2: OK
    * {'id': 'd128e213-c71a-48ac-82c7-3d8526f00cc5', 'name': 'app.tasks.search.reindex_database', 'args': '()', 'kwargs': '{}', 'type': 'app.tasks.search.reindex_database', 'hostname': 'w1@ppcg-v2', 'time_start': 1537244276.9397058, 'acknowledged': True, 'delivery_info': {'exchange': '', 'routing_key': 'celery', 'priority': 0, 'redelivered': None}, 'worker_pid': 25174}
(venv) travis-deploy@ppcg-v2:/var/www/ppcg-v2$ date -d @1537244276
Tue Sep 18 04:17:56 UTC 2018
(venv) travis-deploy@ppcg-v2:/var/www/ppcg-v2$ date
Tue Sep 18 04:45:23 UTC 2018
 
Anonymous
Oh dear
 
in the logs i can see the task has completed
so I have no idea why celery keeps it in the queue
 
Anonymous
4:51 AM
I'll purge it
 
Not to interrupt your discussion, but I'm here to point out that lately I've had issues with login (that is, I had to try a few times before it worked - with errrors ranging from Bad request to Blah blah... Something related to OAuth ... Blah Blah. Also the Challenges page doesn't seem to load on the production server
 
@Mr.Xcoder yeah we're debugging that right now
 
Anonymous
TBF nothing is loading on the server right now (damnit)
 
all related to the same issues of celery being weird
celery is same version for me locally and on the prod serve
 
Anonymous
I think it's because of this
 
Anonymous
4:52 AM
@Downgoat What's your py3 version?
 
3.7
 
Anonymous
prod is 3.6.3 - that might make a difference
 
@Mego though that definetly returns as I can see in the logs...
hmmm
brb doing debug
issue doesn't occur when I run from CLI
 
Anonymous
I'm going to force it to start over clean
 
Anonymous
I think we're ok again?
 
5:06 AM
ok weird it works when i do from CLI
make 0 sense
 
5:33 AM
ok and it is working so far
 
5:56 AM
@Mego I think we turned off beat and that's why it's not breaking
 
 
12 hours later…
Anonymous
5:45 PM
@Downgoat No, I restarted beat
 
Anonymous
I think
 

« first day (317 days earlier)      last day (317 days later) »