API quota confusion

SmokeDetector

Feb 16, 2016 19:24

API quota rolled over with 1365 requests remaining.

stackoverflow: 1516
superuser: 566
askubuntu: 493
physics: 278
math: 265
drupal: 237
unix: 211
gis: 205
english: 181
magento: 167
electronics: 166
salesforce: 159
apple: 132
dba: 123
wordpress: 118
scifi: 118
money: 117
academia: 116
meta: 106
gaming: 102
mathoverflow.net: 100
mathematica: 93
travel: 91
sharepoint: 89
ell: 86
tex: 85
webapps: 79
worldbuilding: 75
android: 73
security: 72
diy: 69
stats: 69
ru.stackoverflow: 68
chemistry: 66
movies: 64
programmers: 63
codereview: 60
gamedev: 54
cs: 51

SmokeDetector

Feb 16, 2016 19:38

[ SmokeDetector ] Bad keyword in title: Is there any Anti-diuretic or vasopressin supplements by Dr. James on cogsci.stackexchange.com

Sally

sd 2f

SmokeDetector

[ SmokeDetector ] Bad keyword in answer: File name too long when copying to a directory with shorter file path by pkmarko sarnych on superuser.com

Ferrybig

@SmokeDetector k

Sally

Feb 16, 2016 20:14

in Worldbuilders' General Chat, 13 mins ago, by ElectionBot

Unofficial results: Winners are Tim B, HDE 226868, Monica Cellio, and Serban Tanasa.

@SmokeDetector f

3/4 are already moderating something. Not much new blood. 4th site for Monica, 3rd for HDE.

Sally

Feb 16, 2016 20:57

I get "This method requires an access_token" when calling /answers/{ids}... what's up with that?

Andy

What's in your filter?

Some of those require the token

I think it's the accepted, downvoted, and upvoted fields

Sally

Yeah, I just now realized that I marked "accepted" instead of "is_accepted"...

SmokeDetector

Feb 16, 2016 21:36

[ SmokeDetector ] Offensive body detected: MSSQL SELECT replace with second table by Thomas Dutoit on stackoverflow.com

Ferrybig

@SmokeDetector Edited

SmokeDetector

API quota rolled over with 4583 requests remaining.

stackoverflow: 158
askubuntu: 41
math: 39
superuser: 35
gis: 27
physics: 26
english: 23
scifi: 22
electronics: 20
mathematica: 20
unix: 19
drupal: 18
programmers: 16
academia: 14
dba: 14
sharepoint: 14
ell: 14
codereview: 13
codegolf: 12
salesforce: 12
security: 12
mathoverflow.net: 12
magento: 12
diy: 11
travel: 10
worldbuilding: 10
movies: 9
wordpress: 9
money: 9
aviation: 9
judaism: 8
webapps: 8
gaming: 8
rpg: 8
graphicdesign: 7
engineering: 7
stats: 7
serverfault: 7
tex: 7
writers: 7
mechanics: 7

API quota rolled over with 4581 requests remaining.

movies: 1
rpg: 1
stackoverflow: 1

API quota rolled over with 4566 requests remaining.

stackoverflow: 3
superuser: 2
math: 1
mathoverflow.net: 1
windowsphone: 1
skeptics: 1
magento: 1
chemistry: 1
money: 1
philosophy: 1
programmers: 1
diy: 1
hsm: 1

Ferrybig

Why so many rollovers?

Sally

Feb 16, 2016 21:52

!!/errorlogs 100

SmokeDetector

    self._br.login_site(self.host)

  File "/media/sda2/SmokeDetector/ChatExchange/chatexchange/browser.py", line 162, in login_site
    'openid_identifier': 'https://openid.stackexchange.com/'

  File "/media/sda2/SmokeDetector/ChatExchange/chatexchange/browser.py", line 178, in _se_openid_login_with_fkey
    fkey = fkey_input['value']

  File "/usr/local/lib/python2.7/dist-packages/bs4/element.py", line 958, in __getitem__
    return self.attrs[key]


KeyError: 'value'
2016-02-14 19:16:20.368604 UTC

Sally

Nothing relevant. The API must be returning wrong quota values at times.

!!/apiquota

SmokeDetector

@Sally The current API quota remaining is 4557.

Sally

This can't even be right. It rolled over 2 hours ago.

SmokeDetector

API quota rolled over with 4554 requests remaining.

stackoverflow: 2
math: 1
tex: 1
ux: 1
sqa: 1
askubuntu: 1
japanese: 1
serverfault: 1
wordpress: 1
meta: 1
sharepoint: 1
movies: 1
arduino: 1

Sally

Feb 16, 2016 21:54

!!/apiquota

SmokeDetector

@Sally The current API quota remaining is 4550.

Sally

Did they decide to reduce quota from 10000 to 5000?

SmokeDetector

API quota rolled over with 4535 requests remaining.

stackoverflow: 2
math: 1
gardening: 1
superuser: 1
academia: 1
law: 1
electronics: 1
travel: 1
askubuntu: 1
gaming: 1
mathematica: 1
wordpress: 1
codereview: 1
rpg: 1
ebooks: 1
dba: 1
movies: 1
unix: 1
english: 1
worldbuilding: 1

Ferrybig

Should we restart smokey to see if it solves the problem?

SmokeDetector

API quota rolled over with 4520 requests remaining.

stackoverflow: 2
mathoverflow.net: 1
superuser: 1
english: 1
physics: 1
dba: 1
aviation: 1
ru.stackoverflow: 1
money: 1
poker: 1
programmers: 1
apple: 1
history: 1
judaism: 1

Sally

Feb 16, 2016 21:58

!!/reboot

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev 734ca7e (Normal Human: fix word boundaries) (running on Raspberry Pi)

Restart: API quota is 4518.

Sally

I'm also considering increasing API queues for math and physics as a general quota-saving measure, in case there is indeed a reduction. Though it won't help much if we have to run for 22 hours with half the usual quota.

SmokeDetector

API quota rolled over with 4517 requests remaining.

biology: 1
electronics: 1
mechanics: 1
bitcoin: 1

Sally

To be better informed, I've added the renewed quota to the "rolled over" message. We'll at least see what discrepancy caused the message to appear.

And bumped Physics to 5, Math to 15: two sites that consume a lot of API requests without much spam.

!!/pull

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev b7109b9 (Normal Human: report renewed quota at rollover) (running on Raspberry Pi)

Restart: API quota is 4500.

API quota rolled over with 4494 requests remaining. Current quota: 9050.

stackoverflow: 5
scifi: 2
math: 1
cs: 1
ell: 1
english: 1
physics: 1
networkengineering: 1
sqa: 1
askubuntu: 1
blender: 1
mathematica: 1
magento: 1
sharepoint: 1
travel: 1
cooking: 1
german: 1
pm: 1
dba: 1

API quota rolled over with 4489 requests remaining. Current quota: 9049.

math: 1
superuser: 1
anime: 1
stackoverflow: 1
travel: 1
philosophy: 1

Sally

Feb 16, 2016 22:07

@SmokeDetector Aha! The server occasionally reports a lot less than the actual number. 9050 sounds a lot more like the true story.

So I'm not so worried about running out, although maybe we should stop flooding chat with these.

SmokeDetector

API quota rolled over with 4483 requests remaining. Current quota: 9048.

superuser: 1
english: 1
softwarerecs: 1
stackoverflow: 1
stats: 1
worldbuilding: 1
diy: 1

API quota rolled over with 4480 requests remaining. Current quota: 9047.

Sally

@hichris123 Any idea about this weird quota behavior?

SmokeDetector

mathematica: 1
chemistry: 1
stackoverflow: 1
crypto: 1

Sally

It's as if we have two different ones now, from different types of API requests.

Killing the reports is easy enough, but it'd be nice to find the cause first...

SmokeDetector

API quota rolled over with 4469 requests remaining. Current quota: 9046.

math: 1
raspberrypi: 1
physics: 1
electronics: 1
askubuntu: 1
stackoverflow: 1
hermeneutics: 1
writers: 1
cstheory: 1
computergraphics: 1
apple: 1
unix: 1

API quota rolled over with 4455 requests remaining. Current quota: 9045.

stackoverflow: 3
superuser: 2
academia: 1
stats: 1
electronics: 1
scifi: 1
codegolf: 1
salesforce: 1
wordpress: 1
dba: 1
unix: 1
italian: 1

Sally

Feb 16, 2016 22:14

Alright, for now raising the required difference to 5000 from 1000 will fix it. The issue remains whether the lower quota will bite us.

!!/pull

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev 005ea7f (Normal Human: raise criterion for rollover to 5000) (running on Raspberry Pi)

Restart: API quota is 4448.

Sally

Let's see.. while the "current" was reporting 9050, 9049, 9048, 9047, 9046, 9045, the smaller numbers (from the preceding run?) were 4494, 4489, 4483, 4480, 4469, 4455.

!!/status

@Sally Running since 22:25:56 UTC (0 minutes)

SmokeDetector

@Sally Running since 22:14:43 UTC (11 minutes)

Sally

Restart: API quota is 9999.

Started a local copy just in case.

And to see how its API quota will measure up.

We are taking down the Stack Overflow secondary SQL servers for hardware upgrades now. There should be no user impact.

Tweeted by Nick_Craver on February 16, 2016 at 8:31 PM

Possible cause?

!!/apiquota

@Sally The current API quota remaining is 9956.

SmokeDetector

Feb 16, 2016 22:37

@Sally The current API quota remaining is 9028.

Sally

@Sally ignore

@Sally That message is not a report.

SmokeDetector

[ SmokeDetector ] All-caps body: What's wrong with this sentence? I know there is something wrong by Grover on english.stackexchange.com

Sally

[ SmokeDetector ] All-caps body: What's wrong with this sentence? I know there is something wrong by Grover on english.stackexchange.com

Stopped to avoid duplicate reports.

!!/apiquota

SmokeDetector

@Sally The current API quota remaining is 4286.

hichris123

@Sally weird. Maybe because of the server-switching. shrug

Sally

Feb 16, 2016 22:50

Restart: API quota is 9911.

It keeps happening, as you can see: 4286 is definitely wrong.

I'll try logging (locally) the quota on every response to see what's up.

Locally the numbers decrease as expected by 1 with every request. Usually

9877
no answers
got answer from owner with name Taejang
got answer from owner with name KorvinStarmast
9988

The later quote value is way higher here.

hichris123

That's... really weird.

I've never seen anything like it.

Sally

Restart: API quota is 9987.

Yes, now I see the same thing locally: jumping between 99xx and 98xx in response.

Sally

Feb 16, 2016 23:13

Restart: API quota is 9958.

Props to Alastair for writing a thorough answer to an old FAQ (Python encode/decode troubles): stackoverflow.com/a/35444608

Something rather relevant to SD

!!/apiquota

SmokeDetector

@Sally The current API quota remaining is 8936.

Sally

@Sally The current API quota remaining is 9934.

Sally

Feb 16, 2016 23:35

[ SmokeDetector ] Email in body: arglin A and other optimization problem by PAUL on math.stackexchange.com

SmokeDetector

[ SmokeDetector ] Email in body: arglin A and other optimization problem by PAUL on math.stackexchange.com

Sally

Was email in body excluded from Tavern?

hichris123

@Sally Hmm, maybe it's two database fighting with each other?

@Sally nope.

Sally

Maybe it should be, then? One of weaker signals.

I gave up on trying to understand the quota thing.

!!/apiquota

SmokeDetector

@Sally The current API quota remaining is 8822.

hichris123

Feb 16, 2016 23:40

Smokey seems to be fine now...

Charcoal HQ

Participants