@Makyen ... didn't I see a general bug or smth about 'caching requests to send to MS when MS is down'? I could've sworn i saw something on it but can't find the issue.
because I had some ideas on that. But it's... pain :P
@ThomasWard Yep, this issue in the MS repo. I should probably create an associated issue in the SD issue where it would be more appropriate to discuss SD specific issues with the actual caching, rather than what to do on MS with data that has an indicator that it's creation date should be offset from its transmission date/time. There's another issue I should create in SD's repo, so I'll see if I can get to creating both shortly.
@Makyen cool. I had some ideas about it, implement a MSQueue object that gets iterated over if it's got contents, basically a FIFO queue object (yes it exists! docs.python.org/3/library/queue.html#module-queue)
so basically MSRequestsQueue is just a Queue object :P
that gets stuff added to it if ms is down
i'll have to dig this weekend with my thoughts on it and prototyping but ERR: Busy today and tomorrow
@ThomasWard Yeah, that's largely along the same lines I've thought about, with the addition of adding a timestamp of when the item enters the queue and transmitting to MS the difference between when it entered the queue and when it's actually transmitted. Obviously, the queue also needs to be saved to disk/storage and restored upon reboot, in order for the queue not to be lost upon reboot/crash.
@tripleee That's strange. I wouldn't expect there to be any differences between the different pages on which it's displayed on the MS side. It's not a value which should get any processing when being displayed.
Playing around a bit, when adding a new API key, it is a value that might be changed slightly after being displayed, if you've previously been to the "new API key" page, then used the back button and re-clicked the link to that page. However, in that case, I'd expect the values to be substantially different, rather than partially similar.
@ThomasWard Yes, of course. However, it's not something which should be updated to disk with every attempt to transmit when MS is UP, as we don't want/need to be unnecessarily increasing the amount of writing we do to the filesystem, and don't need to save it when we can/will immediately transmit it. In other words, the queue should be bypassed when there's A) nothing in the queue and B) we immediately successfully transmit it to MS.
@Ollie IIRC, the mods there have said that red flags are OK, but I'm not certain of that at this point. I'd need to search to see if I can find something in chat about that.
@Makyen this is my thought on a prototype for the queue during "MS Is Down" and how when MS is back up we trigger things. Not sure this is thread safe though.
(MSRequest is a namedtuple that has a default timestamp added of "now" converted to epoch and trimming any fractional bits off)
something like that, we just have to figure the 'write to disk' bits so that any object we pull from the queue is properly stored on disk in a way that can be retrieved
just me thinking out loud :)
(this is a Python 3.9 environment but SHOULD work in 3.6 I think. I'll test)
@tripleee You're doing it right. The domain is being truncated to not include the . character and everything after it. It appears to be a bug in MS. Actually it's the re-emergence of an older bug. I assume the change in the Ruby version we're using has resulted in the work-around no longer working which we were using to allow . to be part of the name (and other fields in the MS API). cc: @thesecretmaster, as you requested pings regarding things that broke (admittedly only about JS).
It ends up trying to do: ... WHERE `spam_domains`.`domain` = 'israelbigmarket' ... among other database accesses, which, of course, don't match the domain you're looking for.
np. I was vaguely remembering having issues like this for similar things. We had "fixed" them, but it's broken again, now.
@tripleee For something like this, it's better to construct a baseline query which includes variables for the post ID(s), and then send the post ID(s) as variables.
@tripleee As you've determined, yes there is. I generally play around with GraphiQL to determine the query I desire (and see what the maximum number of items is based on complexity).
@Ollie An invalid pattern was provided, please check your command.
@Ollie That pattern looks like it's already caught by Potentially bad keyword in body and Potentially bad keyword in answer Append -force to the command if you really want to add this pattern.
