@ArtOfCode and @quartata you are both getting way ahead of what Helios will do initially. It is designed to pull out the things that can be shared between instances - blacklists and notifications initially. Both can be accesssed without a key. Updating without a key will not be possible
@quartata to be fair, that does weaken my reasoning slightly. A persistence layer may still be useful to cover the unauthorized case, but it'd just need to merge the Helios list into a pickle with any additions made.
@quartata Very brief work flow: Smokey starts (or an update command is issued) -> Query Helios end point -> Get JSON data back -> This data is put into the exact same formats as the existing blacklist/watchlists/notifications as currently exist. Smokey uses those files to do it's work...just like today
To update: Send data to Helios end point -> helios magic to save it -> Add record to local file for caching. This saves a repull after you update and will just "be faster".
Those local files are completely overwritten when you issues an update command to pull the new data or when Smokey comes out of standby/start up
If, for whatever reason, Helios isn't available then don't update those files with nothing
From a Smokey perspective, it is still using the same files
The notifications become global though, because those aren't currently saved. We lose the git complications we currently have
We have two options for persistence: pickles and Helios. If someone desn't have a Helios key, it should do the former. If they do, it should do the latter and only the latter: pull from Helios on restart and store in menory like normal
Smokey starts -> Query Helios for data. No Helios available? Use the last saved notifications.txt, blacklist-websites.txt, etc files. Helios available? Replace those files with the next data
It's not supposed to be about handling. It's "how does it behave if wrtten like this"
sorry give me a second
OK. Scenario 2, pickles (the current method) Smokey starts. Someone registers a notification, it's put in memory and the whole structure is flushed to a pickle. Smokey restarts, reads the pickle once, the notification is there in memory.
On save to notifications -> Send notification to Helios & write to the local file. No Helios? Well...world is burning. Write to text file. Important note here It does need a way to handle this unwritten record to Helios when it is available again.
Scenario 3, just Helios with no local files: Smokey starts. Someone registers a notification, it's put in memory. If a Helios key is available, it is also written to a Helios endpoint. Smokey restarts, reads the notifications from Helios once. If we had a key, the new notifiation is there in memory. If we didn't, it is not (just the old ones)
Ah. A light bulb has flickered. Let's make sure I'm understanding a bit. Dev (let's call them NoobSmoke - yes, a mortal kombat reference) clones smokey and wants to develop a thing. NoobSmoke sets a notification. No write access? So they just write to the local file. NoobSmoke goes to bed and shuts down. Next morning they restart. On query of Helios, their local file is overwritten.
Assuming that is correct 1.) Is it a big deal that they need to recreate a notification for their local instance? 2.) Would a command line parameter to not update from Helios be a viable work around? So, on that second start up, if the parameter is passed Smokey just skips the Helios check all together? This would allow them to still issue the update command if they want to
@Andy that's a possibility, but I feel like it can't hurt to also account for the "world on fire" possibility which would presumably occur with an instance that would be expecting to have access to Helios
Smokey starts. Someone registers a notification, it's put in memory. If a Helios key is available, it is written to Helios. If it isn't (or Helios is down inexplicably), it's written to a new pickle instead. Smokey restarts, reads the notifications from Helios and read in the pickle if it's there. If we have write access, write the contents of the pickle to Helios. Either way, smoosh the contents of the pickle and what we got from Helios together and keep it in memory
This also has the benefit of automagically converting our existing pickles into Helios records
If we keep the pickles backwards compatible
When Smokey starts, it'll see the pickle there, read it in and then if it has a Helios key write it all to Helios
The good news is that it doesn't sound that difficult either
My plan is to start a Smokey fork in the next couple days. The problem is december is a busy money for me. I may start but the majority probably won't really get done until the new year
