So when I run get_twitter_api() within itself, it works with no problem at all. But when I import it and run it from the other file I get a FileNotFoundError ...
I think the easiest thing to do is just to reuse dirname()
So you can call
os.path.dirname(os.path.dirname( __file__ ))
if you file is at /Users/hobbes3/Sites/mysite/templates/method.py
This will return "/Users/hobbes3/Sites/mysite"
This technique worked, although it looks wonky as hell
Various fixes to un-fuck handling of file paths when importing modules within each other
The use of manually building file paths was causing various FileNotFoundError problems. I decided to instead use the os.path module to let the paths build themselves using the location of __file__
So, now I need to think about & juggle 2 things. Input appreciated. Here's a bit of code for context:
_input_file_name = f'user_followers_ids {screen_name} {datetime.date.today()}.json'
_followers_ids = []
for user_id in tweepy.Cursor(_api.followers_ids, screen_name=screen_name).items():
_followers_ids.append(user_id)
print(len(_followers_ids))
print(_followers_ids)
That of course doesn't do anything beyond printing to console after querying the API
The two things are: 1. I need to open the JSON file for writing the IDs to it 2. I need to not keep the file open needlessly while waiting for the API throttling to replenish
So I need to introduce a try/except to watch for rate limit errors, I just can't really think what order the instructions should go
So I need to keep the cursor going (I think) so I don't lose where it last stopped (assuming there are more than 5000 followers to go through on the account)
If I call .page() instead of .items() on the cursor, it gives me a full page of IDs as a list, instead of each ID one at a time. I suppose once I have a page, I could open the file, then iterate each item in the list and write it, then close the file...
And try/catch as I attempt to get another page, if I get a rate limit error, just sleep the cursor for a while...
Does that make sense?
Actually why do I even need a JSON file. A newline separated TXT file will do perfectly fine for storing this, since it's all just a single ID number per user.
For anyone who stumbles upon this on Google, tweepy 3.2+ has additional parameters for the tweepy.api class, in particular:
wait_on_rate_limit – Whether or not to automatically wait for rate limits to replenish
wait_on_rate_limit_notify – Whether or not to print a notification when Tweepy is wa...
Oh look the rate limit wait is built in to tweepy. Guess that makes the whole try/catch moot
Just need to set my computer not to go to sleep if I'm going to run a long query so it doesn't drop the connection
_output_file_name = f'user_followers_ids {screen_name} {datetime.date.today()}.txt'
_output_file_path = os.path.join(PATH_TO_APP_DATA, _output_file_name)
for page in tweepy.Cursor(_api.followers_ids, screen_name=screen_name).pages():
_output_file: TextIO
with open(_output_file_path, 'w') as _output_file:
for item in page:
_output_file.write(f'{item}\n')
# For testing
with open(_output_file_path, 'r') as _output_file:
num_lines: int = 0
for line in _output_file.read().splitlines():
Add functionality to query all of an account's followers' user IDs and save them locally
Note that is has not been tested for an account with large (tens of thousands or more) amount of followers, and will need to be tested or may take a long time, depending on the use case.
[Phrancis/TwitterTools]Phrancis pushed commit 8c925807 to master: Fixed order of operations where subsequent pages would overwrite previous pages, due to re-opening the file for writing
@IvenBach Yeah, did that last night, today's been a caffeine roller coaster. I think I'll save the "hydrating user accounts" part for tomorrow, and just Netflix-chill for a little bit before bed.
Wu Assassins on there is quite good; not near as violent as the title suggests, though a little violent; cook-becomes-warrior-but-doesn't-want-to-kill; some martial arts, some detective stuff. Recommend it.
Also tested with a much larger account (100K+ followers) and found no problems as long as the connection from the user computer to the API remains open, which it should unless the computer goes to sleep or crashes.
Currently running rounds with 5 AIs and 10 games in each pair-up. This might take a while...
AlphaBeta algorithm is slow...
using depths of 3, 4, 5 and 6
the fifth AI is using another approach
Hmm... no wonder it's slow... something's not right. One of the AIs keeps trying to make a move at the same position over and over again, which apparently isn't allowed.
Ah, the AI is not analyzing the same game as is actually being played...
@Marc-Andre With the computer? No, not really. Suddenly it just shuts down within a second. I have considered overheating of the CPU, but I am not sure it is this.
> The computer has been restarted without having to shut down properly first. This error can occur if the system stops responding, if it crashes or if the power supply unexpectedly breaks down.
@SimonForsberg “this student would be better off working as a janitor” said one teachers review. When asked for comment we received no reply. Back to you in the news room FooBar.
@Phrancis Yeah, as @Marc-Andre says, potentially all students and all teachers in Stockholm. At least that was what was exposed, how much data that has actually been accessed? Who knows.
@bruglesco When my friend and I took Trig our teacher announced "Today we get to work with logs" at which point we both announced "It's better than bad it's good" which left everyone completely confused, including the professor.
I think the easiest thing to do is just to reuse dirname()
So you can call
os.path.dirname(os.path.dirname( __file__ ))
if you file is at /Users/hobbes3/Sites/mysite/templates/method.py
This will return "/Users/hobbes3/Sites/mysite"
