0
This is just one bit that jumped out at me:
page_text = page_html.decode("utf8", "ignore")
page_text = converter.handle(page_text)
page_text = re.sub("[^a-zA-Z0-9_!]+[ ,.\?!]", "", page_text)
It seems you are converting as utf8, ignoring errors, and then forcing it into ascii with ...