you may have the most common problem: page may use
JavaScript
to add/update elements but
BeautifulSoup
/
lxml
,
requests
/
urllib
can't run
JS
. You may need
Selenium to control real web browser which can run
JS
. OR use (manually)
DevTools
in
Firefox
/
Chrome
(tab
Network
) to see if
JavaScript
reads data from some URL. And try to use this URL with
requests
.
JS
usually gets
JSON
which can be easy converted to Python dictionary (without
BS
). You can also check if page has (free)
API
for programmers. —
furas 22 secs ago