Comments (6)
aldnav
commented on August 15, 2024
I kind of saw similar tweet format too where links aren't in the tweet text because it belonged on a different element on the markup. Will try to hash out something later.
For getting only the first tweet you can do,
# your_app.py
print('Getting tweets for %s' % user)
print(list(get_tweets(user, pages=1))[0]) # convert gen to list first then access subscriptsGetting tweets for nikitavoloboev
Wrote about my workflow in making workflows https://www.alfredforum.com/topic/11274-my-workflow-in-making-workflows …from twitter-scraper.
nikitavoloboev
commented on August 15, 2024
Thank you @aldnav
from twitter-scraper.
nikitavoloboev
commented on August 15, 2024
Hey @aldnav
I tried running your example with this code:
from twitter_scraper import get_tweets
user = "nikitavoloboev"
print(list(get_tweets(user, pages=1))[0])But when ran, I get this error:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/pyquery/pyquery.py", line 95, in fromstring
result = getattr(etree, meth)(context)
File "src/lxml/etree.pyx", line 3230, in lxml.etree.fromstring (src/lxml/etree.c:81070)
File "src/lxml/parser.pxi", line 1871, in lxml.etree._parseMemoryDocument (src/lxml/etree.c:121250)
File "src/lxml/parser.pxi", line 1752, in lxml.etree._parseDoc (src/lxml/etree.c:119804)
File "src/lxml/parser.pxi", line 1066, in lxml.etree._BaseParser._parseUnicodeDoc (src/lxml/etree.c:113546)
File "src/lxml/parser.pxi", line 598, in lxml.etree._ParserContext._handleParseResultDoc (src/lxml/etree.c:107738)
File "src/lxml/parser.pxi", line 709, in lxml.etree._handleParseResult (src/lxml/etree.c:109447)
File "src/lxml/parser.pxi", line 638, in lxml.etree._raiseParseError (src/lxml/etree.c:108301)
File "<string>", line 2
lxml.etree.XMLSyntaxError: Start tag expected, '<' not found, line 2, column 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "run.py", line 5, in <module>
print(list(get_tweets(user, pages=1))[0])
File "/usr/local/lib/python3.7/site-packages/twitter_scraper.py", line 30, in get_tweets
yield from gen_tweets(pages)
File "/usr/local/lib/python3.7/site-packages/twitter_scraper.py", line 18, in gen_tweets
d = pq(r.json()['items_html'])
File "/usr/local/lib/python3.7/site-packages/pyquery/pyquery.py", line 255, in __init__
elements = fromstring(context, self.parser)
File "/usr/local/lib/python3.7/site-packages/pyquery/pyquery.py", line 99, in fromstring
result = getattr(lxml.html, meth)(context)
File "/usr/local/lib/python3.7/site-packages/lxml/html/__init__.py", line 876, in fromstring
doc = document_fromstring(html, parser=parser, base_url=base_url, **kw)
File "/usr/local/lib/python3.7/site-packages/lxml/html/__init__.py", line 765, in document_fromstring
"Document is empty")
lxml.etree.ParserError: Document is emptyfrom twitter-scraper.
aldnav
commented on August 15, 2024
Hello @nikitavoloboev . Have you pulled from the latest master?
python test.py nikitavoloboev
https://www.youtube.com/watch?v=IAXE8wAjSmA …from twitter-scraper.
nikitavoloboev
commented on August 15, 2024
Have you pulled from the latest master?
Not sure what you mean. I ran pip install twitter_scraper then put the code above in a file and tried to ran it.
pipenv is broken for me for some reason.
from twitter-scraper.
aldnav
commented on August 15, 2024
I see. I don't think the latest fixes are in the release yet. I'm not sure what's the best course of action here. If you may, you can opt to pull this repository to keep updated.
from twitter-scraper.
Related Issues (20)
- f'Oops! Either "{query}" does not exist or is private.' HOT 3
- UnboundLocalError: local variable 'html' referenced before assignment HOT 4
- DeprecationWarning: invalid escape sequence \g
- Getting historical tweet volume data with specific hashtags
- Not get Profile Info HOT 3
- Feature: Get User Following/Friend (and Followers)
- Documentation: Addressing shadow-bans HOT 3
- Tweet Links and Media Files HOT 2
- Doesn't scrap anything. HOT 8
- Does this work? HOT 10
- get_tweets in your example is not working HOT 1
- Twitter scrapy
- I am getting this issue : UnboundLocalError HOT 3
- json.decoder.JSONDecodeError: in get_trends() HOT 5
- json.decoder.JSONDecodeError: in get_tweets() HOT 7
- UnboundLocalError: local variable 'html' referenced before assignment
- setup.py fails
- pip3 install fails
- Enable running on Apify cloud platform
- does anyone got an 403 error ? HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from twitter-scraper.