tommeagher / heroku_ebooks Goto Github PK

Hello! I created a bot following the steps and it's working flawlessly, but the 99,9% of the time it tweets nonsense. I think it's because the tweets I'm using are in spanish, and the code is intended for text in english and its grammar. I want to adapt my bot to my needs. I know about programming, but not enough to know what files I need to retouch in this case. What should I change in my side of the code to make my bot full adapted to spanish?

Hello! When trying to use Twitter source accounts with at least 3600K Tweets in local_settings.py I receive the following error:

1713 tweets found in JoseCanseco
Traceback (most recent call last):
File "ebooks.py", line 199, in
ebook_status = mine.generate_sentence()
File "/app/markov.py", line 55, in generate_sentence
restup = (res[-2], res[-1])
IndexError: list index out of range

The JoseCanseco Twitter account specified in the stack trace has over 11K tweets. Any suggestions on how to proceed?

It looks like the script sometimes generates multiline output but only outputs the final line. Is this expected behavior or a bug?

Obviously the more tweets to work from, the better, but this does seem like a relatively simple bug to fix.

error read as follows, despite closely following directions (and as far as I can tell, adding the buildpack correctly through heroku):

remote: Building source:
remote:
remote: -----> Failed to detect set buildpack https://github.com/bear/python-twitter
remote: More info: https://devcenter.heroku.com/articles/buildpacks#detection-failure
remote:
remote: ! Push failed
remote: Verifying deploy...
remote:
remote: ! Push rejected to
! [remote rejected] master -> master (pre-receive hook declined)

any help (even just pointing me in the right direction to sort it out on my own) would be much appreciated, I'm a bit of a novice here.

Hi. I was able to get the bot working once and then after fixing something custom for the theme of my ebooks account, I have either been getting the "x, No, sorry, not this time." error or this Index error, I do see the bot grabbing tweets though. Is this related to the low-tweet count?

Traceback (most recent call last):
  File "ebooks.py", line 203, in <module>
    ebook_status = mine.generate_sentence()
  File "/app/markov.py", line 58, in generate_sentence
    restup = (res[-2], res[-1])
IndexError: list index out of range

1394 tweets found in boredelonmusk
2360 tweets found in tesla
4604 tweets found in spacex
5405 tweets found in elonmusk
5657 tweets found in dogecoin
6867 tweets found in Dogebrarian

After pushing the files to the master app on Heroku and running "worker", I get these two errors:

Running worker on ⬢ secret-taiga-39619... up, run.2390 Traceback (most recent call last): File "ebooks.py", line 7, in <module> from local_settings import * File "/app/local_settings.py", line 12 SOURCE_ACCOUNTS = [“frontend_bot”] ^ SyntaxError: invalid syntax

I have python 3.7, the latest version of git and everything. Working on windows 10. I try running the twittereater in git bash, and I get this:

Traceback (most recent call last): 
 File "twittereater.py", line 8, in <module>
    next(reader) #get rid of the twitter header
  File "C:\Users\micha\AppData\Local\Programs\Python\Python37-32\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x8f in position 775: character maps to <undefined>

Furthermore, I have looked on other forums for resolutions, and entering encoding="utf8" on line 11 of twittereater.py, but this has resulted in nothing but creating a corrupt looking python file which resembles local_settings with error artifacts everywhere, and wiping the CSV file I was trying to make in to a corpus totally clean.

I am new at this all, so I hope I explained the issue clearly. I have been successful in getting a bot working from these files and having the app sweep my existing recent tweets, however I feel the fun will compound if I can add my twitter archive (stretching back to 2011) to the mix.

Thanks!

i'm running into this problem while doing "heroku run worker" in terminal I am not sure what to do about this.

sorry, i have no idea of what i have to do with this
Running worker on ⬢ lit-bayou-19362... up, run.6460 (Free)
Traceback (most recent call last):
File "ebooks.py", line 7, in
from local_settings import *
File "/app/local_settings.py", line 11
SyntaxError: Non-ASCII character '\xe2' in file /app/local_settings.py on line 11, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

Just for short tweets, as horse_ebooks does from time to time.

https://twitter.com/Horse_ebooks/status/366640345049604097

For some reason, it's saying that I have an invalid character in the identifier even though I haven't edited anything in ebooks.py and there are no invalid characters?

So you get messy tweets like this one. I think instead of stripping newlines, all consecutive "runs" of whitespace (newlines, tabs, and spaces) should be collapsed to single spaces. Then the original meaning of the tweets is preserved and passed into the Markov generator, but spurious whitespace is avoided.

This can be achieved with something like

tweet.text = re.sub('\s+', ' ', tweet.text)

I've made it an issue rather than jut doing it as a pull req since it might be worth discussing first.

Hey there, I'm running into the issue below. I'm not familiar with the language so don't understand how to fix your syntax. Can you please advise?

@tommeagher

$ heroku run worker
Running worker on ⬢ guarded-brushlands-52122... up, run.7914 (Free)
File "ebooks.py", line 69
print ">>> Generating from {0}".format(file)
^
SyntaxError: invalid syntax

Whenever I type "heroku run worker" everything works fine yet it always says "TOO SIMILAR: ". It happens 100% of the time. I have created other bots using this exact code and it works fine. I have no idea what is different. The account being posted to is completely empty

UPDATE: it fixed itself. no idea what happened

(caveat, I am not super familiar with coding jargon)

I get this error every few times I run the heroku run worker command:

Traceback (most recent call last):
File "ebooks.py", line 77, in
handle_stats = api.GetUser(screen_name=user)
File "/app/.heroku/python/lib/python2.7/site-packages/twitter/api.py", line 2908, in GetUser
data = self._ParseAndCheckTwitter(resp.content.decode('utf-8'))
File "/app/.heroku/python/lib/python2.7/site-packages/twitter/api.py", line 4862, in _ParseAndCheckTwitter
self._CheckForTwitterError(data)
File "/app/.heroku/python/lib/python2.7/site-packages/twitter/api.py", line 4882, in _CheckForTwitterError
raise TwitterError(data['errors'])
twitter.error.TwitterError: [{u'code': 32, u'message': u'Could not authenticate you.'}]

The other times, I get the # No, sorry, not this time error.

My app permissions are set to read and write in dev. I also set ODDS to 0 yet I keep getting both error messages. I'm not sure what to do to make my bot work. The account I'm pulling from is one I've had forever and has 92k tweets.

Thanks in advance!

What's happening there in stevebuttry_ebx?

Chop off any tail ends that say ', via' or ', says' or whatever. But then check to see if the whole tweet comes verbatim from any tweet in the corpus.

Hi, when I try to run this I get this after running heroku worker start:
Running worker on ⬢ nameofapp... up, run.2970 (Free) 1282 tweets found in NovaPhoenix500 TOO SIMILAR: Will the Switch be a commercial failure?
Edit: After regenerating keys, the issue is mostly fixed. It has happened about once.

With popularity of Mastodon rising, add support to pull source statuses from Mastodon and the option to send the resulting Markov sentence to Mastodon in addition to, or in lieu of, Twitter.

The bot works fine when I run with ORDER = 2/3.

But when I run ORDER = 1 I get the following :

Traceback (most recent call last):
File "ebooks.py", line 94, in
ebook_tweet = mine.generate_sentence()
File "/app/markov.py", line 54, in generate_sentence
restup = (res[-2], res[-1])
IndexError: list index out of range

From @lukethighwalker in #59:

Is there a way in Local_settings.py where I can make the app search my existing timeline and my archive file? or is it simply one or the other?

getting this error whenever I run the heroku run worker command. source account has near 10k tweets. I followed the code correctly, all that. what can I do to fix this?

thank you!

hi, getting error on latest version:

heroku run worker

File "ebooks.py", line 106
    max_id = user_tweets[-1].id - 
                                 ^
SyntaxError: invalid syntax

Today's news from Heroku means that it will no longer allow free dynos, the kind that this repo relies on. I'm not sure yet if it's worth the effort to redesign this app for another platform, nor do I really have the time to support such an effort.

What do you all think? Would you be willing to pay for a $7/month Heroku dyno to keep your ebooks account going? Or should we let this app die with dignity after 9 years? What would you want to see happen?

keep running into this issue when running heroku worker

when i run "heroku config" my API keys show up fine

i did change the local settings file because a tutorial i followed prompted me to replace the Haroku variables with the keys themselves (Tom himself said not to do so) but still having the same issue

have a feeling it may have to do with this Twitter request, i don't see steps for filling it in any tutorial but it's requested of me when I create a new dev app. I had it previously filled with "http://127.0.0.1/twitter/callback" per a Google search but think that might be the culprit as it relates to oAuth

Because of the Twitter API limits and my source accounts being over 40,000 tweets I have ended up combining the archives of my source accounts into one large CSV. I wish that static sources were treated as a first class source instead only treating it as a testing source.

The main issue I have encountered is that filter_status is not being run on static sources which is resulting in some awkward @'s. Could each string from static sources be run through filter_status the same as API tweets and toots?

I have little experience with Python but I am happy to take a stab at a PR if that is preferred.

Thanks

I've tried multiple twitter accounts both with more than 3,600 tweets and less than that. Nothing works, it always produces "No statuses found!" message. I've run "heroku run worker" many times, and every single time it fails with this message.

http://prntscr.com/ind2nv

I'm fairly new to python and Twitter API development, so this is likely something simple I am missing. I double and triple checked the documentation so I believe I have all of the keys and tokens setup correctly in both local_settings.py and in Heroku.

On heroku run worker I see the following log:

File "ebooks.py", line 161, in
handle_stats = api.GetUser(screen_name=user)
File "/app/.heroku/python/lib/python3.7/site-packages/twitter/api.py", line 2868, in GetUser
resp = self._RequestUrl(url, 'GET', data=parameters)
File "/app/.heroku/python/lib/python3.7/site-packages/twitter/api.py", line 4959, in _RequestUrl
raise TwitterError("The twitter.Api instance must be authenticated.")
twitter.error.TwitterError: The twitter.Api instance must be authenticated.

Twitter Dev dash looks like it has changed since docs, so I setup OAuth1.0a as that seemed closest to what was described. Is there something I may have missed? Should I be using a different setup on Twitter Dev?

Thanks in advance.

I definitely have tweets on my source account and I double checked my API keys. Any clue what's going wrong?

This happens whenever I attempt to run git push heroku master. I saw a closed issue about it (#16) , tried the solution offered ran command to switch buildpack to python. I also set the buildpack to python in heroku settings. Now it tells me the app is not compatible with buildpack. I then get push failed and [remote rejected] master -> master (pre-receive hook declined) error: failed to push some refs

A lot of my generated tweets are coming out quite short. Are there any variables within the Markov.py that can be tweaked to produce different outcomes or is it pretty set in stone?

In the original twitter_ebooks bot, there's an option to download and then incrementally update a local json archive of a user's tweets, and also to convert the .csv file of your Twitter Archive files to .json so it can properly work.

It would be nice to allow someone to manually download their twitter archive and then converting the included tweets.csv in the archive to a .json file that the bot can read and publish tweets with.

Check first to see if the word .isupper() or .istitle() before making changes.

When I run ebooks.py with DEBUG set to true, and 3 twitter accounts in source - i recieve this error.
It is successfully authenticating but failing to find...somewhere?
I don't know enough python to work it out.

I have set up this bot successfully once, in February this year using Twitter but I am now running into problems using it for Mastodon, but I don't think the problem is with either service.

I have followed all the steps described multiple times but I keep getting the following error when trying the first heroku run worker command:

Running worker on gentle-ridge-53367... up, run.1762 (Free)
Traceback (most recent call last):
  File "ebooks.py", line 17, in <module>
    from local_settings import *
ModuleNotFoundError: No module named 'local_settings'

I've definitely got the correct file name etc and I've tried changing the runtime.txt several different versions of Python to see if it makes any difference but it doesn't? Have attached my local_settings.py file in case there's something obvious that I've somehow missed. For the record I am very new to coding so I apologise if I'm just wasting your time with a mistake I've made.

local_settings (copy).txt

I'm getting this message whenever I try to run the code. Sometimes I get "2 No, sorry, not this time." and sometimes "IndexError: list index out of range". I've read that this is cause by a less-than-3600-tweets account but the one I'm trying to source (@Tscchnnell) has more than 7k tweets.

Here's the traceback:
Traceback (most recent call last):
File "ebooks.py", line 135, in
twitter_tweets_iter, max_id = grab_tweets(api, max_id)
File "ebooks.py", line 96, in grab_tweets
max_id = user_tweets[-1].id - 1
IndexError: list index out of range

Would love to see a version of this bot running with GPT2. I’m not super familiar with what’s involved in doing that though so I figured I’d put in an issue and it can be closed if it’s not within the scope of the project.

Hi! I'm quite new to this, so I apologise if this is a very obvious and fixable thing to people with python experience, but: I keep getting an error that api is not defined. I can see in ebooks.py that it is defined. As I understand it, it should handle so long as I have pip and python-twitter installed (which I do). Am I missing something?

Full error
Traceback (most recent call last):
File "ebooks.py", line 143, in
status = api.PostUpdate(ebook_tweet)
NameError: name 'api' is not defined

Some other relevant information may be that I intend to run this completely static

Any chance this can be updated to work with v2 endpoints? As of right now, the endpoints in this code don't work unless the user's app is approved for "Elevated" developer access.

Traceback (most recent call last):
  File "/app/ebooks.py", line 161, in <module>
    handle_stats = api.GetUser(screen_name=user)
  File "/app/.heroku/python/lib/python3.9/site-packages/twitter/api.py", line 2869, in GetUser
    data = self._ParseAndCheckTwitter(resp.content.decode('utf-8'))
  File "/app/.heroku/python/lib/python3.9/site-packages/twitter/api.py", line 4908, in _ParseAndCheckTwitter
    self._CheckForTwitterError(data)
  File "/app/.heroku/python/lib/python3.9/site-packages/twitter/api.py", line 4928, in _CheckForTwitterError
    raise TwitterError(data['errors'])
twitter.error.TwitterError: [{'message': 'You currently have Essential access which includes access to Twitter API v2 endpoints only. If you need access to this endpoint, you’ll need to apply for Elevated access via the Developer Portal. You can learn more here: https://developer.twitter.com/en/docs/twitter-api/getting-started/about-twitter-api#v2-access-leve', 'code': 453}] 

The error is as follows:

Running worker on safe-earth-67442.... up, run.1755
Traceback (most recent call last):
  File "ebooks.py", line 79, in <module>
    source_tweets_iter, max_id = grab_tweets(api,max_id)
  File "ebooks.py", line 51, in grab_tweets
    max_id = user_tweets[len(user_tweets)-1].id-1
IndexError: list index out of range

These are my settings with keys left out:

SOURCE_ACCOUNTS = ["GwingleWizard", "Darkstr_", "badsums", "belly_rose", "noh_jgjb", "loserice", "andrewhussie", "billwurtz", "kanyewest", "berniethoughts", "sylferino", "sass_o_matic", "boburnham", "wolfpupy"] #A list of comma-separated, quote-enclosed Twitter handles of account that you'll generate tweets based on. It should look like ["account1", "account2"]. If you want just one account, no comma needed.
ODDS = 1 #How often do you want this to run? 1/8 times?
ORDER = 2 #how closely do you want this to hew to sensical? 1 is low and 3 is high.
DEBUG = True #Set this to False to start Tweeting live
STATIC_TEST = False #Set this to True if you want to test Markov generation from a static file instead of the API.
TEST_SOURCE = ".txt" #The name of a text file of a string-ified list for testing. To avoid unnecessarily hitting Twitter API.
TWEET_ACCOUNT = "GwingleBot" #The name of the account you're tweeting to.

Hi! I'm not familiar with Python so I'm having a hard time trying to fix this error.

Running worker on pure-badlands-73935... up, run.3341 (Free)
Traceback (most recent call last):
File "ebooks.py", line 78, in
handle_stats = api.GetUser(screen_name=user)
File "/app/.heroku/python/lib/python2.7/site-packages/twitter/api.py", line 2928, in GetUser
data = self._ParseAndCheckTwitter(resp.content.decode('utf-8'))
File "/app/.heroku/python/lib/python2.7/site-packages/twitter/api.py", line 4894, in _ParseAndCheckTwitter
self._CheckForTwitterError(data)
File "/app/.heroku/python/lib/python2.7/site-packages/twitter/api.py", line 4914, in _CheckForTwitterError
raise TwitterError(data['errors'])
twitter.error.TwitterError: [{u'message': u'Could not authenticate you.', u'code': 32}]

Thanks!

Just created a bot with this and I realized that it oftens add three dots (...) when retrieving tweets longer than 140 characters - this seems connected to the API changes Twitter made after extending tweet length to 280 characters - the API still outputs a truncated 140-character version unless it is not explicitly requested the full tweet. I'm having troubles implementing this into heroku_ebooks, any idea?