horhik / instagram2fedi Goto Github PK

i had been using this just fine but on running today i was faced with a couple of fun errors

• instaloader.exceptions.ProfileNotExistsException: Profile username does not exist.
  and then on retrying
• instaloader.exceptions.ConnectionException: JSON Query to username/feed/: Could not find "window._sharedData" in html response.

is this a me issue or an ig issue? sorry, not too hot on python so limited capacity to troubleshoot myself.

It looks like whenever the Docker container is redeployed, it posts the last few Instagram posts (as per limits set).
It would be great if it first checked whether the last post was already cross posted before duplicating it.

Please provide some docs how and where to generate the access token...

Hello!

I have just deployed via docker

Under the instagram variable I have attempted a few accounts including my own without luck
When the container runs I always get

JSON Query to justdalek_/feed/: Could not find "window._sharedData" in html response. [retrying; skip with ^C]

Is there something that I need to change to resolve this error message?

Here, what logs look like after release:

2022-05-12T20:31:33.561030+00:00 heroku[web.1]: Starting process with command `python /app/src/main.py`
2022-05-12T20:31:34.577497+00:00 app[web.1]: ['/app/src/main.py', 'python', '/app/src/main.py']
2022-05-12T20:31:34.577514+00:00 app[web.1]: ARGUMENTS
2022-05-12T20:31:34.577514+00:00 app[web.1]: ❗ -> Wrong Argument Name!...
2022-05-12T20:31:34.577514+00:00 app[web.1]: 
2022-05-12T20:31:34.577515+00:00 app[web.1]: 2022-05-12 20:31:34.576732
2022-05-12T20:31:34.577524+00:00 app[web.1]: {'instance': None, 'instagram-user': None, 'token': None, 'check-interval': 3600, 'post-interval': 3600, 'fetch-count': 10, 'carousel-limit': 4, 'use-docker': True}
2022-05-12T20:31:34.577525+00:00 app[web.1]: 🚀 > Connecting to Mastodon/Pixelfed...
2022-05-12T20:31:34.577525+00:00 app[web.1]: 
2022-05-12T20:31:34.577525+00:00 app[web.1]: 2022-05-12 20:31:34.576844
2022-05-12T20:31:34.577525+00:00 app[web.1]: 🚀 > Connecting to Instagram...
2022-05-12T20:31:34.577526+00:00 app[web.1]: 
2022-05-12T20:31:34.577526+00:00 app[web.1]: 2022-05-12 20:31:34.577390
2022-05-12T20:31:34.577531+00:00 app[web.1]: Traceback (most recent call last):
2022-05-12T20:31:34.577532+00:00 app[web.1]: File "/app/src/main.py", line 66, in <module>
2022-05-12T20:31:34.577645+00:00 app[web.1]: get_new_posts(mastodon, mastodon_carousel_size, post_limit, id_filename, using_mastodon, mastodon_carousel_size, post_interval, fetched_user)
2022-05-12T20:31:34.577647+00:00 app[web.1]: File "/app/src/network.py", line 77, in get_new_posts
2022-05-12T20:31:34.577744+00:00 app[web.1]: profile = get_instagram_user(fetched_user)
2022-05-12T20:31:34.577746+00:00 app[web.1]: File "/app/src/network.py", line 18, in get_instagram_user
2022-05-12T20:31:34.577817+00:00 app[web.1]: return Profile.from_username(L.context, user)
2022-05-12T20:31:34.577819+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/instaloader/structures.py", line 691, in from_username
2022-05-12T20:31:34.578018+00:00 app[web.1]: profile = cls(context, {'username': username.lower()})
2022-05-12T20:31:34.578033+00:00 app[web.1]: AttributeError: 'NoneType' object has no attribute 'lower'
2022-05-12T20:31:34.742759+00:00 heroku[web.1]: Process exited with status 1
2022-05-12T20:31:34.852473+00:00 heroku[web.1]: State changed from starting to crashed
2022-05-12T20:31:34.889132+00:00 heroku[web.1]: State changed from crashed to starting
2022-05-12T20:31:46.309290+00:00 heroku[web.1]: Starting process with command `python /app/src/main.py`
2022-05-12T20:31:47.285607+00:00 app[web.1]: ['/app/src/main.py', 'python', '/app/src/main.py']
2022-05-12T20:31:47.285619+00:00 app[web.1]: ARGUMENTS
2022-05-12T20:31:47.285620+00:00 app[web.1]: ❗ -> Wrong Argument Name!...
2022-05-12T20:31:47.285620+00:00 app[web.1]: 
2022-05-12T20:31:47.285620+00:00 app[web.1]: 2022-05-12 20:31:47.284780
2022-05-12T20:31:47.286053+00:00 app[web.1]: {'instance': None, 'instagram-user': None, 'token': None, 'check-interval': 3600, 'post-interval': 3600, 'fetch-count': 10, 'carousel-limit': 4, 'use-docker': True}
2022-05-12T20:31:47.286054+00:00 app[web.1]: 🚀 > Connecting to Mastodon/Pixelfed...
2022-05-12T20:31:47.286054+00:00 app[web.1]: 
2022-05-12T20:31:47.286054+00:00 app[web.1]: 2022-05-12 20:31:47.284902
2022-05-12T20:31:47.286055+00:00 app[web.1]: 🚀 > Connecting to Instagram...
2022-05-12T20:31:47.286055+00:00 app[web.1]: 
2022-05-12T20:31:47.286055+00:00 app[web.1]: 2022-05-12 20:31:47.285511
2022-05-12T20:31:47.286063+00:00 app[web.1]: Traceback (most recent call last):
2022-05-12T20:31:47.286063+00:00 app[web.1]: File "/app/src/main.py", line 66, in <module>
2022-05-12T20:31:47.286064+00:00 app[web.1]: get_new_posts(mastodon, mastodon_carousel_size, post_limit, id_filename, using_mastodon, mastodon_carousel_size, post_interval, fetched_user)
2022-05-12T20:31:47.286064+00:00 app[web.1]: File "/app/src/network.py", line 77, in get_new_posts
2022-05-12T20:31:47.286064+00:00 app[web.1]: profile = get_instagram_user(fetched_user)
2022-05-12T20:31:47.286065+00:00 app[web.1]: File "/app/src/network.py", line 18, in get_instagram_user
2022-05-12T20:31:47.286065+00:00 app[web.1]: return Profile.from_username(L.context, user)
2022-05-12T20:31:47.286067+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/instaloader/structures.py", line 691, in from_username
2022-05-12T20:31:47.286093+00:00 app[web.1]: profile = cls(context, {'username': username.lower()})
2022-05-12T20:31:47.286114+00:00 app[web.1]: AttributeError: 'NoneType' object has no attribute 'lower'
2022-05-12T20:31:47.451100+00:00 heroku[web.1]: Process exited with status 1
2022-05-12T20:31:47.494216+00:00 heroku[web.1]: State changed from starting to crashed

bot_1  | HTTP redirect from https://www.instagram.com/quelili_collective/feed/ to https://www.instagram.com/accounts/login/?next=/quelili_collective/feed/
bot_1  | Traceback (most recent call last):
bot_1  |   File "/app/src/main.py", line 65, in <module>
bot_1  |     get_new_posts(mastodon, mastodon_carousel_size, post_limit, id_filename, using_mastodon, mastodon_carousel_size, post_interval, fetched_user)
bot_1  |   File "/app/src/network.py", line 77, in get_new_posts
bot_1  |     profile = get_instagram_user(fetched_user)
bot_1  |   File "/app/src/network.py", line 18, in get_instagram_user
bot_1  |     return Profile.from_username(L.context, user)
bot_1  |   File "/usr/local/lib/python3.9/site-packages/instaloader/structures.py", line 635, in from_username
bot_1  |     profile._obtain_metadata()  # to raise ProfileNotExistsException now in case username is invalid
bot_1  |   File "/usr/local/lib/python3.9/site-packages/instaloader/structures.py", line 690, in _obtain_metadata
bot_1  |     metadata = self._context.get_json('{}/feed/'.format(self.username), params={})
bot_1  |   File "/usr/local/lib/python3.9/site-packages/instaloader/instaloadercontext.py", line 336, in get_json
bot_1  |     raise LoginRequiredException("Redirected to login page. Use --login.")
bot_1  | instaloader.exceptions.LoginRequiredException: Redirected to login page. Use --login.
insta2fedi_bot_1 exited with code 1

The docker-compose.yml:

version: '3'
services:
  bot:
    build:
      context: .
    image: "horhik/instagram2fedi:latest"
    environment:
      - YOUR_CONTAINER_NAME=xxxxxxxxxxxxxxxxxxxxx
      - I2M_INSTAGRAM_USER=xxxxxxxxxxxxxxxxxxx
      - I2M_INSTANCE=xxxxxxxxxxxxxxxxxxx
      - I2M_TOKEN=xxxxxxxxxxxxxxxxxxxxxxxxxxx
      - I2M_CHECK_INTERVAL=3600 #1 hour    
      - I2M_POST_INTERVAL=3600 #1 hour   
      - I2M_USE_MASTODON=4 #max carouse    - is 4, if there's no limit set to -1
      - I2M_FETCH_COUNT=5 # how many instagram posts to fetch per check_interval   -
      - I2M_USER_NAME=xxxxxxxxxxxxxxxxxxx
      - I2M_USER_PASSWORD=xxxxxxxxxxxxxxxxxxxx

Posted this sequence of images to Instagram. When I2M posted it to PixelFed, the images ended up out-of-order.

I like the script but would like to run it stateless (i.e. not as daemon and without a local file to store already posted IDs). The target is to run it as FaaS with Heroku, Scaleway or any other FaaS provider trigger via cron (i.e. hourly).

Running one-shot instead of deamonized would be straight forward, but how could we avoid double-posting over and over?

1. only look at post in past (timeframe) and make sure script only runs once per timeframe (with the obvious disadvantage that we'd miss posts if there was an unsuccessful invocation).
2. Embed Insta post.mediaid in Toot to detect duplicates. Also introduce a timeframe to reduce the number of posts that need to be compared (with the advantage over option 1 that the script can and should run multiple times in the defined timeframe)

Any other ideas how to achieve the said objective?

Instagram2Fedi is failing for me because Instagram wants to create a checkpoint; however, since the container isn't saving session state, I'm just in this continual loop of failure.

Traceback (most recent call last):
--
Tue, Nov 29 2022 9:04:16 am | File "/app/src/main.py", line 70, in <module>
Tue, Nov 29 2022 9:04:16 am | get_new_posts(mastodon, mastodon_carousel_size, post_limit, id_filename, using_mastodon, mastodon_carousel_size, post_interval, fetched_user, user)
Tue, Nov 29 2022 9:04:16 am | File "/app/src/network.py", line 78, in get_new_posts
Tue, Nov 29 2022 9:04:16 am | profile = get_instagram_user(user, fetched_user)
Tue, Nov 29 2022 9:04:16 am | File "/app/src/network.py", line 18, in get_instagram_user
Tue, Nov 29 2022 9:04:16 am | L.login(user["name"], user["password"])
Tue, Nov 29 2022 9:04:16 am | File "/usr/local/lib/python3.9/site-packages/instaloader/instaloader.py", line 634, in login
Tue, Nov 29 2022 9:04:16 am | self.context.login(user, passwd)
Tue, Nov 29 2022 9:04:16 am | File "/usr/local/lib/python3.9/site-packages/instaloader/instaloadercontext.py", line 239, in login
Tue, Nov 29 2022 9:04:16 am | raise ConnectionException("Login: Checkpoint required. Point your browser to "
Tue, Nov 29 2022 9:04:16 am | instaloader.exceptions.ConnectionException: Login: Checkpoint required. Point your browser to https://www.instagram.com/challenge/action/...

Instaloader discusses the problem at https://instaloader.github.io/troubleshooting.html#login-error

https://github.com/Horhik/Instagram2Fedi/blob/main/src/requipments.txt

If bot is launched, it fetching new posts, but it can't fetch next new posts which has appeared after its launching

Hello!
Is there any way to crosspost from several insta accounts to one pixelfed?

Instagram is blocking everything, even Biblio.

But somehow https://www.picuki.com/ always seems to work... (It's just sometimes a bit slow at refreshing)

So maybe you could get it working with this site?

There is also a RSS Bridge for that, if that would help...

https://github.com/RSS-Bridge/rss-bridge/blob/master/bridges/PicukiBridge.php

Is it possible to turn this around and post from Pixelfed to Instagram?

I see you have no LICENSE file for this project. The default is copyright.

I would suggest releasing the code under the GPL-3.0-or-later or AGPL-3.0-or-later license so that others are encouraged to contribute changes back to your project.

Sorry for bothering you again. Here are logs of JSON content errors. Don't know, maybe it is related to libraries. First run was successful and from the second run I got this:

2022-07-12T07:16:28.005134+00:00 app[web.1]: Could not parse response as JSON, response code was 200, bad json content was 'b'<!DOCTYPE html>\n\n<html lang="en">\n<head>\n\t<meta charset="utf-8">\n\t<meta http-equiv="X-UA-Compatible" content="IE=edge">\n\t<meta name="viewport" content="width=device-width, initial-scale=1">\n\t<meta name="mobile-web-app-capable" content="yes">\n\n\t<title>pixelfed</title>\n\t<link rel="manifest" href="/manifest.json">\n\n\t<meta property="og:site_name" content="pixelfed">\n\t<meta property="og:title" content="pixelfed">\n\t<meta property="og:type" content="article">\n\t<meta property="og:url" content="https://pixelfed.social/login">\n\t\n\t<meta name="medium" content="image">\n\t<meta name="theme-color" content="#10c5f8">\n\t<meta name="apple-mobile-web-app-capable" content="yes">\n\t<link rel="shortcut icon" type="image/png" href="/img/favicon.png?v=2">\n\t<link rel="apple-touch-icon" type="image/png" href="/img/favicon.png?v=2">\n\t<link rel="canonical" href="https://pixelfed.social/login">\n\t<link href="/css/app.css?id=66c10c9502955c88d471277fface8c32" rel="stylesheet" data-stylesheet="light">\n\t<script type="text/javascript">window._sharedData = {curUser: {}, version: 0}; window.App = {config: {"open_registration":true,"uploader":{"max_photo_size":"15000","max_caption_length":"2000","album_limit":"10","image_quality":"85","max_collection_length":100,"optimize_image":true,"optimize_video":true,"media_types":"image\\/jpeg,image\\/gif,image\\/png,video\\/mp4,image\\/webp","enforce_account_limit":true},"activitypub":{"enabled":true,"remote_follow":true},"ab":{"lc":true,"rec":false,"loops":false,"top":false,"polls":false,"cached_public_timeline":true,"gps":false,"spa":true,"emc":true},"site":{"name":"pixelfed","domain":"pixelfed.social","url":"https:\\/\\/pixelfed.social","description":"The original Pixelfed instance, operated by the main developer @dansup"},"username":{"remote":{"formats":{"0":"@","1":"from","2":"custom"},"format":"@","custom":null}},"features":{"timelines":{"local":true,"network":true},"mobile_apis":true,"stories":true,"video":true,"import":{"instagram":false,"mastodon":false,"pixelfed":false},"label":{"covid":{"enabled":false,"org":"visit the WHO website","url":"https:\\/\\/www.who.int\\/emergencies\\/diseases\\/novel-coronavirus-2019\\/advice-for-public"}}}}};</script>\n\t</head>\n<body>\n\t<nav class="navbar navbar-expand navbar-light navbar-laravel shadow-none border-bottom sticky-top py-1">\n\t<div class="container">\n\t\t\t<a class="navbar-brand d-flex align-items-center" href="/" title="Logo">\n\t\t\t\t<img src="/img/pixelfed-icon-color.svg" height="30px" class="px-2" loading="eager" alt="Pixelfed logo">\n\t\t\t\t<span class="font-weight-bold mb-0 d-none d-sm-block" style="font-size:20px;">pixelfed</span>\n\t\t\t</a>\n\n\t\t\t<div class="collapse navbar-collapse">\n\t\t\t\n\t\t\t\n\t\t\t\t<ul class="navbar-nav ml-auto">\n\t\t\t\t\t<li>\n\t\t\t\t\t\t<a class="nav-link font-weight-bold text-dark" href="/login" title="Login">\n\t\t\t\t\t\t\tLogin\n\t\t\t\t\t\t</a>\n\t\t\t\t\t</li>\n\t\t\t\t\t\t\t\t\t\t\t</ul>\n\t\t\t</div>\n\t</div>\n</nav>\n\t<main id="content">\n\t\t<div class="container mt-4">\n <div class="row justify-content-center">\n <div class="col-lg-5">\n <div class="">\n <div class="card-header bg-transparent p-3 text-center font-weight-bold h3">Login</div>\n\n <div class="card-body">\n <form method="POST" action="https://pixelfed.social/login">\n <input type="hidden" name="_token" value="0PJX1cvO8fKTPWAUaNnT9top4AwyZdLzcpWCFCka">\n <div class="form-group row">\n\n <div class="col-md-12">\n <input id="email" type="email" class="form-control" name="email" value="" placeholder="Email" required autofocus>\n\n </div>\n </div>\n\n <div class="form-group row">\n\n <div class="col-md-12">\n <input id="password" type="password" class="form-control" name="password" placeholder="Password" required>\n\n </div>\n </div>\n\n <div class="form-group row">\n <div class="col-md-12">\n <div class="checkbox">\n <label>\n <input type="checkbox" name="remember" > \n <span class="font-weight-bold ml-1 text-muted">\n Remember Me\n </span>\n </label>\n </div>\n </div>\n </div>\n\n \n <div class="form-group row mb-0">\n <div class="col-md-12">\n <button type="submit" class="btn btn-primary btn-block btn-lg font-weight-bold">\n Login\n </button>\n\n </div>\n </div>\n </form>\n\n <hr>\n\n <p class="text-center font-weight-bold">\n <a href="https://pixelfed.social/password/reset">\n Forgot Password\n </a>\n </p>\n </div>\n </div>\n </div>\n </div>\n</div>\n\t</main>\n\t<footer>\n <div class="container py-5">\n <p class="text-center text-uppercase font-weight-bold small text-justify">\n <a href="https://pixelfed.social/site/about" class="text-dark p-2">About</a>\n <a href="https://pixelfed.social/site/help" class="text-dark p-2">Help</a>\n <a href="https://pixelfed.social/site/terms" class="text-dark p-2">Terms</a>\n <a href="https://pixelfed.social/site/privacy" class="text-dark p-2">Privacy</a>\n <a href="https://pixelfed.social/site/language" class="text-dark p-2">Language</a>\n </p>\n <p class="text-center text-muted small mb-0">\n <span class="text-muted">\xc2\xa9 2022 pixelfed.social</span>\n <span class="mx-2">\xc2\xb7</span>\n <a href="https://pixelfed.org" class="text-muted font-weight-bold" rel="noopener">Powered by Pixelfed</a>\n <span class="mx-2">\xc2\xb7</span>\n <span class="text-muted">v0.11.3</span>\n </p>\n </div>\n </footer>\n \t<script type="text/javascript" src="/js/manifest.js?id=4e6dd9cb251d9698bfccb781db000cca"></script>\n\t<script type="text/javascript" src="/js/vendor.js?id=467b36b2c099089a7639448443158ceb"></script>\n\t<script type="text/javascript" src="/js/app.js?id=855c8b0847137dd56c72229ad305c032"></script>\n\t<script type="text/javascript" src="/js/components.js?id=35beb0748dd5518371686a5f00348fbf"></script>\n\t</body>\n</html>\n'' 2022-07-12T07:16:28.005145+00:00 app[web.1]: �[0m 2022-07-12T07:16:28.005146+00:00 app[web.1]: 2022-07-12 07:16:27.907973 2022-07-12T07:16:28.005146+00:00 app[web.1]: [None, None, None, None, None, None] 2022-07-12T07:16:28.005147+00:00 app[web.1]: �[31m😿 > Failed to create toot 2022-07-12T07:17:24.241098+00:00 heroku[web.1]: Error R10 (Boot timeout) -> Web process failed to bind to $PORT within 60 seconds of launch 2022-07-12T07:17:24.385766+00:00 heroku[web.1]: Stopping process with SIGKILL 2022-07-12T07:17:24.515034+00:00 app[web.1]: 2022-07-12T07:17:24.519491+00:00 heroku[web.1]: Process exited with status 137 2022-07-12T07:17:24.797258+00:00 heroku[web.1]: State changed from starting to crashed

BTW, thank you so much for helping me figure out the last issue. It was all about correct composing the container locally and then pushing it to Heroku.