sixcious / infy-scroll Goto Github PK

Hi everyone,

I'm looking for feedback on the removal of these two fields. It's these two inputs outlined in red in the UI:

Right now, this is just about removing them from the UI. They would still be in the Save object, which you will be able to edit in the Options screen. Removing these two fields would allow the Next Link Path and the Page Element Path inputs to have more room and be 100% full width.

Next Link Property
I added this because I thought there may have been a use-case for a site secretly storing the next link in a property/attribute besides href. However, since releasing Infy, I haven't encountered a website yet that actually does this. Has anyone?

Insert Before
There are currently only 50 Database URLs that have an Insert Before path entered for them, and many of them aren't working anymore. Has anyone used this field and would prefer that it remain in the UI?

Thank you for any feedback!

Feature request
The "My Saved URLs" page displays the URL patterns and their dates, but not the names assigned to them.
People are better at recognizing names than regex. Arguably they are at least as important as the dates.

Screenshots

Thank you!

Describe the bug
I like the functions of this pluin. I am using edge edition.
When I click infy Scroll button on the tool button of my browser, right and bottom part content is invisable due to scale and crop. I cannot do right operations in this situation.
I try to resize visiting page to 100% but it does not affect scale rate of this extension.

URL
For all urls

Environment
If you can, please fill out as much as you can about your environment.

• Device: PC
• OS: Windows 10
• Browser Edge
• Browser Version 95
• Extension Version 0.6.06

Hi! Where can i find the previous versions of your app? i need one that actually works with fennec f-droid 68.12.0 for android. (with actual version 0.6.0.6 says addon downloading, but can't install)
Thanks in advance!!

What happened?

I can't seem to get the extension to function on LinkedIn.

URL

https://www.linkedin.com/search/results/people/?keywords=informatique&origin=SWITCH_SEARCH_VERTICAL&sid=Fpy

Version

Version 0.7 β

Browser

Brave Version 1.48.167 Chromium: 110.0.5481.104 (Official Build) (64-bit)

OS

Windows 11

Device

PC

Describe the bug
On google, without any scrolling by the user, the addon begins loading page 2 but pushes what was on page 1 (the most relevant search results) to the bottom of the page, it keeps loading more pages infinitely making you unable to see the main results from page 1

URL
https://www.google.com/

Environment
If you can, please fill out as much as you can about your environment.

• Device: Mac
• OS: MacOS 12.3
• Browser Chrome
• Browser Version 99.0.4844.84
• Extension Version 0.7

Screenshots

This is the top of the google page, it immediately loaded page 2 and overlapped it on top of page 1

关于保存的网址匹配规则，遇到的困难。
例如
对于一条规则需要生效的网址是，www.example.com/[any word]/，其中[any word]有许多无规律的字符。
另一条规则需要生效的网址是，www.example.com/[any word]/[any word]/，其中[any word]有许多无规律的字符。
如何只保存两条自动滚动规则 ，为了分别对大量网址起作用，而且不冲突。

Hello there, I am trying to get the next button to load the next page and add the results under the results from the current page. Could you please tell me how this can be achieved?

When I navigate to a page that matches the urls of two saved rules, only one of them is attempted, even if it cannot find the scrollElementRule on the page. This makes it much more difficult to make "generic" rules.

Suggestion:

• Find all the saved rules with a URL match
• Check each one in sequence, until both a scrollElementRule and a nextSelector/nextXpath are found.

I believe this is how DB rules work, I'd like saved rules to work similarly.

Thank you.

Currently, clicking on this link will open that page on a new tab. Is there an option to open it on current tab?

The same issue also happens on any link on the newly loaded page (not on the original page)

I'm having trouble trying to replicate the pixiv settings used on the main page. It would be great to have a screenshot of the settings you used for your pixiv example.

Describe the bug
I downloaded the main branch zip from github and unzipped it on windows 10 to manually install it on chromium.

I loaded the unpacked extension successfully using this location 'infy-scroll-main\src\base'

the version is 0.6

I realized something was amiss after getting this reply on an issue. #25 (comment)

I don't know how to get version 0.7.

Environment

• OS: [e.g. Windows 10]
  Version 97.0.4692.71 (Developer Build, ungoogled-chromium) (64-bit)

Screenshots
If applicable, please add any screenshots to help explain the problem.
However, please do not post any screenshots of the extension. For example, please do not post screenshots of the User Interface, Options Screen, or of the Mascot Character.

please add "show-more" to the list of default next keywords.

the addition of the "show-more" keyword works on nitter.net urls.

https://nitter.net/Microsoft

search results on etsy.com are not working. It seems as if the elements are loaded but then they are put into an invisible container :(

As I'm scrolling down an infy scroll of pictures using the Increment > Media function, every time a new image in the series loads, it jumps the scroll right back up to the top, and I have to scroll back down to find my place again.

Firefox Nightly on Android.

Describe the bug
A page that has an associated DB rule is not automatically processed.

1. I go to https://xenforo.com/community/threads/job-xf-emailunsubscribe-invalid-header-value-detected.177611/

2. The next page is not fetched.

3. I click on the extension icon and get the following:

4. I click [OK] and [ACCEPT]

5. The next page is fetched

6. I refresh the page with [F5]

7. I'm back to step 2.

Expected behaviour:
If the page is in the DB, the associated rule should be automatically invoked.

Website
https://xenforo.com/community/threads/job-xf-emailunsubscribe-invalid-header-value-detected.177611/

Environment

• Device: PC
• OS: Windows 10
• Browser: FF 105.0b9
• Extension Version: 0.7

Screenshots
See above, and also:

Hi!

Great extension - congratulations!!! What I'm really missing is a whitelist, though! I just need the functionality on a few websites, on most others I prefer the regular page separation. So it would be really great to have a whitelist where I can enter just the pages I want Infy-Scroll to handle!

Like e.g.:
https://www.google.*

etc.

Thanks!

I have used several autopager extensions/scripts and the problem with most of them is that they use a "blacklist" model. That is, they are on by default and allow opting-out for specific sites.

The problem with that approach is that the autopaging code tends to break some sites, sometimes silently, and it is usually the ones where breakage causes the most unfortunate consequences: financial sites, job interview sites, medical/insurance, etc.

Unfortunately, it is impossible to know in advance if a web site will break with autopaging, so the safest approach is the opposite one - a "whitelist" (aka "opt-in") , where the default is not to try to autopage unless the user marks the site as "safe".

Does Infy support this?

I am having problems making Pixiv galleries work with Infy scroll. Some galleries work, others don't, others work for a certain number of pages then stop, yet others cause Infy to hang (the animated icon does not stop).

Website
Here are a couple of art galleries that I tried it on
https://www.pixiv.net/en/users/415/artworks
https://www.pixiv.net/en/users/39/artworks

Environment
This is required, so please fill out as much as you can about your environment.

• Device: PC
• OS: Windows 10
• Browser: FF
• Browser Version: 108.0b6 (64-bit)
• Extension Version: 0.7

Exported entry

    {
      "id": 14,
      "type": "pattern",
      "url": "https://www.pixiv.net/*",
      "title": "pixiv",
      "date": "2022-11-28T05:06:57.960Z",
      "order": 2,
      "action": "next",
      "scrollAppend": "element",
      "scrollAppendScripts": false,
      "scrollAppendStyles": false,
      "scrollLazyLoad": true,
      "scrollLazyLoadMode": "auto",
      "scrollLazyLoadAttributeSource": "data-src",
      "scrollLazyLoadAttributeDestination": "src",
      "scrollElementRule": "//section//div[ul]",
      "scrollElementInsertRule": "",
      "scrollElementType": "xpath",
      "scrollElementIframe": true,
      "scrollMediaType": "image",
      "scrollWrapFirstPage": false,
      "scrollHeightWait": 0,
      "nextType": "xpath",
      "nextSelector": "//button[@aria-current='true']/following-sibling::a[@aria-disabled='false']",
      "nextXpath": "//button[@aria-current='true']/following-sibling::a[@aria-disabled='false']",
      "nextProperty": [],
      "nextKeywordsEnabled": false,
      "prevType": "selector",
      "prevSelector": "[rel=\"prev\"],[rel=\"previous\"]",
      "prevXpath": "//*[@rel=\"prev\"]|//*[@rel=\"previous\"]",
      "prevProperty": [
        "href"
      ],
      "prevKeywordsEnabled": false,
      "buttonType": "xpath",
      "buttonRule": "//button[@class=\"page-link\"]/span[@class=\"fa fa-chevron-right\"]",
      "buttonMethod": "click",
      "buttonScrollPixels": 1000,
      "list": "",
      "listArray": [],
      "selectionStart": 31,
      "selectionPriority": "smart",
      "selectionCustom": {
        "url": "",
        "regex": "",
        "flags": "",
        "group": 0,
        "index": 0
      },
      "leadingZeros": false,
      "interval": 1,
      "base": 10,
      "baseCase": "lowercase",
      "baseDateFormat": "yyyy/mm/dd",
      "baseRoman": "latin",
      "baseCustom": "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz",
      "errorSkip": 0,
      "errorCodes": [
        "404",
        "3XX"
      ],
      "errorCodesCustom": [],
      "shuffleURLs": false,
      "shuffleLimit": 100,
      "decodeURIEnabled": false
    },

Feature request
Please provide an option (button) to temporarily disable automatic loading on a page. A temporary disable takes effect only on that page, and the auto-load feature restarts after the page is refreshed.
Thanks.

I found a site where images will not load for the appended pages. Can this be corrected?

https://www.eonline.com/news/page/3

• Device: Dell PC
• OS: Windows 10 (64-bit)
• Browser: Firefox 96.0.1 (64-bit)

I'm having trouble figuring out the rule necessary to infy-scroll on

https://camas.github.io/reddit-search/#{%22resultSize%22:100

the xpath copied from chromium inspector gave me:
//*[@id="react-root"]/div[2]/button

Environment
If you can, please fill out as much as you can about your environment.

• Device: [e.g. PC]
• OS: [e.g. Windows 10]
• Browser [ungoogled-chromium]
• Browser Version [97.0.4692.71-1.1]
• Extension Version [e.g. 0.1]

Describe the bug
when you open a post on danbooru.donmai.us, infy activate and starts to add pages without pictures or not relatade to the one you are trying to see

Website
https://danbooru.donmai.us/posts/5497931

Environment
Please fill out as much as you can about your environment.

• Device:PC
• OS: win 10
• Browser: firefox
• Browser Version: 102.0.1
• Extension Version: 0.7

Screenshots
https://imgur.com/a/pyCkZxQ

Feature request
Add an id/name to each page separator, and links to previous separator, next separator, top of page (href="#"), bottom of page (can it be done with js?)

Feature request
Hi I was wondering if you could add support for the ability to set contents of the url bar to be the latest page which has been clicked on and extended with, so that if I re-start my browser my position is at least saved and I do not need to start scrolling from the top.

Much appreciated! :)

Screenshots
If applicable, please add any screenshots that might help explain your feature request better.
However, please do not post any screenshots of the mascot character.

Describe the bug
I get the error message "Oops — Sorry! Infy had trouble loading on this page. Infy might not have permission to work on this page (for example: Browser Pages, Options Page, or New Tab Page), or you might need to try refreshing this page again."

I tried turning the addon off and on again. Reloaded the pages. Nothing helped.

URL
Everywhere

Environment
Win 10 - Firefox 87.0b8

I think this started with the v87 Beta

Describe the feature request you'd like
First of all thank you very much for making this extension, it's by far the most featured of the bunch in this category. While the options your provide in the extension are exhaustive and fulfill most use cases, sometimes I need to do something extra. Rather than cater to each individual case, maybe an option could be added for the user to execute some Javascript (could be purely cosmetic or functional) after Infy appends the next page?

Alternatively, if the above doesn't appeal to you, you can maybe add events/hooks to the script that the end user can hook into and do things with? I would appreciate that very much. Thank you.

Screenshots
N/A

Hi,
I installed the extension under Vivaldi browser, but when I set a rule and Accept it it works, but if I revisit that site nothing happens.
It seems that it doesn't save any rule I set at all.

Environment
If you can, please fill out as much as you can about your environment.

• Device: PC
• OS: Windows 10
• Browser Vivaldi
• Browser Version 4.3.2431.3 (Official Build) (64-bit)
• Extension Version 0.6 β (03/30/21)

It'd be nice to have a monospace font (or just monospace, save for a configurable option) for textarea fields expecting code, or code-like text—e.g. the Database Blacklist and other fields receiving regexes, and maybe Next Rule in the popup. I thought about a custom CSS input option similar to Tampermonkey as an larger scope alternative, but that seems somewhat overkill as the current UI is pretty great to use otherwise.

Screenshots

If applicable, please add any screenshots that might help explain your feature request better. However, please do not post any screenshots of the extension.

Hopefully a screenshot of just the content of a text box is fine for reference? I'll remove the before if requested.

Current:

Monospaced:

Feature request
Please add the option to edit the saved URLs to the the "My Saved URLs" page. It is more convenient than going through a page that matches the pattern, especially if the page matches more than one pattern.

Can be done with a little "pencil" icon that will open the existing rule configuration dialog with the URL/pattern data pre-loaded.

Thank you!

Awesome extension 👍 It would be even better if it can handle local url lists ( a list of url that begins like file:///)

Sometimes I'd like to disable saved URLs temporarily. Being able to do that from the setup page will be helpful. Currently the only available operation is "delete".

Hi everyone,

I'm currently working on the next version of Infy, Version 0.8. This is going to be a pretty big update. If anyone would like to help test this version before it's released, please watch or subscribe to this issue. I'll update the repository and this post with instructions when it's ready.

Thank you!

Describe the feature request you'd like
When I open the portable version of Chrome under the new Windows 10 or 11, the "save" under Infy Scroll Options is empty.
This kind of experience is not good, after all, it was created and preserved with great effort before.
The data saved by some other extension programs can still be valid, such as
Tab Session Manager, uBlock, Tampermonkey, SwithyOmega.
uBlock saves the settings in the cloud, and Tab session Manager saves it locally. These experiences are all very good.
In addition, is it possible for users to submit custom rules to the database?

Feature request
JS event after loading a new page.
I want to replace/remove some content/char from the page. it would be great if I can subscribe to a event and do modification.
Or can add a class on the div.

In Infy Scroll 0.3, there are a few issues that will be fixed in the next 0.4 update.

Issue 1 - Slow Appending

The append delay timeout is incorrectly placed before the append actually happens, resulting in an unnecessary delay (2 seconds is the default). This will be adjusted so that the timeout for the append delay is placed after the append actually occurs.

Issue 2 - Appending Too Early (Page 2)

Currently, Infy will append Page 2 (the first page it appends) as soon as you start scrolling. This will be changed so that it will only append Page 2 if you are near the bottom.

Issue 3 - Appending Too Early (AutoPagerize Mode)

There is a bug in Append Element (AutoPagerize Mode) where you may notice Infy will append pages earlier than it should. This is because we are not factoring in the offset, or basically the distance from the very top of the HTML document to the very top of the parent element we are appending to. This will be fixed in the next update to factor in the offset so that pages are appended only when you're near the bottom.

Issue 4 - Incorrect AutoPagerize Page Element Rule Implementation for Selector

The page element rule when using CSS Selectors is incorrectly expecting the "parent element," not the children element like the XPath rule does. This will be changed to match the XPath implementation. Any saves that use the Page Element Selector rule type will have an " > * " added to them to point to the children element in order to keep them working in Version 0.4+.

Other Notes
The scrollbar detection option is no longer relevant and needed, as the Append Threshold check (Pixels) will automatically do this check.

More minor bug fixes and performance improvements will also be coming.

It would be nice to allow user import custom AutoPagerize rule from urls, I means not only the wedata.
And custom css to fixed sites like 'javascript-hide' on https://yande.re/ is necessary for me.

因为看不懂，或许我遇到的问题是可以通过设置进行解决的？
Screenshots

谢谢

I have some questions that I think should be addressed in the documentation.

• Where are the new elements inserted in the Element mode?

• What happens if a page matches several rule patterns?
  Which of the Saved URLs, DB, and Custom DB take precedence?
  What is several patterns in the same category (above) match? Does Infy Scroll try them sequentially until it finds one that works? In that case, does order matter (if it does, consider it to be a request to allow reordering of Saved URLs)

• It seems that the custom DB and the Saved URLs address similar needs (user-defined rules).
  Are there any differences between them that would suggest using one over the other?
  Are there plans to merge them in the future?

Thank you!

In Infy Scroll 0.2, there are a few issues that will be fixed in the next 0.3 update.

Issue 1 - Aggressive Append Threshold

• The default settings (2 Pages, 85%) are much too aggressive for appending the next page. They will be reset to much more relaxed defaults so the extension will typically only append the next page if you're on the last page. This can be adjusted back for those who prefer the more aggressive settings in the Options.
• The scrollbar percentage append threshold doesn't scale as well as intended when there's a lot of pages already appended. For example, 85% works well when there's 10 pages, but not when there's 100 pages. This will be replaced with something like a pixels from bottom offset. That will always scale right no matter how many pages there are.
• Finally, the extension currently will start auto-appending the next page immediately upon loading the page. However, this may be too aggressive when you only just care about viewing the first page's screen. So it will be coded to only start appending as you start scrolling, or if there is no scrollbar present.

Issue 2 - Lack of Append Scripts Option (May fix broken images/html)
There's a new option that will be added: the ability to append each page's scripts for certain websites. The extension normally removes all scripts from each appended page, as some sites may produce bad results if they're appended (for example: the page may reload/refresh itself due to a script). However, there are use-cases where some websites may benefit from this; for example, on sites where the image thumbnails are broken, the scripts can set them. This can be enabled on a case-by-case basis per site.

Issue 3 - Saving URLs Is Unclear
After you click Yes to save a URL, the extension actually requires you to also click the Accept Button to finalize the save. This isn't mentioned anywhere, so a note will be added in the Save URL Dialog to click the Accept Button.

Issue 4 - Firefox Private Window Installation Error Dialog
Firefox is the only browser that lets you install extensions in Private/Incognito Windows, but this causes issues with Infy because it tries to open the Options Screen with a welcome dialog and expects to have access to the chrome.* api. So in the next update, an error dialog will be displayed instead of the welcome dialog explaining the problem about Private Window installations so users aren't left in the dark.

Some more minor fixes/improvements will be coming as well!

Feature request
infy doesnt work with presearch.com i tried to find their xpath but failed

Screenshots
https://imgur.com/a/MKmg1iV

I have a list of urls that I'm autopaging through, say 5 urls.
But I turn on the Auto -> Set Auto Settings, and if it's on 10 items, then it just repeats the last item over and over.
Can't the Auto option stop if it reaches the end of the list please? So that I don't have to know exactly how many items there are each time in my list. (Because the URL List doesn't show how items is in its list, maybe that can be added too).

As an aside, why not have an autoincrement on the URL list option too?
The auto increment option works OK, except that it changes the page url which I'm trying to avoid.

Describe the bug

1. Go to https://github.com/sixcious/infy-scroll/issues?q=is%3Aissue
2. Scroll down to see the extension working
3. click in Filters textbox, add a space at the end of the query string, and hit ENTER
4. Scroll down to see the extension NOT working
5. Refresh the page ( url changes to https://github.com/sixcious/infy-scroll/issues?q=is%3Aissue+ ) and it works again.

Website
https://github.com/sixcious/infy-scroll/issues?q=is%3Aissue

Environment
Please fill out as much as you can about your environment.

• Device: PC
• OS: Windows 10
• Browser: FF
• Browser Version: 105.0b9
• Extension Version: 0.7

or make "dark reader" extension enable in setting page

Describe the bug
I use ublacklist app on Chromum
after I installed this app.
from the second page of google. "Block this site" button doesn't show up
ublacklist (https://chrome.google.com/webstore/detail/ublacklist/pncfbmialoiaghdehhbnbhkkgmjanfhe)

URL

Environment
If you can, please fill out as much as you can about your environment.

• Device: [e.g. PC] PC
• OS: [e.g. Windows 10] Win10
• Browser [e.g. Chrome, Edge, Firefox] Chromum
• Browser Version [e.g. 84.0.4147.125]
• Extension Version [e.g. 0.1]

Screenshots
https://imgur.com/a/goWw5Gu

Hi!

Thank you so much for taking the time to read this sticky post. 💜

When posting or commenting on an issue, I kindly ask that you follow the below guidelines:

1. Please be nice and respectful
2. Please do not post any NSFW (Not Safe For Work) URLs or content
3. Please only open issues related to Infy Scroll (the app itself), such as bug reports or feature requests. If you need help making Infy work with a specific website, please do not open an issue here on GitHub and read the section below

Can you help me make it work with a specific website?

I really wish I could! I've spent the last several years now helping people both privately and on GitHub but there's just too many websites out there for one person to help everyone with. So, I'm going to try and make this a community effort. If you need help making Infy work on a specific website, I kindly request that you join the Discord and hopefully someone from the community can help you. Alternatively, if you need help with Selectors, XPath, or other web development topics, you could try asking your question on Stack Overflow. Lastly, I've written a Help Guide that might be able to help.

Links

• Discord (Beta)
• Stack Overflow
• Help Guide

Thank you so much! 💜

Recently Infy Scroll will always stop at 7 pages on Amazon, even when I explicitly request Auto mode. It always stopped at 7 pages by default, but used to go beyond that when I invoked Auto mode.

Also, where is the 7 page limit set for normal scroll mode? Can I change it?

Thanks a lot for this great and useful Tool!

Describe the feature request you'd like

Better default handling for the Scroll Mode (Next Link / Increment URL / ...) and Append Mode (Page / Iframe / Element / ...)

Each time as I open Infy Scroll, I do the same thing:

1. Select: "Increment URL"
2. Select Append Mode: Element

Suggestion 1: UI remembers last used settings or the Options defines the default

It would be great if Infy Scroll:

• either remember the last used Infy Scroll Mode (Next Link / Increment URL / ...) and the last used Append Mode (Page / Iframe / Element / ...)
• or if one could define those default values in the Infy Scroll Options

Suggestion 2: Saved URLs knows which mode to use

It would be great if saved URLs automatically reuses the right Infy Scroll Mode (Next Link / Increment URL / ...) and the right Append Mode (Page / Iframe / Element / ...).
It would be useful if the Saved URLs saves those modes, too.

Thanks a lot, kind regards,
Thomas

Describe the bug
On some forums, images are not resized correctly on fetched pages, and they spill over the text.
See screenshots below.

Website
https://forums.redflagdeals.com/fido-bf-special-phone-plan-20gb-lte-30-m-first-line-then-25-m-second-line-2581484/

Environment

• Device: Desktop
• OS: Win 10
• Browser: Firefox
• Browser Version: 108.0b5 64-bit (but happened on earlier versions as well)
• Extension Version: 0.7

Screenshots (Optional)

Actual:

Expected:

Issue

This is not a very big deal and I was only able to look through the first page of open issues to see if it's been reported before and I wasn't able to comb through the closed issues at all but as the title suggests at least on some sites the first page is getting duplicated upon activation of the extension specifically I've noticed it consistently with amazon.com and amazon.com/vine period I have these whitelisted and everything works as expected/ intended as far as the extension activating when and where it's supposed to which is really the important part but a minor annoyance is that I always have to scroll through the first page of results again without fail. If this is something that I'm doing or not doing then I'd love to know so I can make the necessary adjustments and I wouldn't be surprised if it were user error because this extension is flawless Imo at least invaluable so if I have to put up with this minor annoyance then so be it. I'll try to determine if it's happening elsewhere, as well.

And again, I apologize if this has already been reported or addressed elsewhere as I had barely enough time to write this issue out let alone delve into the backlog of clothes issues ETC I thought about not leaving it until I had time to do so but all that would probably result in is me not leaving it at all so I made the Judgment call that I felt was more prudent. We'll see what hindsight has to say on the matter before too long, I suppose...

PS - I went to grab a specific URL and since Vine is an invite only program, I elected to provide an Amazon.com URL example, however I found that the issue was not presenting, so maybe it is specific to Amazon Vine, after all. In which case, I would completely understand if this falls under "can't/won't fix" for no other reason than, unless you're a "Vine Voice" (and swimming in free Amazon shite while simultaneously drowning in "items awaiting reviews"), you're hard pressed to even be able to replicate it, let alone do much about it. Either way, thanks for an awesome extension! It absolutely improves my browsing experience (i.e., improves life) several fold.

URL

https://www.amazon.com/vine/vine-items?queue=encore

Version

0.7 ß

Browser

Chrome 111.0.5563.14 (Official Build) beta (64-bit)

OS

Noticed on ChromeOS but I think I've encountered it on Linux, as well(Arch-based distro Garuda Linux)

Device

PC