Git Product home page Git Product logo

garywill / bigsearch Goto Github PK

View Code? Open in Web Editor NEW
187.0 5.0 14.0 703 KB

Browser extension. Definitly more than a GET/POST sender. Handily use search engines via a Flexible Tool! UI has Vimium-like feature 🌐🔍 (Pure-client. No 3rd-party server needed) 大术专搜 既专又广 手敲几下 纵横去往

License: GNU Affero General Public License v3.0

JavaScript 70.75% CSS 10.57% HTML 18.69%
metasearch metasearch-engine search-engine search-tool web-search firefox firefox-addon firefox-extension firefox-webextension search-history

bigsearch's Introduction

firefox chrome

English | 中文

Big Search

Handily use / switch any one or multiple (uninterruptedly) search engines (or search any websites).

Friendly to novices, satisfying to experts. For daily, entertainment & work.

Includes basic features that a multi-engine web search tool should have:

  • Built-in Google,DuckDuckGo,Youtube,eBay,Github etc
  • GET/POST
  • Custom engines

and so on. Besides above basic features, it can also✨:

  • 🖋️ Single-line or multi-line text
  • ⌨️ Every search step can be done by just keyboard, without mouse. It's UI has built-in Vimium-like feature. Keys can quickly call different search engines (details)
  • 🖥️ Desktop (browser extension / web app) and mobile (web app)
  • 🔎 Even works with websites that don't provide GET/POST interface (so-called In-page-Ajax-render websites) (see FAQ below)
  • 🔎 Do many operations at once by one button. Cross-engine search calling
  • 💪 User JSON programatical advanced customization (GUI easy editing also). Great flexibility in engine data & search method (details)

and so on... Go on for readme

signboard

Search everywhere 🗺️ for everything 👨‍💻. Breadth & Focus.

Table of Contents

Start to Install & Use

Ways to use:

  1. Browser extension (recommended)

  2. Web App : For demo purpose. Web app can't function fully like extension. Web app can be used on mobile browsers.

Basic Screenshots, Demo, Video

Use Extension Breadth & Focus UI style choosable
en breadth themes
Context selection search Different UI adaptions Vem (Vimium-like)
context 3ui vem
Edit search engines Special search methods, flexibility & extensibility
edit edit-add

demo_gif

Video of full introduction & instructions (Youtube) (choose the chapter you want to watch):

Features

Basic

  • 🔎 Use any search engines or inquery websites (multiple uninterruptedly) from one UI. Any that support GET/POST. (60+ built-in)
  • 🔎 User custom search engine (details) (syncronizable in extension)
  • 🔎 Use browser-installed search engines (browser extension, so can directly use those you've added into browser, Firefox only)

More

  • 🖋️ Single-line or multi-line text input & send

    Useful when e.g. want to translate article

  • 🗂️ Search engines categorization
  • 📋 Save (locally), reuse and manage search history (won't save when in incognito mode). Reusing method: double-click behavior
    • In single-line edit: replace input box text
    • In multi-line edit: insert to cursor
  • 🖱️ Quickly use selected text (single-line or multi-line) as search term
  • ⌨️ Configurable keyboard shortcuts. Default keys (may need changing manually):
    • Open popup. Firefox: Ctrl+Alt+S Chrome & others:Ctrl+Shift+S
    • Set selected text as search term. Firefox: Ctrl+Alt+D Chrome & others:Ctrl+Shift+D

    change on Firefox | change on Chrome: chrome://extensions/shortcuts

  • 🖥️ UI adapted multiply: Desktop (browser extension or web app) and mobile (web app only). Extention can show UI in: toolbar button popup UI, standalond tab, or sidebar (browser native)

Even more: What's special about it

  • ⌨️ Vem feature: Built-in Vimium-like feature. You can use keys to quickly call different search engines (details)
  • 🔎 Even works with websites that don't provide GET/POST interface (so-called In-page-Ajax-render websites) (details)
  • 🔎 Do many operations at once by one button
  • ✨ Good-looking, powerful & lightweight (details)
  • 💪 User JSON programatical advanced customization (GUI easy editing also). Both built-in & user-defined search engines data use same format (details). Great flexibility in engine data & search method:
    • 🔲 Multiple buttons for one engine: more than one operations for one engine. (Buttons inherit data from engine. Some key-value data ​​in button can override the ones in engine name)
    • 📞 Cross-engine use: Call another engine (or it's certain button action) to do the action
    • 🔏 String-formatting or character-replacing user's input, according to engine's need. And, charset specifying.
    • 🔎 Combining above two to search websites that don't provide search

Vem feature (built-in Vimium-like)

If you've used !bang or omnibox, they require you to memorize codes.

While, Big Search's Vem mode visually gives you convinience & ability to use keyboard to (uninterruptedly) click different search engines' different buttons. Press Shift+Enter to enter Vem mode

vem

Key Function
Shift+Enter Finish input and enter Vem mode (used when not in Vem mode)
Shift+Enter Opposite to above (used when in Vem mode)
I Quit Vem mode and focus inputbox
1 - 9 Switch catagory (in Vem mode)
A - Z Press corresponding button (in Vem mode)
J & K Scroll table down/up
more to be added ...

Above key bindings are not formally determined. May change in the future according to user feedback

Note if new engines added, button sequence may change, so as button key (A - Z).

Safety & Privacy

  • 🛡️ No injection to web (except when searching In-page-Ajax-render websites)
  • 🛡️ Client side functions and includes data fully. No necessary 3rd-party server. Completely no collecting user data
  • 🛡️ User custom engine GUI editor is used online (to shrink extension size). If you want extreme security, you can choose to disable communication between them, and use manual edit.

FAQ

Q: What is "In-page-Ajax-render" ?

A:

General GET/POST search flow is:

  1. You input search term and click search button
  2. Your browser loads the search term into a GET/POST query then sends it to target URL you specify
  3. Your browser opens a new tab and loads the HTML responded by the target website

Big Search's In-page-Ajax-render search is:

  1. You input search term and click search button
  2. Your browser opens target website page you specified (search term not sent)
  3. Big Search injects a simple JS to the page, filling your search term into the text box on page (according to css selector you specified). Then JS triggers clicking the submit button (also according to css selector you specified) action
  4. The website page itself then uses Ajax to fetch search results, then your browser renders them on page

The In-page-Ajax-render search feature can be used to search websites that do not provide GET/POST search interface to public (e.g. SPA websites), or those who require random token assigned at a search entrance form.

"In-page-Ajax-render" is short for "In-page-submit-form-via-Ajax-then-XHR-gets-search-result-then-render-and-show-in-page-without-web-frame-navigation"

This part is also explained in the video description

List of engines

60+ , currently. See list of build-in search engines

Similar tools & methods comparison

Comparison of open source web searching (multi-engines) tools

Experienced users may prefer intuitive horizontal comparison to quickly know what's special. (Also know about others btw)

How to edit search engines (GUI or JSON)

Online GUI engine-editing tool (link 1) (link 2)

Generally, use above GUI editor. If you don't want to use GUI (e.g. you are advanced user and want to use advanced features), please go on and read below.

Big Search users can edit search engines programatically. Following paragraphs of this section are about JSON-format engine data specification. For both:

  1. User-defined private engines
  2. Big Search built-in search engines (enginesdata.js)

Examples

Short Format

You only need to write very simple JSON and have basic HTTP knowledge on GET Method.

{
    "Google": "https://www.google.com/search?q={0}",
    "Yahoo Search": "https://search.yahoo.com/search?q={0}"
}
More about short format

Although above is simple and correct, if you're programmer it's not recommended to use display name as key. We should at least, for example:

{
    "yahoo": {
        "dname": "Yahoo Search",
        "full_url": "https://search.yahoo.com/search?q={0}"
    }
}

Full Format

Using full format you have opportunity to use all the features of this tool.

Also, mixing short formatted and full formatted elements is supported.

Full format examples
{
    "yahoo": {
        "dname": "Yahoo Search",
        "addr": "https://search.yahoo.com",
        "action": "https://search.yahoo.com/search",
        "kw_key": "q"
    },

    "google": {
        "dname": "Google",
        "addr": "https://www.google.com",
        "action": "https://www.google.com/search",
        "kw_key": "q",
        "btns": {
            "search": {
                "label": "Google Search"
            },
            "lucky": {
                "label": "I'm Feeling Lucky",
                "params": [
                    {"key":"btnI", "val": "1"}
                ]
            }
        }
    },

    "label_mbap" : { "lstr": "Cross-engine" },
    "itunesapps": {
        "dname": "iTunes Apps (Google)",
        "addr": "https://www.apple.com/itunes/charts/free-apps/",
        "btns": {
            "search_apps": {
                "label": "Search Apps",
                "kw_format": "{0} site:apple.com/*app",
                "use_other_engine": {
                    "engine": "google",
                    "btn": "search"
                }
            }
        }
    },

    "label_usaj": { "lstr": "Search In-page-Ajax-render web" },
    "chrome_ext_dev": {
        "dname": "Chrome Ext Dev Doc",
        "addr": "https://developer.chrome.com/docs/extensions/reference/",
        "action": "https://developer.chrome.com/docs/extensions/reference/",
        "ajax": ".search-box__input"
    },

    "label_many": { "lstr": "Many Engines at once" },
    "many_once" : {
        "dname": "Many Engines at once",
        "btns": {
            "gg_ddg": {
                "label": "Google + DDG",
                "use_other_engine": ["google", "duckduckgo"]
            }
        }
    }
}

This part is also explained in the video description

Specification of Editing Engines Data and Special Search Methods

Specifications

In JSON format.

Engine data in full format can contain following key-values (special search methods included):

Specifications
// # Some key-value ​​in the button can override the key-value in the engine name
{
    "engine_name": {
        "dname": "Engine display name",
        "addr": "Homepage URL",  // # optional
        "tip": "Engine tip text", // # optional
        
        "action": "form action url",
        // # For example, https://search-engine.com/search?q=input_content,
        // # The action is https://search-engine.com/search
        "kw_key": "The key name of the keyword in the query string", // # In above example, it is q
        
        "full_url": "http://www.example.com/search/{0}", // # optional. Only when GET method. Will override above two. The entire url 
        
        "method": "get/post", // # optional. default is get
        
        "charset": "UTF-8/gb2312/gb18030/big5/iso-xxxx....", // # optional. default UTF-8 (NOTICE: full_url not compatible with this
        
        "allow_referer": false, // # false(default)/true optional
        
        "kw_replace": [[" ", "-"]], // # Optional, characters that need to be replaced in the search term. In this example, replace spaces with '-'
        "kw_format": "formatted string with {0}", // # optional. {0} is like %s

        "params":[ // # Optional, the key/value other than the keyword in the query string required for this operation. POST method may need
            {"key": "key", "val": "value"},
            // # For example, https://search-engine.com/search?q=input_content&option=searchall
            // # so {key: "option", val: "searchall"},
        ],
        
        "use_other_engine": {   // # optional, use another engine to do the operation.
            "dbname": "bigsearch/user/browser",   // # Optional. Where the engines database come from (3 available databases): BigSearch build-in database (default) / User custom database / Browser-installed database
            "engine": "engine name", 
            "btn": "button name"   // # Optional. Absence will make fallback to the first button
            // # this entire object can be simplified to an engine name string
        },
        // #  (if use array here, then do many operations at once)
        
        "ajax": ......  // # Optional. Read the In-page-Ajax-render websites instructions
            
        "btns": { // # optional. If there is no such item, a "Search" button is displayed, and clicking the button will do the default action
            "Button name": {
                "label": "Button display text",
                "btn_tip": "Tip text", // # optional
                
                "..." : "..." // # here in a btn can be key-value pairs, which will override those in engine
            },
            ....
        }
    },
}

This part is also explained in the video description

Instructions for searching In-page-Ajax-render websites

Some websites doesn't provide GET/POST search (or not respond results in HTML). Visitor need to open their page then input, they then show results via Ajax on their page. (see FAQ)

Big Search can deal with such In-page-Ajax-render websites (by injecting js to your browser web, to automate inputting, clicking, form submitting). And easy to configure:

Eg 1: Specify the css selector of input box. It will automatically input search term and trigger pressing Enter event.

"ajax": "#search-box-input"

Eg 2: Delay 2s -> Input -> Delay 1s -> Trigger clicking button event

"ajax": [2000, "#search-box-input", 1000, "#submit-button"]

This part is also explained in the video description

For Developers

We encourage user to submit their customized search engines data to source code after they format as JSON (submit to file enginesdata.js)

What's Next Step

This tool still can be improved to do something more:

  • Omnibox
  • Be compatible with others e.g. OpenSearch. Add or convert by 1-click

Technical Diagram

tech_diagram

Third-party libraries and components used

Fast and lightweight: NO heavy framework or library dependencies. Although it has UI themes of both simplicify & fancy, all main features and UI are pure JS + CSS.

  • LZ-UTF8.js (38kB not minified. Data compression library, only for user-custom engines sync)

    Copyright (c) 2021, Rotem Dan
    Released under the MIT license.
    
  • Foggy Lake (37kB webp. Default background photo)

    by Quang Nguyen Vinh

  • Unicons icon (svg)

    Unicons by Iconscout

Globalization

Because there are currently only 2 languages supported, no framework used yet, only a simple function to implement multi-language.

For strings that you want it to be multi-language (it's okay to leave it English only, also), use JS function i18n(), whose input parameters can be:

  • An array of strings (for only Chinese and English 2 languages). [0] is Chinese, [1] is English
  • An Object like {zh: "This is Chinese, en: "This is English", fr: "This is French"}

It will return a string of the corresponding language

If you want to add a search engine that only targets users in a certain language, you can use visible_lang to make it visible only to a certain language.

History, Code Status, and License

A part of the code of this tool can be traced back to around 2008. Web app published to be usable to the public on Internet in 2015. Not until early 2020s, I found webExtension and JS standarized, so made this tool a browser extension and setup a Github repo for it. (Yes it's developed slowly and gently, not full-time.)

Some code has been ever (and may still getting) refactored. Although there's still some some aged parts, this tool has always been modern, handy & useful.

Code is licensed under AGPL.

Change log

bigsearch's People

Contributors

flyingicedragon avatar garywill avatar gitoffthelawn avatar yang-kyle avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

bigsearch's Issues

对话搜索无法触发输入和搜索

尝试添加kimi,打开新网页后,无法正常触发输入和搜索。在用的油猴脚本使用粘贴方式可以正常触发。

站点:https://kimi.moonshot.cn/

以下配置无法触发输入和搜索

"kimi": {
    "dname": "kimi",
    "addr": "https://kimi.moonshot.cn",
    "ajax": [
      3000,
      "[data-testid='msh-chatinput-editor']",
      500,
      "[data-testid='msh-chatinput-send-button']"
    ],
    "action": "https://kimi.moonshot.cn/chat"
  },

使用js脚本可以完成输入和搜索

  function saveTextToClipboard(text) {
      navigator.clipboard.writeText(text)
          .then(() => {
              console.log('文本已成功保存到剪贴板');
          })
          .catch((error) => {
              console.error('保存文本到剪贴板时出现错误:', error);
          });
  }

  function triggerPasteEvent(element, value) {
      const event = new ClipboardEvent('paste', {
          dataType: 'text/plain',
          clipboardData: new DataTransfer(),
          bubbles: true,
          cancelable: true,
      });

      event.clipboardData.setData('text/plain', value);

      element.dispatchEvent(event);
  }

  var arxiv_paper = urlParams.get('arxiv');
  var editorElement = document.querySelector('[data-testid="msh-chatinput-editor"]');

  if (!editorElement && retryCount < maxRetries) {
      retryCount++;
      setTimeout(getElementWithRetry, retryInterval);
  } else if (editorElement) {
      editorElement.focus();
      saveTextToClipboard(arxiv_paper);

      // 获取剪贴板中的值
      navigator.clipboard.readText().then((clipboardValue) => {
          triggerPasteEvent(document.querySelector('[data-testid="msh-chatinput-editor"]'), clipboardValue);
          setTimeout(function () {
              document.querySelector('button[data-testid="msh-chatinput-send-button"]').click();
              
          }, 500);
      });

  }

同样的bing copilot也无法完成输入和搜索: https://www.bing.com/search?form=MY0291&OCID=MY0291&q=Bing+AI&showconv=1

"bing_chat": {
    "dname": "bing_chat",
    "addr": "https://www.bing.com/search?form=MY0291&OCID=MY0291&q=Bing+AI&showconv=1",
    "ajax": [
      5000,
      "#searchbox",
      500,
      "[aria-label='Submit']"
    ],
    "action": "https://www.bing.com/search?q=Bing+AI&showconv=1"
  }

谢谢!

能否添加自动开启/禁用vem的选项?

既然nvm模式是这个扩展的特色, 为什么不能在打开扩展的时候自动开启vem? 还需要再按快捷键切换状态.
为了更便捷点,能否添加一个开关选项, 可以让用户在弹出界面决定是否开启vem?

打开拓展自动选中文本

目前的大术专搜是打开拓展或者用快捷键唤醒后,光标在文本的最后
image

打开拓展后,自动选中文本应该会更加人性化(图-火狐浏览器)
image

一是打开后可以再次进行搜索,与原有的功能不冲突
二是粘贴新的文本也更加方便(进行新搜索的使用频率应该更高,若是想在原有基础上更改只需要用键盘方向键移动一下光标,或者鼠标点击一下,输入历史的功能也能恢复搜索过的文本)

综上,希望添加打开拓展自动选中文本的功能

Allow the user to create his own catagories

The extensions is very helpful but, if we add a long list of search engines, it will be hard to use.
It would be good if it is possible to add categories as well as shown here:
image

Base64 encode search term

Slack requires that search terms be Base64 encoded. Is this a feature you would be interested in adding?

Web of Science 失效

Web of Science 页面变化,原本的搜索方式失效。目前只想到 ajex 的办法解决,体验很差。不仅需要添加等待,而且需要输入检索式。

"webofscience": {
    "dname": "Web of Science",
    "addr": "https://webofscience.clarivate.cn",
    "action": "https://webofscience.clarivate.cn/wos/alldb/advanced-search",
    "ajax": [
        2000,
        "#advancedSearchInputArea"
    ]
}

Features request

Hi there!

  1. No Icons.
  2. No multiple links are open in one click. How can I select 5 or 7 links and open them at one time???
  3. No Russian language.
  4. No folders and SubFolders (infinity nesting).
  5. The visual design sucks 🤢🤮
  6. Can't get how to use it for this site?

https://andisearch.com

The product is still raw.

Right now I'm using Trufflepiggy and Selection Search.

Take care 🖤

Bad UI on Ubuntu Chrome x64

Hi Gary,

Congrats for putting this amazing extension finally on the play store.

I just installed it in Ubuntu, and I noticed a wrong UI display, so basically you need to scroll left/right to see the whole content of the pop-up:

ext1

ext2

PS: for the sake of completeness, I tested the FireFox version even though I don't use it, and it is working fine there (the UI part at least)

Coridally,

Search multiple engines at once

Is it possible to set this so that I can choose and search multiple engines at once rather than having to click search for each engine separately?

Thanks

Not a Bug! Alternative but as a app for Windows

Hello there! Sorry for asking here. Does anybody know a BigSearch like but as a Win installable/portable application? If Open Source even better.

I only know one alternative but as a Windows app but the developer or the one who bought the project did a 🐀 move. If interested read the reviews to know why: SnipDo.

Hope someone can help me. Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.