Git Product home page Git Product logo

bitcoinsearch-app's Introduction

BTC TECHNICAL MATERIALS SEARCH

This project will serve as a search engine for searching Bitcoin technical related materials. It will aggregate all Bitcoin related materials and performs search on those materials instead of going through thousands of unrelated Google search results.

It will classify materials according to:

  • Relevance
  • Domains
  • Tags
  • Authors

Getting started

The search engine is built using NextJS and connects to elasticsearch

The .env looks like:

API_KEY="0000000000000000000164dbb81fbea0a98f09eae1ff2a51493cb3a633523891=="
CLOUD_ID="Deployment_name:0000000000000000000365ff7535528e43b5c6793e840c2b2a0a38e1648c930f"
INDEX="index-name"

Installation

npm install
npm run dev

Contribute 🚀

We welcome contributors to the project. Open an issue or PR to help us out.

License 📗

This is MIT licensed.

bitcoinsearch-app's People

Contributors

adamjonas avatar aureleoules avatar balogunofafrica avatar emmanuel-develops avatar tvpeter avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

bitcoinsearch-app's Issues

Improving Relevance and Clarity of Search Result Snippets

In our current search system, when results are displayed, a snippet is shown by truncating the first 300 characters of the body or the AI-generated summary for that source. While this method provides a quick glimpse of the content, there are notable areas where it falls short in terms of relevance and user expectations:

  1. Lack of Contextual Snippets: Unlike other search engines which dynamically display snippets highlighting the keywords or phrases searched by the user, our system merely shows the beginning of the document. This often fails to immediately provide the most relevant section of the text to the user.
  2. Confusion Due to Summary Snippets: Our approach to prioritize summaries in search results can confuse users who, based on their experiences with other search engines, expect to see a snippet from the actual document rather than a summarized version.

Proposed Solutions

  • Contextual Snippet Generation: To address the first issue, we need to research and possibly develop a method to extract snippets that specifically highlight the search terms within the context of the document. This approach ensures that the snippet shown is directly relevant to the user's query.
    • possible inspiration: how does stract does it?
  • Clearer Summary Indication: For the second issue, while summaries are valuable for quick understanding, it's crucial to clearly indicate when a snippet is a summary rather than a direct excerpt. Alternatively, we could provide both—the relevant snippet and a button to view the summary. This dual approach would cater to both quick overviews and deeper context.
    • possible inspiration: all search engines have some extra options at the top right corner of the individual search result.

To see how we currently compare, you can review the difference in search result presentation across various platforms:

Improving how we generate and present snippets will significantly enhance user experience by making search results more relevant and understandable, aligning with common user expectations and industry standards.

Do not show side bar when there are no search results

The app preserves the state of the side bar when there is no search result.

Expected behaviour: When there is no search results, we should see the side bar. Also, if I previously searched for an query with result and I make a new query that doesn't return a result, the state of the side bar should be updated so that we do not see the author or sort by date side bar of the previous search.

See images below for reference.

Screenshot 2023-02-18 at 08 09 25

Screenshot 2023-02-18 at 08 10 29

staging feedback

based on https://bitcoinsearch-43t9q2gbz-btc-knowledge-base.vercel.app/

Screenshot 2024-05-01 at 4 45 32 PM This leaks too far down (below the fold, shorten it) Screenshot 2024-05-01 at 4 46 07 PM Github item isn't visible Screenshot 2024-05-01 at 4 47 57 PM Some spans are links and some aren't. Why is the date a link but the icon or source isn't? Remove the https:// from all urls. It makes them unnecessarily long. Screenshot 2024-05-01 at 4 50 07 PM Are the app descriptions the same font as the rest of the site? Why is the dev project in grey and the others in black? Screenshot 2024-05-01 at 4 52 32 PM Sources leak onto a second row

Here they are all two rows and but don't fill up the width of the div
Screenshot 2024-05-01 at 4 54 20 PM

Screenshot 2024-05-01 at 4 53 09 PM Gnusha should be bitcoin-dev mailing list

Results on a page

Let's bump the default results on a page from 20 to 50 to see what the performance differences are.

grouping mailing list posts

  1. Ideal is to group the mailing lists like we had before the migration from app search to elastic search
  2. A initial fix would be to label posts as original posts versus replies

Add author to main results

As a searcher I'd like to see who the author of the content is for each result to better judge its quality

test issue

Testing an issue to see if it would appear on two project boards,

  • BTC Search App, and
  • The BTC Dev Project Roadmap

Add keyboard shortcuts

  • / for highlight of the main searchbar
  • Return key to select checkboxes or author search (maybe just map to space?)
  • arrow to navigate the sidebar (though tabs work now)

The first seems like an easy addition, maybe worth discussing the other two.

remove tags

Tags do not seem to be useful at the moment. Remove the tags pills in the results and the facets on the sidebar.

User submitted content additions

As a searcher, if I don't find a source that I think should be indexed, I should be able to submit the URL for review to be added.

Not sure what the copy and interface are here, but a simple form submitted to a google sheet or something like that would be enough to get started.

Automate code linting (development)

Currently, you have to manually run each lint script (e.g. npm run eslint:fix or npm run prettier:fix) to detect and fix code styles issues. This will reduce development time and performance and can lead to different linting errors being introduced (unintentionally) in the code base.

We can mitigate this by automating the process as this will reduce the scripts the developer has to run and will ensure faster development. It will also ensure that there is uniform code style in the repository.

QA Review by Product

First off - great work so far. We've very close!

Overall

  • A Can someone please show me the link preview image and copy?
  • B Let's name our buttons to boost accessibility
    • Pasted image 20240418173416
  • C In terms of metadata, let's make sure we tackle all the usual OpenGraph/OG stuff. We did this well in bitcoindevs.xyz
    • Let's change preview link to "Search the depths of bitcoin's technical ecosystem"
  • D When returning back to the website - let's say i'm coming off the Search Page like this one... it flashes the homepage for a few seconds and then goes to the appropriate link.
  • E Remove Bitcoin Search from Secondary Product Menu
    • Pasted image 20240418172700

Homepage

  • A Make the search input box font even brighter
    • Pasted image 20240418173134
  • B Bottom of page CTA should be at least 2x smaller
    • image
  • C Remove icons in the 3 value props and pull up the text, for example
    • image
    • image
  • D The "search" button in the searchbox shouldn't be clickable if the input field is empty

Search Page

  • A Same as landing page - make the search bar much brighter
    • image
  • Let's keep the "showing x" results section
  • B Let's increase the font of the holocat section, it's smaller than any other font we're using
    • image
  • C Let's make sure that clicking a search result opens a new tab
  • D Source formatting should follow Title Case
    • "Bitcoin dev mailing list" -> "Bitcoin Dev Mailing List"
    • "BTC transcripts" -> "BTC Transcripts"
    • "LN dev mailing list" -> "LN Dev Mailing List", and the like
    • If in doubt, let's use something like this, with APA https://capitalizemytitle.com/
  • E When searching for something in the authors/ sources section, it doesn't remove non-relevant options. For example - as i search for "githu", i should only see sources with "githu" in the title string
    • Pasted image 20240418174740
  • F For some reason i'm seeing duplicates when searching for a query in the Source list
    • Pasted image 20240418174804

Footer

Feedback for Designer

  • A Some of our logos look pretty invisible in dark mode - we have to design a light mode version for them. For example, Delvingbitcoin & GitHub
    • image
  • B We need a different favicon for the product
    • Pasted image 20240418172744
  • C The second image on the Landing Page should still be an image of the search page with "lightning" being the query
    • It doesn't need to be perfect or the final version of the screen - we can change it after. Just needs to be about lightning
    • Pasted image 20240418171644
  • D This image looks unintentionally blurry. It doesn't look like you're trying to make the "newest sort" and "sources list" pop, which is the intention. Maybe darkening out or making even more fuzzy will help? Not a blocker for launch.
    • image
    • image

So close lads!

Facet selection on home page without a search term

As a user, I'd like to select all docs with facets available on the results page but without searching a specific term.

For example, I'd like to see all mailing list posts by Greg Maxwell or all docs by Pieter Wuille from stack exchange.

Maybe we can do this with a collection of the pills for the domains and authors. Let's leave tags out.

Extend Thread Grouping Functionality to All Sources

Currently, our search functionality in Bitcoin Search groups results from the same source under a single initial post, which users can expand to see the rest of the posts. This feature is beneficial as it organizes information neatly and enhances user experience by decluttering the results page.

However, this grouping is only implemented for mailing list & Stack Exchange and is not yet available for other significant sources such as DelvingBitcoin and BitcoinTalk.

You can verify this behavior by searching for anything and then filter by the specific sources.

To provide a consistent and user-friendly search experience across all our platforms, I propose we extend this thread grouping feature to include all other sources. This would mean that posts from DelvingBitcoin, BitcoinTalk, and any other selected sources would also be displayed in a grouped format.

Add github actions to enforce global code check on pull request (development)

There is currently no check for errors in the code for every pull request to main on github. This requires that the reviewer manually check for these errors or check out the PR locally to review.
Hence, this can silently introduce bugs or silent errors in the app.

This can be mitigated by setting up an automated github actions workflow to check the code for errors (linting or spelling errors and later unit or e2e tests) on every pull request to main. If there are any errors, it will be flagged, thereby narrowing the workload of the reviewer as he only has to focus on the code functionality review and other parameters.

Typeahead fragments

The typeahead has a lot of fragments in its results. Can we tune it better or not display so many?

Screen Shot 2023-03-07 at 9 23 21 PM

Mobile design aspect ratio might be too small

I compared our app vs Google search focusing on the layout and text of the result page on mobile. One thing that stood out to me was that Google's result text is much easier to read and the colors are vivid even in dark mode as opposed to ours.
The buttons showing the authors, dates, and the reference link at the top of the main highlighted link are too small. Please see the screenshots below

Just so you know, this is just my opinion and observation. This is a design choice so best to include the designer in this conversation.

Screenshot 1: Google
google-iOS

Screenshot 2: Bitcoinsearch iOS
bitcoinsearch-iOS

Screenshot 3: Bitcoinsearch Android
bitcoinsearch-android

mailing list groupings

On the backend, I've boosted original_posts from the mailing list but I also think we can do a better job ordering the groupings of threads.

Let's display the original_post as the parent of the grouping and if there are no original_posts, let's go in date order which you can pull from the created_at field OR you can pop the resource identifier off after the last slash (e.g. as in 018045 of https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-July/018045.html) and use that number to order them (lowest first).

Mailing list results repetition

The repetition of the mailing list makes the results noisey.

Let's only show the original post and show just the titles of the thread urls below it. Sort of like the mailing list itself:
Screen Shot 2023-01-23 at 3 27 24 PM

However, when someone requests an author or other facet revert replies back to the expanded result (or at least the one first one).

This can be filtered using the type field (they are labeled either "original_post" or "reply").

Show result number at the top

Screen Shot 2023-01-11 at 1 57 17 PM

Remove the "1-20" on left and default pull-down for the number of results listed on right and default it to the maximum (why not?). Replace with the total number of results (will have to check if that raw number is delivered from the API).

Add tags to results

Add tags to the pills at the bottom of each result.

Not sure on the colors here. Maybe it makes sense to differentiate.

A click on a pill should highlight that facet both in the sidebar and in the results. Clicking it again should remove the filter.

Screen Shot 2023-01-16 at 8 08 39 PM

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.