Git Product home page Git Product logo

noogle's Introduction

Website GitHub top language GitHub Workflow Status

Noogle is a simple Nix API search engine. It lets you search nix functions.

Current Features

  • Beginners-friendly search for nix and nix-related functions
    • Render documentation comments optimized for readability
    • Search optimized for English language and nix function-names.
  • Filter by type signatures.
    • Function types are parsed and interpreted.
  • Noogle also knows stuff that is not (yet) officially documented.
    • Types of builtins. (including builtins.derivation)
    • Can be extended via markdown contributions to noogle.
    • Always shows the latest docs based on the main branch of nixpkgs.
  • Outputs Pre-rendered static html that is indexable by other search engines.
  • Wasm based performant search.

Available data

There are the following subsets of nix and nixpkgs functions available.

Recursively indexed:

  • lib
  • pkgs.rustPackages

Recursively means all deeply nested sub-attributes.

Normally indexed:

  • builtins
  • pkgs.stdenv
  • pkgs.dockerTools
  • pkgs.writers
  • pkgs.pythonPackages
  • pkgs.haskell.lib
  • pkgs.haskell.lib

Off the tree functions

Some function are not part of the evaluation value of default.nix in nixpkgs. They must be imported individually.

  • make-disk-image

All Indexing is done via the pasta module. PRs welcome!

Contribute

Contributions are very welcome just file a PR or issue. Indexed data can be added very easily in ./pasta/src/eval.nix

Note: Indexed data must evaluate!

Build this page

nix build .#ui

This page generates static html pages. One page per api function.

It automatically includes meta tags for other search engines like google or bing.

Searching within the page is done via pagefind which is only available in the production build.

Develop

nix develop

This command creates the node_modules folder with all needed dependencies based on dream2nix.

noogle's People

Contributors

domenkozar avatar github-actions[bot] avatar hsjobeki avatar phaer avatar renovate[bot] avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

noogle's Issues

missing lib/strings-with-deps

https://github.com/NixOS/nixpkgs/blob/master/lib/strings-with-deps.nix

I think the types are

type TextEntryName = string;
type TextEntry = { deps :: [TextEntryName]; text :: string; supportsDryActivation? :: boolean; };
type TextEntries = { "$textEntryName" :: TextEntry };

textClosureList :: TextEntries -> (TextEntry | TextEntryName) -> TextEntries;
textClosureMap :: (TextEntry -> string) -> TextEntries -> [TextEntryName] -> string;
noDepEntry :: string -> TextEntry;
fullDepEntry :: string -> [TextEntryName] -> TextEntry;
packEntry :: [TextEntryName] -> TextEntry;
stringAfter :: [TextEntryName] -> string -> TextEntry;

Not sure how to denote types and that supportsDryActivation is an optional property

FR: search by name?

Being new to nix, I'm so grateful to have a way to search by function signature and have already found this very useful. Thank you!

I was surprised that searching by fully qualified function name didn't work, for example I wanted to look at builtins.elem. It shows up if I search for a -> [b] -> bool, but searching for elem gives me 7 pages of results and builtins.elem gives 0 results. While I don't haskell, if keeping similarity to Hoogle is of concern, searching for elem seems to yield expected results.

If there's the possibility of adding a feature to search by function name in addition to signature I think it would be helpful. Thank again for providing an already very useful site!

Builtin functions have invalid source code links

What happened
Clicking a "browse source code" link on a builtin function's documentation opens an invalid link in the nixpkgs repo.

Steps to Reproduce
Steps to reproduce the behavior:

  1. Go to a builtin function's documentation.
  2. Click on "Browse source code".
  3. Note that the opened page does not exist.

Expected behavior
Either

  • No link should be shown in the documentation
  • The link should open the C++ code implementing the builtin
  • The link should open the nix code where the builtin is stubbed & documented
Screenshots

image
image

document fetch*

the fetch functions are in pkgs, not lib, and they're a bit mysterious :-)

use buttons instead of drop down for search filter selection

Is your feature request related to a problem? Please describe.

image

With the new design, it takes quite some clicks to filter the functions for the desired input/output combination.

I really liked that the old design was really fast to handle when filtering for input/output type.

Describe the solution you'd like

Maybe noogle could use buttons on the horizontal (like the blue "any" element) instead of a drop down menu?
That would safe 2 clicks when selecting input and output (one on each drop down menu line to open the drop down options)

It would also be nice to enable selection of multiple types for input/output, and using buttons instead of drop down could be a first step in this direction. ;)

Describe alternatives you've considered

Moving the input/output selection from behind the "settings" symbol in the search bar to a place where it is directly accessible could be an alternative/additional solution to save a click.

feature request: noogle cli

Describe the solution you'd like

I would love to be able to interact with Noogle as a command-line tool so I don't have to switch to my browser and open the website.

Describe alternatives you've considered

A couple CLI tools like this exist already, but I far prefer the information and UI of Noogle in the browser.

Additional context

This request might be totally out of scope since an interactive CLI is not a website.

deprecated notice

I'd love deprecated items to show up as such. For example, It seems that this one is deprecated, because the source file it's implemented in is lib/deprecated.nix.

Add nixos modules support

As nixos modules have an interface like declaration we could display interfaces for nixos modules like „drv-parts“ which is an interface for mkDerivation and derivation

Need to fork nixpkgs and make mkOptionType noogle compatible

Add trivial builders

The trivial builders are easy to be added. But the comments there are not complete nor make sense to render as markdown.

  1. Fork nixpkgs and fix the doc-strings
  2. Create dataset for noogle
  3. Create PR in nixpkgs

Add deeplinks

Currently you can't bookmark or share a page with someone because the current information are not stored in the URL.

Edit: apparently there is a share button next to search bar which I missed first because of the not so great contrast. It would be nice if there would be no need for such button and the URL is just always updated.

lib.pipe shows wrong documentation

What happened

entry for lib.pipe shows documentation of lib.lists.unique

Steps to Reproduce

https://noogle.dev/f/lib/pipe

Screenshots

image

Expected behavior
A clear and concise description of what you expected to happen.

Desktop (please complete the following information):

  • OS: x86_64-linux
  • Browser chrome
  • Version 122.0.6261.111

Add mkDerivation

mkDerivation is only one of the important stdenv

It would be super nice to display them here

term argument in URL ignored if not surrounded by %22

What happened
I tried to set up a shortcut to prompt for text and then open the URL https://noogle.dev/?term=<text I entered> but found it didn't work as expected. By trial and error, I found I needed to surround the search text with URL-encoded double quotes %22 to make it work.

Steps to Reproduce
Steps to reproduce the behavior:

  1. Enter https://noogle.dev/?term=getattrs in browser location bar and press enter
  2. Noogle loads with empty search box and no results, the same as simply opening https://noogle.dev
  3. Enter https://noogle.dev/?term=%22getattrs%22 in browser location bar and press enter
  4. Noogle correctly shows search results for getattrs

Expected behavior
Noogle should perform the search given in the URL in both cases.

Desktop (please complete the following information):

  • OS: Linux
  • Browser Chromium
  • Version 118

Display more than 8 results

When only displaying 8 results per page, you can't really search through all your search results to find exactly what you want. I would propose to increase the default and maybe add a view which just display everything.
Also the pager is not nice to use. It only displays the first, last, current and current-1 and current+1. If you want to navigate from the 7th to the 20th page you need to click a lot.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

  • fix(deps): update all dependencies (@emotion/react, @mui/icons-material, @mui/material, @mui/material-nextjs, @next/mdx, @types/node, @types/react, @types/react-dom, @types/react-highlight, @types/seedrandom, @typescript-eslint/eslint-plugin, @typescript-eslint/parser, cachix/cachix-action, cachix/install-nix-action, clap, eslint, eslint-config-next, expect-test, highlight.js, next, react-markdown, react-minisearch, regex, remark-rehype, serde, serde_json, serde_with, serde_yaml, usehooks-ts)
  • chore(deps): lock file maintenance

Detected dependencies

cargo
codemod/Cargo.toml
  • regex 1.9.5
  • rnix 0.11.0
  • rowan *
  • textwrap 0.16.0
  • walkdir 2.4.0
  • expect-test 1.4.0
indexer/Cargo.toml
  • structopt 0.3
  • serde_json 1.0.107
  • serde 1.0.188
pesto/Cargo.toml
  • rnix 0.11.0
  • rowan *
  • regex 1.9.5
  • textwrap 0.16.0
  • walkdir 2.4.0
  • clap 4.4.4
  • serde 1.0
  • serde_json 1.0
  • expect-test 1.4.0
  • serde_with 3.4.0
  • serde_yaml 0.9.27
  • comrak 0.20.0
github-actions
.github/workflows/main.yml
  • actions/checkout v4
  • cachix/install-nix-action v23
  • cachix/cachix-action v12
  • cloudflare/pages-action 1
  • cloudflare/pages-action 1
  • peter-evans/create-or-update-comment v3
npm
scripts/package.json
website/package.json
  • @emotion/cache ^11.11.0
  • @emotion/react ^11.10.5
  • @emotion/styled ^11.10.5
  • @fontsource/roboto ^5.0.0
  • @jsdevtools/rehype-toc ^3.0.2
  • @mdx-js/loader ^3.0.0
  • @mdx-js/react ^3.0.0
  • @mui/icons-material ^5.10.9
  • @mui/material ^5.10.13
  • @mui/material-nextjs ^5.15.0
  • @next/mdx ^14.0.4
  • @types/mdx ^2.0.10
  • @vcarl/remark-headings ^0.1.0
  • hast ^1.0.0
  • hast-util-to-string ^3.0.0
  • highlight.js ^11.7.0
  • minisearch ^6.3.0
  • next ^14.0.3
  • next-mdx-remote ^4.4.1
  • next-sitemap ^4.2.3
  • notistack ^3.0.0
  • pagefind ^1.0.4
  • parse5 ^7.1.2
  • react ^18.2.0
  • react-dom ^18.2.0
  • react-highlight ^0.15.0
  • react-hot-toast ^2.4.1
  • react-mark.js ^9.0.7
  • react-markdown ^9.0.0
  • react-minisearch ^6.0.2
  • rehype ^13.0.1
  • rehype-autolink-headings ^7.1.0
  • rehype-highlight ^7.0.0
  • rehype-slug ^6.0.0
  • rehype-stringify ^10.0.0
  • remark-frontmatter ^5.0.0
  • remark-gfm ^4.0.0
  • remark-heading-id ^1.0.1
  • remark-parse ^11.0.0
  • remark-rehype ^11.0.0
  • remark-stringify ^11.0.0
  • remark-unlink ^5.0.0
  • seedrandom ^3.0.5
  • unified ^11.0.4
  • unist-util-visit ^5.0.0
  • usehooks-ts ^2.9.1
  • @types/node 18.18.1
  • @types/react ^18.0.25
  • @types/react-dom 18.2.8
  • @types/react-highlight ^0.12.5
  • @types/seedrandom ^3.0.4
  • @typescript-eslint/eslint-plugin ^6.7.3
  • @typescript-eslint/parser ^6.7.3
  • eslint 8.50.0
  • eslint-config-next ^14.0.3

  • Check this box to trigger a request for Renovate to run again on this repository

Add nixos.lib

nixos also has some important lib functions, that are not yet documented

  • nixosOptionsDoc
  • systemd-lib
    • unitConfig
    • makeUnit
    • makeJobScript
    • ...etc.
  • make-iso9660Image
  • ...etc.

Building system images is not very well described and requires a lot of research.

FR: Result types

If your Feature is adding / extending more functions

Something I liked about the old site was it had a type of the results in the description. It was nice to see what the function or such required to be called from the site. But I would like something similar in the new site.

Changing to front page or function flashes website

What happened
Navigating to the front page or pkgs.fetchFromGitHub flashes website.

Steps to Reproduce
Steps to reproduce the behavior:

See video

Expected behavior
Background should stay in the theme color.

Screenshots

chrome_Td3JsrGsTs.mp4

Desktop (please complete the following information):

  • OS: Windows
  • Browser Chrome
  • Version 120.0.6099.200

Ability to search for drv builder functions?

Is your feature request related to a problem? Please describe.
I'd like a search functions that build derivations, like the functions that return derivation for python, rust, vim-plugins, ...
(the trivial builders are already indexed so that's good, but I think we can do better!)

This would potentially need special support from nixpkgs, to have (in the documentation?) structured metadata about what that function helps to build, like Rust Package, Python Package, Python Environment, Vim Plugin, Nvim Config.
Might need to add more usability & attrset option documentation to those as well.

Relevant RFC on docstring format: NixOS/rfcs#145

Describe the solution you'd like
Ability to filter by ecosystem (rust, python, java, ..)
Ability to filter by category of well-known things to build (config, env, pkg/module, plugin, misc, ..)

See usage documentation (& attribute docs?) or at least link to documentation (potentially generated from structured doc metadata?)
...

Describe alternatives you've considered
Grepping nixpkgs, looking at existing packages

Standartization of output marker for Example section

Is your feature request related to a problem? Please describe.
It is not clear what is format to distinguish part of Nix expression and output of evaluation. See question in NixOS/nixpkgs#225496 (comment)

From lib.attrsets.foldAttrs:

Example

foldAttrs (item: acc: [item] ++ acc) [] [{ a = 2; } { a = 3; }]
=> { a = [ 2 3 ]; }

From lib.attrsets.foldlAttrs

Example

foldlAttrs
  (acc: name: value: {
    sum = acc.sum + value;
    names = acc.names ++ [name];
  })
  { sum = 0; names = []; }
  {
    foo = 1;
    bar = 10;
  }
->
  {
    sum = 11;
    names = ["bar" "foo"];
  }

foldlAttrs
  (throw "function not needed")
  123
  {};
->
  123

Notice that in first example it is => (looks like double line arrow pointing to right) that separates expression to run from expected output. While second one uses -> (looks like single line arrow to right).

Describe the solution you'd like
Before growing coverage of comment-based documentation it worth to agree on some format.

I have nothing specific, but something popular might be more preferred. E.g. >>> prefix for what needs to be evaluated as Nix expression.

E.g. Haddoc/doctest uses prefix

-- | Compute Fibonacci numbers
--
-- Examples:
--
-- >>> fib 10
-- 55

Same in Python doctest

 """Compute and return the sum of two numbers.

 Usage examples:
 >>> add(4.0, 2.0)
 6.0
 >>> add(4, 2)
 6.0
 """

Inject missing documentation.

As people request more and more functions that are not documented or pieces of documentation missing.

I'd like to use gray-matter to extend existing documentation for functions. Or inject missing documentation. Until PRs about that would get accepted. Also, i'd like to keep controversial content locally.

Machine-readable format

Hello!

I am one of the moderators of the nixos discord (https://discord.gg/RbvHtGa), and we thought it would be nice to integrate noogle into a bot, so people can /search <function_name>

From what I get from the readme, noogle is build as a static webpage, so could it be possible to get a machine-readadable format (e.g. json) that we can embed into the bot?

Maybe this already exists, or could be possible to implement?

autofocus input on landing page

Is your feature request related to a problem? Please describe.
When visiting noogle.dev, I'd like the main search input to be autofocused, so I can start searching immediately.

Describe the solution you'd like
Use the autofocus HTML attribute.

Describe alternatives you've considered
Leave as is, which is still fine.

Additional context
None.

Some short forms are wrong for lib functions

What happened
lib.generators.toJSON's documentation says its short form is lib.toJSON, but they actually have different signatures
The same thing happens for functions like lib.modules.importJSON

Steps to Reproduce
Steps to reproduce the behavior:

  1. search for lib.generators.toJSON
  2. profit

Expected behavior
lib.generators.toJSON should not show a short form of lib.toJSON

Screenshots

Desktop (please complete the following information):

  • OS: x86_64 NixOS
  • Browser: Firefox
  • Version: 112.0.2

Handle mkOverride functions

If your Feature is adding / extending more functions
https://github.com/hsjobeki/nixpkgs/blob/migrate-doc-comments/lib/modules.nix#L1086C1-L1093C1

Is your feature request related to a problem? Please describe.
Functions that set priority (e.g. mkOptionDefault and`mkVMOverride) are not documented on noogle. Leading to empty pages when searched.

Describe the solution you'd like
One solution could be to create a category that manually describe the function, and append at the bottom, under a "Priority" title instead of "Detected Type", the priority it will set when used.

Describe alternatives you've considered
N/A

Additional context
swappy-20240124_123258

markdown code blocks in description don't keep indentation

What happened
The indentation of markdown code blocks seems to get lost, here are two examples:

Expected behavior
Keep the indentation as in the doc comment.

Screenshots

image
https://noogle.dev/f/lib/packagesFromDirectoryRecursive
https://github.com/NixOS/nixpkgs/blob/master/lib/filesystem.nix#L174-L206

image
https://noogle.dev/f/lib/fileset/toSource
https://github.com/NixOS/nixpkgs/blob/master/lib/fileset/default.nix#L275-L280

Desktop (please complete the following information):

  • OS: NixOS
  • Browser Chromium
  • Version 121.0.6167.85
  • Screen resolution (optional)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.