Git Product home page Git Product logo

fselect's Introduction

fselect

Find files with SQL-like queries

Crates.io build

Why use fselect?

While it doesn't tend to fully replace traditional find and ls, fselect has these nice features:

  • SQL-like (not real SQL, but highly relaxed!) grammar easily understandable by humans
  • complex queries
  • aggregate, statistics, date, and other functions
  • search within archives
  • .gitignore, .hgignore, and .dockerignore support (experimental)
  • search by width and height of images, EXIF metadata
  • search by MP3 info
  • search by extended file attributes
  • search by file hashes
  • search by MIME type
  • shortcuts to common file types
  • interactive mode
  • various output formatting (CSV, JSON, and others)

More is under way!

Installation

Latest release from source

  • Install Rust with Cargo and its dependencies to build a binary
  • Run cargo install fselect

Arch Linux

AUR package, thanks to @asm0dey

NixOS

fselect in nixpkgs, thanks to @filalex77

Other Linux

Static build with musl.

Windows 64bit

A statically precompiled binary is available at Github downloads.

Windows via winget

  • Install winget
  • Run winget install -e --id fselect.fselect

Windows via Chocolatey

Windows via Scoop

  • Install Scoop
  • Run scoop install fselect

Mac via Homebrew

  • Install brew
  • Run brew install fselect

Mac via MacPorts

  • Install MacPorts
  • Run:
    sudo port selfupdate
    sudo port install fselect
    

Usage

fselect [ARGS] COLUMN[, COLUMN...] [from ROOT[, ROOT...]] [where EXPR] [group by COLUMNS] [order by COLUMNS] [limit N] [into FORMAT]

Interactive mode

fselect -i

Documentation

More detailed description. Look at examples first.

Examples

Find temporary or config files (full path and size):

fselect size, path from /home/user where name = '*.cfg' or name = '*.tmp'

Windows users may omit the quotes:

fselect size, path from C:\Users\user where name = *.cfg or name = *.tmp

Or put all the arguments into the quotes like this:

fselect "name from /home/user/tmp where size > 0"

Search within a directory name with spaces (backticks are also supported):

fselect "name from '/home/user/dir with spaces' where size > 0"
fselect "name from `/home/user/dir with spaces` where size > 0"

Or simply escape the single quotes:

fselect name from \'/home/user/dir with spaces\' where size gt 0

Specify file size, get absolute path, and add it to the results:

cd /home/user
fselect size, abspath from ./tmp where size gt 2g
fselect fsize, abspath from ./tmp where size = 5m
fselect hsize, abspath from ./tmp where size lt 8k
fselect name, size from ./tmp where size between 5mb and 6mb

More complex query:

fselect "name from /tmp where (name = *.tmp and size = 0) or (name = *.cfg and size > 1000000)"

Aggregate functions (you can use curly braces if you want, and even combine them with the regular parentheses):

fselect "MIN(size), MAX{size}, AVG(size), SUM{size}, COUNT(*) from /home/user/Downloads"

Formatting functions:

fselect "LOWER(name), UPPER(name), LENGTH(name), YEAR(modified) from /home/user/Downloads"

Get the year of an oldest file:

fselect "MIN(YEAR(modified)) from /home/user"

Use single quotes if you need to address files with spaces:

fselect "path from '/home/user/Misc stuff' where name != 'Some file'"

Regular expressions of Rust flavor are supported:

fselect name from /home/user where path =~ '.*Rust.*'

Negate regular expressions:

fselect "name from . where path !=~ '^\./config'"

Simple globs expand automatically and work with = and != operators:

fselect name from /home/user where path = '*Rust*'

Classic LIKE:

fselect "path from /home/user where name like '%report-2018-__-__???'"

Exact match operators to search with regexps disabled:

fselect "path from /home/user where name === 'some_*_weird_*_name'"

Find files by date:

fselect path from /home/user where created = 2017-05-01
fselect path from /home/user where modified = today
fselect path from /home/user where accessed = yesterday
fselect "path from /home/user where modified = 'apr 1'"
fselect "path from /home/user where modified = 'last fri'"

Be more specific to match all files created at interval between 3PM and 4PM:

fselect path from /home/user where created = '2017-05-01 15'

And even more specific:

fselect path from /home/user where created = '2017-05-01 15:10'
fselect path from /home/user where created = '2017-05-01 15:10:30'

Date and time intervals possible (find everything updated since May 1st):

fselect path from /home/user where modified gte 2017-05-01

Default is current directory:

fselect path, size where name = '*.jpg'

Search within multiple locations:

fselect path from /home/user/oldstuff, /home/user/newstuff where name = '*.jpg'

With minimum and/or maximum depth specified (depth is a synonym for maxdepth):

fselect path from /home/user/oldstuff depth 5 where name = '*.jpg'
fselect path from /home/user/oldstuff mindepth 2 maxdepth 5, /home/user/newstuff depth 10 where name = '*.jpg'

Optionally follow symlinks:

fselect path, size from /home/user symlinks where name = '*.jpg'

Search within archives (currently only zip-archives are supported):

fselect path, size from /home/user archives where name = '*.jpg'

Or in combination:

fselect size, path from /home/user depth 5 archives symlinks where name = '*.jpg' limit 100

Enable .gitignore or .hgignore support:

fselect size, path from /home/user/projects gitignore where name = '*.cpp'
fselect size, path from /home/user/projects hgignore where name = '*.py'        

Search by image dimensions:

fselect CONCAT(width, 'x', height), path from /home/user/photos where width gte 2000 or height gte 2000

Find square images:

fselect path from /home/user/Photos where width = height

Find images with a known name part but unknown extension:

fselect path from /home/user/projects where name = "*RDS*" and width gte 1

Find old-school rap MP3 files:

fselect duration, path from /home/user/music where genre = Rap and bitrate = 320 and mp3_year lt 2000  

Shortcuts to common file extensions:

fselect path from /home/user where is_archive = true
fselect path, mime from /home/user where is_audio = 1
fselect path, mime from /home/user where is_book != false

Even simpler way of using boolean columns:

fselect path from /home/user where is_doc
fselect path from /home/user where is_image
fselect path from /home/user where is_video

Find files with dangerous permissions:

fselect mode, path from /home/user where other_write or other_exec
fselect mode, path from /home/user where other_all

Simple glob-like expressions or even regular expressions on file mode are possible:

fselect mode, path from /home/user where mode = '*rwx'
fselect mode, path from /home/user where mode =~ '.*rwx$'

Find files by owner's uid or gid:

fselect uid, gid, path from /home/user where uid != 1000 or gid != 1000

Or by owner's or group's name:

fselect user, group, path from /home/user where user = mike or group = mike

Find special files:

fselect name from /usr/bin where suid
fselect path from /tmp where is_pipe
fselect path from /tmp where is_socket

Find files with xattrs, check if particular xattr exists, or get its value:

fselect "path, has_xattrs, has_xattr(user.test), xattr(user.test) from /home/user"

Include arbitrary text as columns:

fselect "name, ' has size of ', size, ' bytes'"

Group results:

fselect "ext, count(*) from /tmp group by ext"            

Order results:

fselect path from /tmp order by size desc, name
fselect modified, fsize, path from ~ order by 1 desc, 3

Finally limit the results:

fselect name from /home/user/samples limit 5 

Format output:

fselect size, path from /home/user limit 5 into json
fselect size, path from /home/user limit 5 into csv
fselect size, path from /home/user limit 5 into html

License

MIT/Apache-2.0


Supported by JetBrains IDEA open source license

fselect's People

Contributors

4censord avatar br1ght0ne avatar chenrui333 avatar ciuncan avatar ethanthemaster avatar herbygillot avatar jhspetersson avatar johnstonskj avatar lcook avatar luxzi avatar matthieu-laurent39 avatar natrys avatar pavlus avatar qaidvoid avatar rasa avatar rleungx avatar roguelazer avatar sirikid avatar thechampagne avatar umaxfun avatar walery avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

fselect's Issues

Superfluous tab after the last column

For example,

>fselect path,name FROM . where name="*.py\-e"  | sed -e 's/\(.*\)/|\1|/'
|./test/__init__.py-e	__init__.py-e	|
|./test/ldapsync.py-e	ldapsync.py-e	|
|./test/00-import.py-e	00-import.py-e	|
|./test/tools.py-e	tools.py-e	|

I understand that tabs is the default format, but it makes little sense to print the last tab.

Space around operators matter

fselect 'path,size from . where size=0'
query: Error parsing condition, no operator found


where size = 0

that works

Expand right hand side of expression

In attempt to pick square images, I tried to run the request:

fselect path where width = height

But it didn't work (I suppose, it's not supported). Maybe that would be a good feature to have. Or even more advanced:

fselect path where width gte (height / 2)

Panic when pipe breaks on standard output

Hey thanks for making such a neat tool! While using it today I discovered that the SIGPIPE signal being passed back from CLI tools that only consume a certain amount of stdout results in a panic. Here's a minimal repro of the bug:

$ RUST_BACKTRACE=1 fselect path, size from /usr | head
/usr/sbin       12288
/usr/sbin/cupsaddsmb    14328
/usr/sbin/pppoe-discovery       22528
/usr/sbin/exim_lock     18424
/usr/sbin/update-passwd 31136
/usr/sbin/genccode      10608
/usr/sbin/netscsid      118688
/usr/sbin/split-logfile 2415
/usr/sbin/cache_dump    11
/usr/sbin/cpgr  4
thread 'main' panicked at 'failed printing to stdout: Broken pipe (os error 32)', src/libstd/io/stdio.rs:690:9
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:70
   2: std::panicking::default_hook::{{closure}}
             at src/libstd/sys_common/backtrace.rs:58
             at src/libstd/panicking.rs:200
   3: std::panicking::default_hook
             at src/libstd/panicking.rs:215
   4: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:478
   5: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:385
   6: std::panicking::begin_panic_fmt
             at src/libstd/panicking.rs:340
   7: std::io::stdio::_print
             at src/libstd/io/stdio.rs:690
             at src/libstd/io/stdio.rs:699
   8: fselect::searcher::Searcher::check_file
   9: fselect::searcher::Searcher::visit_dir
  10: fselect::searcher::Searcher::visit_dir
  11: fselect::searcher::Searcher::list_search_results
  12: fselect::main
  13: std::rt::lang_start::{{closure}}
  14: std::panicking::try::do_call
             at src/libstd/rt.rs:49
             at src/libstd/panicking.rs:297
  15: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:92
  16: std::rt::lang_start_internal
             at src/libstd/panicking.rs:276
             at src/libstd/panic.rs:388
             at src/libstd/rt.rs:48
  17: main
  18: __libc_start_main
  19: _start

Of course, using fselect path, size from /usr limit 10 is a valid workaround for this use-case- I'm just using head for a quick example. Any similar command that uses SIGPIPE has produced equivalent results. Here's an example that just uses sleep:

$ RUST_BACKTRACE=1 fselect path | sleep 1
thread 'main' panicked at 'failed printing to stdout: Broken pipe (os error 32)', src/libstd/io/stdio.rs:690:9
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:70
   2: std::panicking::default_hook::{{closure}}
             at src/libstd/sys_common/backtrace.rs:58
             at src/libstd/panicking.rs:200
   3: std::panicking::default_hook
             at src/libstd/panicking.rs:215
   4: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:478
   5: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:385
   6: std::panicking::begin_panic_fmt
             at src/libstd/panicking.rs:340
   7: std::io::stdio::_print
             at src/libstd/io/stdio.rs:690
             at src/libstd/io/stdio.rs:699
   8: fselect::searcher::Searcher::check_file
   9: fselect::searcher::Searcher::visit_dir
  10: fselect::searcher::Searcher::visit_dir
  11: fselect::searcher::Searcher::visit_dir
  12: fselect::searcher::Searcher::visit_dir
  13: fselect::searcher::Searcher::visit_dir
  14: fselect::searcher::Searcher::visit_dir
  15: fselect::searcher::Searcher::list_search_results
  16: fselect::main
  17: std::rt::lang_start::{{closure}}
  18: std::panicking::try::do_call
             at src/libstd/rt.rs:49
             at src/libstd/panicking.rs:297
  19: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:92
  20: std::rt::lang_start_internal
             at src/libstd/panicking.rs:276
             at src/libstd/panic.rs:388
             at src/libstd/rt.rs:48
  21: main
  22: __libc_start_main
  23: _start

I'm not a rust programmer but from skimming the documentation about std::io::stdio::_print and the source of the check_file function, I think it's not set up to handle SIGPIPE and terminate early. I'm running fselect version 0.6.3 installed via source using cargo.

"order by" does not work correctly

Hi fselect-developers,

thanks for this great tool !
I'm a big fan of using SQL.

But I've just tried to find huge mbox files:
fselect fsize, path from /home/user/Mail
where size gt 10mb
and path like '%tr%'
order by 1 asc

But the order by does not sort correctly.
It's not ordered by fsize as a number value.
I expected at worst that I get a list in lexicographical order.
But it's a randomly ordered list.

Could you explain or better fix this?

Thanks a lot in advance

UD

False positives when matching `name` against a regex starting with the `^` anchor

There seems to be a bug with regex matching. To reproduce:

$ mkdir fselect-test
$ cd fselect-test
$ touch some-file
$ fselect name where name =~ '^\.'
some-file

I would think that the regex ^\. would match a string whose first character is a literal .. Yet fselect thinks the name some-file matches, though it doesn’t start with a dot.

By comparison, fselect name where name =~ '\.', without a starting ^, correctly outputs no rows. fselect name where name =~ '\.$', with the end anchor $, also correctly outputs no rows.

I see that ^ is supported, as I expected, by the regular expressions library you use.

I am on macOS 10.14, using the Fish shell.

Ideas for fixing the problem

I see that the code for doing regex matches against fields is here:

fselect/src/searcher.rs

Lines 1472 to 1491 in bf0ba6d

Op::Rx => {
let regex = self.regex_cache.get(&val);
match regex {
Some(ref regex) => {
return regex.is_match(&field_value.to_string());
},
None => {
let regex = Regex::new(&val);
match regex {
Ok(ref regex) => {
self.regex_cache.insert(val.clone(), regex.clone());
return regex.is_match(&field_value.to_string());
},
_ => {
panic!("Incorrect regex expression")
}
}
}
}
},

I don’t see a problem with that part of the code, though I’m not that familiar with Rust.

Lack of documentation

I think that such a good tool needs a good manpage (or info). Also I will try to write it, if I can recall syntax.

raspberry pi 3 "error: Could not compile `libc`."

Hi,

I wanted to try out fselect on my raspberry pi 3B but it gives me the error mentioned in the title.
rust was installed just fine, this is the version:

rustc --version --verbose
rustc 1.38.0 (625451e37 2019-09-23)
binary: rustc
commit-hash: 625451e376bb2e5283fc4741caa0a3e8a2ca4d54
commit-date: 2019-09-23
host: armv7-unknown-linux-gnueabihf
release: 1.38.0
LLVM version: 9.0

and I am on "rustup default stable"

changed it to nightly as some answers on the internet suggested, did not work out for me, same issue.

Attached is the whole output of cargo install fselect

fselect.txt

thread 'main' panicked

thread 'main' panicked at 'called Result::unwrap() on an Err value: Os { code: 63, kind: Other, message: "File name too long" }', libcore/result.rs:945:5
note: Run with RUST_BACKTRACE=1 for a backtrace.

License

Please, add license to your package for me to be able to figure out rules of usage and to have ability to build package.

Trailing backslash causes search with where clause to not return any results

Hej,

This may not be a bug but it caused me some confusion. When fselecting from a folder and using a where clause, the trailing backslash of the folder causes the where clause to not return any results.
It happens both when using a quoted string after fselect and when not using quotes.
This is not the case for forward slashes:

PS C:\Users\Sebastian> fselect "name from d:\e-books\ where name = *.mobi"

PS C:\Users\Sebastian> fselect "name from d:\e-books where name = *.mobi"
Kling, Marc-Uwe - 03 Die Kaenguru-Offenbarung.mobi
The inmates are running the asylum - Alan Cooper.mobi
The Martian - Andy Weir.mobi
If Hemingway Wrote Javascript - Angus Croll.mobi
[...]

PS C:\Users\Sebastian> fselect "name from d:/e-books/ where name = *.mobi"
Kling, Marc-Uwe - 03 Die Kaenguru-Offenbarung.mobi
The inmates are running the asylum - Alan Cooper.mobi
The Martian - Andy Weir.mobi
If Hemingway Wrote Javascript - Angus Croll.mobi
[...]

(newlines for readability)

It's especially annoying because tab-complete in PowerShell automatically adds the final backslash

exclude folder

On the from, possibility to ignore folder or subfolder.

newline spam on query without selectors

version: 0.3.2
query: fselect from ~/src where name like "%rust%"
result: newline spam
expected result: no output or warning about not having any selectors specified

gitignore option only uses `.gitignore` from current directory

When using the gitignore option it only reads the .gitignore file from the current directory. It is pretty common to have multiple tiers of gitignore files effecting different part's of a repo.

It would be great if fselect could use the repo's true gitignore list. You can get the list of ignored files and folders with git status --ignored --porcelain (git v1.7.6 and later). There are also some other options.

Looking at gitignore.rs it looks like you are manually parsing the ignore file and basically implementing the logic, so shelling out to git to get the files would be a bit of a change. There is also a Rust libgit2 binding that might be an option, although that is probably even more work and a new dependency.

Thread panic due to "incorrect regex expression"

It's pretty short, so I'll leave the trace here:

% RUST_BACKTRACE=1 fselect "select path from . where path regexp \(NOT '^./code'\) and name regexp '\.conf$'

thread 'main' panicked at 'Incorrect regex expression', src/searcher.rs:1246:45
stack backtrace:
   0: <unknown>
   1: <unknown>
   2: <unknown>
   3: <unknown>
   4: <unknown>
   5: <unknown>
   6: <unknown>
   7: <unknown>
   8: <unknown>
   9: <unknown>
  10: <unknown>
  11: <unknown>
  12: <unknown>
  13: <unknown>
  14: <unknown>
  15: <unknown>
  16: __libc_start_main
  17: <unknown>

Colorize output when printing to terminal

Just like some other file-searching related tools do (fd, ripgrep, exa - to mention some), fselect could use colors too. Since fselect doesn't support command line options, there might be a simple rule, colorize the output if it prints to terminal, but disable colors if the output is redirected.

Personally, I'd like to see paths colorized according to LS_COLORS variable, that would help to visually distinguish the files by types.


Thank you for a great tool! 🔧

I really like it, but can I trust its quality?

First of all, I like what it can do and was about to install it when I noticed that there doesn't seem to be a single test.

Something that came to my mind is to test it entirely on CLI level to emulate how people would use it. Here is an example of journey level tests, which looks like this when run (it's all just bash).

Do you consider this this as much of a maintenance problem as I do?
Please don't feel pressured or criticized, I really am interested to learn about the way this tool is/was developed.

Ubuntu/Debian binary?

Hiya! I've been using this on my mac and it's simply brilliant! Any chance someone has built this for Ubuntu as well?

Matches always produce an error, with JSON output

Does JSON support work? If there's no match, it prints [], but if there is a match (either using like or = I get an error.

The same queries work fine with csv or lines.

Environment

  • macOS 10.13.6
  • fselect 0.4.3
  • Installed from homebrew
  • LANG="en_GB.UTF-8"
RUST_BACKTRACE=1 fselect path from . where name like %.yaml and is_file = true into json
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("key must be a string", line: 0, column: 0)', libcore/result.rs:945:5
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
   1: std::sys_common::backtrace::print
   2: std::panicking::default_hook::{{closure}}
   3: std::panicking::default_hook
   4: std::panicking::rust_panic_with_hook
   5: std::panicking::continue_panic_fmt
   6: rust_begin_unwind
   7: core::panicking::panic_fmt
   8: core::result::unwrap_failed
   9: fselect::searcher::Searcher::check_file
  10: fselect::searcher::Searcher::visit_dirs
  11: fselect::searcher::Searcher::list_search_results
  12: fselect::main
  13: std::rt::lang_start::{{closure}}
  14: std::panicking::try::do_call
  15: __rust_maybe_catch_panic
  16: std::rt::lang_start_internal
  17: main
[%

Case-insensitive regular expressions

Seems that regular expressions are case-sensitive by default (which is fine) but I cannot find any way to make them case-insensitive (using regex with operators like /.../i does not work).

Examples:

  • fselect name where name =~ '.*substring.*' -> works (case sensitive)
  • fselect name where name =~ '/.*substring.*/i' -> does not work (should be a case-insensitive search)

Is it a limitation or did I miss something?

Support for HEIF EXIF data

Newer iPhones default to capturing photos in *.heic files using the HEIF format. These photos still use EXIF metadata, and it looks like the libheif-rs crate can parse it out of such files. Could the EXIF support in fselect be reasonably extended to cover HEIF files as well? libheif-rs depends on the libheif C++ library and its dependencies, so this probably would have an impact on build complexity unless it was a compile-time option.

I'm trying to find anything that can sort a folder of mixed HEIF and JPEG photos by date taken, and not finding much smaller than full-blown photo management databases.

Field: is_source

How do you feel about having an attribute called is_source? It will match any files which can be classified as source files e.g. .rs. I've already implemented it, but I didn't want to waste your time with a PR if you didn't think it was useful.

select by modified not correct

fselect name,modified from . where modified le '2019-04-18 18:50:00'

it list file which modified is 2019-04-18 18:51:06

Special character issue in Powershell under windows 10

Hej! I have some special character issues when using powershell in windows 10:
image

PS C:\Users\Sebastian> fselect
FSelect utility �[33m0.6.1�[0m
Find files with SQL-like queries.
�[4;36mhttps://github.com/jhspetersson/fselect�[0m

This happens all over the place, in help messages, error messages, results, ...

Select absolute path?

How can I select the absolute path, when selecting from a relative path?
If it's not possible, could this be implemented?
e.g. fselect abspath from ../foo where ...

where clause is ignored in simple query

fselect size,modified,path from . where modified gt 2018-08-01 and name='*.txt' order by modified

This seems pretty straightforward but it looks like the everything in the where clause is ignored and the sorting is not working either. It just dumps the same output as if I did

fselect size,modified,path from .

I am using zsh and I installed from source on Ubuntu 18.10

AVG function - attempt to divide by zero

If there're no files in a directory AVG function fails with an error:

thread 'main' panicked at 'attempt to divide by zero', src/function.rs:382:20
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

Also MIN function returns -1 in such case, which looks confusing.

Accept lists of files via stdin?

Just a random thought, in a true unix tradition, be nice to be able to do things like

$ find <whatever> | fselect max(size)

This way fselect would be more composable with other tools. E.g., could do things like fd | fselect | fzf (with fd being lightning-fast when going through massive numbers of files, so you could narrow it down before filtering it with more detail). In theory, you could also pipe fselect into fselect, why not? :)

Syntax-wise, maybe the whole from clause could be then omitted, and if there's anything being piped in, it would filter those results instead of searching in the current folder.

Support count(*)

Would be nice to have built-in support for counting the resultset.

fselect 'count(*) from node_modules where name = package.json'

gets the total amount of installed packages in a JS project, for example.

Error message "could not canonicalize path" on Windows 10

Windows 10:
I just tried to select some files from my temp folder which in my case is a RAM drive mounted as drive T:. fselect always responds with the error:
T:: could not canonicalize path

This is my command line:
fselect abspath from T: where name = 'hb*'

I can add a backslash to "T:", but the error message is the same. The temp drive is created by the tool ImDisk-Toolkit, but it should not matter what type of drive should be search, right?

Support `1 hours ago`

I often need fselect path by duration, can support some duration variable,
like 1 hours ago

find can do this like

find . -mmin 60 #exactly 60 minutes old

Field: is_hidden

For dotfiles on unix, and perhaps hidden files on NTFS/Windows?

Columns are printed in separate lines

When I select multiple columns, they get printed in separate lines:

$ fselect name size from /usr depth 1
lib32
36864
etc
4096
share
20480
lib64
270336
include
69632
man
4096
bin
139264
lib
270336
local
4096
sbin
139264
src
4096

I would have expected a tabular format:

lib32   36864
etc     4096
share   20480
lib64   270336
include 69632
man     4096
bin     139264
lib     270336
local   4096
sbin    139264
src     4096

Or, at least, CSV (or even tab separated):

lib32,36864
etc,4096
share,20480
lib64,270336
include,69632
man,4096
bin,139264
lib,270336
local,4096
sbin,139264
src,4096

Some suggestions

If fselect has a way to distinguish textual files from binary ones, it can be interesting to have a is_text search field (based on content, not on file extension).
And in this case, I would like to have something like:
where line_count gt 2000 or where line_length gt 160
which, among other things, can be useful for source code files...
Of course, fselect has to be agnostic for line end markers, even accepting a mix of CRLF, CR and LF...

I see / guess that some things are hard-coded, like the list of extensions for is_xxx search fields.
Perhaps you could support a .fselectrc file.
On Windows, I suggest the same place than .gitconfig and .npmrc (among others), at root of user profile (usually at C:\User\<user name>).
Among other things, this would allow to add our favorite language extensions to is_source, this specific archive extension to is_archive, and so on.

I don't know if it is easy to add (or even possible without bloating) but supporting a duration for audio and video files would be very useful, I think.

Maybe you can add some explanations to some obscure column names. Particularly hsize and fsize (look the same for me on Windows), (s)uid and (s)gid (Unix / Mac only?) and so on (I understand is_block or is_socket and some others are probably not supported on Windows).
Of course, one can assume that if they don't understand the name, they don't need it... 😄

From clause does not allow dash `-` in dirname

It is not possible to query if the directory in the FROM clause has a dash in it, e.g.,

> fselect path from './foo-bar'
./foo: entity not found

even though this is a valid path (part) name.

gitignore

First time with fselect:

Actual

when i exec fselect size, path from . gitignore
then fselect not display files switch .gitignore
but fselect display files in .git directory

Expected

when i exec fselect size, path from . gitignore
then fselect not display files switch .gitignore
and fselect not display files in .git directory

add .git in .gitignore is not usual.

Thanks (fselect is great)
Ami44

thread 'main' panicked at 'byte index 42 is out of bounds ...', src/libcore/str/mod.rs:2017:9

I've just installed static (with musl) FSelect 0.6.4 and started it in some random directory via
RUST_BACKTRACE=1 ~/fselect name to get:

thread 'main' panicked at 'byte index 42 is out of bounds of `abandonedpsychiatricinsaneasylumroom.jpg`', src/libcore/str/mod.rs:2017:9
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:71
   2: std::panicking::default_hook::{{closure}}
             at src/libstd/sys_common/backtrace.rs:59
             at src/libstd/panicking.rs:197
   3: std::panicking::default_hook
             at src/libstd/panicking.rs:211
   4: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:474
   5: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:381
   6: rust_begin_unwind
             at src/libstd/panicking.rs:308
   7: core::panicking::panic_fmt
             at src/libcore/panicking.rs:85
   8: core::str::slice_error_fail
             at src/libcore/str/mod.rs:0
   9: core::str::traits::<impl core::slice::SliceIndex<str> for core::ops::range::RangeTo<usize>>::index::{{closure}}
  10: fselect::util::parse_filesize
  11: fselect::searcher::Searcher::get_field_value
  12: fselect::searcher::Searcher::check_file
  13: fselect::searcher::Searcher::visit_dir
  14: fselect::searcher::Searcher::visit_dir
  15: fselect::searcher::Searcher::visit_dir
  16: fselect::searcher::Searcher::list_search_results
  17: fselect::main
  18: std::rt::lang_start::{{closure}}
  19: std::panicking::try::do_call
             at src/libstd/rt.rs:49
             at src/libstd/panicking.rs:293
  20: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:85
  21: std::rt::lang_start_internal
             at src/libstd/panicking.rs:272
             at src/libstd/panic.rs:394
             at src/libstd/rt.rs:48
  22: main

Support order by clause

Example(I wish to find 10 largest files in ~/Downloads):
fselect hsize, path from ~/Downloads order by size desc limit 10

gitignore doesn't work on Windows

I am doing a quick test of fselect on Windows 10 on the top-level of an Angular project.
I started with the simple request adapted from readme:
size, path from . where name = *.ts or name = *.css or name = *.html
but it started to collect all files in node_modules.
So I changed the request to:
size, path from . gitignore where name = *.ts or name = *.css or name = *.html
but I get the same.

1087    .\node_modules\ignore\index.d.ts
2861    .\node_modules\ipaddr.js\lib\ipaddr.js.d.ts
110     .\node_modules\is-plain-object\index.d.ts
97      .\node_modules\isobject\index.d.ts

and so on.
In .gitignore, we have the classical paths:

/build/
/bin/
/node_modules

My suspicion is that you check against slashes but get backslashes out of the filesystem API calls.
If I change the request to:
size, path from ./src where name = *.ts or name = *.css or name = *.html
I get a listing of:

30      ./src\shared\file\index.ts
2750    ./src\shared\file\item.model.ts
60      ./src\shared\main\index.ts
1297    ./src\shared\main\modal.service.ts
66      ./src\shared\sharing\index.ts

with mixed slashes and backslashes, which isn't very nice... :-)
Since forward slashes are well understood overall in Windows, I suggest to normalize on them.

Thanks for this very interesting and flexible utility.

panic opening terminal

fselect panics when opening stdout when newer terminfo/ncurses is present. It’s a issue with the term crate, due to a change in the ncurses. The issue in term is now fixed but fselect will need to be updated to use the new version to fix the issue.

See the term issue for more info: Stebalien/term#81 (comment)

Thanks again for building a neat tool in Rust.

Text search in files [feature request]

Hey guys,
I've been using fselect for a while now. It is a great tool and it helps me a lot. The only thing I miss is a "contains" statement which can be used to match files which contain certain text string. It would be great if you could add this function.

Add Cargo.lock file

In the process of making changes to formulae in the Homebrew package manager, I noticed that fselect was one of a handful of Rust binary projects without a Cargo.lock file in version control. The Cargo book recommends the following (source):

If you’re building an end product, which are executable like command-line tool or an application, or a system library with crate-type of staticlib or cdylib, check Cargo.lock into git.

More information about the reasoning can be found in the "Why do binaries have Cargo.lock in version control, but not libraries?" section of the Cargo FAQ.

The Cargo.lock file helps package managers to keep builds reproducible, since cargo install simply uses the latest dependency versions unless the --locked flag is added to the command, in which case it will use the versions outlined in Cargo.lock. Without a Cargo.lock file, there's a chance that a dependency update will break the build sometime in the future, which is something I've already encountered with other Rust binary projects.

Would you please consider checking Cargo.lock into version control?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.