Git Product home page Git Product logo

grepme's Introduction

GrepMe

Grep for GroupMe

Build Status Code style: black

For something similar that runs in the browser, take a look at https://www.searchme.co/ instead.

For Users

Elevator pitch

Do you wish GroupMe had a search function? Me too. That's why I wrote GrepMe, a command line search tool for GroupMe that's featureful and easy to use.

Installing

  1. pip install grepme
  2. Create your login token on https://dev.groupme.com/applications -> Create Application None of the info there is used in grepme, you can give garbage values.
  3. Run grepme. You should be prompted for your login token.

If you type your token wrong, you can use -D and grepme will prompt you again, e.g. grepme -D some_text

Examples

  • Search case-insensitive for 'school': grepme -i school
  • Show every message from every group you're in: grepme '.*'
  • Search in a specific group: grepme --group USCCyber api
  • Filter by date: grepme -d '.*' | grep 2018
  • Search by user: grepme -u Joshua '.*'
  • Show all available groups: grepme -l
  • Show version: grepme -V
  • Show messages newer than 1 week: grepme --json '.*' | jq -r "select(.created_at > $(date -d '1 week ago' +%s)) | .text"
  • Show at most 10 messages: grepme --json '.*' | head -n 10 | jq -r '.name, .text'

See it in action

$ grepme -i swear --group 'ACM$'
Huиter Damroи: I work in the IBM building but I can meet you at Swearingen or anywhere.
Matthew Clapp: Is anybody in Swearingen?
ℬℜΔƉѰ: Can someone confirm that the Airport monitors in Swearingen have a Code-a-thon announcement?
Justin Baum: Hey does anyone know who I should email so my Carolina Card can get me into Swearingen?
^C

Configuration

GrepMe uses configparse for configuration, see its documentation for details. For example, I use the following configuration in ~/.config/grepme.toml, which searches only in 'ACM' by default:

group = "^ACM$"

Full usage

usage: grepme [-h] [-g GROUP] [-l] [-q] [-d] [-i] [-a AFTER_CONTEXT]
              [-b BEFORE_CONTEXT] [-c CONTEXT] [-u USER] [-o] [-v] [-V] [-D]
              [--clear-cache] [--color | --no-color] [--json] [-f | -F]
              regex [regex ...]

grep for groupme, version 1.3.5

positional arguments:
  regex                 text to search

optional arguments:
  -h, --help            show this help message and exit
  -g GROUP, --group GROUP
                        group to search. can be specified multiple times
  -l, --list            show all available groups and exit
  -q, --quiet           don't show who said something
  -d, --date            show the date a message was sent
  -i, --ignore-case     ignore case distinctions in both text and groups
  -a AFTER_CONTEXT, -A AFTER_CONTEXT, --after-context AFTER_CONTEXT
                        show the following n messages after a match
  -b BEFORE_CONTEXT, -B BEFORE_CONTEXT, --before-context BEFORE_CONTEXT
                        show the previous n messages before a match
  -c CONTEXT, -C CONTEXT, --context CONTEXT
                        show n messages around a match. overrides -A and -B.
  -u USER, --user USER  search by username. can be specified multiple times
  -o, --only-matching   only show text that matched, not the whole message
  -v, --reverse-matching
                        only show messages that didn't match
  -V, --version         show version
  -D, --delete-cached   delete cached credentials. useful if you mistype in
                        the inital login prompt
  --clear-cache         delete cached message. you should very rarely have to
                        use this option
  --color               always color output
  --no-color            never color output
  --json                print messages as JSON
  -f, --favorited, --liked
                        only show liked messages
  -F, --not-favorited, --not-liked
                        never show liked messages

Unicode is handled fine, see examples above.

For Developers

Testing

  1. pip install -r dev-requirements.txt
  2. test/pre-commit

If you see any test failures, it's a bug! Please let me know: https://github.com/jyn514/GrepMe/issues If you have suggestions for more tests, those are also welcome.

Contributing

Fork the repository, make some changes, make a pull request. Note: the script test/pre-commit will be run on any commit. You may want to run it automatically yourself: ln -s ../test/pre-commit .git/hooks

grepme's People

Contributors

hdamron17 avatar jyn514 avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar

grepme's Issues

Allow arbitrary users for liked/not_liked

Currently we only allow filtering by messages you've liked, not by what other people have liked.

Steps:

  • figure out how to do this with argparse (nargs=* is a disaster)
  • map usernames to user ids with get(/groups/:id)['members']

TypeError in context

If context goes out of the bounds of a chat (or possibly if there is a failed request where context should be), there is an exception where the message becomes NoneType and breaks. See following stack trace for grepme -g "Oak Pointe Elementary STEM Night" -A10 "~5" (I chose this chat because it is not likely to change much and it seems to only break on context after):

--- Oak Pointe Elementary STEM Night ---
Joshua Nelson: We're making a u turn, we'll be there in ~5 minutes
Samyu Comandur: where are y’all
Samyu Comandur: @Nathan Kilcoyne
Samyu Comandur: Traceback (most recent call last):
  File "/usr/local/opt/pyenv/versions/3.7.3/bin/grepme", line 10, in <module>
    sys.exit(main())
  File "/usr/local/opt/pyenv/versions/3.7.3/Python.framework/Versions/3.7/lib/python3.7/site-packages/grepme/__main__.py", line 38, in main
    search_all(config)
  File "/usr/local/opt/pyenv/versions/3.7.3/Python.framework/Versions/3.7/lib/python3.7/site-packages/grepme/lib.py", line 300, in search_all
    print_message(buffer, i, args)
  File "/usr/local/opt/pyenv/versions/3.7.3/Python.framework/Versions/3.7/lib/python3.7/site-packages/grepme/lib.py", line 188, in print_message
    add_attachments(message)
  File "/usr/local/opt/pyenv/versions/3.7.3/Python.framework/Versions/3.7/lib/python3.7/site-packages/grepme/lib.py", line 46, in add_attachments
    message['text'] += '\n'
TypeError: unsupported operand type(s) for +=: 'NoneType' and 'str'

Inconsistent context flags

Why are flags -a, -b, and -c synonyms for -A, -B, and -C, respectively? This doesn't seem to add anything and takes away from flags which could be reserved for other things (in grep, -a and -b are --text and --byte-offset, respectively, both of which are not needed here). In grep, -c is a synonym for --count which could be a reasonable flag for grepme.

Another flag offered by grep on ubuntu (although possibly not all) is -NUM which is a synonym for -C NUM.

Lastly, if consistency is a concern, flag long names in grep use an equals sign for parameters, e.g. --context=5 rather than --context 5.

Allow enumerating groups

Currently, to find groups available you have to open a webbrowser. Allow enumerating groups with a flag (maybe --list)

Caching

If someone searches a group twice in 2 minutes, it shouldn't take just as long the second time.

The simple way is probably to cache per-message.

Need flag to specify user to search

Currently, the only way to search private messages is to not specify a group to search (as I understand it). Would be useful to add a flag, like -u, to specify the name of a user whose dms can be searched.

Context prints messages twice

$ grepme -C 3 python
--- ACM ---
Not Clay Norris: Eww, Matlab
Imposter David Schmitt: Oh gross
Imposter David Schmitt: I thought it was python :/
Not Clay Norris: https://www.mathworks.com/help/matlab/matlab_external/call-matlab-functions-from-python.html
Imposter David Schmitt: Don't we get free Matlab?
Joshua Nelson: yes it's from the university
Not Clay Norris: Yeah, https://my.sc.edu/software/
Imposter David Schmitt: I think that remains to be seen.
Not Clay Norris: Eww, Matlab
Imposter David Schmitt: Oh gross
Imposter David Schmitt: I thought it was python :/                                           
Not Clay Norris: https://www.mathworks.com/help/matlab/matlab_external/call-matlab-functions-from-python.html                   

notice the mathworks link is printed twice

Make group explicitly regex match

Currently -g specifies the group name as a regular expression. This is inappropriate for certain circumstances so I believe it should not be the default. For example, the "ACM" group is impossible to match without first matching "No pizza 🍕ACM Leadership". Suggested alternatives are:

  1. -g matches explicitly while -G matches regex (possibly even a third flag which matches explicitly but allows wildcards, e.g. "ACM*" would match anything starting with "ACM")
  2. Add a flag, say --best-group, which matches only the group most similar to the specified group name

Improve Tests

  • Make sure API hasn't changed
  • Integration tests on real-world data. Not sure how to do authentication and make this reproducible.
  • Check CLI options are parsed correctly
  • Add past bugs to tests

Some messages are being missed?

Can't replicate this myself but I've gotten multiple bug reports:

Hmmm okay because I'm pretty sure this word was said since I did my own GroupMe analysis with the API, but I wanted to use GrepMe to get the message it was from easily and it gave me no results :/

I’ve noticed a similar issue before. Try requesting again. If it doesn’t get a frame it currently just skips and keeps going if I remember correctly. 

@hdamron17 @nglaeser

[meta] improve startup time

Partially addressed in ea2b5a7.

The main offenders are keyring and urllib3, which coincidentally are doing of all the hard work.

Times are generated with python -X importtime $(which grepme).

Import times
(bash) ~/.../python/grepme ▶️ python -X importtime /home/joshua/.local/bin/grepme
import time: self [us] | cumulative | imported package
import time:       161 |        161 | zipimport
import time:      3324 |       3324 | _frozen_importlib_external
import time:       111 |        111 |     _codecs
import time:       745 |        855 |   codecs
import time:       737 |        737 |   encodings.aliases
import time:      1081 |       2673 | encodings
import time:       383 |        383 | encodings.utf_8
import time:       232 |        232 | _signal
import time:       537 |        537 | encodings.latin_1
import time:        92 |         92 |     _abc
import time:       480 |        571 |   abc
import time:       561 |       1132 | io
import time:       236 |        236 |       _stat
import time:       330 |        565 |     stat
import time:       250 |        250 |       genericpath
import time:       392 |        641 |     posixpath
import time:      2074 |       2074 |     _collections_abc
import time:       955 |       4235 |   os
import time:       378 |        378 |   _sitebuiltins
import time:       141 |        141 |     _locale
import time:       268 |        409 |   _bootlocale
import time:       644 |        644 |   types
import time:       426 |        426 |       warnings
import time:       326 |        752 |     importlib
import time:       234 |        234 |       importlib.machinery
import time:       773 |       1007 |     importlib.abc
import time:       216 |        216 |           _operator
import time:      1556 |       1772 |         operator
import time:       271 |        271 |         keyword
import time:        86 |         86 |           _heapq
import time:       279 |        364 |         heapq
import time:       188 |        188 |         itertools
import time:       311 |        311 |         reprlib
import time:       130 |        130 |         _collections
import time:      1310 |       4342 |       collections
import time:       109 |        109 |         _functools
import time:       859 |        967 |       functools
import time:      1014 |       6322 |     contextlib
import time:       591 |       8670 |   importlib.util
import time:       331 |        331 |   sitecustomize
import time:       128 |        128 |   usercustomize
import time:      4039 |      18831 | site
import time:      1121 |       1121 |   enum
import time:       131 |        131 |     _sre
import time:       533 |        533 |       sre_constants
import time:       787 |       1319 |     sre_parse
import time:       522 |       1972 |   sre_compile
import time:       318 |        318 |   copyreg
import time:      1353 |       4763 | re
import time:       346 |        346 |       __future__
import time:       439 |        439 |         termios
import time:       444 |        883 |       getpass
import time:       303 |        303 |             collections.abc
import time:      4173 |       4476 |           configparser
import time:       233 |        233 |             time
import time:       301 |        301 |                   token
import time:      1495 |       1796 |                 tokenize
import time:       285 |       2080 |               linecache
import time:       641 |       2720 |             traceback
import time:       516 |        516 |               _weakrefset
import time:       758 |       1273 |             weakref
import time:        67 |         67 |               _string
import time:      1441 |       1507 |             string
import time:      1198 |       1198 |             threading
import time:       115 |        115 |             atexit
import time:      1967 |       9012 |           logging
import time:       458 |        458 |                 fnmatch
import time:       706 |       1163 |               glob
import time:       167 |        167 |                   errno
import time:       149 |        149 |                   zlib
import time:       445 |        445 |                     _compression
import time:       430 |        430 |                     _bz2
import time:       677 |       1551 |                   bz2
import time:       502 |        502 |                     _lzma
import time:       525 |       1026 |                   lzma
import time:       116 |        116 |                   pwd
import time:        76 |         76 |                   grp
import time:      1157 |       4240 |                 shutil
import time:       144 |        144 |                   _struct
import time:       337 |        480 |                 struct
import time:       131 |        131 |                 binascii
import time:      1670 |       6520 |               zipfile
import time:      1686 |       9369 |             entrypoints
import time:       410 |        410 |             keyring.credentials
import time:       407 |        407 |             keyring.errors
import time:       277 |        277 |             keyring.util
import time:       443 |        443 |             keyring.util.properties
import time:       899 |      11801 |           keyring.backend
import time:      1884 |       1884 |                 signal
import time:       136 |        136 |                 _posixsubprocess
import time:       108 |        108 |                 select
import time:       128 |        128 |                   math
import time:      1022 |       1150 |                 selectors
import time:      2249 |       5525 |               subprocess
import time:      6304 |      11829 |             platform
import time:     17043 |      28871 |           keyring.util.platform_
import time:       460 |        460 |           keyring.backends
import time:       523 |        523 |           keyring.backends.fail
import time:       538 |        538 |             dbus._compat
import time:      1537 |       1537 |             dbus.exceptions
import time:       230 |        230 |               _dbus_bindings
import time:       802 |       1032 |             dbus.types
import time:      2987 |       6092 |           dbus
import time:       420 |        420 |                   jeepney.auth
import time:      1419 |       1419 |                   jeepney.low_level
import time:       600 |        600 |                   jeepney.bus
import time:      3059 |       3059 |                     typing
import time:       657 |       3715 |                   jeepney.wrappers
import time:       512 |       6663 |                 jeepney
import time:       307 |       6970 |               jeepney.integrate
import time:       385 |        385 |                 _socket
import time:      3660 |       4045 |               socket
import time:       364 |        364 |               jeepney.routing
import time:       480 |        480 |               jeepney.bus_messages
import time:       745 |      12601 |             jeepney.integrate.blocking
import time:       234 |        234 |               secretstorage.defines
import time:      2363 |       2363 |                   _hashlib
import time:       160 |        160 |                     _blake2
import time:       154 |        154 |                     _sha3
import time:       554 |        867 |                   hashlib
import time:       503 |       3732 |                 hmac
import time:       340 |        340 |                     cryptography.__about__
import time:       414 |        753 |                   cryptography
import time:       437 |        437 |                         _opcode
import time:       556 |        992 |                       opcode
import time:       909 |       1901 |                     dis
import time:      3075 |       4975 |                   inspect
import time:       523 |       6251 |                 cryptography.utils
import time:       339 |      10321 |               secretstorage.dhcrypto
import time:       413 |        413 |               secretstorage.exceptions
import time:       286 |        286 |                       cryptography.hazmat
import time:       530 |        816 |                     cryptography.hazmat.primitives
import time:     11273 |      11273 |                       six
import time:       789 |        789 |                       cryptography.exceptions
import time:       270 |        270 |                         cryptography.hazmat.backends
import time:      1050 |       1319 |                       cryptography.hazmat.backends.interfaces
import time:      1514 |       1514 |                       cryptography.hazmat.primitives.ciphers.modes
import time:      1448 |      16341 |                     cryptography.hazmat.primitives.ciphers.base
import time:       431 |      17586 |                   cryptography.hazmat.primitives.ciphers
import time:       574 |        574 |                   cryptography.hazmat.primitives.ciphers.algorithms
import time:       608 |      18767 |                 secretstorage.util
import time:       487 |      19253 |               secretstorage.item
import time:      2491 |      32710 |             secretstorage.collection
import time:       443 |      45753 |           secretstorage
import time:       202 |        202 |             win32ctypes
import time:        68 |        270 |           win32ctypes.pywin32
import time:       205 |        205 |           pywintypes
import time:       899 |        899 |               _ctypes
import time:       518 |        518 |               ctypes._endian
import time:      1774 |       3191 |             ctypes
import time:       445 |       3636 |           keyring.backends._OS_X_API
import time:     55445 |     166539 |         keyring.core
import time:       383 |     166922 |       keyring
import time:       436 |     168585 |     grepme.login
import time:      1808 |       1808 |           locale
import time:      1902 |       3710 |         gettext
import time:      1421 |       5131 |       argparse
import time:       248 |        248 |         _datetime
import time:      1608 |       1855 |       datetime
import time:       340 |        340 |             _json
import time:       853 |       1192 |           json.scanner
import time:       908 |       2099 |         json.decoder
import time:      4758 |       4758 |         json.encoder
import time:       452 |       7307 |       json
import time:       637 |        637 |         certifi.core
import time:       671 |       1308 |       certifi
import time:        80 |         80 |               six.moves
import time:      1662 |       1662 |                 http
import time:       303 |        303 |                   email
import time:      1048 |       1048 |                     email.errors
import time:       476 |        476 |                         email.quoprimime
import time:       413 |        413 |                           base64
import time:       287 |        699 |                         email.base64mime
import time:       471 |        471 |                             quopri
import time:       274 |        744 |                           email.encoders
import time:       363 |       1106 |                         email.charset
import time:      1555 |       3834 |                       email.header
import time:        90 |         90 |                             _bisect
import time:       369 |        458 |                           bisect
import time:        85 |         85 |                           _random
import time:      1332 |       1874 |                         random
import time:       445 |        445 |                           urllib
import time:      1919 |       2363 |                         urllib.parse
import time:       890 |        890 |                           calendar
import time:       481 |       1371 |                         email._parseaddr
import time:      1206 |       6812 |                       email.utils
import time:       666 |      11312 |                     email._policybase
import time:      1414 |      13773 |                   email.feedparser
import time:       528 |      14604 |                 email.parser
import time:       380 |        380 |                   uu
import time:       542 |        542 |                   email._encoded_words
import time:       377 |        377 |                   email.iterators
import time:      1087 |       2385 |                 email.message
import time:      4597 |       4597 |                   _ssl
import time:      4793 |       9390 |                 ssl
import time:      1736 |      29774 |               http.client
import time:       189 |      30042 |             six.moves.http_client
import time:      2707 |      32749 |           urllib3.exceptions
import time:       611 |        611 |             urllib3.packages
import time:       765 |       1375 |           urllib3.packages.ssl_match_hostname
import time:       356 |        356 |             _queue
import time:       534 |        890 |           queue
import time:      6007 |       6007 |                     urllib3.util.selectors
import time:       724 |       6731 |                   urllib3.util.wait
import time:      1298 |       8028 |                 urllib3.util.connection
import time:       975 |        975 |                 urllib3.util.request
import time:       760 |        760 |                 urllib3.util.response
import time:      2343 |       2343 |                 urllib3.util.ssl_
import time:      1675 |       1675 |                 urllib3.util.timeout
import time:      2938 |       2938 |                 urllib3.util.retry
import time:        81 |         81 |                     six.moves.urllib
import time:        98 |        178 |                   six.moves.urllib.parse
import time:      2166 |       2343 |                 urllib3.util.url
import time:       779 |      19838 |               urllib3.util
import time:        56 |      19893 |             urllib3.util.ssl_
import time:      2901 |       2901 |             urllib3._collections
import time:      3210 |      26003 |           urllib3.connection
import time:       240 |        240 |                 _uuid
import time:       816 |       1055 |               uuid
import time:       226 |        226 |                   winreg
import time:       516 |        742 |                 mimetypes
import time:      1523 |       2264 |               urllib3.fields
import time:      1000 |       4319 |             urllib3.filepost
import time:      1360 |       5678 |           urllib3.request
import time:      4809 |       4809 |           urllib3.response
import time:      5912 |      77413 |         urllib3.connectionpool
import time:      3803 |       3803 |         urllib3.poolmanager
import time:      1315 |      82530 |       urllib3
import time:       249 |        249 |       grepme.constants
import time:       667 |      99044 |     grepme.lib
import time:       401 |     268028 |   grepme
import time:       282 |     268310 | grepme.__main__

Refactor code

  • Use filter function instead of lots of keyword arguments: The search_messages and print_message functions take an obscene amount of arguments, most of which are only used in one place. A lot of the logic is duplicated as well. It would be nice to have a filter function that abstracts those arguments. - done for a while now
  • The main function is ridiculously big, maybe split the parser into its own function? - done for a while now
  • Very little documentation
  • nested lambdas? really?
  • arg parsing is hacky: make text optional (defaulting to .*)? That would let everything go through argparse
  • find a way to combine the two big loops in main - done since 40560e3

Support images

Right now we completely ignore posted images. It would be nice to at least show the URL.

Deliminate groups

If you search in multiple chats, you see results from all of them intermingled. grepme should deliminate groups when there are multiple you're searching through. This would be fairly easy to do in the main function.

.* only highlights first line

I don't really mind this because it'd be annoying to fix and I don't think anyone cares, but it is a bug.

Example: in the following message, only "here's ... do?" will be highlighted.

Joshua Nelson: here's a fun c++ bug: what does this line of code do?

std::string foo = "ba" + 'r';

Change default group

Having it be ^ACM$ is very strange for people not in ACM. It should either be a configurable default or search the last group you've used.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.