schell / steeloverseer Goto Github PK
View Code? Open in Web Editor NEWA file watcher and development tool.
License: BSD 3-Clause "New" or "Revised" License
A file watcher and development tool.
License: BSD 3-Clause "New" or "Revised" License
$ g++ 000.cpp && diff output.txt <(./a.out < input.txt)
1c1
< 9
\ No newline at end of file
---
> Hello
$ sos -p 000.cpp -c "g++ 000.cpp && diff output.txt <(./a.out < input.txt)"
Hit Ctrl+C to quit.
Modified: 000.cpp
[1/1] g++ 000.cpp && diff output.txt <(./a.out < input.txt)
/bin/sh: -c: line 0: syntax error near unexpected token `('
/bin/sh: -c: line 0: `g++ 000.cpp && diff output.txt <(./a.out < input.txt)'
Failure ✗ (1)
It would be nice if there were an option to run the commands on first start before any of the files change. This is useful when starting up a server.
On 85d4008, e.g.:
$ echo "cabal test --test-options='1234'\\'' 567'"
cabal test --test-options='1234'\'' 567'
$ sos . -c "cabal test --test-options='1234'\\'' 567'"
sos: Error parsing command 'cabal test --test-options='1234'\'' 567''
(1)$
I can’t quite get how this happens, but look at this screenshot:
For all following modifications of that .hs
file, nothing happens: sos
doesn’t output interrupted
or anything like that. And the commands are not run.
In htop
it has 8 instances of Cabal as its child processes, but they are not really Cabal, only named so (look at their mem stats):
Getting closer to stackage release, but still some errors:
After running a command, a zombie process stays.
32430 robin 20 0 127208 4812 3068 S 0.0 0.0 0:00.24 `- bash
23581 robin 20 0 1.001t 34584 30164 S 0.0 0.3 0:00.09 `- sos
23661 robin 20 0 0 0 0 Z 0.0 0.0 0:00.04 `- cabal
24087 robin 20 0 0 0 0 Z 0.0 0.0 0:00.04 `- cabal
24213 robin 20 0 0 0 0 Z 0.0 0.0 0:00.05 `- cabal
im using Steel Overseer 2.0.1.0
and the command sos -p '.*\.hs' -c 'cabal run'
While editing a file, emacs creates a lock file (symbolic link) of the form .#filename → [email protected]:timestamp
(not sure if it's a real timestamp or not).
While checking file status on files in the target directory, sos
crashes with the following message:
sos: /path/to/file: getFileStatus: does not exist (No such file or directory)
Steps to reproduce:
Create an invalid link.
$ cd /tmp
$ ln -s /invalid/target link
Run sos
$ sos .
sos: /tmp/link: getFileStatus: does not exist (No such file or directory)
On Hackage https://hackage.haskell.org/package/steeloverseer-2.1.0.0 the version bounds for Aeson are
aeson (>=0.8 && <1.6)
but this commit 70d3006 provides support for Aeson 2.
Hackage is probably up-to-date; the latest revision has the same date as the commit above:
Revision 1 made by Bodigrim at 2022-04-16T16:57:02Z
Can the version bound on Hackage be fixed somehow? Would checking in a Cabal file fix this?
Thank you
I’m using the latest HEAD, 85d4008.
I’m first modifying Main.hs 3 times in Emacs, and then switch to some other terminal and do git checkout src/
and:
$ cd some-project/
$ sos src/ -p '\.hs$' -e '/\.#' -e '/flycheck_' -c : Hit Ctrl+C to quit.
Modified: src/Main.hs
[1/1] :
Success ✓
Modified: src/Main.hs
[1/1] :
Success ✓
Modified: src/Main.hs
[1/1] :
Success ✓
# now I’ll do git checkout src/
# nothing happened
^C
Right now sos
doesn't deal very well with commands that read on stdin (like some stack exec my-thing
at the end of a list of commands).
Somehow, these interactive processes aren't actually killed right now, so sos
both leaks pids and also subsequent runs compete for input from stdin.
~ $ stack install steeloverseer
Run from outside a project, using implicit global project config
Using resolver: nightly-2016-06-21 from implicit global project's config file: /home/simon/.stack/global-project/stack.yaml
megaparsec-5.0.0: configure
megaparsec-5.0.0: build
megaparsec-5.0.0: copy/register
steeloverseer-2.0: download
steeloverseer-2.0: configure
steeloverseer-2.0: build
Completed 2 action(s).
-- While building package steeloverseer-2.0 using:
/home/simon/.stack/setup-exe-cache/x86_64-linux/setup-Simple-Cabal-1.24.0.0-ghc-8.0.1 --builddir=.stack-work/dist/x86_64-linux/Cabal-1.24.0.0 build --ghc-options " -ddump-hi -ddump-to-file"
Process exited with code: ExitFailure 1
Logs have been written to: /home/simon/.stack/global-project/.stack-work/logs/steeloverseer-2.0.log
Configuring steeloverseer-2.0...
Building steeloverseer-2.0...
Preprocessing library steeloverseer-2.0...
[1 of 5] Compiling Sos ( src/Sos.hs, .stack-work/dist/x86_64-linux/Cabal-1.24.0.0/build/Sos.o )
/tmp/stack27432/steeloverseer-2.0/src/Sos.hs:20:11: error:
• Expecting two more arguments to ‘ParseError’
Expected a type, but ‘ParseError’ has kind ‘* -> * -> *’
• In the type ‘ParseError’
In the definition of data constructor ‘SosCommandParseException’
In the data declaration for ‘SosException’
If this is due to a missing or too large upper bound on megaparsec, it would be great if you could add a revision for that bound on Hackage!
I'm thinking of expressing the steeloverseer "server" as a stream of typed "messages", where each one is responded to by the client (front-end: terminal, ncurses, etc).
Haven't though about it too hard but here's what I've come up with:
Use a typed GADT for the message type, produced by the server process and tagged with the expected type of the client response.
data MessageF a where
-- "I just started a job"
JobStarted :: JobDescription -> MessageF () -- client responds with (), aka "ok"
-- "The current job produced a line of output"
Stdout :: Text -> MessageF ()
-- "A job exited unsuccessfully, continue?"
JobDied :: JobDescription -> MessageF Bool
The exact messages obviously need to be fleshed out a bit. Anyways, with this GADT, I think we can reuse the streaming
machinery with a clever functor:
data Message a = forall x. Message (MessageF x) (x -> a)
Thus, the overall type of the server is something like
Stream Message IO ()
which expands to (basically)
Pure ()
| Effect (IO (Stream Message IO ()))
| Step (forall x. (MessageF x) (x -> Stream (Message IO ())))
When I'm working on a script named "foo.py", I'd like to use sos foo.py
as a shorthand for sos foo.py -c foo.py
. The former invocation does nothing, so this change should not break anything.
Right now \0
refers to the entire match of the regex, which is not necessarily the entire file.
It might be nice to launch sos
like so:
sos -p 'hs$' -c 'stack ghc <the-file>'
and have it run stack ghc
on any modified file that ends in hs
. However, I believe the simplest way to do this (currently) is:
sos -p '.*hs$' -c 'stack ghc \0'
Not too bad, but that extra .*
can be annoying. Would it make sense to have magic variable names for the absolute path matched and/or the basename matched?
I just stumbled on this and I sort of wish I had found it sooner! I just implemented a similar tool at https://github.com/grafted-in/twitch-cli. Mine is not as powerful but perhaps a bit simpler to use. I'm posting here in case there are ideas we can share.
twitch-cli is able to set up many pattern/command patterns and simply relies on you using something like ;
or &&
to run commands in sequence. It uses Glob instead of regex and is therefore a bit simpler but less powerful.
Aside from that: Amazing tool! I love it. Keep it up!
The version in master seems to be working well, and has updated depencencies, would it be possible to upload it to Hackage?
Thank you!
I wonder if a typical ghci workflow can be automated with sos.
Usually you run a ghci process and:
*.hs
changes, you send :r<CR>
or possibly :r<CR>:main<CR>
to ghci*.cabal
or *.yaml
changes, you restart ghciRight now 2. is working perfectly, 3. works kind of weird (I'm not even sure how to describe it) and 1. is not covered at all.
Meanwhile, ghcid supports 1 and 2, but not 3 (by current design) and naturally only supports ghci. Ideally I'd like to be able to use sos to automate a REPL workflow for any language, e.g. restart the process on project.clj
changes and send (run-all-tests)<CR>
on other changes.
Maybe it’d make sense to allow reusing project’s .gitignore, as these are the files one’s most likely to manually be adding to -e
exclusion patterns.
Hi, currently installing haskellPackages.steeloverseer adds up to 2,5Gb of disk storage because it has GHC in closure. The size makes this package hard to add to a development environment, because sharing it with people will imply them downloading 2,5Gb of data. Maybe there is a way to package this wonderful app as they packaged pandoc (it weighs around 200Mb)?
See NixOS/nixpkgs#34376
sos output.raw -c "sleep 0.15; echo 'test'"
Hit Ctrl+C to quit.
Modified: output.raw
[1/1] sleep 0.15; echo 'test'
AsyncCancelled
AsyncCancelled
Modified: output.raw
[1/1] sleep 0.15; echo 'test'
AsyncCancelled
AsyncCancelled
Modified: output.raw
[1/1] sleep 0.15; echo 'test'
AsyncCancelled
AsyncCancelled
Modified: output.raw
[1/1] sleep 0.15; echo 'test'
AsyncCancelled
AsyncCancelled
This only occurs if sleep 0.15
is present. echo 'test'
is never run. This used to work without issue (I might have been on an older commit / version).
This occurs with the master
branch (3eee0ec).
If I understand correctly, currently it is possible to execute commands just one after the other.
Would it be interesting/possible to configure certain commands to run in parallel?
For example, I could have four commands a
, b
, c
and d
and I would like to run a
, then b
and c
in parallel and only after they both complete, run d
This app is very IO-heavy and side-effecty. Would be nice to refactor it to write some pure tests, be able to feed in pure (mocked) filesystem events, and stuff like that.
Some simple high-level properties to test:
Then there's all the untested IO ugliness:
SIGINT
(not sure if Haskell can do anything about these at all)If a file change triggers a longer task, and an other file is changed sos
interrupts the current task with an AsyncCancelled
message.
I'd then expect it to immediately restart the relevant actions, as a watch file has changed. Instead, it waits until an other change is made, and then restarts.
I'm not sure if this is by design or a bug, but either way it would be nice to have the option to restart immediately after cancellation.
If this is a bug, it might be related to #38.
/tmp/stack19783/steeloverseer-2.0.2.0/app/Main.hs:258:5: error:
• The constructor ‘FSNotify.Added’ should have 3 arguments, but has been given 2
• In the pattern: FSNotify.Added path _
In a case alternative:
FSNotify.Added path _ -> S.yield (FileAdded (go cwd path))
In the second argument of ‘S.for’, namely
‘(\case
FSNotify.Added path _ -> S.yield (FileAdded (go cwd path))
FSNotify.Modified path _ -> S.yield (FileModified (go cwd path))
FSNotify.Removed _ _ -> pure ())’
|
258 | FSNotify.Added path _ -> S.yield (FileAdded (go cwd path))
| ^^^^^^^^^^^^^^^^^^^^^^^^
I don't think anyone uses it, and I don't use it.
Resolving dependencies...
Configuring steeloverseer-1.1.0.1...
Building steeloverseer-1.1.0.1...
Preprocessing executable 'sos' for steeloverseer-1.1.0.1...
src/Main.hs:11:8:
Could not find module `SOS'
Use -v to see a list of the files searched for.
ghc --make: /usr/hs/ghc/7.6.3/bin/ghc failure (return code=1)
Failed to install steeloverseer-1.1.0.1
cabal unpack
confirms that there's no SOS.hs in hackage tarball.
Hi!
I have emacs creating files like .#myfile.txt
when I am editing a file, and sos triggers on those (runs the command). This is not the behaviour I want, so I tried using -e to exclude those, but I just can't get it to work!
I tried following:
-e "#"
-e "\.#"
-e ".*\.#.*"
and non of those excludes the files in question from triggering sos.
The message I get is Added: foo/bar/.#yourfile.txt
. So it seems to be triggering only on adding the emacs lock file, not modifying it. Is it possible that -e
matches only modified files and not added ones?
Am I doing smth wrong? Thank you!
Btw. I am on Archlinux, I am running sos on a directory, and sos version is 2.0.2.
Success <stdout>: commitBuffer: invalid argument (invalid character)
prints after every command execution.
Looking at locale
: commercialhaskell/stack#793 (comment)
It would be great to be able to exclude patterns explicitly so we don't have to model that with regular expressions. Excluding based on a string makes for a pretty tough regex.
sos -p ".*\.l?hs$" -e "#" -e "flycheck" -c "stack build"
This sos
command would only stack build
when changes are made to a haskell file (possibly literate haskell) that doesn't contain a #
or flymake
, which are both emacs artifacts.
Please include at least some attribution for the image of the Steel Overseer. I doubt you drew it ;)
This patch works for me:
diff --git a/app/Main.hs b/app/Main.hs
index 5811ff5..339c1f7 100644
--- a/app/Main.hs
+++ b/app/Main.hs
@@ -14,6 +14,7 @@ import Control.Concurrent.STM.TMVar
import Control.Concurrent.STM.TQueue.Extra
import Control.Exception
import Control.Monad
+import Control.Monad.Catch (MonadThrow(..))
import Control.Monad.Managed
import Data.ByteString (ByteString)
import Data.List.NonEmpty (NonEmpty(..))
diff --git a/steeloverseer.cabal b/steeloverseer.cabal
index e6bea6c..63e6a82 100644
--- a/steeloverseer.cabal
+++ b/steeloverseer.cabal
@@ -47,13 +47,13 @@ library
, regex-tdfa >= 1.2
, semigroups >= 0.16
, stm >= 2.4
- , streaming >= 0.1.0 && < 0.1.5
+ , streaming
, text >= 1.2
, yaml >= 0.8
, aeson >= 0.8
, ansi-terminal >= 0.6.2
, containers >= 0.5
- , process >= 1.6.0 && < 1.6.1
+ , process
, unix
hs-source-dirs:
src
@@ -73,7 +73,7 @@ executable sos
, regex-tdfa >= 1.2
, semigroups >= 0.16
, stm >= 2.4
- , streaming >= 0.1.0 && < 0.1.5
+ , streaming
, text >= 1.2
, yaml >= 0.8
, steeloverseer
@@ -107,7 +107,7 @@ test-suite spec
, regex-tdfa >= 1.2
, semigroups >= 0.16
, stm >= 2.4
- , streaming >= 0.1.0 && < 0.1.5
+ , streaming
, text >= 1.2
, yaml >= 0.8
, steeloverseer
When running sos
on Ubuntu-17.10 (x64), I occasionally get sos: addWatch: resource exhausted (No space left on device)
at first I thought this was only when I was in tmux, but it looks like it happens in some non-deterministic fashion. Memory usage is <25% utilization and disk space is ~66% -- so I'm not sure what kind of space is being referred to in this error. I've never had this problem on Archlinux or Debian (so far).
If someone could point me in a direction, I can try to fix this myself (I imagine it is a bit difficult to simulate).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.