Git Product home page Git Product logo

zsh2xonsh's Introduction

zsh2xonsh pypy powered-by-xonsh

Have you heard of xonsh? It's a Python-powered shell.

You can do amazing things like this:

# Interpolate python -> shell
echo @(i for i in range(42))

# Interpolate shell -> python
for filename in `.*`:
    print(filename)
    du -sh @(filename)

# Execute regular shell commands too
cat /etc/passwd | grep root

As you can immagine, this awesomeness is not POSIX-compliant :(

This makes it difficult to setup your $PATH and do things like eval $(brew shellenv)

This package exists to translate traditional zsh scripts into xonsh code.

Compatibility (and how it works)

The goal is 100% compatibility for a subset of shell.

Compatibility is achived by delegating most of the work to zsh.

That is, export FOO="$(echo bar)" in a shell script becomes (essentially) $FOO=$(zsh -c 'echo bar') in xonsh.

We have zsh handle all the globbing/quoting/bizzare POSIX quirks.

In the face of ambiguity, or if we encounter an unsupported feature (like a for loop), then we fail-fast.

This is the most important feature. If something can't be supported 100%, then it will throw a descriptive error. Anything else is a bug :)

Features

The included shell features include:

  1. Quoted expressions "$VAR glob/*" (zsh does expansion here)
  2. Unquoted literals 12, foo ~/foo (mostly translated directly)
  3. Command substitutions "$(cat file.txt | grep bar)"
    • zsh does all the work here
    • Supports both quoted and unquoted forms
  4. If/then statements
    • Conditionals are executed by zsh (so [[ -d "foo" ]] works perfectly)
    • Translated into python if (so body will not run unless conditional passes)
  5. Exporting variables export FOO=$BAR
    • Translates $PATH correctly (xonsh thinks it's a list, zsh thinks it's a string)
    • This is where the subprocess approach doesn't work blindly....
      • We support it cleanly by doing the left-hand assignment xonsh, and the right-hand expression in zsh :)
    • Local variables (local var=x) are supported too (with the proper scoping)
  6. Support alias foo="bar"
    • This even supports globbing in the alias, so alias lsdot="echo .*" would glob in the same way that zsh does (experimental)
  7. Basic support for function declarations (and having positional arguments)
    • Local variables are scoped properly inside the function :)
  8. Support for 'echo' builtin as a python 'print'

You can also add "external builtins", which are extra commands that the script can invoke. In my own files, I use this to add an extend_path command (even though zsh2xonsh properly translates $PATH variables already, it's still a nice utility)

My macbook config is a good example of the full power of the translator. It supports function declarations, if/then statements and conditionals (all in an attempt to cleanup homebrew cludges lol).

All of these behaviors should be 100% compatible with their zsh equivalents. If you try anything else (or encounter an unsupported corner case), then you will get a descripitive error :)

Installation & Usage

This is a pypi package, install it with pip install zsh2xonsh.

The API is simple, run translate_to_xonsh(str) -> str to translate from zsh -> xonsh code. This does not require xonsh at runtime, and can be done ahead of time.

If you want to evaluate the code immediately after translating it (for example in a .xonshrc), you can use . This requires xonsh at runtime (obviously) and uses the evalx builtin.

Additionally you can use the CLI (python -m zsh2xonsh), which accepts an import.

If you want to provide extra utility functions to your code, you can define extra_builtins.

Example

In my .xonshrc, I dynamically translate and evaluate the output of brew shellenv:

zsh2xonsh.translate_to_xonsh_and_eval($(/opt/homebrew/bin/brew shellenv))

Likewise, I do the same with my machine-specific .config.zsh files (in the example directory).

Motiviation

First of all, I need support for eval $(brew shellenv) .

Second of all, I still use zsh as my backup shell (by the use of my wrap-shell utility).

This means I need to setup $PATH and other enviornment variables for both of these shells, indepenently of each other.

The natural way to set these up is by using shell scripts. I have a seperate one for each of my different machines (server, macbook, old lapotop, etc)

For each of these (tiny) shell scripts, zsh2xonsh works very well :)

So in addition to properly translating $(brew shellenv), it also needs to translate these basic shell "environement files".

Turns out, dealing with platform specific quirks can get pretty intense and requires things like function definitions and if/then statements. In order to properly support exports, control flow has to be emulated in Python (can't just delegate to zsh).

zsh2xonsh's People

Contributors

techcable avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

zsh2xonsh's Issues

TODO: Add tests

This is hard (not just because I'm lazy), but also because we often want to execute the translated commands under a "clean" xonsh environment and not just verify they parse.....

However, getting a clean xonsh environment for testing would be difficult, because we're often running under xonsh or with an already modified $PATH.

This is especially true in my case because I use zsh2xonsh already in my .xonshrc ๐Ÿ˜‰. Maybe we could use --no-rc and scrub the environment variables back to the original?

Alternatively, maybe we can spawn up zsh -f -c 'echo $PATH' to get a semi-clean path (or xonsh --no-rc)? We have to be careful not to inherit any environment variables besides the default one.

EDIT: xonsh --no-rc seems to work well using the following command.

from subprocess import run, PIPE, DEVNULL
run([sys.executable, '-m', 'xonsh', '--no-rc', '-c' 'echo $PATH'], env={}, stdout=PIPE, stderr=DEVNULL, encoding='utf-8').stdout.rstrip('\n')

I suggest we use this to bootstrap $PATH and $PYTHON_PATH, wiping other non-essential environment vars :)

This should give us a somewhat clean slate (while also respecting homebrew python).

Updating $PATH early on doesn't affect later `zsh` calls

Early in my .xonshrc I run eval $(brew shellenv) by doing

zsh2xonsh.translate_to_xonsh_and_eval($(/opt/homebrew/bin/brew shellenv))

Later on, I try to run my ~/.config.zsh

Currently ~/.config.zsh invokes the fd command to detect the best $JAVA_HOME.

In the config,

local preferred_java_version=17
local preferred_java_home=$(fd "jdk-${prefered_java_version}.*\.jdk" /Library/Java/JavaVirtualMachines --maxdepth 1)

is translated into

preferred_java_version=17
preferred_java_home=runtime.zsh('fd "jdk-${prefered_java_version}.*\\.jdk" /Library/Java/JavaVirtualMachines --maxdepth 1')`

Unfortunately, fd is only available from homebrew.

Based on my debugging, it appears the initial $(brew shellenv) does not affect the $PATH in runtime.zsh_expand_quote(). That is, os.environ() does not see the correct path.

The ultimate reason for this is that $UPDATE_OS_ENVIRON = False by default.

We need to find a way to work around this (or convince xonsh to change the defaults). Why was it set this way?

Move from recursive-decent to auto-generated parser

The recursive-descent parser I have in place works, but it's kind of clunky.

We need to either move to an auto-generated parser (ply or PEG) or some sort of parser-combinators library.

I've run into a couple of stalls during debugging and I think that's probably the #1 problem with the reliability right now :)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.