Git Product home page Git Product logo

tunnel's Introduction

Tunnel

Tunnel is a set of scripts that simplify launching remote Mathematica kernels where all established MathLink connections are automatically tunneled through a secure shell connection.

Features

  • Supports launching of remote Mathematica controller kernels and remote Mathematica compute kernels.
  • MathLink connections to the remote kernel are tunneled over SSH. This simplifies connecting to a remote kernel behind a firewall or a NAT router over the Internet.
  • Works with Windows, Linux and OS X versions of Mathematica.
  • Compatible with Mathematica versions from 8.0 to 11.0.

Requirements

  • A Wolfram Mathematica product (Mathematica, gridMathematica, Wolfram Finance Platform).
  • OpenSSH client and server under Linux and OS X.
  • PuTTY on a Windows Mathematica front end machine.
  • An SSH server on a remote Windows Mathematica kernel machine. See the Tunnel manual for a list of supported SSH servers.

Installation

Under Linux or OS X, execute the shell script install-tunnel.sh to install the required scripts to the correct locations for an existing Mathematica installation. Under Windows, execute the batch script install-tunnel.bat to install the required scripts.

The Tunnel related scripts can also be installed manually. See the Tunnel manual for more information.

The installation must be performed on both the local machine that runs the Mathematica front end, and on the remote machine that runs the Mathematica kernel.

Usage

See the Tunnel manual for information on

  • How to set up remote controller kernel configurations using the Tunnel scripts in the Mathematica front end kernel configuration options dialog.
  • How to set up remote compute kernels with the Mathematica Parallel package.
  • Technical background information.

Known Issues

Tunnel enabled remote controller kernel launching no longer works for Mathematica 11.1, because Wolfram has switched to a new method of establishing the parallel links between the front end and the controller kernel. Tunnel enabled remote compute controller kernel launching works, though.

tunnel's People

Contributors

sakra avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

tunnel's Issues

Questions about the port forwarding

Hi Sakra,

Thanks for sharing Tunnel, it helped me a lot!

I have a question about the SSH port forwarding used in Tunnel. In the Tunnel, the ssh server is the same machine that the remote kernel is running on. Is it possible to use a different machine as the SSH server? For example, the front end is running on my local machine A, and the kernel is running at B behind a firewall. And Tunnel applies in this situation. Consider if the kernel is required to run on a different machine C, and B and C have unblocked TCP/IP access between them. Is it possible to create a tunnel between A and C through B, so that the remote kernel runs on C? Thanks

multiple IP address

Hi Sakra,

When I use the Tunnel on a remote machine with multiple IP addresses, it seems that the links are only partially connected.

Here are the details:
The machine on which the remote kernel is running on has multiple IP addresses:

204.90.40.21
172.17.40.21
104.141.1.34

and only IP address 204.90.40.21 is the public IP address that I can ssh to. When I try using Tunnel to start a remote (main) kernel, it shows only two of the six links are established. Here are the ports on the remote machine for the kernel:

[mwu3@smic1 ssh]$ lsof -i 4 -a
COMMAND     PID USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
MathKerne 29666 mwu3    4u  IPv4 33257804      0t0  TCP localhost.localdomain:59040->localhost.localdomain:51676 (ESTABLISHED)
MathKerne 29666 mwu3    5u  IPv4 33257801      0t0  TCP localhost.localdomain:60247->localhost.localdomain:51675 (ESTABLISHED)
[mwu3@smic1 ssh]$

And ports on my local machine for the frontend are:

ssh       1324 xslittlegrass    3u  IPv4 0xc9651f6c7f393a47      0t0  TCP mwu3-2.lsu.edu:50961->smic1.hpc.lsu.edu:ssh (ESTABLISHED)
Mathemati 1798 xslittlegrass   36u  IPv4 0xc9651f6c6a728317      0t0  TCP localhost:51474->localhost:51475 (ESTABLISHED)
Mathemati 1798 xslittlegrass   48u  IPv4 0xc9651f6c7f90ba47      0t0  TCP localhost:51675->localhost:51681 (ESTABLISHED)
Mathemati 1798 xslittlegrass   49u  IPv4 0xc9651f6c6a21b4b7      0t0  TCP localhost:51676->localhost:51682 (ESTABLISHED)
Mathemati 1798 xslittlegrass   81u  IPv4 0xc9651f6c7f531657      0t0  TCP localhost:51480->localhost:51478 (ESTABLISHED)
Mathemati 1798 xslittlegrass  101u  IPv4 0xc9651f6c6a244d87      0t0  TCP localhost:51481->localhost:51479 (ESTABLISHED)
MathKerne 1802 xslittlegrass   22u  IPv4 0xc9651f6c7f4e2be7      0t0  TCP localhost:51475->localhost:51474 (ESTABLISHED)
MathKerne 1802 xslittlegrass   36u  IPv4 0xc9651f6c7fa3ed87      0t0  TCP localhost:51478->localhost:51480 (ESTABLISHED)
MathKerne 1802 xslittlegrass   50u  IPv4 0xc9651f6c7f391f27      0t0  TCP localhost:51479->localhost:51481 (ESTABLISHED)
bash      2053 xslittlegrass   36u  IPv4 0xc9651f6c6a728317      0t0  TCP localhost:51474->localhost:51475 (ESTABLISHED)
bash      2053 xslittlegrass   48u  IPv4 0xc9651f6c7f8660c7      0t0  TCP localhost:51675 (LISTEN)
bash      2053 xslittlegrass   49u  IPv4 0xc9651f6c7f864657      0t0  TCP localhost:51676 (LISTEN)
bash      2053 xslittlegrass   81u  IPv4 0xc9651f6c7f531657      0t0  TCP localhost:51480->localhost:51478 (ESTABLISHED)
bash      2053 xslittlegrass  101u  IPv4 0xc9651f6c6a244d87      0t0  TCP localhost:51481->localhost:51479 (ESTABLISHED)
ssh       2085 xslittlegrass    3u  IPv4 0xc9651f6c69e2f317      0t0  TCP mwu3-2.lsu.edu:51677->smic1.hpc.lsu.edu:ssh (ESTABLISHED)
ssh       2085 xslittlegrass    7u  IPv4 0xc9651f6c7f4e3d87      0t0  TCP localhost:51677 (LISTEN)
ssh       2085 xslittlegrass    9u  IPv4 0xc9651f6c6a245657      0t0  TCP localhost:51678 (LISTEN)
ssh       2085 xslittlegrass   11u  IPv4 0xc9651f6c6a1faa47      0t0  TCP localhost:51679 (LISTEN)
ssh       2085 xslittlegrass   13u  IPv4 0xc9651f6c698e1d87      0t0  TCP localhost:51680 (LISTEN)
ssh       2085 xslittlegrass   14u  IPv4 0xc9651f6c6a245f27      0t0  TCP localhost:51681->localhost:51675 (ESTABLISHED)
ssh       2085 xslittlegrass   17u  IPv4 0xc9651f6c68174be7      0t0  TCP localhost:51682->localhost:51676 (ESTABLISHED)

We can see that four links at ports 51677-51680 are not established.

This partial connection results of no graphics nor code highlight in Mathematica, and the frontend acts like a terminal for the kernel:

snip20150830_1

I'm not sure whether this is because ssh is set to forward the port on the wrong interface. In tunnel, the ssh is set to forward the port on 127.0.0.1, but I'm not sure which IP address ssh will select from. I see that in the package, it can choose the right IP address for subkernels, so multiple IP address will not cause problems for subkernels. But here the problem is at the main kernel.

Any idea how to solve this?


Update

It looks like the ssh port forwarding works fine. I can create six links without problem mannully between local and remote kernels:

I start the ssh port forwarding first using

ssh -R 52848:127.0.0.1:52848 -R 52849:127.0.0.1:52849 -L 52850:127.0.0.1:52850 -L 52851:127.0.0.1:52851 -L 52852:127.0.0.1:52852 -L 52853:127.0.0.1:52853 smic

Local:

In[1]:= linkToRemote1=LinkCreate["[email protected],[email protected]",LinkProtocol->"TCPIP"]
Out[1]= LinkObject[[email protected],[email protected], 62, 1]

In[2]:= linkToRemote2=LinkConnect["[email protected],[email protected]",LinkProtocol->"TCPIP"]
Out[2]= LinkObject[[email protected],[email protected], 63, 2]

In[3]:= linkToRemote3=LinkConnect["[email protected],[email protected]",LinkProtocol->"TCPIP"]
Out[3]= LinkObject[[email protected],[email protected], 64, 3]

In[4]:= LinkWrite[linkToRemote1,"hello 1"]
In[5]:= LinkWrite[linkToRemote2,"hello 2"]
In[6]:= LinkWrite[linkToRemote3,"hello 3"]

Remote:

In[1]:= linkToLocal1=LinkConnect["[email protected],[email protected]",LinkProtocol->"TCPIP"]
Out[1]= LinkObject[[email protected],[email protected], 60, 1]

In[2]:= linkToLocal2=LinkCreate["[email protected],[email protected]",LinkProtocol->"TCPIP"]
Out[2]= LinkObject[[email protected],[email protected], 61, 2]

In[3]:= linkToLocal3=LinkCreate["[email protected],[email protected]",LinkProtocol->"TCPIP"]
Out[3]= LinkObject[[email protected],[email protected], 62, 3]

In[4]:= LinkRead[linkToLocal1]
Out[4]= hello 1

In[5]:= LinkRead[linkToLocal2]
Out[5]= hello 2

In[6]:= LinkRead[linkToLocal3]
Out[6]= hello 3

And I think Szabolcs may have had the same probem

http://forums.wolfram.com/mathgroup/archive/2011/Jun/msg00283.html

Dynamics are not working

I realize 11.1 is not officially supported. However, I followed your instructions exactly and everything works. All except for Dynamic behavior as this screenshot shows:

image

Is there anything I can do to get around it?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.