Git Product home page Git Product logo

sshtunnel's Introduction

CircleCI AppVeyor Documentation Status coveralls version

pyversions license

Author: Pahaz

Repo: https://github.com/pahaz/sshtunnel/

Inspired by https://github.com/jmagnusson/bgtunnel, which doesn't work on Windows.

See also: https://github.com/paramiko/paramiko/blob/master/demos/forward.py

Requirements

Installation

sshtunnel is on PyPI, so simply run:

pip install sshtunnel

or :

easy_install sshtunnel

or :

conda install -c conda-forge sshtunnel

to have it installed in your environment.

For installing from source, clone the repo and run:

python setup.py install

Testing the package

In order to run the tests you first need tox and run:

python setup.py test

Usage scenarios

One of the typical scenarios where sshtunnel is helpful is depicted in the figure below. User may need to connect a port of a remote server (i.e. 8080) where only SSH port (usually port 22) is reachable. :

----------------------------------------------------------------------

                            |
-------------+              |    +----------+
    LOCAL    |              |    |  REMOTE  | :22 SSH
    CLIENT   | <== SSH ========> |  SERVER  | :8080 web service
-------------+              |    +----------+
                            |
                         FIREWALL (only port 22 is open)

----------------------------------------------------------------------

Fig1: How to connect to a service blocked by a firewall through SSH tunnel.

If allowed by the SSH server, it is also possible to reach a private server (from the perspective of REMOTE SERVER) not directly visible from the outside (LOCAL CLIENT's perspective). :

----------------------------------------------------------------------

                            |
-------------+              |    +----------+               +---------
    LOCAL    |              |    |  REMOTE  |               | PRIVATE
    CLIENT   | <== SSH ========> |  SERVER  | <== local ==> | SERVER
-------------+              |    +----------+               +---------
                            |
                         FIREWALL (only port 443 is open)

----------------------------------------------------------------------

Fig2: How to connect to PRIVATE SERVER through SSH tunnel.

Usage examples

API allows either initializing the tunnel and starting it or using a with context, which will take care of starting and stopping the tunnel:

Example 1

Code corresponding to Fig1 above follows, given remote server's address is pahaz.urfuclub.ru, password authentication and randomly assigned local bind port.

Example 2

Example of a port forwarding to a private server not directly reachable, assuming password protected pkey authentication, remote server's SSH service is listening on port 443 and that port is open in the firewall (Fig2):

Example 3

Example of a port forwarding for the Vagrant MySQL local port:

Or simply using the CLI:

(bash)$ python -m sshtunnel -U vagrant -P vagrant -L :3306 -R 127.0.0.1:3306 -p 2222 localhost

Example 4

Opening an SSH session jumping over two tunnels. SSH transport and tunnels will be daemonised, which will not wait for the connections to stop at close time.

CLI usage

$ sshtunnel --help
usage: sshtunnel [-h] [-U SSH_USERNAME] [-p SSH_PORT] [-P SSH_PASSWORD] -R
                 IP:PORT [IP:PORT ...] [-L [IP:PORT ...]] [-k SSH_HOST_KEY]
                 [-K KEY_FILE] [-S KEY_PASSWORD] [-t] [-v] [-V] [-x IP:PORT]
                 [-c SSH_CONFIG_FILE] [-z] [-n] [-d [FOLDER ...]]
                 ssh_address

Pure python ssh tunnel utils
Version 0.4.0

positional arguments:
  ssh_address           SSH server IP address (GW for SSH tunnels)
                        set with "-- ssh_address" if immediately after -R or -L

options:
  -h, --help            show this help message and exit
  -U SSH_USERNAME, --username SSH_USERNAME
                        SSH server account username
  -p SSH_PORT, --server_port SSH_PORT
                        SSH server TCP port (default: 22)
  -P SSH_PASSWORD, --password SSH_PASSWORD
                        SSH server account password
  -R IP:PORT [IP:PORT ...], --remote_bind_address IP:PORT [IP:PORT ...]
                        Remote bind address sequence: ip_1:port_1 ip_2:port_2 ... ip_n:port_n
                        Equivalent to ssh -Lxxxx:IP_ADDRESS:PORT
                        If port is omitted, defaults to 22.
                        Example: -R 10.10.10.10: 10.10.10.10:5900
  -L [IP:PORT ...], --local_bind_address [IP:PORT ...]
                        Local bind address sequence: ip_1:port_1 ip_2:port_2 ... ip_n:port_n
                        Elements may also be valid UNIX socket domains:
                        /tmp/foo.sock /tmp/bar.sock ... /tmp/baz.sock
                        Equivalent to ssh -LPORT:xxxxxxxxx:xxxx, being the local IP address optional.
                        By default it will listen in all interfaces (0.0.0.0) and choose a random port.
                        Example: -L :40000
  -k SSH_HOST_KEY, --ssh_host_key SSH_HOST_KEY
                        Gateway's host key
  -K KEY_FILE, --private_key_file KEY_FILE
                        RSA/DSS/ECDSA private key file
  -S KEY_PASSWORD, --private_key_password KEY_PASSWORD
                        RSA/DSS/ECDSA private key password
  -t, --threaded        Allow concurrent connections to each tunnel
  -v, --verbose         Increase output verbosity (default: ERROR)
  -V, --version         Show version number and quit
  -x IP:PORT, --proxy IP:PORT
                        IP and port of SSH proxy to destination
  -c SSH_CONFIG_FILE, --config SSH_CONFIG_FILE
                        SSH configuration file, defaults to ~/.ssh/config
  -z, --compress        Request server for compression over SSH transport
  -n, --noagent         Disable looking for keys from an SSH agent
  -d [FOLDER ...], --host_pkey_directories [FOLDER ...]
                        List of directories where SSH pkeys (in the format `id_*`) may be found

sshtunnel's People

Contributors

alex3d avatar basnijholt avatar bsamuel-ui avatar cameronmaske avatar chronial avatar cjermain avatar dirtysalt avatar drewfustin avatar expobrain avatar fernandezcuesta avatar g0djan avatar gdmachado avatar ipeluffo avatar mrakitin avatar nalvarez1 avatar pahaz avatar pschmitt avatar rasterburn avatar stavxyz avatar sweenu avatar tkhyn avatar v0idk avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sshtunnel's Issues

Add support reconnection of transport.

Sometimes transport have socker error: PARAMIKO: 1430@transport | Socket exception: An existing connection was forcibly closed by the remote host (10054)
Required add reconnection mechanism for solving this problem.

2015-07-08 11:01:34,346 | DEBUG   | Incoming request #30
2015-07-08 11:01:53,002 | DEBUG   | Incoming request #29 connection closed.
2015-07-08 11:03:18,878 | WARNING | Incoming request #30 sending RST
2015-07-08 11:03:18,882 | DEBUG   | Incoming request #30 connection closed.
2015-07-08 11:11:26,091 | ERROR   | PARAMIKO: 1430@transport | Socket exception: An existing connection was forcibly closed by the remote host (10054)
2015-07-08 11:11:26,094 | DEBUG   | Incoming request #6 connection closed.
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 2446)
2015-07-08 11:11:26,095 | DEBUG   | Incoming request #28 connection closed.
Traceback (most recent call last):
2015-07-08 11:11:26,095 | DEBUG   | Incoming request #27 connection closed.
2015-07-08 11:11:26,096 | DEBUG   | Incoming request #32 to ('127.0.0.1', 8888) failed: SSHException('Unable to open channel.',)
  File "C:\Python34\lib\socketserver.py", line 617, in process_request_thread
    self.finish_request(request, client_address)
----------------------------------------
  File "C:\Python34\lib\socketserver.py", line 344, in finish_request
    self.RequestHandlerClass(request, client_address, self)
Exception happened during processing of request from ('127.0.0.1', 2447)
  File "C:\Python34\lib\socketserver.py", line 673, in __init__
    self.handle()
Traceback (most recent call last):
  File "C:\Python34\lib\site-packages\sshtunnel.py", line 242, in handle
    self.request.getpeername())
  File "C:\Python34\lib\site-packages\sshtunnel.py", line 242, in handle
    self.request.getpeername())
  File "C:\Python34\lib\site-packages\paramiko\transport.py", line 731, in open_channel
    raise e
  File "C:\Python34\lib\site-packages\paramiko\transport.py", line 731, in open_channel
    raise e
  File "C:\Python34\lib\site-packages\paramiko\transport.py", line 1592, in run
    ptype, m = self.packetizer.read_message()
paramiko.ssh_exception.SSHException: Unable to open channel.
  File "C:\Python34\lib\site-packages\paramiko\packet.py", line 341, in read_message
    header = self.read_all(self.__block_size_in, check_rekey=True)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python34\lib\site-packages\paramiko\packet.py", line 204, in read_all
    x = self.__socket.recv(n)
  File "C:\Python34\lib\socketserver.py", line 617, in process_request_thread
    self.finish_request(request, client_address)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
----------------------------------------
  File "C:\Python34\lib\socketserver.py", line 344, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Python34\lib\socketserver.py", line 673, in __init__
    self.handle()
  File "C:\Python34\lib\site-packages\sshtunnel.py", line 252, in handle
    raise HandlerSSHTunnelForwarderError(msg)
HandlerSSHTunnelForwarderError: Incoming request #32 to ('127.0.0.1', 8888) failed: SSHException('Unable to open channel.',)
----------------------------------------
2015-07-08 11:11:27,122 | DEBUG   | Incoming request #33 to ('127.0.0.1', 8888) failed: SSHException('SSH session not active',)
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 2453)
Traceback (most recent call last):
  File "C:\Python34\lib\site-packages\sshtunnel.py", line 242, in handle
    self.request.getpeername())
  File "C:\Python34\lib\site-packages\paramiko\transport.py", line 696, in open_channel
    raise SSHException('SSH session not active')
paramiko.ssh_exception.SSHException: SSH session not active

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python34\lib\socketserver.py", line 617, in process_request_thread
    self.finish_request(request, client_address)
  File "C:\Python34\lib\socketserver.py", line 344, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Python34\lib\socketserver.py", line 673, in __init__
    self.handle()
  File "C:\Python34\lib\site-packages\sshtunnel.py", line 252, in handle
    raise HandlerSSHTunnelForwarderError(msg)
HandlerSSHTunnelForwarderError: Incoming request #33 to ('127.0.0.1', 8888) failed: SSHException('SSH session not active',)
----------------------------------------
2015-07-08 11:11:28,181 | DEBUG   | Incoming request #34 to ('127.0.0.1', 8888) failed: SSHException('SSH session not active',)
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 2455)
Traceback (most recent call last):
  File "C:\Python34\lib\site-packages\sshtunnel.py", line 242, in handle
    self.request.getpeername())
  File "C:\Python34\lib\site-packages\paramiko\transport.py", line 696, in open_channel
    raise SSHException('SSH session not active')
paramiko.ssh_exception.SSHException: SSH session not active

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python34\lib\socketserver.py", line 617, in process_request_thread
    self.finish_request(request, client_address)
  File "C:\Python34\lib\socketserver.py", line 344, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Python34\lib\socketserver.py", line 673, in __init__
    self.handle()
  File "C:\Python34\lib\site-packages\sshtunnel.py", line 252, in handle
    raise HandlerSSHTunnelForwarderError(msg)
HandlerSSHTunnelForwarderError: Incoming request #34 to ('127.0.0.1', 8888) failed: SSHException('SSH session not active',)
----------------------------------------
2015-07-08 11:11:29,275 | DEBUG   | Incoming request #35 to ('127.0.0.1', 8888) failed: SSHException('SSH session not active',)
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 2457)
Traceback (most recent call last):
  File "C:\Python34\lib\site-packages\sshtunnel.py", line 242, in handle
    self.request.getpeername())
  File "C:\Python34\lib\site-packages\paramiko\transport.py", line 696, in open_channel
    raise SSHException('SSH session not active')
paramiko.ssh_exception.SSHException: SSH session not active

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python34\lib\socketserver.py", line 617, in process_request_thread
    self.finish_request(request, client_address)
  File "C:\Python34\lib\socketserver.py", line 344, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Python34\lib\socketserver.py", line 673, in __init__
    self.handle()
  File "C:\Python34\lib\site-packages\sshtunnel.py", line 252, in handle
    raise HandlerSSHTunnelForwarderError(msg)
HandlerSSHTunnelForwarderError: Incoming request #35 to ('127.0.0.1', 8888) failed: SSHException('SSH session not active',)
----------------------------------------
2015-07-08 11:11:30,336 | DEBUG   | Incoming request #36 to ('127.0.0.1', 8888) failed: SSHException('SSH session not active',)
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 2459)
Traceback (most recent call last):
  File "C:\Python34\lib\site-packages\sshtunnel.py", line 242, in handle
    self.request.getpeername())
  File "C:\Python34\lib\site-packages\paramiko\transport.py", line 696, in open_channel
    raise SSHException('SSH session not active')
paramiko.ssh_exception.SSHException: SSH session not active

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python34\lib\socketserver.py", line 617, in process_request_thread
    self.finish_request(request, client_address)
  File "C:\Python34\lib\socketserver.py", line 344, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Python34\lib\socketserver.py", line 673, in __init__
    self.handle()
  File "C:\Python34\lib\site-packages\sshtunnel.py", line 252, in handle
    raise HandlerSSHTunnelForwarderError(msg)
HandlerSSHTunnelForwarderError: Incoming request #36 to ('127.0.0.1', 8888) failed: SSHException('SSH session not active',)

Tests intermitently fail

Despite trying to figure out what's going on with the test suite, it still gets locked during teardown. This leads to timeout on CI or just a never ending test on local.

Script won't exit because paramiko thread is still running

The following script works and "FINISH" is printed, but it never exits to the shell:

import threading

from sshtunnel import SSHTunnelForwarder

with SSHTunnelForwarder(
    "localhost",
    ssh_username="localuser",
    ssh_password="localpass",
    remote_bind_address=('127.0.0.1', 5984),
    local_bind_address=('127.0.0.1', 9000)
) as server:

    print(server.local_bind_port)
    # these don't help
    # server.stop()
    # server.close()

# [t.close() for t in threading.enumerate() if t.__class__.__name__ == "Transport"]

print('FINISH!')

Python is waiting on threads to finish, but they never do. After running this script in iPython I can see the threads:

threading.enumerate()
[<_MainThread(MainThread, started 140735243972608)>,
 <HistorySavingThread(IPythonHistorySavingThread, started 123145307557888)>,
 <paramiko.Transport at 0x3880470 (cipher aes128-ctr, 128 bits) (connected; awaiting auth)>,
 <paramiko.Transport at 0x3874630 (cipher aes128-ctr, 128 bits) (connected; awaiting auth)>,
 <paramiko.Transport at 0x3191ba8 (cipher aes128-ctr, 128 bits) (connected; awaiting auth)>,
 <paramiko.Transport at 0x3874550 (cipher aes128-ctr, 128 bits) (active; 0 open channel(s))>,
 <paramiko.Transport at 0x3191c88 (cipher aes128-ctr, 128 bits) (connected; awaiting auth)>,
 <paramiko.Transport at 0x387bac8 (cipher aes128-ctr, 128 bits) (connected; awaiting auth)>,
 <paramiko.Transport at 0x3803e48 (cipher aes128-ctr, 128 bits) (connected; awaiting auth)>,
 <paramiko.Transport at 0x3874f60 (cipher aes128-ctr, 128 bits) (connected; awaiting auth)>]

If I uncomment the last line which explicitly closes the paramiko.Transport threads, the script exits correctly. It seems like I'm missing something, but even so this default behavior is confusing.

Please release version 0.0.8.3 on pypi

Hi,

I'd like to be able to use 0.0.8.3 (with the fix to gethostbyname_ex / gethostname for OS X, but I'd like to install from pypi, rather that pip install git+https. Can you please create a formal release? Thanks!

Is there a way to make it work with Pageant on Windows ?

I could not manage it.. Has anybody tried that ? Which options should be used ?

I was (seemingly) able to connect to SSH using paramiko with Pageant, but I could not establish working tunnel for MySQL

I can provide more details if anyone is interested.

If debug_level argument is passed to constructor then unknown argument error is raised

SSHTunnelForwarder( '1.2.3.4', 22, debug_level='ERROR')

Results in an error (similar to this):

Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/user/pyenvs/application/__main__.py", line 213, in <module>
    args.profile
  File "/Users/user/pyenvs/application/ssh_tools/ssh.py", line 47, in forward_necessary_ports
    debug_level='ERROR'
  File "/usr/local/lib/python3.4/site-packages/sshtunnel.py", line 552, in __init__
    raise ValueError('Unknown arguments {0:!r}'.format(kwargs))
TypeError: non-empty format string passed to object.__format__

debug_error is a configurable kwarg so this shouldn't be the case. A pop occurring after the argument is read should occur

Use three digits for the version

I think that this project is not so much to use four digits for versioning. It seems that the time has come to do release 0.1.0 and then up the last digit for new versions.

Test coverage error

The following error is shown during tests:

INTERNALERROR> AssertionError: Test coverage should never be `None`

Full log:

GLOB sdist-make: /data/virtualenv/sshtunnel/sshtunnel/setup.py
py27 inst-nodeps: /data/virtualenv/sshtunnel/sshtunnel/.tox/dist/sshtunnel-0.0.8.3.zip
py27 installed: apipkg==1.4,cffi==1.7.0,coverage==4.1,cryptography==1.4,enum34==1.1.6,execnet==1.4.1,funcsigs==1.0.2,idna==2.1,ipaddress==1.0.16,mock==2.0.0,paramiko==2.0.1,pbr==1.10.0,py==1.4.31,pyasn1==0.1.9,pycparser==2.14,pytest==2.9.2,pytest-cov==2.3.0,pytest-xdist==1.14,six==1.10.0,sshtunnel==0.0.8.3
py27 runtests: PYTHONHASHSEED='2974953662'
py27 runtests: commands[0] | python -V
Python 2.7.12
py27 runtests: commands[1] | py.test --showlocals --cov sshtunnel --durations=10 -n4 tests
================================================= test session starts =================================================
platform linux2 -- Python 2.7.12, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /data/virtualenv/sshtunnel/sshtunnel, inifile:
plugins: cov-2.3.0, xdist-1.14
gw0 [44] / gw1 [44] / gw2 [44] / gw3 [44]
scheduling tests via LoadScheduling
......................................X....INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/data/virtualenv/sshtunnel/sshtunnel/.tox/py27/lib/python2.7/site-packages/_pytest/main.py", line 94, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/data/virtualenv/sshtunnel/sshtunnel/.tox/py27/lib/python2.7/site-packages/_pytest/main.py", line 125, in _main
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/data/virtualenv/sshtunnel/sshtunnel/.tox/py27/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 724, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/data/virtualenv/sshtunnel/sshtunnel/.tox/py27/lib/python2.7/site-packages/_pytest/main.py", line 94, in wrap_session
INTERNALERROR>   File "/data/virtualenv/sshtunnel/sshtunnel/.tox/py27/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/data/virtualenv/sshtunnel/sshtunnel/.tox/py27/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/data/virtualenv/sshtunnel/sshtunnel/.tox/py27/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 595, in execute
INTERNALERROR>   File "/data/virtualenv/sshtunnel/sshtunnel/.tox/py27/lib/python2.7/site-packages/_pytest/main.py", line 125, in _main
INTERNALERROR>     return _wrapped_call(hook_impl.function(*args), self.execute)
INTERNALERROR>   File "/data/virtualenv/sshtunnel/sshtunnel/.tox/py27/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 249, in _wrapped_call
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>     wrap_controller.send(call_outcome)
INTERNALERROR>   File "/data/virtualenv/sshtunnel/sshtunnel/.tox/py27/lib/python2.7/site-packages/pytest_cov/plugin.py", line 221, in pytest_runtestloop
INTERNALERROR>     assert self.cov_total is not None, 'Test coverage should never be `None`'
INTERNALERROR> AssertionError: Test coverage should never be `None`
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/data/virtualenv/sshtunnel/sshtunnel/.tox/py27/lib/python2.7/site-packages/_pytest/main.py", line 94, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/data/virtualenv/sshtunnel/sshtunnel/.tox/py27/lib/python2.7/site-packages/_pytest/main.py", line 125, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/data/virtualenv/sshtunnel/sshtunnel/.tox/py27/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 724, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/data/virtualenv/sshtunnel/sshtunnel/.tox/py27/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/data/virtualenv/sshtunnel/sshtunnel/.tox/py27/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/data/virtualenv/sshtunnel/sshtunnel/.tox/py27/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 595, in execute
INTERNALERROR>     return _wrapped_call(hook_impl.function(*args), self.execute)
INTERNALERROR>   File "/data/virtualenv/sshtunnel/sshtunnel/.tox/py27/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 249, in _wrapped_call
INTERNALERROR>     wrap_controller.send(call_outcome)
INTERNALERROR>   File "/data/virtualenv/sshtunnel/sshtunnel/.tox/py27/lib/python2.7/site-packages/pytest_cov/plugin.py", line 221, in pytest_runtestloop
INTERNALERROR>     assert self.cov_total is not None, 'Test coverage should never be `None`'
INTERNALERROR> AssertionError: Test coverage should never be `None`
.Coverage.py warning: Module sshtunnel was never imported.
Coverage.py warning: No data was collected.

`sshtunnel` command causes ImportError

sshtunnel installs an sshtunnel command via the console_scripts entry point, but it doesn't point to an existent callable.

$ sshtunnel
Traceback (most recent call last):
  File "~/.virtualenvs/sshtunnel-dev/bin/sshtunnel", line 9, in <module>
    load_entry_point('sshtunnel', 'console_scripts', 'sshtunnel')()
  File "~/.virtualenvs/sshtunnel-dev/lib/python2.7/site-packages/pkg_resources/__init__.py", line 542, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "~/.virtualenvs/sshtunnel-dev/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2542, in load_entry_point
    return ep.load()
  File "~/.virtualenvs/sshtunnel-dev/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2202, in load
    return self.resolve()
  File "~/.virtualenvs/sshtunnel-dev/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2212, in resolve
    raise ImportError(str(exc))
ImportError: 'module' object has no attribute 'main'

port is incorrectly read from ssh config file as a string, leading to AssertionError: PORT is not a number

When initializing SSHTunnelForwarder() with the default ssh config file, an AssertionError is raised:

Traceback (most recent call last):
File "test_ssh.py", line 99, in
remote_bind_address=(mysql_host, int(mysql_port))) as server:
File "/usr/local/lib/python2.7/dist-packages/sshtunnel.py", line 877, in init
check_port(self.ssh_port)
File "/usr/local/lib/python2.7/dist-packages/sshtunnel.py", line 76, in check_port
assert isinstance(port, int), 'PORT is not a number'
AssertionError: PORT is not a number

Looking through the code, it seems that in _read_ssh_config(), ssh_port is taken "as is" from hostname_info. hostname_info, in turn, is the result of paramiko.SSHConfig().lookup() which returns port as string. Thus int assertion fails on ssh_port.

I was able to fix the issue by simply converting ssh_port to int in _read_ssh_config():
sshtunnel.py, line 911: ssh_port = ssh_port or hostname_info.get('port')
change to
sshtunnel.py, line 911: ssh_port = ssh_port or int(hostname_info.get('port'))

Let me know if I'm missing something. Thanks.

New pip release for ssh_private_key_password

I am using sshtunnel with certificates with passwords. It would be awesome to have a new pip release including the ssh_private_key_password code that was merged 15 days ago.

Thanks.

error connecting

Hi,

There is my code:

from sshtunnel import SSHTunnelForwarder

server = SSHTunnelForwarder(
        ssh_address=(config.SSH_HOST,config.SSH_PORT),
        ssh_username=config.SSH_USER,
        ssh_password=config.SSH_PASSWORD,
        remote_bind_address=(config.SSH_REMOTE_HOST, config.SSH_REMOTE_PORT)
        )
server.start()
print server.local_bind_port

And when execute them, i'm receive this error

File "tunel.py", line 34, in
server.start()
File "/usr/local/lib/python2.7/dist-packages/sshtunnel.py", line 664, in start
for _srv in self._server_list
TypeError: init() got an unexpected keyword argument 'daemon'

Can you help me?

Provide install instructions

Wasted a bunch of time wondering why my setup.py install wasn't working, and why setup.py --prefix wasn't recognized. Please provide install instructions within your README.md

Connect Usage scenarios with Usage examples

In the docs, it would be great to provide the figures like in Usage scenarios together with a corresponding code example that covers the scenario depicted in the figure.

Right now, in Figure 1 we have something with port 22 and 8080, but in Example 1 the port numbers are different and the examples are not explained.

tests/test_forwarder.py:656: IndexError

_______________________________________________________________ SSHClientTest.test_deprecate_warnings_are_shown ________________________________________________________________

self = <tests.test_forwarder.SSHClientTest testMethod=test_deprecate_warnings_are_shown>

    @unittest.skipIf(os.name == 'nt',
                     reason='Need to fix test on Windows')
    def test_deprecate_warnings_are_shown(self):
        """Test that when using deprecate arguments a warning is logged"""
        warnings.simplefilter('always')  # don't ignore DeprecationWarnings

        with warnings.catch_warnings(record=True) as w:
            for deprecated_arg in ['ssh_address', 'ssh_host']:
                _kwargs = {
                    deprecated_arg: (self.saddr, self.sport),
                    'ssh_username': SSH_USERNAME,
                    'ssh_password': SSH_PASSWORD,
                    'remote_bind_address': (self.eaddr, self.eport),
                }
                SSHTunnelForwarder(**_kwargs)
                logged_message = "'{0}' is DEPRECATED use '{1}' instead"\
                    .format(deprecated_arg,
                            sshtunnel.DEPRECATIONS[deprecated_arg])
>               self.assertTrue(issubclass(w[-1].category, DeprecationWarning))
E               IndexError: list index out of range

tests/test_forwarder.py:656: IndexError
----------------------------------------------------------------------------- Captured stderr call -----------------------------------------------------------------------------
2016-03-27 10:18:47,934| DEB | MainThrea/0221@test_forw | ********************************************************************************
2016-03-27 10:18:47,934| INF | MainThrea/0222@test_forw | setUp for: TEST_DEPRECATE_WARNINGS_ARE_SHOWN()
2016-03-27 10:18:47,936| INF | MainThrea/0226@test_forw | Socket for ssh-server: 127.0.0.1:50356
2016-03-27 10:18:47,936| INF | MainThrea/0228@test_forw | Socket for echo-server: 127.0.0.1:48965
2016-03-27 10:18:47,937 | INFO    | Skipping loading of ssh config file
2016-03-27 10:18:47,937| INF | MainThrea/0917@sshtunnel | Skipping loading of ssh config file
2016-03-27 10:18:47,992 | INFO    | Connecting to gateway: 127.0.0.1:50356 as user 'ub5zHSih4KsU'
2016-03-27 10:18:47,992| INF | MainThrea/0864@sshtunnel | Connecting to gateway: 127.0.0.1:50356 as user 'ub5zHSih4KsU'
2016-03-27 10:18:47,992 | DEBUG   | Concurrent connections allowed: True
2016-03-27 10:18:47,992| DEB | MainThrea/0867@sshtunnel | Concurrent connections allowed: True
2016-03-27 10:18:47,993| INF | MainThrea/0239@test_forw | tearDown for: TEST_DEPRECATE_WARNINGS_ARE_SHOWN()
2016-03-27 10:18:47,993| INF | MainThrea/0277@test_forw | Sending STOP signal
2016-03-27 10:18:47,994| INF | MainThrea/0256@test_forw | tearDown() ssockl
2016-03-27 10:18:47,994| INF | MainThrea/0256@test_forw | tearDown() esockl
=============================================================== 1 failed, 42 passed, 1 xpassed in 25.28 seconds ================================================================

Script won't exit after one of the loaded ssh keys fails authentication

This is similar to #72. What's different here is that several ssh keys are loaded. The first key used fails authentication. The second key passes. Example script:

import threading
import sshtunnel

with sshtunnel.open_tunnel(
    '<remotehost>',
    debug_level='DEBUG',
    remote_bind_address=('localhost', 5432),
) as server:
    print(server.local_bind_port)

# [t.close() for t in threading.enumerate() if t.__class__.__name__ == "Transport"]

print('FINISH!')

Prints out the following but never exits to the shell:

2016-10-21 11:16:55,045| INF | MainThrea/0981@sshtunnel | 3 keys loaded from agent
2016-10-21 11:16:55,045| INF | MainThrea/0901@sshtunnel | Connecting to gateway: <remotehost>:22 as user 'username'
2016-10-21 11:16:55,045| DEB | MainThrea/0904@sshtunnel | Concurrent connections allowed: True
2016-10-21 11:16:55,045| DEB | MainThrea/1284@sshtunnel | Trying to log in with key: 01234567890123456789012345678901
2016-10-21 11:16:55,133| DEB | MainThrea/1293@sshtunnel | Authentication error
2016-10-21 11:16:55,133| WAR | MainThrea/1333@sshtunnel | Server is not started. Please .start() first!
2016-10-21 11:16:55,133| DEB | MainThrea/1351@sshtunnel | Transport is closed
2016-10-21 11:16:55,134| DEB | MainThrea/1284@sshtunnel | Trying to log in with key: 98765432109876543210987654321098
2016-10-21 11:16:55,222| INF | Srv-37096/1318@sshtunnel | Opening tunnel: 0.0.0.0:37096 <> localhost:5432
37096
2016-10-21 11:16:55,222| INF | MainThrea/1337@sshtunnel | Shutting down tunnel ('0.0.0.0', 37096)
2016-10-21 11:16:55,323| INF | Srv-37096/1324@sshtunnel | Tunnel: 0.0.0.0:37096 <> localhost:5432 released
2016-10-21 11:16:55,442| DEB | MainThrea/1351@sshtunnel | Transport is closed
FINISH!

And similar #72 if the second to last line which explicitly closes the paramiko.Transport threads is uncommented then the script exits correctly.

Using sshtunnel 0.1.1, paramiko 1.17.1

Bad file descriptor error on stop()

Hi I'm seeing an issue while running the following code on a Mac with python v2.7.10 and sshtunnel v0.0.6

I can't cleanly stop the server with 'server.stop()'

server = SSHTunnelForwarder(
  ssh_address=('10.XX.XX.XX',22),
  ssh_username="USERNAME",
  ssh_password="PASSWORD",
  remote_bind_address=('localhost', 8080),
  local_bind_address=('0.0.0.0', 8080)
)

server.start()

server.local_is_up(('localhost', 8080))
True

server.stop()
2016-01-06 10:47:49,554 | ERROR   | Tunnel: 0.0.0.0:8080 <> localhost:8080 error: (9, 'Bad file descriptor')

Ideally I want to be able to create a tunnel, send some traffic. After which the remote server will be restarted, blocking traffic on the tunnel. Then when the remote server powers on again. I want to stop and start the tunnel again. In order to send some more traffic.

After running 'server.start()' the tunnel is shut down but it seems the thread its running on is not stopped.

In [39]: server.start()
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-39-9352b800b946> in <module>()
----> 1 server.start()

/Users/ciaran.whyte/dev/es-tas/venv/lib/python2.7/site-packages/sshtunnel.pyc in start(self)
    693                 self._transport.connect(hostkey=self._ssh_host_key,
    694                                         username=self._ssh_username,
--> 695                                         password=self._ssh_password)
    696             else:
    697                 self.logger.debug('Logging in with RSA key')

/Users/ciaran.whyte/dev/es-tas/venv/lib/python2.7/site-packages/paramiko/transport.pyc in connect(self, hostkey, username, password, pkey, gss_host, gss_auth, gss_kex, gss_deleg_creds)
   1070             self._preferred_keys = [hostkey.get_name()]
   1071
-> 1072         self.start_client()
   1073
   1074         # check host key if we were given one

/Users/ciaran.whyte/dev/es-tas/venv/lib/python2.7/site-packages/paramiko/transport.pyc in start_client(self, event)
    484         # synchronous, wait for a result
    485         self.completion_event = event = threading.Event()
--> 486         self.start()
    487         while True:
    488             event.wait(0.1)

/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.pyc in start(self)
    737             raise RuntimeError("thread.__init__() not called")
    738         if self.__started.is_set():
--> 739             raise RuntimeError("threads can only be started once")
    740         if __debug__:
    741             self._note("%s.start(): starting thread", self)

RuntimeError: threads can only be started once

CLI: AssertionError: ADDRESS is not a tuple (list)

testssh> python -m sshtunnel -U vagrant -P vagrant -L :3306 -R 127.0.0.1:3306 -p 2222 localhost
2015-06-18 00:44:34,637 | WARNING | Could not read SSH configuration file: ~/.ssh/config
Traceback (most recent call last):
  File "C:\Python34\lib\runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Python34\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\pahaz_000\Downloads\!_rm\sshpahaz\sshtunnel.py", line 993, in <module>
    with open_tunnel(**vars(ARGS)) as my_tunnel:
  File "C:\Users\pahaz_000\Downloads\!_rm\sshpahaz\sshtunnel.py", line 883, in open_tunnel
    forwarder = SSHTunnelForwarder(ssh_address, **kwargs)
  File "C:\Users\pahaz_000\Downloads\!_rm\sshpahaz\sshtunnel.py", line 586, in __init__
    check_addresses(remote_bind_addresses)
  File "C:\Users\pahaz_000\Downloads\!_rm\sshpahaz\sshtunnel.py", line 166, in check_addresses
    check_address(address)
  File "C:\Users\pahaz_000\Downloads\!_rm\sshpahaz\sshtunnel.py", line 153, in check_address
    .format(type(address).__name__)
AssertionError: ADDRESS is not a tuple (list)

Is there a way to use the private keys loaded in ssh agent ?

I had my own implementation of the SSH Forwarder using Paramiko, but yours looks a lot more solid. However it doesn't seem to use the the private keys loaded in my ssh agent (the one you can list with ssh-add -l). Is such thing possible in the current version or is this something we would need to develop.

sshtunnel 0.0.8 doesn't work Windows

Release 0.0.8 won't work on Windows.

Specifically, socketserver doesn't expose UnixStreamServer on Windows. Since _UnixStreamForwardServer inherits from socketserver.UnixStreamServer, merely importing sshtunnel results in an Exception being raised on Windows.

Default logger displays ssh password

I'm using getpass to allow the user to enter his ssh password on the command-line, before starting the tunnel. However this is then logged in plain text by the logger (DEBUG level). I can avoid this by enforcing a higher logging level but it would be safer not displaying it at all by default.

When connecting with a proxy command, AttributeError is raised from erroneously calling connect() on paramiko.proxy.ProxyCommand

In sshtunnel.py, in _get_transport(), the code erroneously calls connect() on _socket irregardless of the existence of self.ssh_proxy. This leads to an AttributeError when, for example, a config file has a proxy command. The reason for this is that in this case, _socket is not an instance of socket.socket but paramiko.proxy.ProxyCommand, which has no method connect() and should be passed directly into paramiko.Transport.

I think the change that needs to be made is moving lines 1022-1023 under the else clause, as it was the case with 0.0.8.2.

Traceback (most recent call last):
File "test.py", line 116, in
remote_bind_address=(mysql_host, int(mysql_port))) as server:
File "/usr/local/lib/python2.7/dist-packages/sshtunnel.py", line 1478, in enter
self._raise(exception=exc_class, reason=exc, traceback=tb)
File "/usr/local/lib/python2.7/dist-packages/sshtunnel.py", line 1007, in _raise
raise exception(reason, traceback)
AttributeError: (AttributeError("'ProxyCommand' object has no attribute 'connect'",), <traceback object at 0x7f4c23b98998>)

def _get_transport(self):
    """ Return the SSH transport to the remote gateway """
    if self.ssh_proxy:
        if isinstance(self.ssh_proxy, paramiko.proxy.ProxyCommand):
            proxy_repr = repr(self.ssh_proxy.cmd[1])
        else:
            proxy_repr = repr(self.ssh_proxy)
        self.logger.debug('Connecting via proxy: {0}'.format(proxy_repr))
        _socket = self.ssh_proxy
    else:
        _socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    **_socket.settimeout(SSH_TIMEOUT)**
    **_socket.connect((self.ssh_host, self.ssh_port))**
    transport = paramiko.Transport(_socket)
    transport.set_keepalive(self.set_keepalive)
    transport.use_compression(compress=self.compression)
    transport.daemon = self.daemon_transport

Ignore error

I am using this package in my package to use ipyparallel with our university cluster.

Sometimes if the connection is lost I am getting thousands of lines of the same error printed, however for different ports. This sometimes crashes the notebook I am working in.

Is there an option to silent this error?

Catch “Could not open connection to gateway”

Every now and then setting up a tunnel using sshtunnel.py fails because the gateway (ssh_host) complains the first time I connect to it. I would like to give it a few retries before giving up:

for attempt in range(5):
    try:
        forwarder.start()
    except Exception, e:
        print 'Error (trying again in five seconds):\n' + format(e.message))
        time.sleep(5)
    else:
        break
else:
    print 'Failed to setup a connection to the gateway'
    sys.exit(1)

However, the exception is not caught. I took a peek in the sshtunnel.py code and found that the following code catches the related Paramiko exception:

except paramiko.ssh_exception.AuthenticationException:
    self.logger.error('Could not open connection to gateway')
    return

Does this make it impossible to catch a 'Could not open connection to gateway' exception?

in __init__(), 'local_bind_address=' functions incorrectly

the documentation states that if you pass a tuple like so: local_bind_address=(127.0.0.1, ) it should be fine. however, the program will fail with an assertion error.

Solution:

Either the documentation should be amended, or the functionality of assigning a random port should be instituted.

IOError is raised when `private_key` is not found in file system

When either using CLI or API the file referred in private_key cannot be found, an IOError exception is raised.
Instead, if the private key file is not found, a warning message should be shown. In case a valid password is set, no exception should be raised.

fails to get local interfaces if cannot resolve local hostname

Hey,
I noticed that for some reason, SSHTunnelForwarder would always throw an exception regardless of the parameters,

socket.gaierror: [Errno 8] nodename nor servname provided, or not known

it was caused by _get_local_interfaces trying to resolve the local hostname (in my case, Ich) and failing (probably just a bad /etc/hosts file or something, not really sure)

anyway, this caused me a couple of hours of frustration but i eventually figured out that if i replace

      local_if = socket.gethostbyname_ex(socket.gethostname())[-1]

with

      try:
        local_if = socket.gethostbyname_ex(socket.gethostname())[-1]
    except:
        local_if = []

it will handle this exception.

I can make a pull request if you like.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.