alecmuffett / eotk Goto Github PK

I'm using the recent master of eotk (d25456a) and Tor version 0.3.5.7. When running eotk genkey the script fails, because it seems Tor creates a v3 onion key and lib.d/generate-onion-key.sh tries to mv the file private.key which doesn't exist here. When I do a export ONION_VERSION=3; ./eotk genkey the script works without problems. Also when using Tor 0.3.4.7 /and/ without ONION_VERSION it also works.

Also when trying export ONION_VERSION=3; ./eotk config foo.tconf results in

map: 7nn3d7pc62aisbl6yw2dxtiwhbfcige7xnytuydnu6o43ervefbhihad.v3pub: bad onion address

I guess starting it might be a good idea to test for Tor version. The recent v0.3.5.7 started to default to v3 onions. So if the underlying Tor version is equal or larger than this version it might be better to stick to v3 onions. What do you think?

Add support for Next generation onion service. The control port for V3 was added in the latest alpha of Tor 0.3.3.1. https://gitweb.torproject.org/torspec.git/tree/proposals/284-hsv3-control-port.txt

more info
https://trac.torproject.org/projects/tor/wiki/doc/NextGenOnions?version=50#IntrotoNextGenOnionServicesakaprop224

https://gitweb.torproject.org/torspec.git/tree/rend-spec-v3.txt

to be fixed.

We would like to have the nginx workers be publicly accessible so we can treat them like any other webserver for monitoring purposes. Right now they only listen on a local domain socket (which is cool!) but precludes this.

I've installed EOTK on Rasbian Stretch, and it generates onion keys and configures fine, but when I try to start a project, I get this error:

nginx: [alert] failed to load the 'resty.core' module (https://github.com/openresty/lua-resty-core); ensure you are using an OpenResty release from https://openresty.org/en/download.html (reason: module 'resty.core' not found:
	no field package.preload['resty.core']
	no file './resty/core.lua'
	no file '/usr/local/share/luajit-2.1.0-beta3/resty/core.lua'
	no file '/usr/local/share/lua/5.1/resty/core.lua'
	no file '/usr/local/share/lua/5.1/resty/core/init.lua'
	no file './resty/core.so'
	no file '/usr/local/lib/lua/5.1/resty/core.so'
	no file '/usr/local/lib/lua/5.1/loadall.so'
	no file './resty.so'
	no file '/usr/local/lib/lua/5.1/resty.so'
	no file '/usr/local/lib/lua/5.1/loadall.so') in /home/pi/eotk/projects.d/mp.d/nginx.conf:490

In looking at the Raspbian install script, this should theoretically be working.

see title

I want https://join.foo.onion to redirect to https://join.theintercept.com -- basically, to not load the onion site for that subdomain. In my conf file I have:

set redirect_host ^(join)\\.,301,https://join.theintercept.com

But when I load the https://join.cxppxsjlgsvtq7liq4byprpbp7lun3z2khkv5miumwalfwmnqxcf2ayd.onion, I get the header:

Location: https://join.cxppxsjlgsvtq7liq4byprpbp7lun3z2khkv5miumwalfwmnqxcf2ayd.onion/

When it should be:

Location: https://join.theintercept.com

The connection from nginx to the site should use HTTPS.

Something to keep in mind is that the site might then set cookies as Secure, so you'll have to rewrite the Set-Cookie header.

❤️

If a specific tweak in an nginx.conf file is needed for a particular setup, it would be great if that could be applied globally for all projects, either on config or restart.

see title

    server_name
    %ONION_ADDRESS%
    ~^(?<servernamesubdomain>([-0-9a-z]+\\.)+)%ONION_ADDRESS_RE2%$
    ;

I assume the dashes before 0 are because of subdomains?
However, this regex is wrong. The official Tor regex (for v2 HSes) is [a-z2-7]{16}\.onion
No 1s, 8s or 9s.
Also the {16} will be absolute with v3, which has 56 chars. Do we have the regex for that yet?

I see you have the v1+v2 address regex correct here: https://github.com/alecmuffett/eotk/blob/master/templates.d/nginx.conf.txt#L620

I just installed eotk, then I followed "RUNBOOK.md" but after I typed ./eotk ob-remote-nuke-and-push and it's giving me this:

:::: remote wolfgang: stop -a :::: ssh: Could not resolve hostname wolfgang: Name or service not known :::: rnap wolfgang :::: ssh: Could not resolve hostname wolfgang: Name or service not known rsync: connection unexpectedly closed (0 bytes received so far) [sender] rsync error: unexplained error (code 255) at io.c(235) [sender=3.1.2]

EDIT=I am using a Raspberry Pi 3 with Raspbian
EDIT2=I use only one machine

Recently I wanted to add a domain to a already existing project. I did:

1. eotk genkey
2. Entered the output from above to oldproject.conf. A line like hardmap secrets.d/OUTPUT domainname.
3. Issued eotk config oldproject.conf
4. eotk restart oldproject

I was able to use the onion servie, but got a warning about the certifificate. The certificate just used the old onion service name, but not the newly created ones.

I'd have expected that running config also creates a new certificate. Could this be changed or is it intentional?

fix

Currently, the log files on projects that contain multiple domains mapped do not distinguish between the domains that are requested.

I'm thinking adding something like this:

log_format compression '$remote_user [$time_local] %DNS_DOMAIN%'
                           '"$request" $status $body_bytes_sent '
                           '"$http_referer" "$http_user_agent" ';

to templates.d/nginx.conf.txt might do the trick?

./eotk gen did not work. error: tor failed to launch in /Users/emmettkelleher/eotk/secrets.d/__gok6609.dir cat: /Users/emmettkelleher/eotk/secrets.d/__gok6609.dir/tor.pid: No such file or directory kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]

Hi @alecmuffett,

I need to buy a certificate (or multiple) for an onion site set up with eotk and I'm wondering about the best way to go about it.

The proxy is supposed to cover non-onion domains of the form

• foo.com
• bar.foo.com
• *.bar.foo.com

(but no subdomains other than bar)

I haven't checked yet, but I assume I could buy a single cert to cover all the above cases, or I could get two - one for foo.com and one for [*.]bar.foo.com.

I'm currently testing with just:

hardmap secrets.d/foo.key foo.com

and that works for all subdomains, at all levels (but I only care about the bar subdomain and what's below it). Reading the docs, it seems I should use:

hardmap secrets.d/foo.key foo.com bar

but that conflicts with "you will ignore all hostnames" (bar.foo.com is an address that should work, as well as baz.bar.foo.com).

I'd expect the above to work with one cert. What about multiple? I'm having trouble finding the answer in the docs. Is that supported?

Thanks!

Hi,

why is there not a option to mirror only http onion sites? without the https?

i figured it out with much manual mods but a automated mode would be better

I just installed oetk on my server (Ubuntu 20.04).
I already installed tor before, and I have an onion address for my website.
I had to install the service (SearXNG) via docker with Caddy, and I have access only to Caddyfile.
I bought an SSL certificate from HARICA, and I want to install it to make HTTPS the protocol of the onion address.

Is it possible to install the SSL certificate by eotk without generating a new onion address (using the onion address I already have)?
If possible, I kindly ask you to let me know the steps I will follow.
I appreciate any help you can provide.

Requesting help/brainstorm with deploying EOTK on FSFE websites, see https://git.fsfe.org/FSFE/fsfe-website/pulls/1756

Namely:

1. How should it be deployed over EOTK?
2. Making it easily maintainable
3. Setting HTTP Headers for Onion-Location (if EOTK handles that)

Relevants

FSFE is using SSL/TLS which should be preserved on onions to avoid leaking of sensitive informations[1]

KREYREEN: websites which are HTTPS should remain HTTPS. Websites which are HTTP should remain HTTP. Attempting to convert HTTP to HTTPS (or: vice-versa) on the fly, is painful and inadvisable, because it can break backend expectations. -- Anonymous

SSL Certificate for onion was last time i checked 600USD per year[?] which is hardly an option for any non-profit organization.

We can look at FSFE and work out whether there is a way for it to be Pure-HTTP-without-S in limited circumstances, and then might connect EOTK to that. -- Anonymous

DANE is not an option for cert authority

DANE wont help you in a Tor context since Tor's DNS subsystem is first and foremost not used for .onion's, but also TLSA record lookups aren't supported even for via-exit circuits in Tor. -- Anonymous

And TLSA over proxied end-points where the client doesn't full-chain validation of the DNSSEC signatures will be a very bad combination. -- Anonymous

SOOC - Same Origin Onion Certificate[3]

Idea for the future, not an option atm as it needs changes in the Tor Browser

References

1. https://github.com/alecmuffett/eotk/blob/master/docs.d/security-advisories.d/001-torbrowser.md
2. #21
3. https://github.com/alecmuffett/onion-dv-certificate-proposal/blob/master/text/draft-muffett-same-origin-onion-certificates.txt

Thanks a lot in advance, ^-^

• Krey

./eotk start
2017/02/09 [emerg] 472#472: unknown directive "subs_filter_types" in /home/$USER/Desktop/eotk/projects.d/<website>.d/nginx.conf:41

am i missing nginx dependencies? i'm new to nginx, not sure if pacman -S nginx will include everything i need.

$ uname -a
Linux unknown 4.9.7-1-ARCH #1 SMP PREEMPT Wed Feb 1 19:33:40 CET 2017 x86_64 GNU/Linux

$ nginx -h
nginx version: nginx/1.10.3

My config file has:

hardmap %NEW_V3_ONION% theintercept.com

And also:

set redirect_host ^join\\..*\\.onion$,307,https://join.theintercept.com

This should redirect https://join.[xxx].onion to https://join.theintercept.com. And it appears that it does, except EOTK also see the theintercept.com and rewrites it again to https://join.[xxx].onion, so it ends up in an infinite redirect loop.

harica.gr is issuing Onion SSLs for v3.

To validate, one needs to place a fille in:
http://__________.onion/.well-known/pki-validation/
to validate the CSR

Are there instructions on this floating about? How can someone add these directories?

I am so not a techie and appreciate any help that you may have to offer.
Wish to setup an existing website with an onion address.
And first please let me say - alec muffett, thank you for this!

Computer - MacBook running 10.12.6 with a touchbar
Software - Installed Xcode 8. Installed Homebrew.

Tried to accomplish install using a mixture of How-To_install doc and youtube video.
Everything went groovy until the vi xxxx.conf > set project and hardmap secrets.d...
Now I'm stuck there with all the tilds and can't get back to the $
So, I check the net and what I thought I read is that I am now in Insert mode and I have to get back to Command mode, and to hit ESC key to do that. Hit ESC on my touchbar and nothing (but frog error sounds :)

Questions:

1. Is that what I have to do - get from Insert mode to Command mode?
2. If so, Any solution to my ESC not working and If not, Where I go from hanging here?
3. Why on earth am I doing this instead of surfing right now? Okay that question is for my therapist.

Any help is very very very much appreciated.
thanks!!!
tlel

see title

When I run ./opt.d/install-everything-on-osx.sh I get this error.

Error: homebrew/nginx was deprecated. This tap is now empty as all its formulae were migrated.
+ brew install nginx-full --with-lua-module --with-subs-filter-module --with-headers-more-module

check and warn/fatal if a member of eotk-workers.conf has the same actual hostname as the current working host. People should use localhost instead.

Hi,
I'd like to work on/implement a custom CSP value.
This, to have client-side control over the allowed resources to be requested on an "anonymous" connection.

Will try to submit a Pull Request soon.

Lots of things are needed, I'm opening this as a stub so I don't forget some random ones I encounter.

1. /etc/pki/tls/openssl.cnf is the location on CentOS7

Apologies for the delay:
Gist: https://gist.github.com/CJFWeatherhead/3b907ffc0817a1255658df2195a68a75

Alec completely up to you if you wish to use it

Hello! I'm use this manual for install EOTK on Ubuntu 20.04 server:

git clone https://github.com/alecmuffett/eotk.git
cd eotk
./opt.d/build-ubuntu-20.04.sh

But I have this error when use command ./eotk gen:

root@Atlas:~/eotk# ./eotk gen
error: the only supported value for ONION_VERSION is 3

see title

Want to see all headers, metadata, etc, both as sent, and as rewritten.

Hi everyone,

I tried to compile EOTK on an Ubuntu 20.04, however it always stops with a fatal error during compilation.

Anyone any idea what this could be?

Thx!

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.6 LTS
Release:        20.04
Codename:       focal

+ BuildAndCleanup
+ make
make  all-am
make[1]: Entering directory '/home/user/eotk/opt.d/tor-0.4.5.8'
  CC       src/test/test-test_addr.o
gcc: fatal error: Killed signal terminated program cc1
compilation terminated.
make[1]: *** [Makefile:20144: src/test/test-test_addr.o] Error 1
make[1]: Leaving directory '/home/user/eotk/opt.d/tor-0.4.5.8'
make: *** [Makefile:7343: all] Error 2
+ exit 1

Looks like your builds aren't currently passing any options to GCC but if people are using this in production directly, then it might be a good idea to throw a few optimisations in there if they offer some benefit without compromising how the builds function in any real way.

Might be worth starting a discussion about the kind of options that might be worth adding.

After some experimentation with the Nginx build options, the default CFLAGS from the Makefile are set to:

-pipe -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -D_GLIBCXX_USE_CXX11_ABI=0 -Wno-unused-local-typedefs -Wno-error

Some that might be worth adding, maybe:

• -fPIE - build position-independent. We can do this in the nginx build since we're building using static modules only, rather than dynamic (if we want dynamic ones we need to use -fPIC instead).
• -fstack-protector-all - I think stack protection handles overflow/underrun errors by halting execution, so I'm not sure about this one if you're targeting an enterprise scenario, but I'd probably turn it on anyway.
• --fstack-protector --param=ssp-buffer-size=4 - as above, but more relaxed for performance reasons (canaries only added to functions containing an array larger than 4-bytes)
• -Wp,-D_FORTIFY_SOURCE=2 - see Debian Wiki.

Maybe one for the linker too:

• -Wl,-z,relro - During program load, several ELF memory sections need to be written to by the linker, but can be turned read-only before turning over control to the program. (Debian wiki)

So currently I'm building with:

--with-cc-opt='-fPIE -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-all' \
--with-ld-opt='-Wl,-z,relro' \

I did briefly experiment with -O2 as a performance optimisation, but Nginx defaults to -O and this would need to be overriden in the Makefile directly rather than with configure options. I'm guessing that's for a reason.

But all in all, everything seems to work very nicely using these options.

Can't do anything with the source code in windows. Remember that the general rule is that releases are supposed to be EXECUTABLE INSTALLATION FILES not just mere source code, which should be included under the "code" section of the repository.

Hi, I found a typo error which resulted in a broken link in HOW-TO-INSTALL.md documentation, its about Adding root certificate in tor browser. I have fixed it in #87 Pull request.

I'm pretty sure that this setup is not included in the current configuration options?

The reason is that we are having real trouble getting onion SSL certs for non corporate entities. We will continue to fight obtain them but in the mean time it would be great to have a working solution which would setup an HTTP onion address as a proxy for an HTTPS website.

I will be attempting to submit a pull-request with this code but won't get to it this week so wanted to post it up as an issue in case anyone else fancied doing it :-)

I set it up last night and experimented using a domain name I control.

I implement @ietf RFC 7469 to stipulate specific X.509 credentials in @apache on all the domain names I control. @TheTorProject client doesn't connect to the clear-text host-name through the @TheTorProject domain name because the self-signed X.509 credential set up for the @TheTorProject domain name suffix doesn't match.

It also affects @ietf RFC 6797 if the X.509 credential for the clear-text host-name is preloaded in @TheTorProject client. It therefore stops connecting to the clear-text host-name because of the subsequent mismatch. I'd suggest adding the Secure Hash Algorithm 2 output for @TheTorProject domain name to the stipulated X.509 credentials @apache but it's not going to fix it because the self-signed X.509 credential itself is untrusted.

I realize that the configuration I use is uncommon but it is a useful practical demonstration of the benefit and robustness of @ietf RFC 7469 and @ietf RFC 6797!

@ietf RFC 7469 is a useful mechanism but I feel it also might be detrimental because it could permit identification of a clear-text host-name if it's being used to host both clear-text material and material through @TheTorProject and the Secure Hash Algorithm 2 output is identified through comparing a database of clear-text domain names and their subsequent Secure Hash Algorithm 2 output.