alecmuffett / eotk Goto Github PK
View Code? Open in Web Editor NEWEnterprise Onion Toolkit
License: GNU General Public License v3.0
Enterprise Onion Toolkit
License: GNU General Public License v3.0
I've installed EOTK on Rasbian Stretch, and it generates onion keys and configures fine, but when I try to start a project, I get this error:
nginx: [alert] failed to load the 'resty.core' module (https://github.com/openresty/lua-resty-core); ensure you are using an OpenResty release from https://openresty.org/en/download.html (reason: module 'resty.core' not found:
no field package.preload['resty.core']
no file './resty/core.lua'
no file '/usr/local/share/luajit-2.1.0-beta3/resty/core.lua'
no file '/usr/local/share/lua/5.1/resty/core.lua'
no file '/usr/local/share/lua/5.1/resty/core/init.lua'
no file './resty/core.so'
no file '/usr/local/lib/lua/5.1/resty/core.so'
no file '/usr/local/lib/lua/5.1/loadall.so'
no file './resty.so'
no file '/usr/local/lib/lua/5.1/resty.so'
no file '/usr/local/lib/lua/5.1/loadall.so') in /home/pi/eotk/projects.d/mp.d/nginx.conf:490
In looking at the Raspbian install script, this should theoretically be working.
I set it up last night and experimented using a domain name I control.
I implement @ietf RFC 7469 to stipulate specific X.509 credentials in @apache on all the domain names I control. @TheTorProject client doesn't connect to the clear-text host-name through the @TheTorProject domain name because the self-signed X.509 credential set up for the @TheTorProject domain name suffix doesn't match.
It also affects @ietf RFC 6797 if the X.509 credential for the clear-text host-name is preloaded in @TheTorProject client. It therefore stops connecting to the clear-text host-name because of the subsequent mismatch. I'd suggest adding the Secure Hash Algorithm 2 output for @TheTorProject domain name to the stipulated X.509 credentials @apache but it's not going to fix it because the self-signed X.509 credential itself is untrusted.
I realize that the configuration I use is uncommon but it is a useful practical demonstration of the benefit and robustness of @ietf RFC 7469 and @ietf RFC 6797!
@ietf RFC 7469 is a useful mechanism but I feel it also might be detrimental because it could permit identification of a clear-text host-name if it's being used to host both clear-text material and material through @TheTorProject and the Secure Hash Algorithm 2 output is identified through comparing a database of clear-text domain names and their subsequent Secure Hash Algorithm 2 output.
see title
to be fixed.
see title
My config file has:
hardmap %NEW_V3_ONION% theintercept.com
And also:
set redirect_host ^join\\..*\\.onion$,307,https://join.theintercept.com
This should redirect https://join.[xxx].onion
to https://join.theintercept.com
. And it appears that it does, except EOTK also see the theintercept.com
and rewrites it again to https://join.[xxx].onion
, so it ends up in an infinite redirect loop.
server_name
%ONION_ADDRESS%
~^(?<servernamesubdomain>([-0-9a-z]+\\.)+)%ONION_ADDRESS_RE2%$
;
eotk/templates.d/nginx.conf.txt
Line 524 in 8c040ed
I assume the dashes before 0 are because of subdomains?
However, this regex is wrong. The official Tor regex (for v2 HSes) is [a-z2-7]{16}\.onion
No 1s, 8s or 9s.
Also the {16} will be absolute with v3, which has 56 chars. Do we have the regex for that yet?
I see you have the v1+v2 address regex correct here: https://github.com/alecmuffett/eotk/blob/master/templates.d/nginx.conf.txt#L620
Recently I wanted to add a domain to a already existing project. I did:
eotk genkey
oldproject.conf
. A line like hardmap secrets.d/OUTPUT domainname
.eotk config oldproject.conf
eotk restart oldproject
I was able to use the onion servie, but got a warning about the certifificate. The certificate just used the old onion service name, but not the newly created ones.
I'd have expected that running config
also creates a new certificate. Could this be changed or is it intentional?
Hi,
I'd like to work on/implement a custom CSP value.
This, to have client-side control over the allowed resources to be requested on an "anonymous" connection.
Will try to submit a Pull Request soon.
see title
check and warn/fatal if a member of eotk-workers.conf has the same actual hostname as the current working host. People should use localhost
instead.
fix
Looks like your builds aren't currently passing any options to GCC but if people are using this in production directly, then it might be a good idea to throw a few optimisations in there if they offer some benefit without compromising how the builds function in any real way.
Might be worth starting a discussion about the kind of options that might be worth adding.
After some experimentation with the Nginx build options, the default CFLAGS
from the Makefile are set to:
-pipe -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -D_GLIBCXX_USE_CXX11_ABI=0 -Wno-unused-local-typedefs -Wno-error
Some that might be worth adding, maybe:
-fPIE
- build position-independent. We can do this in the nginx build since we're building using static modules only, rather than dynamic (if we want dynamic ones we need to use -fPIC
instead).-fstack-protector-all
- I think stack protection handles overflow/underrun errors by halting execution, so I'm not sure about this one if you're targeting an enterprise scenario, but I'd probably turn it on anyway.--fstack-protector --param=ssp-buffer-size=4
- as above, but more relaxed for performance reasons (canaries only added to functions containing an array larger than 4-bytes)-Wp,-D_FORTIFY_SOURCE=2
- see Debian Wiki.Maybe one for the linker too:
-Wl,-z,relro
- During program load, several ELF memory sections need to be written to by the linker, but can be turned read-only before turning over control to the program. (Debian wiki)So currently I'm building with:
--with-cc-opt='-fPIE -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-all' \
--with-ld-opt='-Wl,-z,relro' \
I did briefly experiment with -O2
as a performance optimisation, but Nginx defaults to -O
and this would need to be overriden in the Makefile directly rather than with configure
options. I'm guessing that's for a reason.
But all in all, everything seems to work very nicely using these options.
harica.gr is issuing Onion SSLs for v3.
To validate, one needs to place a fille in:
http://__________.onion/.well-known/pki-validation/
to validate the CSR
Are there instructions on this floating about? How can someone add these directories?
see title
Add support for Next generation onion service. The control port for V3 was added in the latest alpha of Tor 0.3.3.1. https://gitweb.torproject.org/torspec.git/tree/proposals/284-hsv3-control-port.txt
https://gitweb.torproject.org/torspec.git/tree/rend-spec-v3.txt
I just installed oetk
on my server (Ubuntu 20.04).
I already installed tor before, and I have an onion address for my website.
I had to install the service (SearXNG) via docker with Caddy, and I have access only to Caddyfile.
I bought an SSL certificate from HARICA, and I want to install it to make HTTPS the protocol of the onion address.
Is it possible to install the SSL certificate by eotk
without generating a new onion address (using the onion address I already have)?
If possible, I kindly ask you to let me know the steps I will follow.
I appreciate any help you can provide.
I am so not a techie and appreciate any help that you may have to offer.
Wish to setup an existing website with an onion address.
And first please let me say - alec muffett, thank you for this!
Computer - MacBook running 10.12.6 with a touchbar
Software - Installed Xcode 8. Installed Homebrew.
Tried to accomplish install using a mixture of How-To_install doc and youtube video.
Everything went groovy until the vi xxxx.conf > set project and hardmap secrets.d...
Now I'm stuck there with all the tilds and can't get back to the $
So, I check the net and what I thought I read is that I am now in Insert mode and I have to get back to Command mode, and to hit ESC key to do that. Hit ESC on my touchbar and nothing (but frog error sounds :)
Questions:
Any help is very very very much appreciated.
thanks!!!
tlel
Hello! I'm use this manual for install EOTK on Ubuntu 20.04 server:
git clone https://github.com/alecmuffett/eotk.git
cd eotk
./opt.d/build-ubuntu-20.04.sh
But I have this error when use command ./eotk gen
:
root@Atlas:~/eotk# ./eotk gen
error: the only supported value for ONION_VERSION is 3
Hi @alecmuffett,
I need to buy a certificate (or multiple) for an onion site set up with eotk and I'm wondering about the best way to go about it.
The proxy is supposed to cover non-onion domains of the form
(but no subdomains other than bar
)
I haven't checked yet, but I assume I could buy a single cert to cover all the above cases, or I could get two - one for foo.com and one for [*.]bar.foo.com.
I'm currently testing with just:
hardmap secrets.d/foo.key foo.com
and that works for all subdomains, at all levels (but I only care about the bar
subdomain and what's below it). Reading the docs, it seems I should use:
hardmap secrets.d/foo.key foo.com bar
but that conflicts with "you will ignore all hostnames" (bar.foo.com is an address that should work, as well as baz.bar.foo.com).
I'd expect the above to work with one cert. What about multiple? I'm having trouble finding the answer in the docs. Is that supported?
Thanks!
Lots of things are needed, I'm opening this as a stub so I don't forget some random ones I encounter.
Requesting help/brainstorm with deploying EOTK on FSFE websites, see https://git.fsfe.org/FSFE/fsfe-website/pulls/1756
Namely:
KREYREEN: websites which are HTTPS should remain HTTPS. Websites which are HTTP should remain HTTP. Attempting to convert HTTP to HTTPS (or: vice-versa) on the fly, is painful and inadvisable, because it can break backend expectations. -- Anonymous
SSL Certificate for onion was last time i checked 600USD per year[?] which is hardly an option for any non-profit organization.
We can look at FSFE and work out whether there is a way for it to be Pure-HTTP-without-S in limited circumstances, and then might connect EOTK to that. -- Anonymous
DANE wont help you in a Tor context since Tor's DNS subsystem is first and foremost not used for .onion's, but also TLSA record lookups aren't supported even for via-exit circuits in Tor. -- Anonymous
And TLSA over proxied end-points where the client doesn't full-chain validation of the DNSSEC signatures will be a very bad combination. -- Anonymous
Idea for the future, not an option atm as it needs changes in the Tor Browser
Thanks a lot in advance, ^-^
I'm pretty sure that this setup is not included in the current configuration options?
The reason is that we are having real trouble getting onion SSL certs for non corporate entities. We will continue to fight obtain them but in the mean time it would be great to have a working solution which would setup an HTTP onion address as a proxy for an HTTPS website.
I will be attempting to submit a pull-request with this code but won't get to it this week so wanted to post it up as an issue in case anyone else fancied doing it :-)
I want https://join.foo.onion
to redirect to https://join.theintercept.com
-- basically, to not load the onion site for that subdomain. In my conf file I have:
set redirect_host ^(join)\\.,301,https://join.theintercept.com
But when I load the https://join.cxppxsjlgsvtq7liq4byprpbp7lun3z2khkv5miumwalfwmnqxcf2ayd.onion
, I get the header:
Location: https://join.cxppxsjlgsvtq7liq4byprpbp7lun3z2khkv5miumwalfwmnqxcf2ayd.onion/
When it should be:
Location: https://join.theintercept.com
Want to see all headers, metadata, etc, both as sent, and as rewritten.
./eotk gen did not work. error: tor failed to launch in /Users/emmettkelleher/eotk/secrets.d/__gok6609.dir cat: /Users/emmettkelleher/eotk/secrets.d/__gok6609.dir/tor.pid: No such file or directory kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
Can't do anything with the source code in windows. Remember that the general rule is that releases are supposed to be EXECUTABLE INSTALLATION FILES not just mere source code, which should be included under the "code" section of the repository.
We would like to have the nginx workers be publicly accessible so we can treat them like any other webserver for monitoring purposes. Right now they only listen on a local domain socket (which is cool!) but precludes this.
I just installed eotk, then I followed "RUNBOOK.md" but after I typed ./eotk ob-remote-nuke-and-push and it's giving me this:
:::: remote wolfgang: stop -a :::: ssh: Could not resolve hostname wolfgang: Name or service not known :::: rnap wolfgang :::: ssh: Could not resolve hostname wolfgang: Name or service not known rsync: connection unexpectedly closed (0 bytes received so far) [sender] rsync error: unexplained error (code 255) at io.c(235) [sender=3.1.2]
EDIT=I am using a Raspberry Pi 3 with Raspbian
EDIT2=I use only one machine
./eotk start
2017/02/09 [emerg] 472#472: unknown directive "subs_filter_types" in /home/$USER/Desktop/eotk/projects.d/<website>.d/nginx.conf:41
am i missing nginx dependencies? i'm new to nginx, not sure if pacman -S nginx
will include everything i need.
$ uname -a
Linux unknown 4.9.7-1-ARCH #1 SMP PREEMPT Wed Feb 1 19:33:40 CET 2017 x86_64 GNU/Linux
$ nginx -h
nginx version: nginx/1.10.3
When I run ./opt.d/install-everything-on-osx.sh
I get this error.
Error: homebrew/nginx was deprecated. This tap is now empty as all its formulae were migrated.
+ brew install nginx-full --with-lua-module --with-subs-filter-module --with-headers-more-module
Currently, the log files on projects that contain multiple domains mapped do not distinguish between the domains that are requested.
I'm thinking adding something like this:
log_format compression '$remote_user [$time_local] %DNS_DOMAIN%'
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent" ';
to templates.d/nginx.conf.txt might do the trick?
Hi, I found a typo error which resulted in a broken link in HOW-TO-INSTALL.md documentation, its about Adding root certificate in tor browser. I have fixed it in #87 Pull request.
Hi everyone,
I tried to compile EOTK on an Ubuntu 20.04, however it always stops with a fatal error during compilation.
Anyone any idea what this could be?
Thx!
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.6 LTS
Release: 20.04
Codename: focal
+ BuildAndCleanup
+ make
make all-am
make[1]: Entering directory '/home/user/eotk/opt.d/tor-0.4.5.8'
CC src/test/test-test_addr.o
gcc: fatal error: Killed signal terminated program cc1
compilation terminated.
make[1]: *** [Makefile:20144: src/test/test-test_addr.o] Error 1
make[1]: Leaving directory '/home/user/eotk/opt.d/tor-0.4.5.8'
make: *** [Makefile:7343: all] Error 2
+ exit 1
If a specific tweak in an nginx.conf file is needed for a particular setup, it would be great if that could be applied globally for all projects, either on config or restart.
I'm using the recent master of eotk (d25456a) and Tor version 0.3.5.7. When running eotk genkey
the script fails, because it seems Tor creates a v3 onion key and lib.d/generate-onion-key.sh
tries to mv
the file private.key
which doesn't exist here. When I do a export ONION_VERSION=3; ./eotk genkey
the script works without problems. Also when using Tor 0.3.4.7 /and/ without ONION_VERSION
it also works.
Also when trying export ONION_VERSION=3; ./eotk config foo.tconf
results in
map: 7nn3d7pc62aisbl6yw2dxtiwhbfcige7xnytuydnu6o43ervefbhihad.v3pub: bad onion address
I guess starting it might be a good idea to test for Tor version. The recent v0.3.5.7 started to default to v3 onions. So if the underlying Tor version is equal or larger than this version it might be better to stick to v3 onions. What do you think?
The connection from nginx to the site should use HTTPS.
Something to keep in mind is that the site might then set cookies as Secure, so you'll have to rewrite the Set-Cookie header.
❤️
Apologies for the delay:
Gist: https://gist.github.com/CJFWeatherhead/3b907ffc0817a1255658df2195a68a75
Alec completely up to you if you wish to use it
Hi,
why is there not a option to mirror only http onion sites? without the https?
i figured it out with much manual mods but a automated mode would be better
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.