Currently all the code is concatenated and printed. Add a CL arg like --beautify to show output like

Code in Python

import requests
.....

I am not sure where the current approach came from, it would be much more useful to simply paste the whole HTTP request once and have the translator translate directly (i.e. in a single step: one paste x translation).

Instead, at present, the translator requires the user to enter the request in 2 steps:
Step 1) Enter request headers (Ctrl+D to finish/Ctrl+C to quit).
Step 2) Enter request body/parameters (Ctrl+D to finish/Ctrl+C to quit).

Given that HTTP requests have a known structure (2 newlines or \r\n\r\n between headers and body), it should be simple enough for the translator to let users paste the whole request once + figure out the rest from there, saving some time to the user ;)

The templates that are used to generate the different scripts are in the same files as the python code to generate the scripts.

It makes it hard for one to read the source code and it is also a poor design to keep it that way.

A better design would be to separate the templates from the source code.

Description

Error on submitting HTTP request for 9gag.com

HTTP Request
GET / HTTP/1.1
Host: 9gag.com
Connection: keep-alive

I think blindly trusting https://gist.github.com/mnordhoff/2213179 was not a good idea.
Error can be found in hrt/util.py

9gag.com which is correct domain failing to satisfy written regex.

The current documentation of the project (that one can find in '/doc/') is far from being complete. It is currently poorly written and does not contain much.

That would be nice to update the documentation to show every person who's interested in the project, what http request translator does for them.

An extremely cool feature to have here would be conversion to send the request via JavaScript only, so that the tester has a fully working initial draft to play with from the browser development tools:
Something that can easily be copy-pasted and executed via F12 / Console tab on most modern browsers, without installing any tools, this is super-handy for PoCs :)

The unit tests for the project do not cover all of the code base, nor properly test everything.

It would be nice to continue updating the tests to increase the coverage while still being meaningful.

The code still has some parts that does not follow PEP8. Mixed spaces and tabs and too long lines of code are some of the primary issues.

ValueError: Invalid URL 'http://www.cmd5.com/'.
is shown.
The regexp in utils.py is wrong.It can't deal the domain with numbers.

A simple request like:

GET http://127.0.0.1:80/robots.txt HTTP/1.1
Host: 127.0.0.1:80
Accept-Encoding: identity
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:6.0) Gecko/20100101 Firefox/15.0

is not considered valid according to check_valid_url in util.py.

The problem is that check_valid_url does not consider the case where the domain name is an IP address.
The solution would be to extend check_valid_url to support IPv4 and IPv6.

The OWASP GSoC Idea related to HTTP Request Translator does not link to this repo.

So, I was under the impression that we might have to create a new tool from scratch, and not update this code. Is that the case?

I think that the existing codebase isn't as clean as it can be and have some plans in mind regarding how we can structure our code so that it has an extensible architecture (something that is mentioned in the idea.)

I've recently added a similar feature (exporting requests as Python Code etc.) to mitmproxy. Here are the relevant pulls: mitmproxy/mitmproxy#884 & mitmproxy/mitmproxy#916

If you're interested in a quick prototype of my design, I can get it done this weekend.

Raw request

b'POST /tslwebapp/example/HelloWorld.action HTTP/1.1\r\nHost: 10.28.194.39:8080\r\nContent-Type: multipart/form-data; boundary=--------429242435\r\nContent-Length: 224\r\n\r\n----------429242435\r\nContent-Disposition: form-data; name="${#_memberAccess["allowStaticMethodAccess"]=true,@java.lang.Runtime@getRuntime().exec(\'touch /foo\')}=1"\r\n\r\n----------429242435--'

Request CURL translated by HRT

#!/usr/bin/env bash
curl --data "----------429242435Content-Disposition: form-data; name=\"${#_memberAccess[\"allowStaticMethodAccess\"]=true,@java.lang.Runtime@getRuntime().exec('touch /foo')}=1\"----------429242435--"  -v --request POST http://172.16.244.210:8000/tslwebapp/example/HelloWorld.action  --header "Host: 172.16.244.210:8000"  --header "Content-Type: multipart/form-data; boundary=--------429242435"  --header "Content-Length: 224"  --include

intercepted at proxy

POST /tslwebapp/example/HelloWorld.action HTTP/1.1
Host: 172.16.244.210:8000
User-Agent: curl/7.54.0
Accept: */*
Content-Type: multipart/form-data; boundary=--------429242435
Content-Length: 82
Connection: close

----------429242435Content-Disposition: form-data; name="0=1"----------429242435--

How it should look like

POST /tslwebapp/example/HelloWorld.action HTTP/1.1
Host: 172.16.244.210:8000
User-Agent: curl/7.54.0
Accept: */*
Content-Type :multipart/form-data; boundary=--------429242435
Content-Length: 224
Content-Type: application/x-www-form-urlencoded
Connection: close

----------429242435
Content-Disposition: form-data; name="${#_memberAccess["allowStaticMethodAccess"]=true,@java.lang.Runtime@getRuntime().exec('touch /foor')}=1"

----------429242435--

Right now, HRT uses pycurl to transform the HTTP request to Python. It should use the more standard requests to do this.

Review setup.py added in the pull request #23 at arunk-s@4ed0b76 for any name changes or other stuff.
Also, test it by installing it locally or in a virutalenv.

@DePierre, can you publish the tool on PyPI? Then we can easily integrate it in OWTF's web interface. (owtf/owtf#829)

It looks like when using interactive mode like this:
http_request_translator --interactive

When unexpected Control + D sequences are used, the following happens, would be more professional to handle this in a more graceful way :)

Steps:

1. Paste a request + convert it
2. Paste another request, play with Control + D at the wrong time
3. The following happens:

>>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> 
>>> Traceback (most recent call last):
  File "/usr/local/bin/http_request_translator", line 4, in <module>
    __import__('pkg_resources').run_script('http-request-translator==0.1', 'http_request_translator')
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 735, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1659, in run_script
    exec(script_code, namespace, namespace)
  File "/usr/local/lib/python2.7/dist-packages/http_request_translator-0.1-py2.7.egg/EGG-INFO/scripts/http_request_translator", line 54, in <module>

  File "/usr/local/lib/python2.7/dist-packages/http_request_translator-0.1-py2.7.egg/EGG-INFO/scripts/http_request_translator", line 49, in take_arguments

  File "build/bdist.linux-i686/egg/http_request_translator/translator.py", line 34, in process_arguments
  File "build/bdist.linux-i686/egg/http_request_translator/translator.py", line 100, in take_headers
  File "build/bdist.linux-i686/egg/http_request_translator/translator.py", line 131, in take_body
  File "build/bdist.linux-i686/egg/http_request_translator/translator.py", line 100, in take_headers
  File "build/bdist.linux-i686/egg/http_request_translator/translator.py", line 122, in take_body
  File "build/bdist.linux-i686/egg/http_request_translator/translator.py", line 170, in parse_raw_request
UnboundLocalError: local variable 'host' referenced before assignment

The custom URI parsing logic written in hrt/url.py should not be used (including the regexes). Instead a reliable parsing module like https://github.com/google/ipaddr-py should be used.

@DePierre ?

It seems that ruby_script fails when dealing with ip addresses. For instance:

$python2 translator.py --output ruby --Request "GET http://127.0.0.1/robots.txt HTTP/1.1
Host: 127.0.0.1"
@@@ DEBUG input check_valid_url 'http://127.0.0.1http://127.0.0.1/robots.txt'
# . . .

This does not happen with python_script for instance:

$python2 translator.py --output python --Request "GET http://127.0.0.1/robots.txt HTTP/1.1   
Host: 127.0.0.1"
@@@ DEBUG input check_valid_url 'http://127.0.0.1'
# . . .

  --interactive, -i     Interactive mode: read raw HTTP request from keyboard,
                        hit enter when ready.Type ':q!' to exit from the
                        interactive mode.

is wrong，actually it should be

  --interactive, -i     Interactive mode: read raw HTTP request from keyboard,
                        hit enter when ready.Press 'Ctrl+D' to finish or 'Ctrl+C'
                        to exit from the interactive mode.

The following URL [::1]:80 (IPv6 localhost on port 80, see https://www.ietf.org/rfc/rfc2732.txt) triggers the following exception:

Traceback (most recent call last):
  File "translator.py", line 138, in <module>
    main()
  File "translator.py", line 135, in main
    args = take_arguments()
  File "translator.py", line 44, in take_arguments
    process_arguments(parser.parse_args())
  File "translator.py", line 76, in process_arguments
    plugin_manager(script_list, parsed_tuple)
  File "/http-request-translator/http_request_translator/plugin_manager.py", line 14, in plugin_manager
    searchString)
  File "/http-request-translator/http_request_translator/ruby_script.py", line 22, in generate_script
    url = get_url(header_dict['Host'])
  File "/http-request-translator/http_request_translator/util.py", line 50, in get_url
    if int(port.strip()) in port_protocol.keys():
ValueError: invalid literal for int() with base 10: ''

It looks like the request translator will fail when an "unusual" http method like PUT is used, the http method might need to be something like "OMG" or anything else, especially given the target user base of this tool who will all be trying to mess with a web server :)
NOTE: Other translation modes are probably also affected I guess?

Step 1) run in interactive mode

http_request_translator --interactive

Step 2) Paste the request
NOTE: The same request with a "GET" method worked
NOTE 2: When fixing, please try other methods like MEC, TEST or OMG, be creative, all strange methods should be translatable (hint for test case!) :)

PUT https://target.com/path/to/target HTTP/1.1
[some headers here...]

Step 3) Control + D and Control + D results in this error:
btw, please fix the horrible typo too :D

Please Enter a Vaild Request!