noscripter / theharvester Goto Github PK

What steps will reproduce the problem?
1. run a command using -b linkedin
2. python theHarvester.py -d securitytube.net -l 500 -b linkedin
3. python theHarvester.py -d microsoft.com -l 500 -b linkedin
4. python theHarvester.py -d microsoft  -l 500 -b linkedin

What is the expected output? What do you see instead?
Expected to find info as in the examples, but nothing is output - 0 results

What version of the product are you using? On what operating system?
Backtrack 5 R3 version 2.2 of theHarvester

Please provide any additional information below.
I have tried several sites, including ones I know have linkedin links and info 
and users (my work - me) came up empty. It also does not create an XML or HTML 
file when using linkedin (-f linkedin.html)

I would like to know what are the steps for the installation of the API-KEY to 
make search in SHODAN 

I am using kali linux in their latest version   

best regards ! 

The regexs in myparser.py arse case sensitive, I think it makes sense to make 
them case insensitive to yield more results.

To do this for emails change like 34 in myparser.py to:

reg_emails = re.compile('[a-zA-Z0-9.-_]*' + '@' + '[a-zA-Z0-9.-]*' + self.word, 
re.I)

^ re.I will also have to be added to the other regexs within myparser.py to 
make them case insensitive.

Hi,

First of all thank you for taking the time to create theHarvester, I use it 
very often.

One thing that I think that it is missing is the ability to output exactly 
where the email addresses were found. For example, if emails were found via a 
Google search, it would be cool to show on which URLs the email addresses were 
found.

Thanks again for such a great tool!

Ryan

What steps will reproduce the problem?
1. theharvester -d cisco.com -b exalead

What is the expected output? What do you see instead?
Invalid search engine, try with: bing, google, linkedin, pgp, exalead, jigsaw, 
bing_api, people123, google-profiles


What version of the product are you using?
Ver. 2.2a 

On what operating system? 
Linux kali 3.14-kali1-486 #1 Debian 3.14.5-1kali1 (2014-06-07) i686 GNU/Linux

Hello sir,
         please sir am a new comer,new student, sir my question goes thus 
1. how can i install ubutu linux 
2. how can i use (theharvester) on windows xp 
3. how can i hack host, smtp,rdp, admin eamils
please sir i will be glad if you can give positive response to all what i have 
whritten up, Thank very much i wich i should be one of your best student 
because i want to learn and good aslo, Hope i will hear from you soon Thanks 
very much 



Best regard
oluwaseun 

What steps will reproduce the problem?
1. Executing a search:

./theHarvester.py -d somedomain.com -l 500 -b all -f temp.html

What is the expected output? What do you see instead?
Output is not being entirely placed into the html output. Emails and hosts 
found are, but the link(s) prior to the email (e.g. google links to pages) are 
not being placed into the html report.

What version of the product are you using? On what operating system?
* TheHarvester Ver. 2.2a 

Please provide any additional information below.

What steps will reproduce the problem?
1. ./theharvester.py -d microsoft -l 200 -b linkedin


What is the expected output? What do you see instead?
List linked users

What version of the product are you using? On what operating system?
theHarvester2.2

Please provide any additional information below.

The problem resides in the unique function in myparser.py and the affected 
statemnt is 

if x[0] != "@":

i think that this can be fixed editing the previous statement with this one:

if x != "" and x[0] != "@":

$ ./theHarvester.py -d XXX.com -l 500 -b all

*******************************************************************
*                                                                 *
* | |_| |__   ___    /\  /\__ _ _ ____   _____  ___| |_ ___ _ __  *
* | __| '_ \ / _ \  / /_/ / _` | '__\ \ / / _ \/ __| __/ _ \ '__| *
* | |_| | | |  __/ / __  / (_| | |   \ V /  __/\__ \ ||  __/ |    *
*  \__|_| |_|\___| \/ /_/ \__,_|_|    \_/ \___||___/\__\___|_|    *
*                                                                 *
* TheHarvester Ver. 2.2a                                          *
* Coded by Christian Martorella                                   *
* Edge-Security Research                                          *
* [email protected]                                   *
*******************************************************************


Full harvest..
[-] Searching in Google..
    Searching 0 results...
    Searching 100 results...
    Searching 200 results...
    Searching 300 results...
    Searching 400 results...
    Searching 500 results...
[-] Searching in PGP Key server..
[-] Searching in Bing..
    Searching 50 results...
    Searching 100 results...
    Searching 150 results...
    Searching 200 results...
    Searching 250 results...
    Searching 300 results...
    Searching 350 results...
    Searching 400 results...
    Searching 450 results...
    Searching 500 results...
[-] Searching in Exalead..




<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" 
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-US" lang="en-US">





  <head >
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
    <meta name="description" content="The best search engine out there"/>
    <meta name="viewport" content="initial-scale = 1, user-scalable = no" />
    <meta http-equiv="X-UA-Compatible" content="IE=8" />
    <link rel="icon" href="http://www.3ds.com/favicon.ico"/>
    <link rel="shortcut icon" href="http://www.3ds.com/favicon.ico"/>

    <link type="application/opensearchdescription+xml" rel="search" title="Exalead" href="/go/opensearchdescription/"/>

    <link type="text/css" rel="stylesheet" href="/content/media/css/base.css?1349082334"/>



    <link rel="help" href="faq/"/>
    <link rel="home" href="/search/web/results/"/>




          <link rel="first" href="/search/web/results/?q=%XXX.com&amp;elements_per_page=15&amp;start_index=0"/>



          <link rel="next" href="/search/web/results/?q=%XXX.com&amp;elements_per_page=15&amp;start_index=15"/>




    <title>
  Web Search - "@XXX.com"
 - Exalead</title>

What steps will reproduce the problem?
1. Download version 2.2 (latest) tar from google code.
2. Untar.
3. Run. ($ ./theHarvester.py -f REDACTED -l 500 -b all)

What is the expected output? 

STDOUT of the results

What do you see instead?

$ ./theHarvester.py -f REDACTED -l 500 -b all

*************************************
*TheHarvester Ver. 2.2              *
*Coded by Christian Martorella      *
*Edge-Security Research             *
*[email protected]      *
*************************************


Full harvest..
[-] Searching in Google..
$

What version of the product are you using?

2.2 TAR from Google Code

On what operating system?

Mac OS X Mountain Lion

Please provide any additional information below.

I think it is probably due to a dependency issue. Would be useful if the 
dependency error was shown to know which is needed.

Hi,

I've been working in some modifications and added support for private search 
engines like ixquick, duckduckgo and disconnect.me (with bing, google, yahoo 
and duckduckgo search capability). How can I do to collaborate with the proyect 
and upload that code?

Gilberto Najera

What steps will reproduce the problem?
1.Run a search on a known site. 
2.
3.

What is the expected output? What do you see instead?
I expected to see all of the emails attached to the organization.
Three important email addresses are missing. These may be being hidden by some 
type of security but I am unaware of it. And I really do not expect that all of 
the three would be hidden. They are openly advertised on the website. Therefore 
they should be harvestable. 


What version of the product are you using? On what operating system?
2.2a Kali

Please provide any additional information below.

It seems that httplib does not support the use of the system proxy variable.

Please can you add authenticated proxy support.

What steps will reproduce the problem?
1. theharvester.py -d gfong.com -l 50 -b google

What is the expected output?

[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]

What do you see instead?

[email protected]
[[email protected]
[[email protected]
[email protected]
[email protected]
[email protected]

What version of the product are you using? On what operating system?

2.2a on Windows 7

Please provide any additional information below.

The problem occurs when an email-address is put inside square brackets.
This is common in mail exports, like this:
From: John Doe [[email protected]]
Sent: Fri 13, 1337
To: D. Evil [[email protected]]

Saw an earlier bug fix, removing preceding @ from addresses. Probably the same 
issue? It should anyhow be easy to filter out the square brackets.

What steps will reproduce the problem?
1. theharvester -d target-domain -b all -v -f target-domain.html -n -c
2.
3.

What is the expected output? What do you see instead?

-Expected output of the command should include a dns brute forcing of said 
target.
-What is seen is an error that the program is having trouble opening 
dns-search.txt


What version of the product are you using? On what operating system?
-Version of theharvester is 2.2a on fully updated Kali Linux

Please provide any additional information below.
-I solved the issue by replacing "f = open(self.file,"r")" in 
dnssearch-threads.py to "f = open(self.file,"dns-search.txt")" and all ran 
great with no errors.

Hopefully this will help others that are having the same issues

What steps will reproduce the problem?
1. Perform a harvester query, for a known organisation
2. Notice that when you attempt to modify the email regex to:
(' ' + '[a-zA-Z0-9.-_]*' + '.' + '[a-zA-Z0-9.-_]*' + '@' + '[a-zA-Z0-9.-]*' + 
self.word)
You will begin to see some results appearing as "... [email protected]"
3. These results are incorrectly being parsed, due to the fact that you are 
creating the results not from the pages, but including truncated google results.

What is the expected output? What do you see instead?

Expected: "[email protected]" - as viewed on webpage.
Actual: "... [email protected]" - From Truncated google result.

What version of the product are you using? On what operating system?
2.2a - Mac OS X

Please provide any additional information below.

I cannot see a fix for this, unless you provide a future command line switch 
e.g. -IF (Investigate further and attempt to curl/ grep the page for the 
corresponding result.)

What steps will reproduce the problem?

1. Trying to run the Harvester but get Permission denied though I have full 
privledge
2. trying to run using ./theHarvester.py

3. Version 2.2a

What is the expected output? What do you see instead?
Should run but get Permission denied

What version of the product are you using? On what operating system?
Harvester 2.2a LinuxMint


Please provide any additional information below.

All I can add

What steps will reproduce the problem?
1. enter any search that is solely based on exalead, e.g. ./theHarvester.py -d 
www.wikipedia.org -b exalead

What is the expected output? What do you see instead?
I should get a valid search, instead I get this:

Invalid search engine, try with: bing, google, linkedin, pgp, exalead, jigsaw, 
bing_api, people123, google-profiles


What version of the product are you using? On what operating system?
Mac OS X 10.6: Version 2.2a

Please provide any additional information below.
I tracked the error at line 89 of theHarvester.py: the list is not complete, 
"exalead" is not listed. When fixing this in my version, it all runs smoothly. 
Also yandex is in the list, but currently unsupported, maybe exclude it for now?

What steps will reproduce the problem?
1. choose google under the -b arguement
2. choose a number over a 1000 under the -l argument 
3. run the tool
What is the expected output? What do you see instead?
i expect to see the application keep counting up the number of pages, instead i 
see it stops at 1000, instead of going over to the spcified -l parametyer. 

What version of the product are you using? On what operating system?
latest on kali linux 

Please provide any additional information below.
you guys kick ass ! 

when i try to generate a XML file as the output i am still getting HTML file  

*the expected outcome should be xml file


* i am using recent harvester version on Ubuntu 


please tell me the option for generating the XML file as output .

command used ::./theHarvester.py -d <HOSTNAME> -l 500 -b all -f har.xml

Hello,

I encounter a problem with theHarvester-2.2a and 2.2.
In both versions, I am not able to generate an html file with the "f" option.

The following message is displayed:

Saving file
local variable 'full' referenced before assignment
Error creating the file
I have also tried several versions of python (python, python2,2.7)
I am running under mint 16.
Please let me know If you need more informations.