YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. about bulk_extractor HOT 27 CLOSED

I've never seen this before. It's in the code being produced by GNU flex. Can you share the input file with me? This is either being caused by scan_email or scan_acct. Can you run -x email and -x acct and let me know which results in the error going away?

from bulk_extractor.

Also, are you having this error in the released bulk_extractor or in the github version?

from bulk_extractor.

 I downloaded the version from github.  I am running version 1.3.1 Rev 10845.

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Friday, April 5, 2013 1:44 AM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Also, are you having this error in the released bulk_extractor or in the github version?
—
Reply to this email directly or view it on GitHub.

from bulk_extractor.

Can you please answer my other questions?

On Apr 5, 2013, at 8:52 AM, rawtalent06 [email protected] wrote:

I downloaded the version from github. I am running version 1.3.1 Rev 10845.

#4 Raw Talent
Rising Suns of Enlightenment
Fall 2000
N.O.T.O.R.I.O.U.S NU MU
Georgia Institute of Technology

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Friday, April 5, 2013 1:44 AM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Also, are you having this error in the released bulk_extractor or in the github version?
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.

from bulk_extractor.

I am running those now.  As of right now, -x email failed.  The other -x accts is still running.

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Friday, April 5, 2013 8:54 AM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Can you please answer my other questions?

On Apr 5, 2013, at 8:52 AM, rawtalent06 [email protected] wrote:

I downloaded the version from github. I am running version 1.3.1 Rev 10845.

#4 Raw Talent
Rising Suns of Enlightenment
Fall 2000
N.O.T.O.R.I.O.U.S NU MU
Georgia Institute of Technology

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Friday, April 5, 2013 1:44 AM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Also, are you having this error in the released bulk_extractor or in the github version?
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.

from bulk_extractor.

I only received this error when using -x email.  What does it actually mean if I use that flag? I know in the help it states it disables the email scanner.

When using the flag -x accts, it completed.  However, I did get another error that did not stop it from completing:  "***carve: Cannot create bulkOutput/vcard/000000.vcf: Permission denied".  Not sure why I am getting a permission denied when it is able to create the other export files/folders.  In addition, it did create the folder vcard and I am not running this command as root.

To give you some more background, I am running bulk extractor on a 32 gb RAM dd file (which I am unable to share).  It has completed before and the only thing I can think of that is causing this issue now is yara/yarascan.  I have another terminal running yara/yarascan and when running bulk_extractor at the same time, bulk extractor seems to error out.
 

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Friday, April 5, 2013 1:42 AM
Subject: Re: [bulk_extractor] Input Buffer Overflow (#20)

I've never seen this before. It's in the code being produced by GNU flex. Can you share the input file with me? This is either being caused by scan_email or scan_acct. Can you run -x email and -x acct and let me know which results in the error going away?
—
Reply to this email directly or view it on GitHub.

from bulk_extractor.

You are using the development version, not the release version, so you are getting development bugs. Can you give me a copy of the disk image that generates the problem?

On Apr 5, 2013, at 9:40 AM, rawtalent06 [email protected] wrote:

I only received this error when using -x email. What does it actually mean if I use that flag? I know in the help it states it disables the email scanner.

When using the flag -x accts, it completed. However, I did get another error that did not stop it from completing: "***carve: Cannot create bulkOutput/vcard/000000.vcf: Permission denied". Not sure why I am getting a permission denied when it is able to create the other export files/folders. In addition, it did create the folder vcard and I am not running this command as root.

To give you some more background, I am running bulk extractor on a 32 gb RAM dd file (which I am unable to share). It has completed before and the only thing I can think of that is causing this issue now is yara/yarascan. I have another terminal running yara/yarascan and when running bulk_extractor at the same time, bulk extractor seems to error out.

#4 Raw Talent
Rising Suns of Enlightenment
Fall 2000
N.O.T.O.R.I.O.U.S NU MU
Georgia Institute of Technology

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Friday, April 5, 2013 1:42 AM
Subject: Re: [bulk_extractor] Input Buffer Overflow (#20)

I've never seen this before. It's in the code being produced by GNU flex. Can you share the input file with me? This is either being caused by scan_email or scan_acct. Can you run -x email and -x acct and let me know which results in the error going away?
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.

from bulk_extractor.

Thanks. So the bug is in the accts scanner flexer

On Apr 5, 2013, at 9:18 AM, rawtalent06 [email protected] wrote:

I am running those now. As of right now, -x email failed. The other -x accts is still running.

#4 Raw Talent
Rising Suns of Enlightenment
Fall 2000
N.O.T.O.R.I.O.U.S NU MU
Georgia Institute of Technology

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Friday, April 5, 2013 8:54 AM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Can you please answer my other questions?

On Apr 5, 2013, at 8:52 AM, rawtalent06 [email protected] wrote:

I downloaded the version from github. I am running version 1.3.1 Rev 10845.

#4 Raw Talent
Rising Suns of Enlightenment
Fall 2000
N.O.T.O.R.I.O.U.S NU MU
Georgia Institute of Technology

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Friday, April 5, 2013 1:44 AM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Also, are you having this error in the released bulk_extractor or in the github version?
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.

from bulk_extractor.

Its the other way around.  The email scanner failed but the accts scanner completed.

 _____________________
From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Friday, April 5, 2013 10:27 AM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Thanks. So the bug is in the accts scanner flexer

On Apr 5, 2013, at 9:18 AM, rawtalent06 [email protected] wrote:

I am running those now. As of right now, -x email failed. The other -x accts is still running.

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Friday, April 5, 2013 8:54 AM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Can you please answer my other questions?

On Apr 5, 2013, at 8:52 AM, rawtalent06 [email protected] wrote:

I downloaded the version from github. I am running version 1.3.1 Rev 10845.

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Friday, April 5, 2013 1:44 AM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Also, are you having this error in the released bulk_extractor or in the github version?
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.

from bulk_extractor.

What is the release version?  Also, have you ever created a python script to kick off bulk_extractor?

Thanks,
 

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Friday, April 5, 2013 10:26 AM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

You are using the development version, not the release version, so you are getting development bugs. Can you give me a copy of the disk image that generates the problem?

On Apr 5, 2013, at 9:40 AM, rawtalent06 [email protected] wrote:

I only received this error when using -x email. What does it actually mean if I use that flag? I know in the help it states it disables the email scanner.

When using the flag -x accts, it completed. However, I did get another error that did not stop it from completing: "***carve: Cannot create bulkOutput/vcard/000000.vcf: Permission denied". Not sure why I am getting a permission denied when it is able to create the other export files/folders. In addition, it did create the folder vcard and I am not running this command as root.

To give you some more background, I am running bulk extractor on a 32 gb RAM dd file (which I am unable to share). It has completed before and the only thing I can think of that is causing this issue now is yara/yarascan. I have another terminal running yara/yarascan and when running bulk_extractor at the same time, bulk extractor seems to error out.

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Friday, April 5, 2013 1:42 AM
Subject: Re: [bulk_extractor] Input Buffer Overflow (#20)

I've never seen this before. It's in the code being produced by GNU flex. Can you share the input file with me? This is either being caused by scan_email or scan_acct. Can you run -x email and -x acct and let me know which results in the error going away?
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.

from bulk_extractor.

-x turns off the scanner.

On Apr 5, 2013, at 10:46 AM, rawtalent06 [email protected] wrote:

Its the other way around. The email scanner failed but the accts scanner completed.

#4 Raw Talent
Rising Suns of Enlightenment
Fall 2000
N.O.T.O.R.I.O.U.S NU MU
Georgia Institute of Technology

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Friday, April 5, 2013 10:27 AM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Thanks. So the bug is in the accts scanner flexer

On Apr 5, 2013, at 9:18 AM, rawtalent06 [email protected] wrote:

I am running those now. As of right now, -x email failed. The other -x accts is still running.

#4 Raw Talent
Rising Suns of Enlightenment
Fall 2000
N.O.T.O.R.I.O.U.S NU MU
Georgia Institute of Technology

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Friday, April 5, 2013 8:54 AM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Can you please answer my other questions?

On Apr 5, 2013, at 8:52 AM, rawtalent06 [email protected] wrote:

I downloaded the version from github. I am running version 1.3.1 Rev 10845.

#4 Raw Talent
Rising Suns of Enlightenment
Fall 2000
N.O.T.O.R.I.O.U.S NU MU
Georgia Institute of Technology

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Friday, April 5, 2013 1:44 AM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Also, are you having this error in the released bulk_extractor or in the github version?
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.

from bulk_extractor.

The script python/regress.py runs bulk_extractor. It is also run by the GUI.

from bulk_extractor.

Thanks again for the bug report. I should have a fix posted shortly.

from bulk_extractor.

Please try a7ba2d6..845ed8f and let me know what you think.

from bulk_extractor.

Actually, try 1154119

from bulk_extractor.

Ok, I redownloadede project using:

    git clone --recursive https://github.com/simsong/bulk_extractor.git

I ran it with the following command from a terminal:  bulk_extractor image_file.001 -o bukOut

I am now receiving the following error:

    YY_FATAL_ERROR :5068 -- fatal flex scanner internal error--end of buffer missed
 

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Sunday, April 7, 2013 12:27 AM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Actually, try 1154119
—
Reply to this email directly or view it on GitHub.

from bulk_extractor.

That's good. Let me make some changes and get back to you. Thanks.

On Apr 7, 2013, at 5:20 PM, rawtalent06 [email protected] wrote:

Ok, I redownloadede project using:

git clone --recursive https://github.com/simsong/bulk_extractor.git

I ran it with the following command from a terminal: bulk_extractor image_file.001 -o bukOut

I am now receiving the following error:

YY_FATAL_ERROR <stdout>:5068 -- fatal flex scanner internal error--end of buffer missed

#4 Raw Talent
Rising Suns of Enlightenment
Fall 2000
N.O.T.O.R.I.O.U.S NU MU
Georgia Institute of Technology

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Sunday, April 7, 2013 12:27 AM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Actually, try 1154119
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.

from bulk_extractor.

Is the program crashing or just printing the error and continuing to execute?

Sent from my iPad

On Apr 7, 2013, at 5:20 PM, rawtalent06 [email protected] wrote:

Ok, I redownloadede project using:

git clone --recursive https://github.com/simsong/bulk_extractor.git

I ran it with the following command from a terminal: bulk_extractor image_file.001 -o bukOut

I am now receiving the following error:

YY_FATAL_ERROR <stdout>:5068 -- fatal flex scanner internal error--end of buffer missed

#4 Raw Talent
Rising Suns of Enlightenment
Fall 2000
N.O.T.O.R.I.O.U.S NU MU
Georgia Institute of Technology

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Sunday, April 7, 2013 12:27 AM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Actually, try 1154119
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.

from bulk_extractor.

It continues to execute.  I just started the program again to see if it will complete.  It will take at least 1 hour on the laptop I am currently using at home.

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Sunday, April 7, 2013 6:23 PM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Is the program crashing or just printing the error and continuing to execute?

Sent from my iPad

On Apr 7, 2013, at 5:20 PM, rawtalent06 [email protected] wrote:

Ok, I redownloadede project using:

git clone --recursive https://github.com/simsong/bulk_extractor.git 

I ran it with the following command from a terminal: bulk_extractor image_file.001 -o bukOut

I am now receiving the following error:

YY_FATAL_ERROR <stdout>:5068 -- fatal flex scanner internal error--end of buffer missed 

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Sunday, April 7, 2013 12:27 AM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Actually, try 1154119
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.

from bulk_extractor.

Oh, if it continues to execute, we're done! It's now just giving a warning, but the FATAL_ERROR is no longer fatal. I'll clean up the error message, but we are done here!

On Apr 7, 2013, at 6:37 PM, rawtalent06 [email protected] wrote:

It continues to execute. I just started the program again to see if it will complete. It will take at least 1 hour on the laptop I am currently using at home.

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Sunday, April 7, 2013 6:23 PM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Is the program crashing or just printing the error and continuing to execute?

Sent from my iPad

On Apr 7, 2013, at 5:20 PM, rawtalent06 [email protected] wrote:

Ok, I redownloadede project using:

git clone --recursive https://github.com/simsong/bulk_extractor.git

I ran it with the following command from a terminal: bulk_extractor image_file.001 -o bukOut

I am now receiving the following error:

YY_FATAL_ERROR :5068 -- fatal flex scanner internal error--end of buffer missed

#4 Raw Talent
Rising Suns of Enlightenment
Fall 2000
N.O.T.O.R.I.O.U.S NU MU
Georgia Institute of Technology

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Sunday, April 7, 2013 12:27 AM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Actually, try 1154119
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.

from bulk_extractor.

It is still executing.  It doesn't appear to be finishing.  its been 2 hours and normally it takes about 1 hour.  I have been getting that same error message for at least 1 hour now.

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Sunday, April 7, 2013 6:51 PM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Oh, if it continues to execute, we're done! It's now just giving a warning, but the FATAL_ERROR is no longer fatal. I'll clean up the error message, but we are done here!

On Apr 7, 2013, at 6:37 PM, rawtalent06 [email protected] wrote:

It continues to execute. I just started the program again to see if it will complete. It will take at least 1 hour on the laptop I am currently using at home.

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Sunday, April 7, 2013 6:23 PM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Is the program crashing or just printing the error and continuing to execute?

Sent from my iPad

On Apr 7, 2013, at 5:20 PM, rawtalent06 [email protected] wrote:

Ok, I redownloadede project using:

git clone --recursive https://github.com/simsong/bulk_extractor.git

I ran it with the following command from a terminal: bulk_extractor image_file.001 -o bukOut

I am now receiving the following error:

YY_FATAL_ERROR :5068 -- fatal flex scanner internal error--end of buffer missed

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Sunday, April 7, 2013 12:27 AM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Actually, try 1154119
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub.

from bulk_extractor.

It's hard to debug based on what you have given me. Is the message repeatedly being generated? That seems odd. How big is your disk image? Would it be possible for you to send me the report.xml file?

On Apr 7, 2013, at 8:42 PM, rawtalent06 [email protected] wrote:

It is still executing. It doesn't appear to be finishing. its been 2 hours and normally it takes about 1 hour. I have been getting that same error message for at least 1 hour now.

#4 Raw Talent
Rising Suns of Enlightenment
Fall 2000
N.O.T.O.R.I.O.U.S NU MU
Georgia Institute of Technology

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Sunday, April 7, 2013 6:51 PM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Oh, if it continues to execute, we're done! It's now just giving a warning, but the FATAL_ERROR is no longer fatal. I'll clean up the error message, but we are done here!

On Apr 7, 2013, at 6:37 PM, rawtalent06 [email protected] wrote:

It continues to execute. I just started the program again to see if it will complete. It will take at least 1 hour on the laptop I am currently using at home.

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Sunday, April 7, 2013 6:23 PM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Is the program crashing or just printing the error and continuing to execute?

Sent from my iPad

On Apr 7, 2013, at 5:20 PM, rawtalent06 [email protected] wrote:

Ok, I redownloadede project using:

git clone --recursive https://github.com/simsong/bulk_extractor.git

I ran it with the following command from a terminal: bulk_extractor image_file.001 -o bukOut

I am now receiving the following error:

YY_FATAL_ERROR :5068 -- fatal flex scanner internal error--end of buffer missed

#4 Raw Talent
Rising Suns of Enlightenment
Fall 2000
N.O.T.O.R.I.O.U.S NU MU
Georgia Institute of Technology

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Sunday, April 7, 2013 12:27 AM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Actually, try 1154119
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.

from bulk_extractor.

Why do you think that it normally takes 1 hour? What is the size of your disk image? When does the program say that it will be finished? Do the status reports indicate what percentage has been handled, and is that percentage increasing?

On Apr 7, 2013, at 8:42 PM, rawtalent06 [email protected] wrote:

It is still executing. It doesn't appear to be finishing. its been 2 hours and normally it takes about 1 hour. I have been getting that same error message for at least 1 hour now.

#4 Raw Talent
Rising Suns of Enlightenment
Fall 2000
N.O.T.O.R.I.O.U.S NU MU
Georgia Institute of Technology

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Sunday, April 7, 2013 6:51 PM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Oh, if it continues to execute, we're done! It's now just giving a warning, but the FATAL_ERROR is no longer fatal. I'll clean up the error message, but we are done here!

On Apr 7, 2013, at 6:37 PM, rawtalent06 [email protected] wrote:

It continues to execute. I just started the program again to see if it will complete. It will take at least 1 hour on the laptop I am currently using at home.

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Sunday, April 7, 2013 6:23 PM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Is the program crashing or just printing the error and continuing to execute?

Sent from my iPad

On Apr 7, 2013, at 5:20 PM, rawtalent06 [email protected] wrote:

Ok, I redownloadede project using:

git clone --recursive https://github.com/simsong/bulk_extractor.git

I ran it with the following command from a terminal: bulk_extractor image_file.001 -o bukOut

I am now receiving the following error:

YY_FATAL_ERROR :5068 -- fatal flex scanner internal error--end of buffer missed

#4 Raw Talent
Rising Suns of Enlightenment
Fall 2000
N.O.T.O.R.I.O.U.S NU MU
Georgia Institute of Technology

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Sunday, April 7, 2013 12:27 AM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Actually, try 1154119
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.

from bulk_extractor.

Below are the answers:

1. The message is repeastedly being generated.
2. My image is about 32 gb
3. When using the -x accts flag on this image before, it took about 1 hour to complete using this laptop.  This was before I updated to the bulk extractor version today.
4. The status usually gets to about 79% and then starts displaying that error message.
5. And yes the status does increase before getting that message.
6. The program doesn't say when it will be finished.  I have never seen that message or option before.

7. The program finally completed but it doesn't display the job stats or that is completed cleanly.  I have attached what it looked like when the program finished running.

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Sunday, April 7, 2013 9:38 PM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Why do you think that it normally takes 1 hour? What is the size of your disk image? When does the program say that it will be finished? Do the status reports indicate what percentage has been handled, and is that percentage increasing?

On Apr 7, 2013, at 8:42 PM, rawtalent06 [email protected] wrote:

It is still executing. It doesn't appear to be finishing. its been 2 hours and normally it takes about 1 hour. I have been getting that same error message for at least 1 hour now.

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Sunday, April 7, 2013 6:51 PM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Oh, if it continues to execute, we're done! It's now just giving a warning, but the FATAL_ERROR is no longer fatal. I'll clean up the error message, but we are done here!

On Apr 7, 2013, at 6:37 PM, rawtalent06 [email protected] wrote:

It continues to execute. I just started the program again to see if it will complete. It will take at least 1 hour on the laptop I am currently using at home.

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Sunday, April 7, 2013 6:23 PM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Is the program crashing or just printing the error and continuing to execute?

Sent from my iPad

On Apr 7, 2013, at 5:20 PM, rawtalent06 [email protected] wrote:

Ok, I redownloadede project using:

git clone --recursive https://github.com/simsong/bulk_extractor.git

I ran it with the following command from a terminal: bulk_extractor image_file.001 -o bukOut

I am now receiving the following error:

YY_FATAL_ERROR :5068 -- fatal flex scanner internal error--end of buffer missed

---

From: Simson L. Garfinkel [email protected]
To: simsong/bulk_extractor [email protected]
Cc: rawtalent06 [email protected]
Sent: Sunday, April 7, 2013 12:27 AM
Subject: Re: [bulk_extractor] YY_FATAL_ERROR macro called in scan_email.cpp or scan_accts.cpp. (#20)

Actually, try 1154119
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub.

from bulk_extractor.

It is unclear to me if this is a bug that has been in scan_accts from the beginning, if this is a bug that was recently introduced, or if this is a bug that is unique to your situation. It seems that yy_get_next_buffer() is generating an error condition because yyg->yy_c_buf_p > yyg->yy_n_chars+1. I don't know enough about the buffer management for the scanner that flex creates.

We have a substitution for flex and that is to use lightgrep, but I do not think that it is ready for production. yy_get_next_buffer() is called in two locations. My next fix will be to have it throw an exception which we will catch elsewhere, rather than providing a return code. This approach should work. I'll have it coded up shortly. Thanks again for your patience and your help on this.

Simson

from bulk_extractor.

Please check out ed2d2a5..c102c4c. I think that this will solve your problem.

from bulk_extractor.

The exception is now thrown with an object (e.g. exception()), rather than with a pointer (e.g. new exception), and as a result the exception is caught properly. Although it is unfortunate that the flex scanner is generating an exception, this is the best we can do at the moment.

from bulk_extractor.