sylphiawindy / fhscanhttplibrary Goto Github PK

Lets see the following function all tree:

1- SendHttpRequest()
2- DispatchHTTPRequest()
3- response = conexion->ReadHTTPResponseData((ConnectionHandling*)
HTTPHandleTable[HTTPHandle]->GetClientConnection(),request,&lock);

Inside ReadHTTPResponseData the "lock" mutex, named as "ExternalMutex",
which is part of the HTTPAPI instance , is called several times:


 ExternalMutex->LockMutex();
  FreeConnection();
 ExternalMutex->UnLockMutex();

and again at: 

ExternalMutex->LockMutex();
shutdown(datasock,2);
closesocket(datasock);
i = StablishConnection();
if (!i)
{
    FreeConnection();
    ExternalMutex->UnLockMutex();
    //UnLockMutex(lock);
    return (NULL);
}
for (i = 0; i <= PENDING_PIPELINE_REQUESTS - 1; i++)
{
    SendHTTPRequest(PIPELINE_Request[i]);
}

ExternalMutex->UnLockMutex();



Each time that more than one thread needs to reestablish an HTTP
connection, they should wait until all previous connection attempts ends.
This feature was added to mantain compatibility with the old C code and
should be modified to increase the speed of the http core.
Its known to be the main speed problem with the http proxy when several
threads attempt to reconnect to a host that issued several "connection:
close" messages.

TODO: 
- Identify if an external global mutex is still required to avoid problems
with the connection table.
- Check if the internal ::ConnectionHandling "lock" mutex is enough.

What steps will reproduce the problem?

1. Compile Linux Version 
2. perform http requests (happened in multithread mode, but will probably 
happen in single-thread mode too)
3. crash will happen anytime

What is the expected output? What do you see instead?
Application crash randomly when trying to lock mutex in threading.cpp .

Terminal message:
tpp.c:63: __pthread_tpp_change_priority: Assertion `new_prio == -1 || (new_prio 
>= __sched_fifo_min_prio && new_prio <= __sched_fifo_max_prio)' failed.

In my custom signal handler log file: 

  Signal=6  Desc=Aborted  DateTime: Fri Mar 18 14:41:49 2011  
Obtained 14 stack frames.
[bt]: (0) ./LinuxTelepmx(_ZN15AppCrashManager16print_StacktraceEPKc+0x25) 
[0x820d4f5]
[bt]: (1) ./LinuxTelepmx() [0x820d814]
[bt]: (2) [0xe73400]
[bt]: (3) [0xe73416]
[bt]: (4) /lib/libc.so.6(gsignal+0x51) [0x984941]
[bt]: (5) /lib/libc.so.6(abort+0x182) [0x987e42]
[bt]: (6) /lib/libc.so.6(__assert_fail+0xf8) [0x97d8e8]
[bt]: (7) /lib/libpthread.so.0(+0xfae4) [0xc47ae4]
[bt]: (8) /lib/libpthread.so.0(+0x7ebd) [0xc3febd]
[bt]: (9) ./LinuxTelepmx(_ZN9Threading9LockMutexEv+0x11) [0x88a7395]
[bt]: (10) ./LinuxTelepmx(_ZN7HTTPAPI20CleanConnectionTableEPPv+0x16) 
[0x889f8ec]
[bt]: (11) ./LinuxTelepmx(_Z10ThreadFuncPv+0x1f) [0x889ec1b]
[bt]: (12) /lib/libpthread.so.0(+0x5cc9) [0xc3dcc9]
[bt]: (13) /lib/libc.so.6(clone+0x5e) [0xa2a69e]

What version of the product are you using? On what operating system?
httpapi 1.3.0

Please provide any additional information below.

Probably similar issue: 
http://sourceware.org/bugzilla/show_bug.cgi?id=3610
http://sourceware.org/ml/libc-help/2008-05/msg00071.html

What steps will reproduce the problem?
1. Scan random HTTP server
2. sometimes HTTP core segfaults
3. If gdb is attached and fhscan compiled with debug information, the bug
points to an invalid memory access at DecodeChunk() function.


What is the expected output? What do you see instead?
run without problems

Please use labels and text to provide additional information.

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb23a0b90 (LWP 22330)]
0x08050c23 in DecodeChunk (lpBuffer=0xffffffff <Address 0xffffffff out of
bounds>, encodedlen=146) at HTTPCore/Encoding_Chunked.cpp:78
78                              memcpy(chunkcode,encoded,MAX_CHUNK_LENGTH);
(gdb) backtrace
#0  0x08050c23 in DecodeChunk (lpBuffer=0xffffffff <Address 0xffffffff out
of bounds>, encodedlen=146) at HTTPCore/Encoding_Chunked.cpp:78
#1  0x08051058 in CBDecodeChunk (cbType=2, api=0x8aa8350, HTTPHandle=8,
request=0x8b9f1e0, response=0x8b9e0c8) at HTTPCore/Encoding_Chunked.cpp:160
#2  0x0804a7e2 in HTTPCALLBACK::DoCallBack (this=0x8aad670, cbType=2,
HTTPHandle=8, request=0x8b9f1e0, response=0x8b9e0c8) at
HTTPCore/CallBacks.cpp:154
#3  0x08056577 in HTTPAPI::DispatchHTTPRequest (this=0x8aa8350,
HTTPHandle=8, request=0x8b9f1e0) at HTTPCore/HTTP.cpp:479
#4  0x08056c97 in HTTPAPI::SendHttpRequest (this=0x8aa8350, HTTPHandle=8,
request=0x8b9f1e0, lpUsername=0x0, lpPassword=0x0) at HTTPCore/HTTP.cpp:747
#5  0x080571ff in HTTPAPI::SendHttpRequest (this=0x8aa8350, HTTPHandle=8,
VHost=0x0, HTTPMethod=0x806e9d1 "GET", lpPath=0x806e9ab "/", PostData=0x0,
PostDataSize=0, lpUsername=0x0, lpPassword=0x0)
    at HTTPCore/HTTP.cpp:852
#6  0x080573be in HTTPAPI::SendHttpRequest (this=0x8aa8350, HTTPHandle=8,
HTTPMethod=0x806e9d1 "GET", lpPath=0x806e9ab "/") at HTTPCore/HTTP.cpp:828
#7  0x0806477f in ScanHosts (ptr=0x8aa8350) at Scanner/FHScan.cpp:140
#8  0x006855ab in start_thread () from /lib/libpthread.so.0
#9  0x00604cfe in clone () from /lib/libc.so.6
(gdb)


(gdb) print response->HeaderSize
$12 = 171
(gdb) print response->Header
$13 = 0x9cd6e60 "HTTP/1.1 401 Unauthorized\r\nWWW-Authenticate: Basic
realm=\"P645ME+\"\r\nContent-Type: text/html\r\nTransfer-Encoding:
chunked\r\nServer: ZyXEL-RomPager/3.02\r\nConnection: close\r\n\r\n"
(gdb) print response->DataSize
$14 = 146
(gdb) print response->Data
$15 = 0xffffffff <Address 0xffffffff out of bounds>
(gdb)


The response->Data information is not valid, however its unsure if this is
caused by the filemapping function or by the ReadHTTPResponseData()

What steps will reproduce the problem?
run fhscan scanner (tested under linux) against a large number of hosts and
capture a pcap

What is the expected output? What do you see instead?
requests against different hosts are performed in the same tcp session


There must be some kind of sincronization error at the AddPipelineRequest()
/ GetPIPELINERequestID() functions

What steps will reproduce the problem?
1. browse a website like www.terra.es with IE8 against fhscan proxy

What is the expected output? What do you see instead?
Pages retrieved. 

Instead lots of pages are returned with invalid status code.
After some deeper inspection... looks like ReadHTTPResponse() fails to
retrieve the complete HTTP response data therefore next reads return
previous http data instead of http headers.

ReadHTTPProxyRequestData() its reading fine the client requests so the
problem must be at ReadHTTPResponseData() ( Maybe its a chunkencoding error )

What steps will reproduce the problem?
execute the application

What is the expected output? What do you see instead?
The application exits

What version of the product are you using? On what operating system?
Windows vista ultimate x86 Russian