rolkey / indyproject Goto Github PK

Update TIdHTTPServer to natively support "multipart/form-data" posts and 
add a new TIdMessageParts property to TIdHTTPRequestInfo to hold the data.

TIdIMAP4.AppendMsg() currently uses TIdMessage.SaveToFile() to save the 
message data to a temporary file on the HDD.  This is used for calculating 
the byte length of the message headers, and the byte length of the entire 
message.  These values are then used when formatting the IMAP 'APPEND' 
command.  AppendMsg() should be updated to not use a temporary file 
anymore.

I think we need to support multii-byte strings in SASL.  In order to do
this, we need to implement RFC 3454 Preparation of Internationalized
Strings ("stringprep")

The lines of a message are not required to be dot-prefixed like in SMTP, 
so TIdIMAP should not care about parsing or generating termination 
sequences.  It tells the server exactly how many bytes the message 
actually is.  If the message data actually contains a valid line with just 
a dot on it, TIdIMAP would end up truncating the message that is stored on 
the server.

TIdHTTPServer does not currently expose the complete request URI (per RFC 
2616 Section 5.1.2).  It needs to do so, especially for the 
OnHeadersAvailable event.

Validate TIdSysLogMessage against RFC 3164

The Trivial FTP components currently use strings to transfer data around.  
This is broken when Unicode strings are used.  They need to be re-written 
to transfer data using raw bytes instead.  This way, there are no Ansi<-
>Unicode encoding issues to deal with.

Architecural changes in Indy 10's I/O system over previous versions have 
broken TIdHL7.  It needs to be fixed.

TIdIMAP4.GetResponse() should be re-written to utilize TIdTCPConnection's 
existing internal logic, and to validate the format of the server's 
replies, raising an exception if an unknown format is encountered.

Delphi 2010 introduces a new 'delayed' keyword for delay-loading of DLL 
functions.  Indy's manual calls to LoadLibrary() and GetProcAddress() can 
be replaced with this feature, allowing for much cleaner code.  In the 
event that a DLL function cannot be found (older OS version, wrong DLL 
version, etc), the System.SetDliFailureHook() callback can be used to 
handle that.

With FIPS-complience, we will need to provide more secure SASL mechanisms 
than what Indy currently has.  Many SASL mechanism such as Digest, OTP, 
CRAM-MD5, and S/Key use compromised hash algorithms.  SHA1 also has been 
partially compromised so we need SASL mechanisms that use stronger hashing 
that are FIPS approved such as SHA224, SHA256, SHA384, and SHA512.  I 
would suggest starting off with SCRAM.

http://tools.ietf.org/html/draft-ietf-sasl-scram-10

Note that I have checked in code that does HMAC-SHA-224, HMAC-SHA-256, 
HMAC-SHA-384, and HMAC-SHA-512 so SCRAM should be doable.

TIdMultipartFormDataStream.Size calculations were broken when Charset 
support was added.  This needs to be fixed.

TIdMessage's LoadFrom...() and SaveTo...() methods currently use 
TIdMessageClient internally to handle message parsing/generating.  
TIdMessageClient blindly assumes that each line involves transparency of 
leading dots, as described in RFC 821 
(http://www.ietf.org/rfc/rfc0821.txt).  However, that transparency only 
applies to SMTP connections.  When reading/writing messages from local 
files or non-SMTP connections, the transparency rules must not be 
applied.  TIdMessage and/or TIdMessageClient need to be updated to allow 
the caller to specify what kind of source/target is being used so 
transparency can be enabled/disabled accordingly.

SMTP Service Extensions for Transmission of Large and Binary MIME Messages
http://www.ietf.org/rfc/rfc3030.txt

Add OnThreadStartup/Shutdown events and virtual methods to 
TIdSchedulerOfThread and have them be called by 
TIdThread.Before/AfterExecute().  This way, users can perform per-thread 
initializations/cleanups, such as calling CoInitialize/Ex() and 
CoUninitialize().

Need to double-check how TIdHTTP, TIdHTTPServer, and TIdHTTPProxyServer 
handle the "Expect" header, particularly in regard to the "100-continue" 
expectation.  Have seen some reports that suggest improper handling may be 
causing timeout problems.

TIdHTTPServer needs to support the "Transfer-Encoding" header when reading 
a client request.

TIdSchedulerOfThreadPool appears to still have the problems that have been 
discussed over the years.

April 2005:
http://newsgroups.cryer.info/atozedsoftware/indy.general/200504/0504253614.
html

September 2005:
http://newsgroups.cryer.info/atozedsoftware/indy.general/200509/0509144041.
html

October 2006:
http://delphi.newswhat.com/geoxml/forumhistorythread?
groupname=borland.public.delphi.internet.winsock&messageid=Xns986296DB1597A
[email protected]

Add support for Simple FTP (RFC 913)

Sometimes, Indy is used in a worker thread with other libraries that 
require message queue processing, such as ActiveX/COM objects.  Currently, 
TIdAntiFreeze ignores message pump requests when calling in the context of 
worker threads.  It could be updated to allow message pumping (not via 
TApplication.ProcessMessages(), though).  Perhaps add a new property that 
is False by default for backwards compatibility, but can be set to True 
when needed.

Indy stores most of its compiler conditional defines in 
IdCompilerDefines.inc, but this file is only usable in Delphi code.  There 
is no corresponding file for use in C++Builder.  If nothing else, Indy 
could use HPPEMIT statements in IdGlobal.pas for the more important 
defines, like the version number.

Update TIdIOHandler to pass its ConnectTimeout value all the way to 
TIdStack.Connect() to get the timeout logic out of 
TIdIOHandler.ConnectClient().  This way, TIdStackWindows can be updated to 
use WSA...() functions when available, and thus avoid having to use the 
TIdConnectThread class to abort the connection after the timeout elapsed.  
Other TIdStack... classes can continue using TIdConnectThread when 
suitable platform-specific techniques are not available.

TIdFTP.List() does the following:

FListResult.Text := ReadStringFromStream(LDest, -1, 
IOHandler.DefStringEncoding);

The IOHandler.DefStringEncoding is set to TIdTextEncoding.UTF8 beforehand 
when communicating a UTF-8 enabled FTP server.

Under D2007 and earlier, the ADestEncoding parameter of 
ReadStringFromStream() is being set to nil.  When it calls BytesToString() 
internally, ADestEncoding is being set to TIdTextEncoding.ASCII, causing 
non-ASCII characters to be lost before TIdFTP has a chance to parse the 
listing.

When receiving directory listings from an FTP server, TIdFTP applies text 
charset decoding, based on whether UTF-8 was negotiated or not, before the 
FTP listing parsers are invoked to parse the data.  The parser classes 
should be updated to allow TIdFTP to pass the raw undecoded data to them 
instead.  TIdFTP itself should not be interpretting the raw data at all.

Update TIdHTTPPRoxyServer to support caching and chunked/compressed 
transfers

TIdTelnet needs to be changed to use String anymore for telnet data.  It 
does not work correctly with UnicodeString in Delphi/C++Builder 2009 and 
later.  Look at RFC 2066.

Update TIdHTTP and TIdHTTPServer to support the "X-HTTP-Method-Override" 
header.

Look into using IcmpCreateFile() and related functions on Windows 2000+.

Add a new TIdMultipart class to the TIdMessageParts collection.  MIME 
parts in email can be nested, sometimes several levels deep. Indy 10 added 
the TIdMessagePart.ParentPart property to begin addressing this, but that 
property is very limited in its functionality, and required a lot of extra 
coding be added inside of TIdMessageClient when parsing and generating 
MIME-based emails.  By introducing a new TIdMessagePart-derived class 
for 'multipart/...' MIME parts 
('multipart/alternative', 'multipart/related', etc), where it would 
contain its own local collection of immediate nested TIdMessagePart 
objects, I believe this would make handling of MIME parts much more 
simplified and straight-forward. Not only for Indy internally, but for 
developers as well when building/consuming emails using the TIdMessage 
component, or handling 'multipart/form-data' posts in HTTP.

http://www.ietf.org/rfc/rfc2640.txt

Look in http://www.indyproject.org/wiki/ and http://qc.embarcadero.com for 
open Indy issues.

In \Protocols\IdCoder3to4.pas line 277

// Reduce to 3 bytes
Result[LOutPos]     := (FDecodeTable[LInBytes[0]] shl 2) or ((FDecodeTable
[LInBytes[1]] shr 4) and 3);
Result[LOutPos + 1] := ((FDecodeTable[LInBytes[1]] and 15) shl 4) or 
((FDecodeTable[LInBytes[2]] shr 2) and 15);
Result[LOutPos + 2] := ((FDecodeTable[LInBytes[2]] and 3) shl 6) or 
(FDecodeTable[LInBytes[3]] and 63);

When the base64 data contains, for whatever reason, garbage (for instance 
a space as first character) above code will generate a range-check error.

In case of a space FDecodeTable[LInBytes[0]] will return 255, which is 
then "shifted" by two times  add eax, eax.

Suggested change:

// Reduce to 3 bytes
Result[LOutPos]     := ((FDecodeTable[LInBytes[0]] and 63) shl 2) or 
((FDecodeTable[LInBytes[1]] shr 4) and 3);
...

TIdAuthenticationNTLM needs to be fixed to work correctly in D2009.

Apparently, String variables in derived C++ classes are default-
constructed after InitComponent() has already been called at the 
TIdInitializerComponent level, wiping out the initialization that 
InitComponent() had performed.  This probably happens for any C++ types 
that need explicit construction by the compiler.  This goes back to 
differences between Delphi and C++ object construction models.

Reimplement TIdCriticalSection using spin-lock implementation by Istvan 
Agoston and was based on this:
http://www.intel.com/cd/ids/developer/asmo-na/eng/dc/threading/333935.htm

Found it from here (while looking something completely different):
http://17slon.com/gp/gp/gplists.htm

Currently, when a timeout occurs in TIdIOHandler.ReadLn(), it sets the 
ReadLnTimedOut property to True and returns a blank string.  Much code 
does not check for that condition when calling ReadLn().  A new 
ReadLnTimeoutAction property should be added to allow callers to decide 
whether ReadLn() should raise a timeout exception or not, similar to how 
the MaxLineAction property operates.

TIdEmailAddress fails for a valid address like '"Sm\"art\"POP@JAM" 
<[email protected]>'

Code to reproduce:

var
  addr: TIdEMailAddressItem;
  decodedFrom, parsedAddr: WideString;
begin
  decodedFrom := '"Sm\"art\"POP@JAM" <[email protected]>';
  addr.Text := decodedFrom;
  parsedAddr := addr.Text;
  Assert(SameText(parsedAddr, decodedFrom), 'Addresses are not equal:' + 
parsedAddr + ' <--> ' + decodedFrom);
end;

The problem is now that name contains the value including the quoted 
(quoted with '\') double quotes. Then the quote symbol '\' is duplicated 
when using the GetText function: '"Sm\\"art\\"POP@JAM" <[email protected]>'

Finish fixing and updating TIdCookie and TIdCookieManager to fully support 
RFCs 2109 and 2965.

TIdURI is not even close to being RFC compliant when handling URIs, and 
its URLEncode() and ParamsEncode() methods have always lead to problems 
and controversy over how they should be implemented.

The new implemenation should be based on RFC 3986 (and even better, RFC 
3987), and various protocol-specific derivitives.  

The new implementation needs to validate against the following bug reports 
as well:

QC #65434 (RAID #264075)
Indy IdUri.URLEncode can't encode '|' (7c) character
http://qc.codegear.com/wc/qcmain.aspx?d=65434

QC #65436 (RAID #264074)
Indy IdUri.URLEncode can't encode to UTF8-encoded format
http://qc.codegear.com/wc/qcmain.aspx?d=65436

TIdMappedFTP needs to support the IPv6 commands and commands such as MLSD.
We might also to support a IMG command.  It's in an Internet Draft and is 
used FTP voyager.

When a client connects and the MaxConnections limit has been reached, the 
client is disconnected and its thread is destroyed rather than being put 
back in the pool.  This is because the thread is still in a suspended 
state, and TIdSchedulerOfThread.TerminateYarn() destroys suspended threads.

[BCC32 Error] Idstackbsdbase.hpp(156): E2139 Declaration missing ;
  Full parser context
    Unit1.cpp(7): #include c:\program files\embarcadero\rad studio\7.0
\include\Indy10\IdStackWindows.hpp
    IdStackWindows.hpp(20): #include c:\program files\embarcadero\rad 
studio\7.0\include\Indy10\Idstackbsdbase.hpp
    Idstackbsdbase.hpp(27): namespace Idstackbsdbase
    Idstackbsdbase.hpp(140): class TIdIn6Addr
    Idstackbsdbase.hpp(143): class TIdIn6Addr:: 
    Idstackbsdbase.hpp(155): class TIdIn6Addr:: :: 

TIdIn6Addr has an "s6_addr" member, but "s6_addr" is defined with a 
#define statement in in6addr.h, which is included by winsock2.h.

See http://en.wikipedia.org/wiki/Category:Binary-to-text_encoding_formats

The classes in IdStackVCLPosix need to be implemented for Unix-like
operating systems such as OS/X and Linux.

The backend IdStackLibc will probably not work because libc.pas not going
to be used in Embarcadero's cross-platform products and the Posix API
definitions will probably NOT be the same as FreePascal's or Kylix.

I have not yet started this as Embarcadero needs to do some work at there
end.  Our work starts when they release some with the BSD Socket API
definitions.

The task itself will probably be based at least partially on IdStackLibc.

Procedure TIdCustomSocksServer.HandleConnectV5 have line:

VHost := BytesToString(LData, Length(LData)-2);

But right code is:

VHost := BytesToString(LData, 0, Length(LData)-2);

Change TIdIMAP4 to not use Capture() anymore.  It needs to use the actual 
byte count that the server specifies instead.

Some FTP servers require a CLNT command be issued before UTF-8 will work 
correctly.  Internet Explorer has a bug in its UTF-8 implementation, so 
those servers rely on the CLNT command to identify IE clients in order to 
work around the problem.  TIdFTP needs to send a CLNT command even when 
its ClientInfo.ClientName property is empty so those servers can identify 
Indy clients and not ignore their UTF-8 requests.

The TIdHMAC Framework needs to use the HMac code from OpenSSL through the 
new IdFIPS software layer.  This is to help ensure FIPS compliencew 
because HMAC is a cryptographic function.

It looks like we may have to update Indy to use the new OpenSSL 1.0 series 
because some Linux distributions now ship with OpenSSL 1.0 beta 4 and I 
think there's a FIPS version of that in some of them.  I did see this in a 
recent version of PCLinuxOS 2009.

In any case, we may soon have to support it given Embarcadero's cross-
platform plans.

Add support for SSH