innfeed error 500 handler about inn HOT 14 OPEN

Provisionally the current behavior is:

When I have more time I'll recheck and when I have even more time I might check innduct.

from inn.

Comment by iulius on 25 Dec 2008 19:34 UTC
Maybe innfeed should not try to propagate such messages (and innd should not have accepted them -- even after TAKETHIS) because the message-ID is not valid.

USEPRO states:

The msg-id MUST NOT be more than 250 octets in length.

NOTE: The length restriction ensures that systems that accept message identifiers as a parameter when referencing an article (e.g. [RFC3977]) can rely on a bounded length.

RFC 3977 states:

• A message-id MUST begin with "<", end with ">", and MUST NOT contain the latter except at the end.
• A message-id MUST be between 3 and 250 octets in length.
• A message-id MUST NOT contain octets other than printable US-ASCII characters.

from inn.

Comment by iulius on 28 Dec 2008 20:23 UTC
Beware that these NNTPRelay and Diablo peers should have returned 501 (and not 500) because the base command is known. It was only a syntax error.

RFC 977 already stated that. And RFC 3977 currently states the same.

There are probably other return codes to catch, besides 500 and 501.

from inn.

Comment by iulius on 8 Feb 2010 22:08 UTC
Also see the definition of a failed connect (should the sleepTimeout be reset in getModeResponse() when innfeed sends MODE STREAM and receives an unknown answer?).

More generally, innfeed should implement RFC 3977.

from inn.

The message ID limit seems to have been fixed in 2009 or so, so I think we won't see the specific issue again.

But I think other disagreements over article validity between INN and other news servers could (in principle) lead to the same troublesome innfeed behaviour, if nothing's changed there since.

from inn.

Yes, innfeed no longer sends invalid Message-IDs. This specific command triggering the issue does not permit reproducing the problem.
This issue is now to properly handle 500 response codes, and more globally to implement NNTP Version 2 (RFC 3977) in innfeed...

from inn.

AFAICS innxmit handles 500 quite ungracefully, preventing subsequent articles from being transmitted. I've not fully characterized its behavior yet though.

Beware that these NNTPRelay and Diablo peers should have returned 501 (and not 500) because the base command is known. It was only a syntax error.

Agreed. I would argue that, in isolation:

• Retrying the article on a 500 response is legitimate behaviour. IHAVE (etc) are not unknown commands. If we get 500 then we are probably talking to a reader server. Either we are misconfigured or the peer is, and either way the misconfiguration will probably soon be corrected, at which point we will start being able to feed to it. Retrying the previously errored articles is, at least probably, what we want.
• Dropping the article on a 500 response is also legitimate behaviour; essentially it says that if configuration is temporarily wrong, we don't propagate articles. It seems particularly appropriate at the start or end of a peering relationship, which is also the time that configuration on end or the other is most like to be transiently wrong. Still, since it may plausibly prevent propagation of articles that would be propagated but for a configuration fix, it's not necessarily the preferred behaviour.
• Retrying the article on a 501 response is incorrect behaviour. IHAVE (or CHECK etc) has been recognized as a legitimate command but there was some disagreement about the message ID syntax. That may be a bug in one implementation or the other, but it's not going to be fixed on a short timescale - in today's diminished Usenet it could be easily be multiple years, or never.
• Dropping the article on a 501 response is correct behavior (for the reasons above).

If some servers are sending 500 for a syntax error then I think that suggests that we should drop in both cases, though there is also IMHO a firm argument for retrying on 500.

from inn.

AFAICS innxmit handles 500 quite ungracefully, preventing subsequent articles from being transmitted. I've not fully characterized its behavior yet though.

For 500 it always exits,which has the effect of retrying later.

For 501:

• in streaming mode, it hopes that it includes the message ID, which it probably won't, so it will exit (again, retrying later)
• in non-streaming mode, it counts it as a refusal, i.e. doesn't retry.

Fixing this is slightly complicated by innxmit throwing away command ordering information, instead using message IDs in responses to match them back to articles, despite RFC3977 s3.5 making it clear that it's allowed to rely on ordering (which should actually be faster!)

from inn.

Do you see different behaviour between innfeed and innxmit?
I agree with your analysis of handling 500 and 501. I think they are the only remaining unknown responses that should be delt with (502 and 40x can be considered as a "retry later").
The ideal would be for 500 to retry during a few days after the attempt or the date of the article (for instance artcutoff days) but it would need to get that information. Otherwise let's just drop the article. Or if there are several 500 in the row, do not drop the articles as it would mean there is a configuration problem. But that's probably overkill to handle, and may lead to another problem (how many articles in the row, and they can still be blocked forever anyway...).
So just not retrying after 500 and 501 seems a good and not overcomplicated approach.

from inn.

Should we have separate issues for the innfeed and innxmit behavior or use this issue for both? I've been working on changes to innxmit (as an easy way to make a start). I don't particularly mind, myself.

from inn.

A separate issue for innxmit can be opened if you prefer. (Or the title of this one changed to mention both innfeed and innxmit.) I also don't particularly mind :)

from inn.

BTW above patch series is not well tested yet - feedback welcome of course but don't try to merge until I've had a chance to do some system-level testing.

from inn.

@ewxrjk I believe you'll be glad to know that your patch for innxmit helped the investigation of a bug in another news server. For testing purposes, I fed my spool, and the innxmit process stopped with an out of order (should be <mid>) response. The remote news server appeared not to answer any response code when receiving an article with an empty continuation line in its headers; it silently discarded the article and just sent a response code for the subsequent pipelined TAKETHIS command.
Hopefully innxmit told me that as soon as it happened, giving the culprit Message-ID. Otherwise it would have been difficult to understand what was going wrong, only finding it out after 3 millions articles sent or maybe never as it seems that innxmit re-tries and if the remote server refuses the already seen Message-ID in the CHECK command, it will just be discarded.
So, thanks again for you valuable latest patch!

from inn.

@Julien-Elie glad to hear it!

from inn.