Per the latest revision of the proposed VTP standard, we've dropped the requirement for subscribers to have IVORNs: jdswinbank/vtp@b786e7e for details. Comet should no longer insist that they do.

Requires making a local comet user, I guess.

When a new event is received, we store its IVORN so that we can use it for de-duplication purposes: we shouldn't process the same event twice.

Currently the IVORNs are stored in a series of anydbm databases, which will grow without limit. When a large number of events are received:

1. These databases will get extremely large;
2. Searching the database for previously seen IVORNs will get very expensive.

The simplest way to avoid this is simply to expire IVORNs from the database after some time (a month?). Alternatively, we could investigate a more sophisticated database system which works around the issues.

Hi all, I see that Comet is supposed to have experimental Python 3.x support, but I'm running into issues installing the py2-ipaddress library in Python 3, which appears to be required:

Collecting py2-ipaddress
  Using cached py2-ipaddress-3.4.1.tar.gz
    Complete output from command python setup.py egg_info:
    py2-ipaddress: For Python 3.x, please use the official ipaddress module.

Is there a workaround? Thanks!

Pierre Le Sidaner reports that Comet rejects VOEvent's submitted by Dakota's sendvo.exe with a GPG error.

Not clear whether this is a fault in Comet, or if Dakota is sending something non-compliant. If it's the latter, consider whether we can be flexible.

This is potentially a much bigger issue than just this broker. How can we authenticate an event so that we are sure who sent it? See, for example, the note by Denny, but be aware of its shortcomings -- who wants to spawn GPG for every event received?!

Important: the broker should be able to limit submission of new events to certain IP addresses.

Less important: the broker should be able to limit subscribers to certain IP addresses.

In both cases, it's desirable to use netmasks or similar to specify a range rather than a single address.

It would be nice to add functionality which sends a duplicate stream of all output to a user specified file, for logging purposes.

I suppose one could do something equivalent currently using some combination of piping and running 'tail' to watch the logfile, but I think it would be handy.

EDIT
Actually, this can be done using the twistd logging facility.

First prototype landed in 35e61ae but is untested.

Currently, we write very verbose logging information. Which is handy for debugging, but probably not generically useful. We should make this customizable, so the user can select how much they want to see.

IVOA Identifiers, version 2.0 deprecates the term "IVORN" in favour of IVOID.

The VOEvent standard continues to refer to IVORNs, as does the VOEvent XML schema.

Comet's documentation & command line options should refer to IVOIDs, but provide an explanation. Internally, we can continue to refer to IVORNs, since that's what we'll actually read out of packets.

When the subscriber fails to connect to a broker, it uses an exponential back off to avoid a series of rapid-fire reconnections. However, as soon as a connection is made, the timer on that back-off is reset to zero.

If the broker is checking for authenticated subscribers, the connection might be successful but the broker shortly thereafter terminates it when the subscriber fails to authenticate. However, the successful connection has reset the back-off, so the subscriber immediately reconnects and tries to re-authenticate, and so on, ad infinitum.

A smarter approach would be to avoid resetting the back off until a few seconds after the connection has been made to give the remote time to disconnect.

The external command handler using the option --cmd appears to fail when using current master (3.0.0a0) and Python 3.

After switching back a few times to 2.0.1 with Python 2, I gave the issue a closer look: it appears spawn.py is still using the text attribute for event, while the corresponding xml_document class in xml.py is using raw_bytes. Simply replacing text with raw_bytes, and removing an .encoding() method call, fixes this for me.

What confuses me somewhat is that changes to xml_document and in spawn.py were made on the same day, which somewhat suggests this should just work and I'm overlooking something.

A very quick test shows that the same issue actually occurs for Python 2, and the same diff below fixes it there as well.

The diff is

@@ -14,12 +14,12 @@
 __all__ = ["SpawnCommand"]

 class SpawnCommandProtocol(ProcessProtocol):
-    def __init__(self, deferred, raw_bytes):
+    def __init__(self, deferred, text):
         self.deferred = deferred
-        self.raw_bytes = raw_bytes
+        self.text = text

     def connectionMade(self):
-        self.transport.write(self.raw_bytes)
+        self.transport.write(self.text.encode('utf-8'))
         self.transport.closeStdin()

     def processEnded(self, reason):
@@ -47,7 +47,7 @@
         d = defer.Deferred()
         log.info("Running external command: %s" % (self.cmd,))
         reactor.spawnProcess(
-            SpawnCommandProtocol(d, event.raw_bytes),
+            SpawnCommandProtocol(d, event.text),
             self.cmd,
             args=self.args,
             env=os.environ

Need more examples, and disambiguation of the term "publisher".

In trying to build a Conda package for Comet, I am seeing test failures on Windows with python 3.7.1. Extract from the the build log:

__________________ SpawnCommandProtocolTestCase.test_no_exec __________________
self = <Process pid=None>
reactor = <twisted.internet.selectreactor.SelectReactor object at 0x0000009DB8029E10>
protocol = <comet.handler.spawn.SpawnCommandProtocol object at 0x0000009DB9649A20>
command = '/not/a/real/executable', args = ['/not/a/real/executable']
environment = {b'7ZIP': b'"C:\\Program Files\\7-Zip\\7z.exe"', b'ALLUSERSPROFILE': b'C:\\ProgramData', b'APPDATA': b'C:\\Users\\appveyor\\AppData\\Roaming', b'APPVEYOR': b'True', ...}
path = None
    def __init__(self, reactor, protocol, command, args, environment, path):
        """
        Create a new child process.
        """
        _pollingfile._PollingTimer.__init__(self, reactor)
        BaseProcess.__init__(self, protocol)
    
        # security attributes for pipes
        sAttrs = win32security.SECURITY_ATTRIBUTES()
        sAttrs.bInheritHandle = 1
    
        # create the pipes which will connect to the secondary process
        self.hStdoutR, hStdoutW = win32pipe.CreatePipe(sAttrs, 0)
        self.hStderrR, hStderrW = win32pipe.CreatePipe(sAttrs, 0)
        hStdinR, self.hStdinW  = win32pipe.CreatePipe(sAttrs, 0)
    
        win32pipe.SetNamedPipeHandleState(self.hStdinW,
                                          win32pipe.PIPE_NOWAIT,
                                          None,
                                          None)
    
        # set the info structure for the new process.
        StartupInfo = win32process.STARTUPINFO()
        StartupInfo.hStdOutput = hStdoutW
        StartupInfo.hStdError  = hStderrW
        StartupInfo.hStdInput  = hStdinR
        StartupInfo.dwFlags = win32process.STARTF_USESTDHANDLES
    
        # Create new handles whose inheritance property is false
        currentPid = win32api.GetCurrentProcess()
    
        tmp = win32api.DuplicateHandle(currentPid, self.hStdoutR, currentPid, 0, 0,
                                       win32con.DUPLICATE_SAME_ACCESS)
        win32file.CloseHandle(self.hStdoutR)
        self.hStdoutR = tmp
    
        tmp = win32api.DuplicateHandle(currentPid, self.hStderrR, currentPid, 0, 0,
                                       win32con.DUPLICATE_SAME_ACCESS)
        win32file.CloseHandle(self.hStderrR)
        self.hStderrR = tmp
    
        tmp = win32api.DuplicateHandle(currentPid, self.hStdinW, currentPid, 0, 0,
                                       win32con.DUPLICATE_SAME_ACCESS)
        win32file.CloseHandle(self.hStdinW)
        self.hStdinW = tmp
    
        # Add the specified environment to the current environment - this is
        # necessary because certain operations are only supported on Windows
        # if certain environment variables are present.
    
        env = os.environ.copy()
        env.update(environment or {})
        newenv = {}
        for key, value in items(env):
    
            key = _fsdecode(key)
            value = _fsdecode(value)
    
            newenv[key] = value
    
        env = newenv
    
        # Make sure all the arguments are Unicode.
        args = [_fsdecode(x) for x in args]
    
        cmdline = quoteArguments(args)
    
        # The command, too, needs to be Unicode, if it is a value.
        command = _fsdecode(command) if command else command
        path = _fsdecode(path) if path else path
    
        # TODO: error detection here.  See #2787 and #4184.
        def doCreate():
            flags = win32con.CREATE_NO_WINDOW
            self.hProcess, self.hThread, self.pid, dwTid = win32process.CreateProcess(
                command, cmdline, None, None, 1, flags, env, path, StartupInfo)
        try:
>           doCreate()
..\_test_env\lib\site-packages\twisted\internet\_dumbwin32proc.py:227: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    def doCreate():
        flags = win32con.CREATE_NO_WINDOW
        self.hProcess, self.hThread, self.pid, dwTid = win32process.CreateProcess(
>           command, cmdline, None, None, 1, flags, env, path, StartupInfo)
E       pywintypes.error: (3, 'CreateProcess', 'The system cannot find the path specified.')
..\_test_env\lib\site-packages\twisted\internet\_dumbwin32proc.py:225: error
During handling of the above exception, another exception occurred:
self = <comet.handler.test.test_spawn.SpawnCommandProtocolTestCase testMethod=test_no_exec>
    def test_no_exec(self):
        """
        Fail when the executable doesn't exist.
        """
        spawn = SpawnCommand("/not/a/real/executable")
>       return self.assertFailure(spawn(DummyEvent()), Exception)
..\_test_env\lib\site-packages\comet\handler\test\test_spawn.py:39: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
..\_test_env\lib\site-packages\comet\handler\spawn.py:64: in __call__
    env=os.environ
..\_test_env\lib\site-packages\twisted\internet\posixbase.py:353: in spawnProcess
    return Process(self, processProtocol, executable, args, env, path)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <Process pid=None>
reactor = <twisted.internet.selectreactor.SelectReactor object at 0x0000009DB8029E10>
protocol = <comet.handler.spawn.SpawnCommandProtocol object at 0x0000009DB9649A20>
command = '/not/a/real/executable', args = ['/not/a/real/executable']
environment = {b'7ZIP': b'"C:\\Program Files\\7-Zip\\7z.exe"', b'ALLUSERSPROFILE': b'C:\\ProgramData', b'APPDATA': b'C:\\Users\\appveyor\\AppData\\Roaming', b'APPVEYOR': b'True', ...}
path = None
    def __init__(self, reactor, protocol, command, args, environment, path):
        """
        Create a new child process.
        """
        _pollingfile._PollingTimer.__init__(self, reactor)
        BaseProcess.__init__(self, protocol)
    
        # security attributes for pipes
        sAttrs = win32security.SECURITY_ATTRIBUTES()
        sAttrs.bInheritHandle = 1
    
        # create the pipes which will connect to the secondary process
        self.hStdoutR, hStdoutW = win32pipe.CreatePipe(sAttrs, 0)
        self.hStderrR, hStderrW = win32pipe.CreatePipe(sAttrs, 0)
        hStdinR, self.hStdinW  = win32pipe.CreatePipe(sAttrs, 0)
    
        win32pipe.SetNamedPipeHandleState(self.hStdinW,
                                          win32pipe.PIPE_NOWAIT,
                                          None,
                                          None)
    
        # set the info structure for the new process.
        StartupInfo = win32process.STARTUPINFO()
        StartupInfo.hStdOutput = hStdoutW
        StartupInfo.hStdError  = hStderrW
        StartupInfo.hStdInput  = hStdinR
        StartupInfo.dwFlags = win32process.STARTF_USESTDHANDLES
    
        # Create new handles whose inheritance property is false
        currentPid = win32api.GetCurrentProcess()
    
        tmp = win32api.DuplicateHandle(currentPid, self.hStdoutR, currentPid, 0, 0,
                                       win32con.DUPLICATE_SAME_ACCESS)
        win32file.CloseHandle(self.hStdoutR)
        self.hStdoutR = tmp
    
        tmp = win32api.DuplicateHandle(currentPid, self.hStderrR, currentPid, 0, 0,
                                       win32con.DUPLICATE_SAME_ACCESS)
        win32file.CloseHandle(self.hStderrR)
        self.hStderrR = tmp
    
        tmp = win32api.DuplicateHandle(currentPid, self.hStdinW, currentPid, 0, 0,
                                       win32con.DUPLICATE_SAME_ACCESS)
        win32file.CloseHandle(self.hStdinW)
        self.hStdinW = tmp
    
        # Add the specified environment to the current environment - this is
        # necessary because certain operations are only supported on Windows
        # if certain environment variables are present.
    
        env = os.environ.copy()
        env.update(environment or {})
        newenv = {}
        for key, value in items(env):
    
            key = _fsdecode(key)
            value = _fsdecode(value)
    
            newenv[key] = value
    
        env = newenv
    
        # Make sure all the arguments are Unicode.
        args = [_fsdecode(x) for x in args]
    
        cmdline = quoteArguments(args)
    
        # The command, too, needs to be Unicode, if it is a value.
        command = _fsdecode(command) if command else command
        path = _fsdecode(path) if path else path
    
        # TODO: error detection here.  See #2787 and #4184.
        def doCreate():
            flags = win32con.CREATE_NO_WINDOW
            self.hProcess, self.hThread, self.pid, dwTid = win32process.CreateProcess(
                command, cmdline, None, None, 1, flags, env, path, StartupInfo)
        try:
            doCreate()
        except pywintypes.error as pwte:
            if not _invalidWin32App(pwte):
                # This behavior isn't _really_ documented, but let's make it
                # consistent with the behavior that is documented.
>               raise OSError(pwte)
E               OSError: (3, 'CreateProcess', 'The system cannot find the path specified.')
..\_test_env\lib\site-packages\twisted\internet\_dumbwin32proc.py:232: OSError

Have these errors been seen before? If its just that the test suite doesn't run on Windows, but the main library does, that can easily be worked around during conda builds.

This could be implemented as for publishers (see #2), but is lower priority.

Hi John,

currently, to send VOEvents programmatically, I simply call out to comet-sendvo via the command line; a la:
https://github.com/timstaley/pysovo/blob/master/pysovo/comms/comet.py

Because essentially the command line is the only stable and user-friendly API, as far as I'm concerned.
While this actually works quite cleanly, would it be sensible to wrap this same functionality internally from Comet and provide a Python API?

Should use a <Param> instead.

When a subscriber connects, it should be possible to challenge them for credentials and drop the connection if the challenge fails.

See the Allan & Denny protocol, and note the relationship to signing packets (#4).

$ twistd comet -b --broadcast-port=9876 --action=save-event --ivorndb=`pwd`/data

I then try and send a test event (received from GCN this afternoon):

$ comet-sendvo --port=9876 --file=./test.xml

and get:

2012-02-29 15:06:07-0800 [-] Log opened.
2012-02-29 15:06:07-0800 [-] Starting factory <__main__.OneShotSender instance at 0x101168638>
2012-02-29 15:06:07-0800 [VOEventSender,client] Got response from IPv4Address(TCP, '127.0.0.1', 9876)
2012-02-29 15:06:07-0800 [VOEventSender,client] "Incomprehensible data received from IPv4Address(TCP, '127.0.0.1', 9876) (role=authenticate)"
2012-02-29 15:06:07-0800 [VOEventSender,client] Stopping factory <__main__.OneShotSender instance at 0x101168638>
2012-02-29 15:06:07-0800 [VOEventSender,client] 'Event was NOT sent successfully'
2012-02-29 15:06:07-0800 [-] Main loop terminated.

The twistd.log shows:

2012-02-29 15:06:07-0800 [comet.tcp.protocol.VOEventBroadcasterFactory] New subscriber at IPv4Address(TCP, '127.0.0.1', 58677)
2012-02-29 15:06:07-0800 [VOEventBroadcaster,1,127.0.0.1] "Incomprehensible data received from IPv4Address(TCP, '127.0.0.1', 58677) (role=utility)"
2012-02-29 15:06:07-0800 [VOEventBroadcaster,1,127.0.0.1] Subscriber at IPv4Address(TCP, '127.0.0.1', 58677) disconnected
2012-02-29 15:06:14-0800 [-] Broadcasting iamalive

Hello,

I am running into a couple issues while trying to use Comet--

I run the below to receive events through Comet:

twistd -n comet --local-ivo=ivo://edu.oswego.team8/comet_broker  --remote=voevent.swinbank.org --save-event --save-event-directory=/home/richardjames/events -r

Unfortunately this is giving me the below error with ipaddress:

ipaddress.AddressValueError: '0.0.0.0/0' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

I have installed comet, however, when I run "trial comet" I get 25 failures here are a couple of the errors:

===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/comet/utility/test/test_whitelist.py", line 57, in test_log_message
    TestFactory(), [ip_network('127.0.0.1/32')], TEST_STRING
  File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
    ' a unicode object?' % address)
ipaddress.AddressValueError: '127.0.0.1/32' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.utility.test.test_whitelist.WhitelistingFactoryTestCase.test_log_message
===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/comet/utility/test/test_whitelist.py", line 47, in test_not_in_whitelist
    factory = WhitelistingFactory(TestFactory(), [ip_network('127.0.0.1/32')])
  File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
    ' a unicode object?' % address)
ipaddress.AddressValueError: '127.0.0.1/32' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.utility.test.test_whitelist.WhitelistingFactoryTestCase.test_not_in_whitelist
-------------------------------------------------------------------------------
Ran 157 tests in 4.891s

FAILED (errors=25, successes=132)

Thank You for any help!
-Rick W

I can start Comet with an XPath filter, like so:

 twistd -n comet --remote localhost \
    --filter='not(starts-with(@ivorn,"ivo://voevent.phys.soton.ac.uk/FOO"))'

which works fine. But, if the XPath is broken, e.g.

--filter='not(starts-with(@ivorn,"ivo://voevent.phys.soton.ac.uk/FOO"'

(note missing close brackets), then Comet will start up quietly and the filter will silently fail. Currently, the only way to check if the XPath translates to a successfully installed filter is to monitor the logs at the server end.

Looks like we try to cancel the timeout counter which has already expired.

2012-03-01 04:00:23+0100 [-] No iamalive received for 120 seconds; disconecting
2012-03-01 04:00:23+0100 [VOEventSubscriber,client] Unhandled Error
    Traceback (most recent call last):
      File "/home/jds/sw/lib/python2.6/site-packages/Twisted-11.1.0-py2.6-linux-x86_64.egg/twisted/application/app.py", line 323, in runReactorWithLogging
        reactor.run()
      File "/home/jds/sw/lib/python2.6/site-packages/Twisted-11.1.0-py2.6-linux-x86_64.egg/twisted/internet/base.py", line 1169, in run
        self.mainLoop()
      File "/home/jds/sw/lib/python2.6/site-packages/Twisted-11.1.0-py2.6-linux-x86_64.egg/twisted/internet/base.py", line 1181, in mainLoop
        self.doIteration(t)
      File "/home/jds/sw/lib/python2.6/site-packages/Twisted-11.1.0-py2.6-linux-x86_64.egg/twisted/internet/pollreactor.py", line 167, in doPoll
        log.callWithLogger(selectable, _drdw, selectable, fd, event)
    --- <exception caught here> ---
      File "/home/jds/sw/lib/python2.6/site-packages/Twisted-11.1.0-py2.6-linux-x86_64.egg/twisted/python/log.py", line 84, in callWithLogger
        return callWithContext({"system": lp}, func, *args, **kw)
      File "/home/jds/sw/lib/python2.6/site-packages/Twisted-11.1.0-py2.6-linux-x86_64.egg/twisted/python/log.py", line 69, in callWithContext
        return context.call({ILogContext: newCtx}, func, *args, **kw)
      File "/home/jds/sw/lib/python2.6/site-packages/Twisted-11.1.0-py2.6-linux-x86_64.egg/twisted/python/context.py", line 118, in callWithContext
        return self.currentContext().callWithContext(ctx, func, *args, **kw)
      File "/home/jds/sw/lib/python2.6/site-packages/Twisted-11.1.0-py2.6-linux-x86_64.egg/twisted/python/context.py", line 81, in callWithContext
        return func(*args,**kw)
      File "/home/jds/sw/lib/python2.6/site-packages/Twisted-11.1.0-py2.6-linux-x86_64.egg/twisted/internet/posixbase.py", line 594, in _doReadOrWrite
        self._disconnectSelectable(selectable, why, inRead)
      File "/home/jds/sw/lib/python2.6/site-packages/Twisted-11.1.0-py2.6-linux-x86_64.egg/twisted/internet/posixbase.py", line 263, in _disconnectSelectable
        selectable.connectionLost(f)
      File "/home/jds/sw/lib/python2.6/site-packages/Twisted-11.1.0-py2.6-linux-x86_64.egg/twisted/internet/tcp.py", line 433, in connectionLost
        Connection.connectionLost(self, reason)
      File "/home/jds/sw/lib/python2.6/site-packages/Twisted-11.1.0-py2.6-linux-x86_64.egg/twisted/internet/tcp.py", line 277, in connectionLost
        protocol.connectionLost(reason)
      File "/home/jds/Work/Comet/comet/tcp/protocol.py", line 170, in connectionLost
        self.check_alive.cancel()
      File "/home/jds/sw/lib/python2.6/site-packages/Twisted-11.1.0-py2.6-linux-x86_64.egg/twisted/internet/base.py", line 89, in cancel
        raise error.AlreadyCalled
    twisted.internet.error.AlreadyCalled: Tried to cancel an already-called event.

[Wishlist item]
Something of a corollary to #32:
(Until such time as we're all submitting VOEvents with GPG fingerprints, and everyone is verifying packets received according to said fingerprints:)
It would be very handy to apply an ivorn-prefix matching component to remote-ip whitelisting, so e.g.
'Only relay events from remote broker @ 123.4.5.6 if their ivorn begins with the prefix ``ivo://foo.bar.acme/```
This would be a simple way of ensuring that remote brokers do not 'pollute' someone else's stream, either through human error or malicious intervention.

I can think of ways to implement this currently using multiple instances of Comet running on one machine (different ports), by applying filtering within the known-good local instances, but it gets a bit baroque.

When subscribing to multiple brokers which all send the same event, Comet should process the first copy of that event to be received and drop the others. However, sometimes an extra copy slips through. For example:

2014-07-05 11:34:36+0200 [INFO VOEventSubscriber,client] VOEvent ivo://nasa.gsfc.gcn/SWIFT#XRT_Proc_Spec_603488-809 received from IPv4Address(TCP, '209.208.78.170', 8099)
2014-07-05 11:34:36+0200 [INFO VOEventSubscriber,client] VOEvent ivo://nasa.gsfc.gcn/SWIFT#XRT_Proc_Spec_603488-809 received from IPv4Address(TCP, '152.78.192.87', 8099)
2014-07-05 11:34:36+0200 [INFO -] Event rejected (Previously seen by this broker); discarding
2014-07-05 11:34:36+0200 [INFO VOEventSubscriber,client] VOEvent ivo://nasa.gsfc.gcn/SWIFT#XRT_Proc_Spec_603488-809 received from IPv4Address(TCP, '50.116.49.68', 8099)
2014-07-05 11:34:36+0200 [INFO -] Event rejected (Previously seen by this broker); discarding
2014-07-05 11:34:36+0200 [INFO VOEventSubscriber,client] VOEvent ivo://nasa.gsfc.gcn/SWIFT#XRT_Proc_Spec_603488-809 received from IPv4Address(TCP, '152.78.192.87', 8099)
2014-07-05 11:34:36+0200 [INFO VOEventSubscriber,client] VOEvent ivo://nasa.gsfc.gcn/SWIFT#XRT_Proc_Spec_603488-809 received from IPv4Address(TCP, '68.169.57.253', 8099)

Here, the same IVORN is received five times. The event comes in both 1.1 and 2.0 formats, so accepting two copies is legitimate. However, two copies are flagged as duplicate, and three are accepted. One must be a mistake.

Cannot properly install comet via pip. I executed the following:

$ virtualenv comet
$ . comet/bin/activate
$ pip install comet

Here is the log file:

cometlog.txt

Hello, I try to get Comet work, but the trial failed:

charlyms@ubuntu:~$ trial comet
comet.handler.test.test_relay
EventRelayTestCase
test_interface ... [OK]
test_name ... [OK]
test_send_event ... [OK]
comet.handler.test.test_spawn
SpawnCommandProtocolTestCase
test_bad_process ... [OK]
test_good_process ... [OK]
test_interface ... [OK]
test_write_data ... [OK]
comet.log.test.test_log
test_comet_logging
test_default_level ... [OK]
test_log_debug_level_debug ... [OK]
test_log_debug_level_info ... [OK]
test_log_debug_level_warning ... [OK]
test_log_info_level_debug ... [OK]
test_log_info_level_info ... [OK]
test_log_info_level_warning ... [OK]
test_log_warning_level_debug ... [OK]
test_log_warning_level_info ... [OK]
test_log_warning_level_warning ... [OK]
comet.plugins.test.test_eventprinter
EventPrinterTestCase
test_interface ... [OK]
test_name ... [OK]
test_print_event ... [OK]
comet.plugins.test.test_eventwriter
EventFileTestCase
test_dup_file ... [OK]
test_file_contents ... [OK]
test_file_created ... [OK]
test_temp_dir ... [OK]
EventWriterTestCase
test_create_directory ... [OK]
test_custom_directory ... [OK]
test_interface ... [OK]
test_name ... [OK]
test_save_event ... [OK]
StringToFilenameTestCase
test_characters ... [OK]
comet.protocol.test.test_base
ElementSenderTestCase
test_lengthLimitExceeded ... [OK]
test_send_xml ... [OK]
EventHandlerTestCase
test_handle_event_fails ... [OK]
test_handle_event_succeeds ... [OK]
test_process_event_invalid_can_nak ... [OK]
test_process_event_invalid_cant_nak ... [OK]
test_process_event_valid_fails ... [OK]
test_process_event_valid_succeeds ... [OK]
test_validate_event_invalid ... [OK]
test_validate_event_valid ... [OK]
comet.protocol.test.test_messages
TransportTestCase
test_ack_valid ... [OK]
test_authenticate_valid ... [OK]
test_authenticateresponse_valid ... [OK]
test_authenticateresponse_valid_filter ... [OK]
test_iamalive_valid ... [OK]
test_iamaliveresponse_valid ... [OK]
test_nak_valid ... [OK]
test_nak_with_result_valid ... [OK]
comet.protocol.test.test_voeventbroadcaster
VOEventBroadcasterFactoryNoTestEventsTestCase
test_DoNotSendTestEvent ... [OK]
VOEventBroadcasterFactoryTestCase
test_protocol ... [OK]
test_sendIamAlive ... [OK]
test_sendTestEvent ... [OK]
VOEventBroadcasterTestCase
test_alive_timeout ... [OK]
test_lack_of_ack ... [OK]
test_receive_ack ... [OK]
test_receive_authenticateresponse_legacy ... [OK]
test_receive_authenticateresponse_with_bad_filter ... [OK]
test_receive_authenticateresponse_with_filters ... [OK]
test_receive_iamalive ... [OK]
test_receive_incomprehensible ... [OK]
test_receive_nak ... [OK]
test_receive_unparsable ... [OK]
test_register_broadcaster ... [OK]
test_sendIAmAlive ... [OK]
test_send_event ... [OK]
test_send_event_with_filter_accept ... [OK]
test_send_event_with_filter_reject ... [OK]
test_sent_authenticate ... [OK]
comet.protocol.test.test_voeventreceiver
VOEventReceiverFactoryTestCase
test_contents ... [OK]
test_protocol ... [OK]
VOEventReceiverTestCase
test_receive_incomprehensible ... [OK]
test_receive_unparsable ... [OK]
test_receive_voevent ... [OK]
test_receive_voevent_invalid ... [OK]
test_timeout ... [OK]
comet.protocol.test.test_voeventsender
VOEventSenderFactoryTestCase
test_no_ack ... [OK]
test_protocol ... [OK]
test_stored_event ... [OK]
VOEventSenderTestCase
test_connectionMade ... [OK]
test_receive_ack ... [OK]
test_receive_incomprehensible ... [OK]
test_receive_nak ... [OK]
test_receive_unparsable ... [OK]
comet.protocol.test.test_voeventsubscriber
VOEventSubscriberFactoryTestCase
test_protocol ... [OK]
test_reconnect_delay ... [OK]
VOEventSubscriberTestCase
test_receive_authenticate ... [OK]
test_receive_iamalive ... [OK]
test_receive_incomprehensible ... [OK]
test_receive_invalid_voevent ... [OK]
test_receive_unparsable ... [OK]
test_receive_valid_voevent ... [OK]
VOEventSubscriberTimeoutTestCase
test_timeout ... [OK]
comet.service.test.test_broker
DefaultOptionsTestCase
test_action_non_extant ... [OK]
test_args_for_disabled_plugin ... [ERROR]
test_cmd ... [ERROR]
test_default_broadcast_whitelist ... [ERROR]
test_default_submission_whitelist ... [ERROR]
test_enable_broadcast ... [ERROR]
test_enable_receive ... [ERROR]
test_faulty_cmd_line ... [OK]
test_filters ... [ERROR]
test_has_print_event_plugin ... [ERROR]
test_has_save_event_plugin ... [ERROR]
test_invalid_filter ... [OK]
test_invalid_ivorn ... [OK]
test_ivorn_missing ... [OK]
test_populated_broadcast_whitelist ... [ERROR]
test_populated_submission_whitelist ... [ERROR]
test_remotes ... [ERROR]
test_save_event_plugin_takes_args ... [ERROR]
test_test_event_loop ... [ERROR]
test_valid_ivorn ... [ERROR]
test_verbose_contradictory ... [ERROR]
test_verbose_default ... [ERROR]
test_verbose_quiet ... [ERROR]
test_verbose_verbose ... [ERROR]
ServiceTestCase
test_has_receiver ... [ERROR]
test_no_service ... [ERROR]
comet.utility.test.test_event_db
Event_DB_TestCase
test_bad_ivorn ... [OK]
test_prune ... [OK]
test_prune_bad_event ... [OK]
test_seen ... [OK]
test_threadsafe ... [OK]
test_unseen ... [OK]
comet.utility.test.test_voevent
broker_test_messageTestCase
test_valid ... [OK]
parse_ivornTestCase
test_authority_id ... [OK]
test_fragment ... [OK]
test_no_fragment ... [OK]
test_partial_ivorn ... [OK]
test_resource_key ... [OK]
test_simple ... [OK]
comet.utility.test.test_whitelist
WhitelistingFactoryTestCase
test_empty_whitelist ... [ERROR]
test_in_whitelist ... [ERROR]
test_log_message ... [ERROR]
test_not_in_whitelist ... [ERROR]
comet.utility.test.test_xml
mutable_element_tests
test_transform_element ... [OK]
test_transform_text ... [OK]
xml_document_encoding
test_encoding_detection ... [OK]
test_from_unicode ... [OK]
xml_document_from_element_TestCase
test_element ... [OK]
test_signature ... [OK]
test_text ... [OK]
xml_document_from_string_TestCase
test_element ... [OK]
test_signature ... [OK]
test_text ... [OK]
xml_security_TestCase
test_billion_laughs ... [OK]
test_quadratic_blowup ... [OK]
comet.validator.test.test_ivorn
CheckSchemaTestCase
test_interface ... [OK]
test_invalid ... [OK]
test_valid ... [OK]
comet.validator.test.test_previously_seen
CheckPreviouslySeenTestCase
test_db_failure ... [OK]
test_interface ... [OK]
test_previously_checked ... [OK]
test_unseen ... [OK]
comet.validator.test.test_schema
CheckSchemaTestCase
test_interface ... [OK]
test_invalid ... [OK]
test_valid ... [OK]

===============================================================================
[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 108, in test_args_for_disabled_plugin
self.config.parseOptions(self.cmd_line)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 277, in parseOptions
self.postOptions()
File "/usr/local/lib/python2.7/dist-packages/comet/service/broker.py", line 119, in postOptions
self['author-whitelist'] = [ip_network(self['author-whitelist'], strict=False)]
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '0.0.0.0/0' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.service.test.test_broker.DefaultOptionsTestCase.test_args_for_disabled_plugin

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 127, in test_cmd
self.config.parseOptions(self.cmd_line)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 277, in parseOptions
self.postOptions()
File "/usr/local/lib/python2.7/dist-packages/comet/service/broker.py", line 119, in postOptions
self['author-whitelist'] = [ip_network(self['author-whitelist'], strict=False)]
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '0.0.0.0/0' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.service.test.test_broker.DefaultOptionsTestCase.test_cmd

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 65, in test_default_broadcast_whitelist
self._test_empty_whitelist('subscriber-whitelist')
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 61, in _test_empty_whitelist
self.config.parseOptions(self.cmd_line)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 277, in parseOptions
self.postOptions()
File "/usr/local/lib/python2.7/dist-packages/comet/service/broker.py", line 119, in postOptions
self['author-whitelist'] = [ip_network(self['author-whitelist'], strict=False)]
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '0.0.0.0/0' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.service.test.test_broker.DefaultOptionsTestCase.test_default_broadcast_whitelist

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 68, in test_default_submission_whitelist
self._test_empty_whitelist('author-whitelist')
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 61, in _test_empty_whitelist
self.config.parseOptions(self.cmd_line)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 277, in parseOptions
self.postOptions()
File "/usr/local/lib/python2.7/dist-packages/comet/service/broker.py", line 119, in postOptions
self['author-whitelist'] = [ip_network(self['author-whitelist'], strict=False)]
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '0.0.0.0/0' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.service.test.test_broker.DefaultOptionsTestCase.test_default_submission_whitelist

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 48, in test_enable_broadcast
self.config.parseOptions(self.cmd_line)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 277, in parseOptions
self.postOptions()
File "/usr/local/lib/python2.7/dist-packages/comet/service/broker.py", line 119, in postOptions
self['author-whitelist'] = [ip_network(self['author-whitelist'], strict=False)]
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '0.0.0.0/0' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.service.test.test_broker.DefaultOptionsTestCase.test_enable_broadcast

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 42, in test_enable_receive
self.config.parseOptions(self.cmd_line)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 277, in parseOptions
self.postOptions()
File "/usr/local/lib/python2.7/dist-packages/comet/service/broker.py", line 119, in postOptions
self['author-whitelist'] = [ip_network(self['author-whitelist'], strict=False)]
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '0.0.0.0/0' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.service.test.test_broker.DefaultOptionsTestCase.test_enable_receive

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 118, in test_filters
self.config.parseOptions(self.cmd_line)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 277, in parseOptions
self.postOptions()
File "/usr/local/lib/python2.7/dist-packages/comet/service/broker.py", line 119, in postOptions
self['author-whitelist'] = [ip_network(self['author-whitelist'], strict=False)]
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '0.0.0.0/0' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.service.test.test_broker.DefaultOptionsTestCase.test_filters

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 89, in test_has_print_event_plugin
self.config.parseOptions(self.cmd_line)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 277, in parseOptions
self.postOptions()
File "/usr/local/lib/python2.7/dist-packages/comet/service/broker.py", line 119, in postOptions
self['author-whitelist'] = [ip_network(self['author-whitelist'], strict=False)]
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '0.0.0.0/0' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.service.test.test_broker.DefaultOptionsTestCase.test_has_print_event_plugin

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 95, in test_has_save_event_plugin
self.config.parseOptions(self.cmd_line)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 277, in parseOptions
self.postOptions()
File "/usr/local/lib/python2.7/dist-packages/comet/service/broker.py", line 119, in postOptions
self['author-whitelist'] = [ip_network(self['author-whitelist'], strict=False)]
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '0.0.0.0/0' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.service.test.test_broker.DefaultOptionsTestCase.test_has_save_event_plugin

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 82, in test_populated_broadcast_whitelist
self._test_populated_whitelist('subscriber-whitelist')
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 74, in _test_populated_whitelist
self.config.parseOptions(self.cmd_line)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 255, in parseOptions
self._dispatch[optMangled](optMangled, arg)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 411, in
fn = lambda name, value, m=method: m(value)
File "/usr/local/lib/python2.7/dist-packages/comet/service/broker.py", line 112, in opt_subscriber_whitelist
self['running_subscriber-whitelist'].append(ip_network(network, strict=False))
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '1.2.3.4/32' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.service.test.test_broker.DefaultOptionsTestCase.test_populated_broadcast_whitelist

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 85, in test_populated_submission_whitelist
self._test_populated_whitelist('author-whitelist')
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 74, in _test_populated_whitelist
self.config.parseOptions(self.cmd_line)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 255, in parseOptions
self._dispatch[optMangled](optMangled, arg)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 411, in
fn = lambda name, value, m=method: m(value)
File "/usr/local/lib/python2.7/dist-packages/comet/service/broker.py", line 108, in opt_author_whitelist
self['running_author-whitelist'].append(ip_network(network, strict=False))
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '1.2.3.4/32' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.service.test.test_broker.DefaultOptionsTestCase.test_populated_submission_whitelist

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 55, in test_remotes
self.config.parseOptions(self.cmd_line)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 277, in parseOptions
self.postOptions()
File "/usr/local/lib/python2.7/dist-packages/comet/service/broker.py", line 119, in postOptions
self['author-whitelist'] = [ip_network(self['author-whitelist'], strict=False)]
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '0.0.0.0/0' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.service.test.test_broker.DefaultOptionsTestCase.test_remotes

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 101, in test_save_event_plugin_takes_args
self.config.parseOptions(self.cmd_line)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 277, in parseOptions
self.postOptions()
File "/usr/local/lib/python2.7/dist-packages/comet/service/broker.py", line 119, in postOptions
self['author-whitelist'] = [ip_network(self['author-whitelist'], strict=False)]
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '0.0.0.0/0' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.service.test.test_broker.DefaultOptionsTestCase.test_save_event_plugin_takes_args

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 36, in test_test_event_loop
self.config.parseOptions(self.cmd_line)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 277, in parseOptions
self.postOptions()
File "/usr/local/lib/python2.7/dist-packages/comet/service/broker.py", line 119, in postOptions
self['author-whitelist'] = [ip_network(self['author-whitelist'], strict=False)]
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '0.0.0.0/0' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.service.test.test_broker.DefaultOptionsTestCase.test_test_event_loop

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 150, in test_valid_ivorn
self.config.parseOptions(self.cmd_line)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 277, in parseOptions
self.postOptions()
File "/usr/local/lib/python2.7/dist-packages/comet/service/broker.py", line 119, in postOptions
self['author-whitelist'] = [ip_network(self['author-whitelist'], strict=False)]
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '0.0.0.0/0' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.service.test.test_broker.DefaultOptionsTestCase.test_valid_ivorn

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 146, in test_verbose_contradictory
self.config.parseOptions(self.cmd_line)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 277, in parseOptions
self.postOptions()
File "/usr/local/lib/python2.7/dist-packages/comet/service/broker.py", line 119, in postOptions
self['author-whitelist'] = [ip_network(self['author-whitelist'], strict=False)]
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '0.0.0.0/0' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.service.test.test_broker.DefaultOptionsTestCase.test_verbose_contradictory

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 131, in test_verbose_default
self.config.parseOptions(self.cmd_line)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 277, in parseOptions
self.postOptions()
File "/usr/local/lib/python2.7/dist-packages/comet/service/broker.py", line 119, in postOptions
self['author-whitelist'] = [ip_network(self['author-whitelist'], strict=False)]
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '0.0.0.0/0' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.service.test.test_broker.DefaultOptionsTestCase.test_verbose_default

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 141, in test_verbose_quiet
self.config.parseOptions(self.cmd_line)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 277, in parseOptions
self.postOptions()
File "/usr/local/lib/python2.7/dist-packages/comet/service/broker.py", line 119, in postOptions
self['author-whitelist'] = [ip_network(self['author-whitelist'], strict=False)]
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '0.0.0.0/0' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.service.test.test_broker.DefaultOptionsTestCase.test_verbose_quiet

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 136, in test_verbose_verbose
self.config.parseOptions(self.cmd_line)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 277, in parseOptions
self.postOptions()
File "/usr/local/lib/python2.7/dist-packages/comet/service/broker.py", line 119, in postOptions
self['author-whitelist'] = [ip_network(self['author-whitelist'], strict=False)]
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '0.0.0.0/0' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.service.test.test_broker.DefaultOptionsTestCase.test_verbose_verbose

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 171, in test_has_receiver
'-b', '--remote', 'dummy'])
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 166, in _make_service
self.config.parseOptions(options)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 277, in parseOptions
self.postOptions()
File "/usr/local/lib/python2.7/dist-packages/comet/service/broker.py", line 119, in postOptions
self['author-whitelist'] = [ip_network(self['author-whitelist'], strict=False)]
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '0.0.0.0/0' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.service.test.test_broker.ServiceTestCase.test_has_receiver

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 194, in test_no_service
service = self._make_service(['--local-ivo', 'ivo://comet/test'])
File "/usr/local/lib/python2.7/dist-packages/comet/service/test/test_broker.py", line 166, in _make_service
self.config.parseOptions(options)
File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 277, in parseOptions
self.postOptions()
File "/usr/local/lib/python2.7/dist-packages/comet/service/broker.py", line 119, in postOptions
self['author-whitelist'] = [ip_network(self['author-whitelist'], strict=False)]
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '0.0.0.0/0' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.service.test.test_broker.ServiceTestCase.test_no_service

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/utility/test/test_whitelist.py", line 30, in test_empty_whitelist
factory.buildProtocol(IPv4Address('TCP', '127.0.0.1', 0)),
File "/usr/local/lib/python2.7/dist-packages/comet/utility/whitelist.py", line 17, in buildProtocol
remote_ip = ip_address(addr.host)
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 163, in ip_address
' a unicode object?' % address)
ipaddress.AddressValueError: '127.0.0.1' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.utility.test.test_whitelist.WhitelistingFactoryTestCase.test_empty_whitelist

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/utility/test/test_whitelist.py", line 38, in test_in_whitelist
factory = WhitelistingFactory(TestFactory(), [ip_network('0.0.0.0/0')])
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '0.0.0.0/0' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.utility.test.test_whitelist.WhitelistingFactoryTestCase.test_in_whitelist

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/utility/test/test_whitelist.py", line 57, in test_log_message
TestFactory(), [ip_network('127.0.0.1/32')], TEST_STRING
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '127.0.0.1/32' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.utility.test.test_whitelist.WhitelistingFactoryTestCase.test_log_message

[ERROR]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/comet/utility/test/test_whitelist.py", line 47, in test_not_in_whitelist
factory = WhitelistingFactory(TestFactory(), [ip_network('127.0.0.1/32')])
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: '127.0.0.1/32' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

comet.utility.test.test_whitelist.WhitelistingFactoryTestCase.test_not_in_whitelist

Ran 157 tests in 1.182s

FAILED (errors=25, successes=132)
Can you please describe, what is going wrong here?
Regards from Germany
Charlyms

What if we (try to) sign with an expired, revoked, non-existent, ..., key?

I get an error when running trial, which says automat is not there; however, automat is there. Please see below:

trial comet
Traceback (most recent call last):
File "/Users/ben/miniconda2/bin/trial", line 6, in
from pkg_resources import load_entry_point
File "/Users/ben/miniconda2/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/init.py", line 2985, in
File "/Users/ben/miniconda2/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/init.py", line 2971, in _call_aside
File "/Users/ben/miniconda2/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/init.py", line 2998, in _initialize_master_working_set
File "/Users/ben/miniconda2/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/init.py", line 662, in _build_master
File "/Users/ben/miniconda2/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/init.py", line 675, in _build_from_requirements
File "/Users/ben/miniconda2/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/init.py", line 854, in resolve
pkg_resources.DistributionNotFound: The 'Automat>=0.3.0' distribution was not found and is required by Twisted
$ sudo conda install automat
Fetching package metadata .............
Solving package specifications: .

All requested packages already installed.

This is the approach for distributing VOEvents taken by eg http://voeventnet.caltech.edu/software/index.html. It would be convenient if Comet could subscribe to these feeds and redistribute over TCP. Publishing to PubSub would seem to be a lower priority.

The PubSub system is very (very!) complex, at least to a neophyte. Twisted/Wokkel offer support, but little documentation.

When the subscriber receives a new event, it currently just prints it to the log. Which is fine to demonstrate things are working, but not very useful. We should allow the end user to specify Python functions (or maybe external scripts?) which are run when an event is received.

Hi John,

quick question on a technicality:
currently IVORNS are given the yay/nay by this regex:

Should the set of fragment characters include '+', or is there a good reason for exclusion?

Seems to be valid for HTTP, cf http://stackoverflow.com/a/1547940/725650, AFAICT, and it is occasionally useful when assigning substream IDs, e.g. if referring to some event generally known via its co-ordinates.

This should be simple enough to implement locally rather than pulling in another external dependency.

Tim writes:

Hi John,

I can't get the comet-sendvo script to run (see below) - when you have time, would you mind testing it your end to check if I'm doing something silly, or if there really is a bug?

Cheers,
Tim.

ts3e11@astro41:~/code/comet$ scripts/comet-sendvo -h voevent -f ~/code/vo_alerts/test_data/soton_test1.1.xml
Traceback (most recent call last):
 File "scripts/comet-sendvo", line 46, in <module>
   log.startLogging(sys.stdout)
AttributeError: 'module' object has no attribute 'startLogging'

ie, it's outdated.

Hi @jdswinbank. I'm interested in distributing Comet with Conda via conda-forge.

Do you have any objections to me doing this?
Do you wish to be named as a maintainer?

Thanks
Duncan

After first installing comet, running "trial comet" gave me lots of error messages like this (running in Python 2.7.10 on Ubuntu 15.10):

exceptions.ValueError: '127.0.0.1/32' does not appear to be an IPv4 or IPv6 network

Updating ipaddress via pip from 1.0.14 (which is in the Ubuntu 15.10 package) to 1.0.16 helped a little, now the message changed to:

ipaddress.AddressValueError: '127.0.0.1/32' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?

So the call to ip_network should be ip_network(u'127.0.0.1/32') instead of ip_network('127.0.0.1/32').
Is this only a problem for me or am I doing something wrong?

Or possibly removed outright -- is it actually useful?

Per jdswinbank/vtp@2fc70f1, timestamps in Transport messages must be UTC (either with or without a "Z" specifier). Ensure that we are properly testing for this.

If we create an event with the IVORN ivo://, then the event db check will attempt to create a database at ${eventdb_root}, which is a directory. This will fail, and the event will be rejected, which is fine.

However, there is now an entry called "" in the list of databases. When we try to prune the databases, that will get locked, but the prune fails and the database is never unlocked. Our event validation then seizes up.

When we refuse to accept a VOEvent message, or for any other event that results in generating a NAK, we should provide the submitter with a reason (eg "event failed schema validation") rather than just a NAK.

Should log a helpful message, so those who aren't clear on what all those factories actually do have some clue about what's going on...

For instance, if the remote immediately disconnects we get:

$ python scripts/comet-sendvo < ~/Downloads/BAT_GRB_Pos_512035-500.xml 
2012-02-09 17:05:21+0100 [-] Log opened.
2012-02-09 17:05:21+0100 [-] Starting factory <__main__.OneShotSender instance at 0x10f591950>
2012-02-09 17:05:21+0100 [VOEventSender,client] Stopping factory <__main__.OneShotSender instance at 0x10f591950>
2012-02-09 17:05:21+0100 [-] Main loop terminated.

Not clear that anything went wrong at all!

First prototype landed in 8d6ed28.

Testing against Comet==1.2.0,

twistd -n comet --receive --broadcast --verbose

runs sort-of OK, except it logs an unhandled error from attempting to use a local IVO set to None.

Turns out that the local-ivo should always be specified, but the None is slipping through the net here:

As an additional note, when an incorrect IVO is specified, it might be nice to explain the required format a la

Invalid IVOA identifier: foo

Which reminds me of 'number invalid, pick again' :-)

Currently, we run the handlers when an event is received, regardless of whether it has a good signature. Should be an option to make signed events mandatory.

Currently, it's possible for clients to request that the server perform some filtering on their behalf.

However, it would be nice for the server admin to be able to impose some blanket filtering, i.e. "do not forward any events matching pattern X to any subscriber".

This would be easy to add to the EventRelay handler. Perhaps EventRelay should just become a plugin rather than a core part of the broker.