isolver / iohub Goto Github PK

Currently, only language sets / keyboards that work with the ascii character set will work with ioHub. Need to replace the module being used on Windows for keyboard event hooking with a unicode version (or patch existing module source to it uses Unicode version of Windows functions).

button events have been added, need to add axis info

Directories generated for each session should contain native device data and the trial list (only if using ConditionVariableProvider class?). Currently the native device data is stored in only the most recent session (and deleted when a subsequent session is started).

Current implementation relies on settings provided to external setup tool.

I am fairly new to python and all the other modules Ive used all tend to have the setup.py file to install it automatically. I have installed all the deps as far as I can tell, but every file or example I run gives this error. Ive added the ioHub path to PYTHONPATH too. I cant find a file called ioHub in the repo so Im a bit confused...

Sorry if this is a dumb question

experiment events now cause the ioHub server to crash, as the format of the list set being set seems to be out of sync with the expected numpy format.

add sendMessages and sendCommands utility methods to the ioHubClient or to the psychopyIOHubRuntime

smi is being worked on, should be done by end of week (by August 31st).

Quartz is a C library, not objC. It should not be hard to wrap the neseccary API functions in ctypes, proving better flexibility and removing the pyobjc depdency. Again, if / when there is time.

Should also check to see if Alex has not already wrapped the Quartz lib in ctypes as an OSX lib file in pyglet.

add ability for experiments to get back messages that were sent to the hub, as a way to check message processing delay and other possible uses.

Get fix for REDm API.

the utility should report any public class methods that are in the implementation that are not in the base class, check that each method signature is correct, that any 'mandatory' methods have been over-ridden, check for commenting / documenting, etc.

make it more user friendly and os-independent.

Save ioHub Server ID to a local file so when a client starts the pid can be read and if it is still
running, it can be killed before starting another instance of the ioHub server. This would happen in the case of a application crash that did not allow the exception handler to terminate the server process properly.

Currently, paths to experiment resources and results are relative to the experiment directory. It would be nice if we could specify absolute paths if someone wanted to store data or pull resources from outside of the working directory.

Only for eye follower model.

Viewed after a calibration, optionally. Save to image and store with session data files as well?

Sometimes currently the ioHub Process does not close properly and shut down, leaving a nasty python process floating around, generally being rude. This normally only happens when the ioHub UDP service dies from an unhandled exception (bug), leaving the ioServer process stranded with no way to be told to shut down. (Other issues may cause this on the OSX implementation as well).

One thing to do to try and do is use weak references where possible, instead of relying on the current method of using a function / method call that breaks cyclical references by setting one side to None.

allows devices to 'start' and 'stop' event reporting.

Update the keyboard event with the following changes:

Event Field changes:

1. Remove 'ascii' field
2. Add ucode field, which should hold the ord(uchar) value for a unicode character (uchar)
3. Change key field to represent the unicode char for key press, when a unicode mapping can be made. Otherwise use the standard internal lookup tables for the OS to map the event to a key label.

** Notes, internally, the unicode key field will be converted to a utf-8 encoded 8-bit char string (i.e. unikey.encode('utf-8') ). When the field is saved to the DataStore or when it is transferred over UDP between processes, it will be in this utf-8 encoded string format. When the experiment process gets the key events, as part of the existing conversion from event lists to event named tuples, the 'key' field will be converted from the utf-8 encoded string to its unicode value using unikey=key.decode('utf-8')

currently, when the ioClient starts up, it also starts the ioServer using the subprocess module. Right now, the client just sleep 5 seconds to give the server 'time' to start up. Obviously this is a hack waiting to fail.

Instead, when the server starts up, it should send a 'ready' message to the client, which the client is waiting to receive. If it does not receive such a packet within X seconds of starting the subprocess, the client can assume the server start failed, generate an exception, and exit.

Two additional features that might improve session parameter usability:

1. Input checking

• dtype (most important)
• list of possible options / satisfies some other criterion (significantly less important)
  If a value is passed incorrectly (passed string when wanted int, or passed string not in the specified list), a dialogue pops up to identify the error, and returns to the GUI for reentry. The "list of possible options" checking is best handled by using dropdown boxes (so it's impossible to have an incorrect entry), but there might be cases (?) where the dropdown box is not adequate.

2. Auto population of last params
   One small but handy feature would be to auto populate the session parameters not with the defaults, but with the values from the previous run. We do this in our lab by .pickle'ing a 'lastParams' dict from the psychopy gui after each participant. Might not be worth it with the .yaml format, but just throwing it out there.

Currently, a 'message' from a client to the ioHub, or the message 'reply', which given we are talking UDP, is itself just a 'message', can not be larger than 1 UDP packet in size. This ends up being about 1400 bytes. While this has not proven to be an issue yet (as long as the event buffer is cleared by the client before read request are made), it is bound to end up being one as more devices are added,

So we need to add support so that ioHub 'messages' between client and ioHub server can be > 1 UDP pack in size.

A little research is needed, as I am not sure it in UDP it can be as simple as :

Sending side:
a) checking number of bytes you are about to stuff into the UDP send. If > max packet size - header size, only send max payload size.
b) redo step a until playload is fully sent.
c) in last packet, send '\r\n' indicating end of message stream

Receiving Side:

a) Start getting packets from a client on a given IP.port
b) buffer packet contents into a dict with ip.port as key until a packet has a '\r\n' as the last 2 bytes of content.
c) when b) occurs, pass message content onto message handler. delete entry for ip:port in Rx dict.

If it can be done like that, it is easy and adds no real overhead.

Other thing to note is msgpack supports streaming unpacker, so each packets payload could be fed into unpacker as it is received instead of waiting until all packets arrive.

This all assumes the UDP packets arrive 'in order'.

Suggestion from Jon:

" Setuptools (aka easy_install) is the way to go. It isn't too hard and you can create 'eggs' for each target platform (which can contain compiled binary code). If you manage to avoid any compiled code then you can distribute just the zip version, which will then be installed by either setuptools or pip. But I'd use setuptools to create the distributions since it keeps your options open."

This could be done instead of the custom python distro idea.

Current defaults for the session parameters are:

• session name (does not need to be unique; saved in the session variables)
• session code (must be a unique code; generates a session folder of the same name (if specified))
• (user defined params) (with syntax for check boxes and drop down boxes)

There may be a bit of redundancy here, as each new run of the experiment requires a unique code, but there is no way to unite two subsequent runs of the same participant. A parsimonious solution, at least in my lab, would be to include the following:

• session id (using new terminology just to distinguish it from name / code above)
• (user defined params) (...)

where "session id" was a string identifier of an individual participant. Instead of preventing duplicate session codes as above, a duplicate session id should pop up a simple dialogue identifying that the session id has already been used, and asking to continue or cancel/return to the GUI to enter a new session id.

Purposefully duplicating a session id could be used if something happens halfway through data collection and the experiment must be exited and restarted, or if an experiment is run at two different time points. The point would be to allow a participant identifier (session id) to link the two sets of data, and some internally generated index, to separate the runs.

Example ioDataStore representation:
sessionID run data cols
SUBJ1 1 1.234...
SUBJ1 1 3.231...
SUBJ1 2 2.342... # data is linked by sessionID, but separated by run
SUBJ1 2 4.239...

The goal is largely to prevent more session id's (or session codes, from before) from being generated than are actually needed to represent the "number" of people. I don't think the run param needs to be handled by the user. In the case of running the same study at two time points, an additional userParam could be specified:

sessionID run userParam1 data cols
SUBJ1 1 test1 1.234...
SUBJ1 1 test1 3.231...
...
SUBJ2 2 test1 5.241...
SUBJ2 2 test1 6.234...
...
SUBJ1 3 test2 2.342...
SUBJ1 3 test2 4.239...

Again, these comments are biased by my own research approach so I'm sure there are many alternatives, but I imagine some others may benefit from this technique as well.

Will give physical settings and calibration settings

some of the eye event types were failing conversion to numpy ndarray definitions at script startup. Most were *EndEvent types. For now I have just commented them out. They need to be looked at and fixed. See eye_event.py in the eyetrackerinterface folder.

Much more user friendly and no significant effect on time before rollover.

Test that interface works in a dual computer setup.

Driven from iohub_config.yaml

The python-xlib package is not ideal in that it does not expose the full API and is not actively maintained. Due to lack of some APIs, doing the unicode keyboard char mapping is not ideal at all. Suggestion it to, when possible, switch to using a ctypes wrapper of the necessary Xlib API areas. I am already doing this to be able to more effectively hide / show the mouse cursor by using the xfixes xlib extension and also to be able to move the muse from one screen index to a location on a different screen.

I have noticed that master Alex Holkner has done an autogenerated xlib wrapper that includes most of what would be needed. I wonder if that xlib.py file could be borrowed and used, adding any missing wrappings needed. Would make the transition faster for sure.

EVENT_TRIGGER =35,
FUNCTION_START =30,
FUNCTION_END =31,
SEQUENCE_START =10,
SEQUENCE_END =11,
EXPERIMENT_START =12,
EXPERIMENT_END =13,
BLOCK_START =14,
BLOCK_END =15,
TRIAL_START =16,
TRIAL_END =17,
EXPERIMENT_LOG =20,
EXPERIMENT_DV =21,
EXPERIMENT_IV =22,
DRAW_START =23,
DRAW_END =24,
SWAP_START =25,
SWAP_END =26,
VBLANK =27, DISPLAY_START =28,
DISPLAY_END =29,
ANALOG_INPUT_SAMPLE =73,
ANALOG_OUTPUT_SAMPLE =74,
ANALOG_INPUT_EVENT =75,
ANALOG_OUTPUT_EVENT =76,
INTEREST_AREA_ACTIVE =90,
INTEREST_AREA_INACTIVE =91,
INTEREST_AREA_POSITION_CHANGE =92
INTEREST_AREA_SHAPE_CHANGE =93
INTEREST_AREA_SHAPE_POSITION_CHANGE =94,
DATA_MISSING =200
DATA_MISSING_START =201
DATA_MISSING_END =202,
DATA_GAP =205,
DATA_GAP_START =206,
DATA_GAP_END =207,
IP_DATA_INPUT =79,
IP_DATA_OUTPUT =80,
SERIAL_DATA_INPUT =77,
SERIAL_DATA_OUTPUT =78,
TTL_INPUT =70,
TTL_OUPUT =71,
PARALLEL_PORT_OUPUT =74,
AUDIO_OUT_START =40,
AUDIO_OUT_END =41,
VOICE_KEY =42,
AUDIO_IN_START =43,
AUDIO_IN_END =44,

The client can the send the offset to the ioHub process and the ioHub process can then use the 'same ' offset as the psychopy process. So wrapping of time field will be 'very' long.

When ever the PsychoPy process changes the clock timetime, it should automatically get sent to the ioHub.

Currently, an ever growing list of configuration settings related to the ioHub and devices are added to a dictionary (see the emrt/examples/psychoTest.py). These should move to a yaml file for better maintainability and separation from the python code.

This would also be how new devices, no part of the python module, could be added by a user.

Test interface in a 2 PC configuration.

Py hook could be replaced quite easily with a small C DLL proving the LL hook bridge to python callbacks. Rest of functionality could be ctypes. The DzlL for the actual hook init. And callback is because the LL hooks must point to functions in a DLL, since they are injected into the event processing pipeline. The C callbacks could inturn call python callbacks though. Or, event queuing could be done in the C DLL and timed polling of an event queue could be done, since we are using a timed poll to pump the msg que anyhow for window events.

The would remove py hook as a dependency, and address any possible unicode issues that may exist with pyhook.

Both in terms of mapping to modifier values and key field when modifier is pressed. Seems to be influenced by caplocks key state; different keysyms are being given based on caplocks state.

This was causing random python int overflow errors due to the use of bisect sort, apparently a known issue in python 2.x. So for now, I have disabled sorting of the global event list so it doe not crash at least.

Will look into alternative options or ways to work around issue so sorting can be re-enabled.

The first run of a new experiment (i.e., no PATHS have been saved) first initializes a .hdf5 in the current working directory, then makes an additional .hdf5 after the PATHs are generated in the specified directory.

Review with Markus.

When an OS generates auto repeat events for keys, ioHub should filter these out and only send key events that match physical key actions.

Since msgpack is being used to encode and decode data sent between processes (it is as fast as ujson, but compresses the messages by about 50% the original size on average), then there is no real need to also use ujson.

ujson right now is being used in a couple places to encode and decode python dictionaries for storage in the datastore file because it is fast and the output is still human readable.

We can use msgpack for the same purpose, however the string output will no longer be human readable if viewed prior to using msgpack to decode it again. This is fine, the API just needs to do the decoding first, which it is already doing but with ujson.

Implement support for saving native EyeGaze files.

You can currently only use pixel coordinates in your scripts. Visual degrees, etc, are broken at the moment, sorry! Will be fixing as a top priority.