Typing strings and using modifiers doesn't work on Linux / my system about autopy2 HOT 8 OPEN

Comment by msanders
Sunday Jan 17, 2010 at 01:25 GMT

bolus,

A few things.

First, why is it that you are using the number 65289 for tab instead of the character '\t' (which should work)?

Second, key.tap() and key.toggle() do not take an ASCII code as its parameter (you don't need ord()). They take either a char, or a keycode as denoted by one of the key.K_* constants (which should NOT be hard-coded in because they differ across platforms).

Also, with regards to modifiers, note that you have to use a key.MOD_ constant, not a key.K_ constants, e.g.: key.tap('\t', key.MOD_ALT).

I'm using a german keyboard layout / os, could this be a problem?

Although I have not tested it on that layout, it should not be. On X11, all character->keycode conversion is done with the standard function XStringToKeysym(), so unless there's a bug with that function, it should work fine.

However, one thing you have pointed out that does not work is special keys. @, for instance, on a US keyboard, is not its own key but the combination of 2 and shift. I have not yet figured out a portable way to fix this (any suggestions would be very much appreciated!).� Currently the only workaround is to manually add the modifiers, for instance key.tap(2, key.MOD_SHIFT), but that is obviously not ideal.

from autopy2.

Comment by bolus
Sunday Jan 17, 2010 at 02:44 GMT

Thanks for the quick reply.

First, why is it that you are using the number 65289 for tab instead of the character '\t' (which should work)?

"\t" doesn't work on my system but 65289 works.

Second, key.tap() and key.toggle() do not take an ASCII code as its parameter (you don't need ord()). They take either a char, or a keycode as denoted by one of the key.K_* constants (which should NOT be hard-coded in because they differ across platforms).

I've tested it before without the ord( ) (=> for char in string: k.tap( char )), but I got the same result as with type_string (half of the chars are ignored). With ord( ), on the other hand, for every char one char gets typed (but sometimes not the right char)

Also, with regards to modifiers, note that you have to use a key.MOD_ constant, not a key.K_ constants, e.g.: key.tap('\t', key.MOD_ALT).

I have used a key.MOD_ constant => k.tap( TAB, k.MOD_ALT )
It failed with "Segmentation fault"

I'm using a german keyboard layout / os, could this be a problem?

Although I have not tested it on that layout, it should not be. On X11, all character->keycode conversion is done with the standard function XStringToKeysym(), so unless there's a bug with that function, it should work fine.

I've tested XStringToKeysym with the help of ctypes. I do not know why, but special chars gets converted to 0.

Here is my ipython session:

In [3]: from ctypes import *
In [4]: f = CDLL( "libX11.so" )
In [5]: f.XStringToKeysym.argtypes = [c_char_p]
In [6]: f.XStringToKeysym.restype  = c_int
In [7]: f.XStringToKeysym("a")
Out[7]: 97
In [8]: f.XStringToKeysym("/") ## should not be 0
Out[8]: 0
In [9]: f.XStringToKeysym("b")
Out[9]: 98
In [10]: f.XStringToKeysym("ß") ## should not be 0
Out[10]: 0
In [11]: f.XStringToKeysym(u"ß")
---------------------------------------------------------------------------
ArgumentError                             Traceback (most recent call last)

/home/dah/<ipython console> in <module>()

ArgumentError: argument 1: <type 'exceptions.UnicodeEncodeError'>: 'ascii' codec can't encode characters in position 0-1: ordinal not in range(128)

That have to be the problem, right? But why doesn't XStringToKeysym work proper. Can you test the command above with an other xorg system?

from autopy2.

Comment by bolus
Wednesday Jan 20, 2010 at 20:03 GMT

Hi,

I've installed the latest Ubuntu version. Unfortunately nothing changed.
But I've found an other tool for gui automation on linux.
The name is autokey (code.google.com/p/autokey/).
It works like a charm with autokey. Maybe you can use it to debug this issue.

Best regards,
Daniel

By the way: I should have translate my chosen user name from English to German.
Big pill?! That sounds like I'm a Ecstasy junkie.

from autopy2.

Comment by grandfather13
Thursday Dec 09, 2010 at 16:41 GMT

I had the same problem under Debian 5.0 and Ubuntu 10.04 with a US keyboard setup. The XStringToKeysym function seems to give NoSymbol responses for any non-alphanumeric character.

I don't know how universal this behavior is for XStringToKeysym, but I've put together a quick fix that works for me. The modifications are in keycode.c I've added a manual lookup for non alphanumeric keys in the case XStringToKeysym returns NoSymbol. I've attached the code in case it is useful for inclusion or for anyone with a similar problem.

Thanks so much msanders for putting together such a clean and useful tool!

modified keycode.c with changes on lines 16-39 and 89-103:

#include "keycode.h"

#if defined(IS_MACOSX)

#include <CoreFoundation/CoreFoundation.h>
#include <Carbon/Carbon.h> /* For kVK_ constants, and TIS functions. */
#define CFStringEqual(s1, s2) (CFStringCompare(s1, s2, 0) == kCFCompareEqualTo)

/* Returns string representation of key, if it is printable.
 * Ownership follows the Create Rule; that is, it is the caller's
 * responsibility to release the returned object. */
CFStringRef createStringForKey(CGKeyCode keyCode);

#endif

/* START CHANGES: Struct to store key mappings not handled by XStringToKeysym 
 * on some Linux systems. */
#if defined(USE_X11)
struct NameSym{
  char Name;
  MMKeyCode Sym;
};

struct NameSym NameSymTable[35]= { /* {Name, Sym}, */
  {'~', XK_asciitilde},     {'_', XK_underscore},
  {'[', XK_bracketleft},        {']', XK_bracketright},     {'!', XK_exclam},
  {'\'', XK_quotedbl},      {'#', XK_numbersign},       {'$', XK_dollar},
  {'%', XK_percent},            {'&', XK_ampersand},        {'\'', XK_quoteright},
  {'*', XK_asterisk},           {'+', XK_plus},             {',', XK_comma},
  {'-', XK_minus},          {'.', XK_period},           {'?', XK_question},
  {'<', XK_less},               {'>', XK_greater},          {'=', XK_equal},
  {'@', XK_at},             {':', XK_colon},            {';', XK_semicolon},
  {'\\', XK_backslash},         {'`', XK_grave},            {'{', XK_braceleft},
  {'}', XK_braceright},     {'|', XK_bar},              {'^', XK_asciicircum},
  {'(', XK_parenleft},      {')', XK_parenright},       {' ', XK_space},
  {'/', XK_slash},          {'\t', XK_Tab},             {'\n', XK_Return},
};
#endif
/* END CHANGES */



MMKeyCode keyCodeForChar(const char c)
{
#if defined(IS_MACOSX)
  /* OS X does not appear to have a built-in function for this, so instead we
   * have to write our own. */
  static CFMutableDictionaryRef charToCodeDict = NULL;
  CGKeyCode code;
  UniChar character = c;
  CFStringRef charStr = NULL;

  /* Generate table of keycodes and characters. */
  if (charToCodeDict == NULL) {
    size_t i;
    charToCodeDict = CFDictionaryCreateMutable(kCFAllocatorDefault,
                                               128,
                                               &kCFCopyStringDictionaryKeyCallBacks,
                                               NULL);
    if (charToCodeDict == NULL) return UINT16_MAX;

    /* Loop through every keycode (0 - 127) to find its current mapping. */
    for (i = 0; i < 128; ++i) {
      CFStringRef string = createStringForKey((CGKeyCode)i);
      if (string != NULL) {
        CFDictionaryAddValue(charToCodeDict, string, (const void *)i);
        CFRelease(string);
      }
    }
  }

  charStr = CFStringCreateWithCharacters(kCFAllocatorDefault, &character, 1);

  /* Our values may be NULL (0), so we need to use this function. */
  if (!CFDictionaryGetValueIfPresent(charToCodeDict, charStr,
                                     (const void **)&code)) {
    code = UINT16_MAX; /* Error */
  }

  CFRelease(charStr);
  return (MMKeyCode)code;
#elif defined(IS_WINDOWS)
  return VkKeyScan(c);
#elif defined(USE_X11)
  char buf[2];
  buf[0] = c;
  buf[1] = '\0';

  /* START CHANGES: Updates to handle XStringToKeysym holes */
  MMKeyCode code =  XStringToKeysym(buf);
  if (code == NoSymbol){
    /* Only handle NoSymbol cases */
    size_t i;
    for ( i = 0; i < 83; ++i) {
      if (c==NameSymTable[i].Name) {
        /* Found It */
        code = NameSymTable[i].Sym;
        return code;
      }
    }
  }
  return code;
  /* END CHANGES */

#endif
}

#if defined(IS_MACOSX)

CFStringRef createStringForKey(CGKeyCode keyCode)
{
  TISInputSourceRef currentKeyboard = TISCopyCurrentKeyboardInputSource();
  CFDataRef layoutData =
    TISGetInputSourceProperty(currentKeyboard,
                              kTISPropertyUnicodeKeyLayoutData);
  const UCKeyboardLayout *keyboardLayout =
    (const UCKeyboardLayout *)CFDataGetBytePtr(layoutData);

  UInt32 keysDown = 0;
  UniChar chars[4];
  UniCharCount realLength;

  UCKeyTranslate(keyboardLayout,
                 keyCode,
                 kUCKeyActionDisplay,
                 0,
                 LMGetKbdType(),
                 kUCKeyTranslateNoDeadKeysBit,
                 &keysDown,
                 sizeof(chars) / sizeof(chars[0]),
                 &realLength,
                 chars);
  CFRelease(currentKeyboard);

  return CFStringCreateWithCharacters(kCFAllocatorDefault, chars, 1);
}

#endif

from autopy2.

Comment by frafra
Sunday Feb 27, 2011 at 23:56 GMT

This patch work for me (RedHat Enterprise Linux 6 x86_64).
Please apply this patch to autopy.

from autopy2.

Comment by scottie
Wednesday May 04, 2011 at 12:47 GMT

I to was having the same problem on latest ubuntu build, the above patch half solved it.

After i apply patch, it will now send a .(dot) to the screen, but still not @ instead of @ it will send a 2(2 with patch, before just nothing).

Also on the same note, when i use autopy.key.tap(2, autopy.key.MOD_SHIFT) i get "Segmentation fault" error.

from autopy2.

Comment by msanders
Thursday May 19, 2011 at 04:24 GMT

Thank you for the patch @grandfather13, I've now applied it (albeit with some slight style changes to fit with the rest of the autopy source code). It seems the original issue is still not completely fixed though, as the test code still does not produce the intended result for me (or @scottie apparently), so I'm still looking into it. I'll look into autokey as well, thanks for pointing that out to me @Bolus.

The segmentation fault seems to be a different issue entirely (on my X11 setup, using any modifier with key.tap() appears to cause it). Still trying to figure out what the deal with that is.

from autopy2.

Comment by msanders
Thursday May 19, 2011 at 04:36 GMT

Got the segmentation fault issue dealt with. Apparently it was just a simple overflow error in the argument's format string.

from autopy2.