Platform- MAC
OS- MAC OS high Sierra

Problem-
I am able to run sikuli script locally on my machine as we jenkins local machine, but if i trigger build to run automation on jenkins, then it got fail. I am performing file upload on mac.

the setup log like this:
[... some stuff ...]
[debug (18-6-28 16:22:27)] RunSetup: Downloads for selected options:
sikulixlibswin-1.1.1.jar sikulixsetupIDE-1.1.2-20180308.083933-100-forsetup.jar sikulixsetupAPI-1.1.2-20180308.083909-100-forsetup.jar jython-standalone-2.7.1.jar
[debug (18-6-28 16:22:27)] RunSetup: adding needed stuff to sikulix.jar
[debug (18-6-28 16:22:27)] FileManager: buildJar: sikulixtemp.jar
[debug (18-6-28 16:22:27)] FileManager: buildJar: adding: sikulix.jar
[debug (18-6-28 16:22:33)] FileManager: buildJar: adding: sikulixjython.jar
[error (18-6-28 16:22:40)] FileManager: buildJar: java.io.EOFException: Unexpected end of ZLIB input stream
[debug (18-6-28 16:22:40)] RunSetup: renaming sikulixtemp.jar to target jar: sikulix.jar
[debug (18-6-28 16:22:40)] RunSetup: rename did not work --- trying copy
[debug (18-6-28 16:22:40)] FileManager: deleteFileOrFolder:
C:\Users\xietian\Desktop\skuli\sikulixtemp.jar
[error (18-6-28 16:22:40)] RunSetup: did not work
[error (18-6-28 16:22:40)] RunSetup:
[error (18-6-28 16:22:40)] RunSetup: ... terminated abnormally :-(
[debug (18-6-28 16:22:40)] RunSetup:
popError: --------------------
Something serious happened! Sikuli not useable!
Check the error log at C:\Users\xietian\Desktop\skuli\SikuliX-1.1.2-SetupLog.txt
[debug (18-6-28 16:22:44)] RunTimeSETUP: final cleanup
[debug (18-6-28 16:22:44)] FileManager: deleteFileOrFolder:
C:\Users\xietian\AppData\Local\Temp\SikuliX-1.1.2-SetupLog.000
[debug (18-6-28 16:22:44)] FileManager: deleteFileOrFolder:
C:\Users\xietian\AppData\Local\Temp\Sikulix_601888068

I succeeded on installing sikulix on windows 7, I run a small script with :

while exists(REGION):

where REGION was a GUI generated region, the program started but had a fatal error soon:

[error (06/06/14 9.51.45)] Region: doFind: invalid parameter: R[315,731 42x21]@s(0)[0,0 1366x768] E:Y, T:3,0
[error (06/06/14 9.51.45)] Terminating SikuliX after a fatal error(1)! Sorry, but it makes no sense to continue!

I am using this code to verify that the package is working. I am using Mac OS

import os
import sikuli
dm = sikuli.DesktopMouse()
dir(dm)
#['__class__', '__cls_storage', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', u'__javaclass__', u'__javaconstructor__', '__module__', '__new__', '__pyx_vtable__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'click', 'doubleClick', 'drag', 'drop', 'equals', 'getClass', 'getLocation', 'hashCode', 'location', 'mouseDown', 'mouseUp', 'notify', 'notifyAll', 'rightClick', 'toString', 'wait', 'wheel']
x = dm.getLocation()

When I try this code, I can see it is trying to load some java lib as shown and gets stuck there. I didn't get any error to know what the package is trying to load. Any help to troubleshoot this would be greatly appreciated.

Here you say

When using the installer version (.exe), the system path entries (path-to-Sikuli\libs and path-to-java\jre6\bin) and the environment variable %SIKULI_HOME% pointing to the folder where Sikuli is installed are setup by the installer.
When using the zip download, you have to do it yourself.
%SIKULI_HOME% is needed for some special cases, when running scripts from command line.

Am I correct thinking that we have to add 2 paths to the environment PATH?

1. <path-to-sikuli>\libs
2. <path-to-java>\jre6\bin

And add one custom environment variable

1. SIKULI_HOME = <path-to-sikuli>

getting this error when i try to select folder using sikuli.
Error: Error running instance method
java.lang.NullPointerException
at org.sikuli.script.Screen.capture(Screen.java:587)
at org.sikuli.script.Screen.capture(Screen.java:607)
at org.sikuli.script.Region.doCheckLastSeenAndCreateFinder(Region.java:3001)
at org.sikuli.script.Region.checkLastSeenAndCreateFinder(Region.java:2996)
at org.sikuli.script.Region.doFind(Region.java:2945)
at org.sikuli.script.Region.access$500(Region.java:21)
at org.sikuli.script.Region$RepeatableFind.run(Region.java:3169)
at org.sikuli.script.Region$Repeatable.repeat(Region.java:3125)
at org.sikuli.script.Region.wait(Region.java:2756)
at org.sikuli.script.Region.find(Region.java:2329)
at org.sikuli.script.Region.getLocationFromTarget(Region.java:3238)
at org.sikuli.script.Region.click(Region.java:3736)
at org.sikuli.script.Region.click(Region.java:3721)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)

Hi all

I am new to sikuli.

this tool very beautiful to test the gui
i don't know the maven structure for sikuli
if any one knows show the sample structure and pom

Thanks in advance
Chelladurai.A

In GUI mode the run(command) should show the console window.

In CLI mode the run(command) could optionally show the console window.

Either way, it should check the exit code and either return it or fail.

Without such improvement, it's just like submodule.call()

Chocolatey is a package manager for Windows. I've created a package that installs the portable Sikuli and puts an environment (x86/x64) specific link on the Desktop. I'd be happy to provide the spec/script file to your repo (so your team can own it).

It would be cool to have this tied into whatever build/publish system you've got going on in Hudson. But, basically, you just increment the version number in the nuspec file. Update the static zip path in the install script. And repackage and publish to http://chocolatey.org.

I can share information, helper scripts, etc with you.

I am able run sikuli scripts on iPhone simulator but not able to execute on iPhone real device...Is it possible with iPhone real device ?

Thanks.

Rationale:
By convention non-zero exit codes indicate failure of terminating programs (Wikipedia: Exit status). This is generally done on all systems, but especially important on UNIX systems using a lot of shell scripting. When automating tasks, instead of parsing stdout and stderr, exit codes often give sufficient information for the system administrator ("did my intent work or not?!").

Now let's consider sikuli.

1. Consider the following sikuli script:
   click("does_not_exist.png")

   Now let's run it:

   % sikuli exitcode.sikuli/
   [info] Sikuli vision engine loaded.
   [info] VDictProxy loaded.
   [error] does_not_exist.png looks like a file, but can't be found on the disk. Assume it's text.
   [info] Text Recognizer inited.
   
   Error: Illegal min or max specification!
   "Fatal error encountered!" == NULL:Error:Assert failed:in file globaloc.cpp, line 75
   #
   # A fatal error has been detected by the Java Runtime Environment:
   #
   #  SIGSEGV (0xb) at pc=0x00007f6259fc465a, pid=16138, tid=140061131474688
   #
   # JRE version: OpenJDK Runtime Environment (7.0_101) (build 1.7.0_101-b00)
   # Java VM: OpenJDK 64-Bit Server VM (24.95-b01 mixed mode linux-amd64 compressed oops)
   # Derivative: IcedTea 2.6.6
   # Distribution: Debian GNU/Linux 8.2 (jessie), package 7u101-2.6.6-1~deb8u1
   # Problematic frame:
   # C  [libtesseract.so.3+0x24465a]  ERRCODE::error(char const*, TessErrorLogCode, char const*, ...) const+0x13a 
   #
   # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
   #
   # An error report file with more information is saved as:
   # /.../poc-tests/hs_err_pid16138.log
   #
   # If you would like to submit a bug report, please include
   # instructions on how to reproduce the bug and visit:
   #   http://icedtea.classpath.org/bugzilla
   # The crash happened outside the Java Virtual Machine in native code.
   # See problematic frame for where to report the bug.
   #
   [1]    16138 abort      sikuli exitcode.sikuli/
   sikuli exitcode.sikuli/  11,29s user 1,04s system 97% cpu 12,677 total
   

   So if the PNG file does not exist in the file system, it starts tesseract and somehow an assertion in C is thrown. What about the exit code? 134. So this test case is fine.

2. Now consider a case where the PNG file exists:

   click("exists_but_cannot_find.png")
   

   What about stdout and stderr?

   % sikuli exitcode.sikuli/
   [info] Sikuli vision engine loaded.
   [info] VDictProxy loaded.
   [error] Can't run this Sikuli script: exitcode.sikuli/
   Traceback (most recent call last):
    File "/.../poc-tests/exitcode.sikuli/exitcode.py", line 1, in <module>
      click("exists_but_cannot_find.png")
   Line 2, in file /.../poc-tests/exitcode.sikuli/exitcode.py
   
      at org.sikuli.script.Region.handleFindFailed(Region.java:421)
      at org.sikuli.script.Region.wait(Region.java:512)
      at org.python.proxies.sikuli.Region$Region$0.super__wait(Unknown Source)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:606)
   
   org.sikuli.script.FindFailed: FindFailed: can not find exists_but_cannot_find.png on the screen.
   Line 2, in file /.../poc-tests/exitcode.sikuli/exitcode.py
   
   sikuli exitcode.sikuli/  13,15s user 0,95s system 94% cpu 14,992 total
   

   What about the exit code? 0. Even though user's initial intent verify correctness of my application could not be met.

3. Now let a PNG file exist and make it findable on the GUI. Intentionally exit with exit code 3.

   click("exists_and_found.png")
   exit(3)
   

   And let's see:

   % sikuli exitcode.sikuli/
   [info] Sikuli vision engine loaded.
   [info] VDictProxy loaded.
   [log] CLICK on (135,64)
   [error] Can't run this Sikuli script: exitcode.sikuli/
   sikuli exitcode.sikuli/  11,18s user 0,70s system 93% cpu 12,743 total
   

   What about the exit code? 3. If you remove exit(3) from the code, it is zero and if you add exit(3), it is 3. Seems legit.

4. And finally consider something like a wrong CLI parameter:

   % sikuli --wrongparameter exitcode.sikuli/
   [error] Can't run this Sikuli script: --wrongparameter
   java.lang.StringIndexOutOfBoundsException: String index out of range: -1
      at java.lang.String.substring(String.java:1911)
      at org.sikuli.script.ScriptRunner.getPyFrom(ScriptRunner.java:109)
      at org.sikuli.script.ScriptRunner.runPython(ScriptRunner.java:103)
      at org.sikuli.script.SikuliScript.main(SikuliScript.java:111)
   [info] Sikuli vision engine loaded.
   ...
   
   sikuli --wrongparameter exitcode.sikuli/  13,09s user 1,26s system 95% cpu 15,095 total
   

   What about the exit code? It is 0.

My point of this issue: Please fix cases 2 and 4 to return non-zero exit codes in case of failure.

Design: Be aware that AFAICS exit codes of exit(...) are passed over to Java and System.Exit is called correspondingly. Hence exit codes might collide (i.e. 1 means test case failed as well as exit(1) was called). Nevertheless, this issue should be resolved by convention. For example all sikuli scripts shall not exit with exit code 255, because it is reserved for failing testcases. You might want to consider this backwards-incompatible, but sikuli goes against conventions established by any other established test framework.

Code: If think most of this fix needs to be done in SikuliScript. For cases 2 and 4 System.exit(1) suffices in case of an exception.

Btw, I think message Can't run this Sikuli script for a script exiting with exit(N) where N!=0 is very non-intuitive, because the script was actually run.

The application I am testing has steps where sometimes the next step response time can be quick but sometimes it can take longer.

"wait(target, timeout)" is really useful as I can request to wait until target appears or until the maximum timeout supplied.

But I have observed that this function does not use the timeout supplied, instead waits until the auto wait timeout. I have actually verified this:

before setting auto wait timeout:-
screen.wait(DATAMART_CREDENTIALS_LABEL, DB_CONNECTION_WAIT_TIME);
FindFailed error occurs. As it waits for the default auto wait not the timeout supplied.

after setting auto wait timeout:-
screen.setAutoWaitTimeout(DB_CONNECTION_WAIT_TIME);
screen.wait(DATAMART_CREDENTIALS_LABEL, DB_CONNECTION_WAIT_TIME);
Waits until the image is found as the auto wait timeout is longer than the default and hence runs successfully.

Can I request the team to kindly fix this issue as It would really be helpful if this function actually works.

I think when installed on Windows 7 overwrites the User PATH variable rather than append to it

**** Known Problem (https://answers.launchpad.net/sikuli/+faq/1766)

**** Pls. report issues, bugs and questions at https://launchpad.net/sikuli

Mac OS X 10.6, Snow Leopard.

If you close the application with a .sikuli file already open then try to launch Sikuli IDE again, you will get an error dialog:

Cannot launch Java application
Uncaught exception in main method: java.lang.NullPointerException

The application will not quit normally and must be killed with the Force Quit command.

Sikuli IDE can only be started again by overwriting the IDE_SESSION key from the preferences file before launching.

defaults write org.sikuli.ide '/org/sikuli/ide/' -dict-add "IDE_SESSION" -string ';'

When I run a test with Sikuli Screen class on a local machine or from a local machine into a remote server, the test runs without error, but when I run the same test from CI (TeamCity), I get this error:

Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M5:test (default-test) on project myProject: There are test failures.
[ERROR]
[ERROR] Please refer to /root/work/myProject/target/surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] There was an error in the forked process
[ERROR]
[ERROR] Cannot instantiate class com.bme.test.SikuliExampleTest
[ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: There was an error in the forked process
[ERROR]
[ERROR] Cannot instantiate class com.bme.test.SikuliExampleTest
[ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:733)
...

Other tests run from TeamCity also work without errors.
POM:

<plugin>
     <groupId>org.apache.maven.plugins</groupId>
     <artifactId>maven-surefire-plugin</artifactId>
     <version>3.0.0-M5</version>
 </plugin>

   <dependency>
        <groupId>com.sikulix</groupId>
        <artifactId>sikulixapi</artifactId>
        <version>2.0.4</version>
    </dependency>

It would be useful if sikuli would save the window position and shows all Windows in the screen where the ide is shown.

For example: The ide is on screen 2, but all other windows like the splash at start, settings, pattern settings will show up on the first (the main) screen.

App.close() seem to nicely ask, which is good, but it should then forcefully terminate/kill the process.

An optional argument, or a App.terminate() or App.kill() may be possibly if an automatic terminate/kill after 15 sec or so seems wrong.

Here you said

you should change the headline to "Sikuli X 1.0.3" and add a short comment, that the user knows, he gets what officially is called Sikuli X 1.0rc3 (because this is how we talk at the launchpad Q&A and bugs about it)

Is the project officially called "Sikuli X"? If so, I can change the Title of the package (that shows up in bold on the web gallery). And the package id "sikulix" is still available if you'd like to use that for new packages.

in file:
./src/main/java/org/sikuli/slides/api/actions/DropAction.java

line 21, it is:
mouse.drag(loc);

but it should be:
mouse.drop(loc);

thanks.

I know this is really pedantic but:

I noticed at 2 points in the Region class documentation that "it's" has been used to refer to something belonging to "it" where "its" should be used - "it's" is a contraction of "it is" - I would guess this is not the only part of the documentation where this mistake has been made; I would suggest getting someone to proofread it all before the next major release.

There's a small problem, NuGet, the underlying technology to Chocolatey, does not support the latest semver spec. Which would have us version the package like this

1.0.0-rc.3+build.905

However, NuGet will only support the old spec, with an arbitrary string appended

1.0.0-rc3build905

If you only publicly release incremented RC versions, then we can make it a little sexier

1.0.0-rc3

Hi. I have never used Sikuli before but am interested to know more. On your http://sikulix-2014.readthedocs.io/en/latest/index.html page, it says, "Python (language level 2.7) supported by the Jython interpreter." I am wondering if Sikuli is compatible with Python 3, specifically 3.5+. Thank you.

Hi,

I really enjoy using Sikuli.
But in my daily scripting there is some missing methods I think it can be useful to add.
Currently I monkey patch some python class (Region, ..) but I think it's a good idea to add these functionnalities in a next built-in Sikuli.

I'm not a python/JPython/Java expert, so I don't send you a bad pull request.

For my examples, some Regions and Location

r0 = Region(10, 10, 10, 10)
r1 = Region(15, 15, 2, 10)
l0 = Location(15, 25)

Here just some useful functions:
Region.union(Region)

# return a new region, new bounds includes r0 and r1 (x=10, y=10, w=10, h=15)
r0.union(r1)

Region.union(Location)

# return a new region, new bounds includes r0 and l0 (x=10, y=10, w=10, h=15)
r0.union(l0)

//

Region.intersects(Region)

# return a Boolean to say if r0 intersects r1 (True)
r0.intersects(r1)

//

Region.intersection(Region)

# return a new region, new bounds is the intersection of r0 and r1 (x=15, y=15, w=2, h=5)
r0.intersection(r1)

//

Region.contains(Region)

# return a Boolean to say if r1 is fully included in r0 (False)
r0.contains(r1)

Region.contains(Location)

# return a Boolean to say if r1.topLeft is included in r0 (True)
r0.contains(r1.topLeft)

//

Also right, left, above, below are useful but I think a optional parameter can be added to extends the current Region instead of always exclude it.

r0 = Region(10, 10, 10, 10)
# return a new region (20, 10, 100, 10)
r1 = r0.right(100) 

if I want include r0, I need to r0.union(r1). That makes 2 steps.
A better way is to have an option, setted by default to False to stay with the same behavior than in previous Sikuli versions

Region.right(range, union=False)

# return a new region (10, 10, 110, 10) So useful !
r1 = r0.right(100, True)

//

Also Region supports 1 or 4 arguments
Region(Region)
Region(x, y, w, h)

Perhaps it's a good idea to add this possibility:

# same of Region(Screen()), take the Screen 0 bounds if no argument setted
reg = Region()

What do you think ?

Thanks,
Jerome

Here you say

The cited text from the home page is a bit misunderstandable: What it wants to say is: on 64Bit Windows do not use the .exe (it simply does not work) but one of the .bat's instead.

So on an x64 environment, we should link to the sikuli-ide-w.bat file. And on an x86 environment, should I link to the same (to keep the package simple) or is there a reason to link right to the sikuli-ide.exe?

NMAKE : fatal error U1064: MAKEFILE not found and no target specified
Stop.

Chocolatey has been updated internally to the latest NuGet version, which supports the latest version specification. If this works, the "newest" Sikuli X package version (1.0.0-rc3) will actually be lower than the one I originally uploaded (1.0.3). I will contact the Chocolatey gallery admin and explain the circumstances.

Is it possible to turn a picture with text into text?

I would like to build Sikuli for win32 from the command line using a script that's included in the root of the repository. I see a build-win32.txt file and I started turning it into a batch script. But, I realized that your Hudson server must have a script or something on it for this already. Can you pull it out of there and check it in?

As i know sikulix 1.1.1 has been released. whether it is published to maven repo? Seems 1.1.1 version could not be found from maven central repo.

thank you very much

I made one...
Sikuli.zip

The Chocolatey package currently specifies a dependency on the Java Runtime (another package I created/uploaded to support this package). If you use the Chocolatey Install command, it will get installed too

cmd> chocolatey install sikuli

or

cmd> cinst sikuli

I can fix up the java runtime version spec to limit it to any version between 6.0 (inclusive) and 7.0 (exclusive).

<dependency id="javaruntime" version="[6.0,7.0)" />

It crash when the path has non-ascii characters.

sikuli jar version is 1.1.2

demo

ADBScreen adbs = ADBScreen.start();
System.out.println(adbs.getDeviceDescription());
try {
Match telephone = adbs.find("pictures/adb/telephone.png");
telephone.click();
try {
hread.sleep(2000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
adbs.aSwipeLeft();
} catch (FindFailed e) {
// TODO Auto-generated catch block
e.printStackTrace();
}

throw exception:

Exception in thread "main" java.lang.UnsatisfiedLinkError: org.opencv.core.Mat.n_Mat(III)J
at org.opencv.core.Mat.n_Mat(Native Method)
at org.opencv.core.Mat.(Mat.java:481)
at org.sikuli.android.ADBDevice.captureDeviceScreenMat(ADBDevice.java:207)
at org.sikuli.android.ADBDevice.captureDeviceScreen(ADBDevice.java:145)
at org.sikuli.android.ADBDevice.captureScreen(ADBDevice.java:136)
at org.sikuli.android.ADBScreen.capture(ADBScreen.java:152)
at org.sikuli.android.ADBScreen.capture(ADBScreen.java:162)
at org.sikuli.script.Region.doCheckLastSeenAndCreateFinder(Region.java:3001)
at org.sikuli.script.Region.checkLastSeenAndCreateFinder(Region.java:2996)
at org.sikuli.script.Region.doFind(Region.java:2925)
at org.sikuli.script.Region.access$500(Region.java:21)
at org.sikuli.script.Region$RepeatableFind.run(Region.java:3169)
at org.sikuli.script.Region$Repeatable.repeat(Region.java:3125)
at org.sikuli.script.Region.wait(Region.java:2756)
at org.sikuli.script.Region.find(Region.java:2329)
at andriod.TestADBDevice.main(TestADBDevice.java:15)

There is the following code in region.rst:

def changed(event):
        print "something changed in ", event.region
        for ch in event.changes:
                ch.highlight() # highlight all changes
        sleep(1)
        for ch in event.changes:
                ch.highlight() # turn off the highlights
with selectRegion("select a region to observe") as r:
    # any change in r larger than 50 pixels would trigger the changed function
    onChange(50, changed)
    observe(background=True)

wait(30) # another way to observe for 30 seconds
r.stopObserver()

The usage of variable r looks weird: shall it be the following instead?

def changed(event):
        print "something changed in ", event.region
        for ch in event.changes:
                ch.highlight() # highlight all changes
        sleep(1)
        for ch in event.changes:
                ch.highlight() # turn off the highlights
with selectRegion("select a region to observe") as r:
    # any change in r larger than 50 pixels would trigger the changed function
    r.onChange(50, changed)
    r.observe(background=True)

    wait(30) # another way to observe for 30 seconds
    r.stopObserver()

We are planning to use in a large scale project. do you recommend using SikuliX or you have some thing else in your bucket which can help us.

Hi,Team,

I have been using Sikuli for my tests and i find it very useful. But i ran into a issue today.
I am running my test on a vnc session, and the test script exits due the following reason:
During my testing process, a button click action failed due to part of the dialog window gets black, which covers the "connect" button.
However, if I move the dialog this won't happen anymore, so I can still run my script just dragging and dropping my dialog before performing my actions on this dialog.

Have you ever met this kind of issue before? Can you give me some advice?Thank you for your time.

After a lot of work, I was finally able to set up Sikuli on my computer, I just had to run the Sikuli-IDE.sh script to run the ide. However nothing happens when I run this script. Please help.

Hi,

The java pattern.exact() suffers from a problem, it does not always match correctly the image at the screen.
The Sikuli IDE works normally with the same image.
I used similar(0.995f) to get the score and I noticed the match score returns numbers like 0.9999996423721313.
I tried other images.
I guess the problem is related with how python and java round the score number before checking the exists..

[]s, ricardo