mmarquee / ui-automation Goto Github PK

It would be great to attract more people on this wonderful project. ;)

Hello,

I am getting the following error during build (Windows 10 build 17134. I would appreciate any help!

testGetFullDescription(mmarquee.automation.AutomationElementTest) Time elapsed: 0.023 sec <<< FAILURE! java.lang.AssertionError: root:Not set at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at mmarquee.automation.AutomationElementTest.testGetFullDescription(AutomationElementTest.java:76) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)

Hi,
I was trying to automate but I am getting an error on the first line.
UIAutomation automation=UIAutomation.getInstance();

Following errors are coming:
java.lang.NoSuchMethodError: com.sun.jna.platform.win32.Guid$GUID.createFieldsOrder([Ljava/lang/String;)Ljava/util/List;
at com.sun.jna.platform.win32.Guid$GUID.(Guid.java:108)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at com.sun.proxy.$Proxy10.(Unknown Source)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:739)
at com.sun.jna.Native.loadLibrary(Native.java:415)
at com.sun.jna.platform.win32.Ole32.(Ole32.java:47)
at mmarquee.automation.Ole32Wrapper.createWrapper(Ole32Wrapper.java:65)
at mmarquee.automation.Ole32Wrapper.(Ole32Wrapper.java:58)
at mmarquee.automation.UIAutomation.(UIAutomation.java:86)
at mmarquee.automation.UIAutomation.getInstance(UIAutomation.java:147)

OS: Win7
UI-automation version is Snapshot version .0.5.1
JDK: 1.8.0_121

however its working fine on Win10.

Thanks

你好啊，我们这个框架不能测试QT应用窗体么？
我用框架调QT应用报这个：
java.io.IOException: Cannot run program "E:\测试Debug\Platform.exe": CreateProcess error=740, 请求的操作需要提升。

Hi,
I'm trying to use mmarquee/ui-automation for a winform application autotest, for the datagrid is an custom elemment, I want to get the item value "test123" to verify, could you give some help for this? thanks a lot.
catched the automation element by UISpy as below:
AutomationElement
General Accessibility
AccessKey: ""
AcceleratorKey: ""
IsKeyboardFocusable: "False"
LabeledBy: "(null)"
HelpText: ""

State
IsEnabled: "True"
HasKeyboardFocus: "False"

Identification
ClassName: ""
ControlType: "ControlType.Custom"
Culture: "(null)"
AutomationId: ""
LocalizedControlType: ""
Name: "name row 0"

Visibility
BoundingRectangle: "(356, 304, 226, 41)"
ClickablePoint: "(null)"
IsOffscreen: "False"

ControlPatterns
Invoke

Value
Value: "test123"
IsReadOnly: "True"

AutomationList list1 = (AutomationList) combo.getList();

Error: The method getList() from the type AutomationComboBox is deprecated

1. The tree structure for an element inside internet explorer does it denote the xpath of that element.?
2. Is there anyway i can fetch the xpath of an IE element.

Relevant Stacktrace:

mmarquee.automation.AutomationException: Error: 0x80004002
        at mmarquee.automation.BaseAutomation.getAutomationElementFromReference(BaseAutomation.java:66)
        at mmarquee.automation.UIAutomation.getElementFromHandle(UIAutomation.java:638)

Device is a Windows 7 (version 6.1) 64 bit machine running JDK 11.

Unfortunately I don't have access to the machine--just the crash reports. Works on a few dozen other machines just fine.

Javadoc goal failure.
Error in mmarquee.automation.controls.LegacyAccessible.setValueFromIAccessible(final String value)
Error: incorrect use of inline tag @value
Should be:
* @param value The string value
Instead of
* @value The string value

Two other warnings:
Add missing @param element in
mmarquee.automation.pattern.LegacyIAccessible.LegacyIAccessible(final AutomationElement element)

Remove <p> and </p> in
mmarquee.automation.uiautomation.IUIAutomationElement3.getCurrentIsPeripheral(Integer retVal)

I have a treeview like the next one and I need to select the bold one.
Is it possible or not?

• Item#1
  • SubItem#1
    • MyItem
  • SubItem#2
  • SubItem#3
• Item#2
• Item#3

I am not able to find a dependency fos this latest Stabel release anywhere. I found only 0.4.3.

Any tips on you guys are using this?

The function UIAutomation.getInstance() frozen my application.
No error, no return, just show it on console:

ago 11, 2016 12:31:12 PM mmarquee.automation.UIAutomation <init>
INFORMAÃıES: Initializing ActiveX
ago 11, 2016 12:31:12 PM mmarquee.automation.UIAutomation <init>
INFORMAÃıES: Creating IUIAutomation instance

I'm using a VM with Win10 to test my application.

maven仓库下载不到 0.5.0啊 只有0.4.3啊，而且包里的方法和仓库的方法不一致

Hello,

I am trying to use your example for Excel 2016 and I am not able to set cell value, though I can read it.

Code sample:

AutomationDataGridCell cell = ...
cell.select(); // Works
cell.setValue("5555"); // Doesn't work

Hi!, thank you so much for this library, but the 0.3.8 version is not in maven repository. Please can you add this version to maven repository? With the version 0.3.7, this example doesn't work:
AutomationDataGrid grid = window.getDataGrid(0);
AutomationDataGridItem item = grid.getCell(0,0);
String itemName = item.name();

This is because the AutomationDataGridItem class is not present in the 0.3.7 version.

Thank you.

Hi,
I'm working on Automation for a MFC application where I have to select a value in ComboBox which is included in a List Control with multiple Columns.

Sample application looks like in the attached image.

I'm using following code:
List myListControl = myWindow.getList("MyListName");
java.util.List myListItems = myListControl .getItems();

Above code is returning only items for First Column (i.e : 1,2,3, and 4) . I'm not able to access combo box present in Second Column.
Could you please let me know how can I do the same?

Thanks
Shashank

The Grid pattern no longer works as expected. Not sure why, but reverting back to a previous version, where the interface was implemented in the classes is still functional, so it is a code change, rather than a breakage in the underlying automation library.

mmarquee.automation.AutomationException: Error: 0x80020003

at mmarquee.automation.pattern.Value.setValue(Value.java:112)
at mmarquee.automation.controls.AutomationEditBox.setValue(AutomationEditBox.java:76)

@markhumphreysjhc @pbi-qfs @mmarquee

Hi guys,

I was able to launch a notepad application on Windows 10 using this API, but when I tried to launch a JNLP using same code (by replacing notepad.exe with path/url to JNLP file) I get below error. Thanks in advance.

Exception in thread "main" java.io.IOException: Cannot run program "C:\Users\adi\Downloads\Diva.jnlp" (in directory "C:\Users\adi\Downloads"): CreateProcess error=193, %1 is not a valid Win32 application
at java.lang.ProcessBuilder.start(Unknown Source)
at mmarquee.automation.utils.Utils.startProcessWithWorkingDirectory(Utils.java:135)
at mmarquee.automation.UIAutomation.launchWithDirectory(UIAutomation.java:162)
at mmarquee.automation.UIAutomation.launchWithWorkingDirectoryOrAttach(UIAutomation.java:215)
at com.mkyong.TestJnlp.main(TestJnlp.java:38)
Caused by: java.io.IOException: CreateProcess error=193, %1 is not a valid Win32 application
at java.lang.ProcessImpl.create(Native Method)
at java.lang.ProcessImpl.(Unknown Source)
at java.lang.ProcessImpl.start(Unknown Source)
... 5 more

Some other ui automation wrappers/frameworks have this feature.

• https://www.autoitscript.com/autoit3/docs/functions/WinSetTrans.htm
• http://pywinauto.github.io/docs/code/pywinauto.controls.HwndWrapper.html?highlight=settransparency#pywinauto.controls.HwndWrapper.HwndWrapper.SetTransparency

It would be interresting to have it in ui-automation.

I don't know if it could help you but this is how to do it in C#:

using System;
using System.Diagnostics;
using System.Threading;
using System.Runtime.InteropServices;

namespace Demo
{
    class Program
    {
        [DllImport("user32.dll")]
        static extern IntPtr FindWindow(string lpClassName, string lpWindowName);

        [DllImport("user32.dll")]
        static extern bool ShowWindow(IntPtr hWnd, int nCmdShow);

        const int SW_HIDE = 0;
        const int SW_SHOW = 5;

        public static void Main(string[] args)
        {
            ProcessStartInfo psi = new ProcessStartInfo();
            psi.FileName = @"C:\madVR\madHcCtrl.exe";
            psi.Arguments = "editLocalSettingsDontWait";
            Process.Start(psi);

            Thread.Sleep(1500);

            IntPtr hWnd = FindWindow("TFMadVRSettings", null);
            ShowWindow(hWnd, SW_HIDE);

            Console.Write("Press any key to continue...");
            Console.ReadKey(true);
        }
    }
}

I intended to run notepad.exe using this framework as a start, however I could not success. My final objective is putting a text into another window's textbox and hit ENTER on MS Windows.

My code is at https://www.pastefs.com/pid/4857

I get "Error: Could not find or load main class dx.uitest.App" in console.

I also get "Missing artifact net.java.dev.jna:jna-platform:jar:4.3.0-SNAPSHOT" in "Problems" tab of Eclipse.

I use Eclipse, M2Eclipse and openjdk 8 on Arch Linux.

I am new to Java and Maven.

Please could you guide me: What is wrong in my code/dependencies?

Good day,

The isOffscreen Value is True if you can the item see and it is False if you can not see it.
And The Value is True if the Item has not this variable.
In Inspector is it True if you can not the Item see and False if you can see it.

Have a nice day,
Sven

I noticed that whenever Chrome re-instantiates, the Chrome://accessbility page resets. This severely limits the interaction between the framework and Chrome, my current workaround is to use another browser such as Firefox.

Use case : Web application testing

Is there a suggested way of enabling this access permission in Chrome within the framework?

I have couple of editbox as a ribbon tab ( COM add-In) in my excel application. I am trying to navigate to that tab and trying to setvalue to the first edit box.
window.getControlByClassName("MyAddinTab", "NetUIRibbonTab").getElement().setFocus(); - This works and this opens up the editboxes in that tab.
window.getEditBox(1).getElement().setFocus(); - This works and set the focus on this edit box
window.getEditBox(1).setValue("Test"); - This seems to be not working. There is no automationException or PatternNotfound exception.
( I am also not able to manually enter anything on the editbox after i run my code. I had to manually click on someother menu and the go back to my add-in tab & then my editbox works. This is totally strange)
Any idea/help on how to fix this issue ? or any workarounds?

As it says above

It always returns root element - "Desktop".

The issue also occurs in demo project.

Hi there,

I recently downloaded the code from the repository "mmarquee/ui-automation" and recreated an Eclipse project. However, the method call to QueryInterface didn't seem to match and gave an error that "The method QueryInterface(Guid.REFIID, PointerByReference) in the type Unknown is not applicable for the arguments (Guid.GUID.ByValue, PointerByReference)".
I am using Neon.1a Release (4.6.1) and Java 1.8.
Can anyone give me some advice on how to solve this error?
Please find the screenshots attached.

Looks like the recent builder changes are defaulting the attached property to always be false (i.e. as below)

public AutomationApplication attach(final Process process)
            throws AutomationException {
        return new AutomationApplication(
                new ElementBuilder(rootElement).process(process).attached(false));
}

Great framework, is helping me a lot.

In need to right-click elements that are clickable. Exists a way currently, or there is an intention to do it?

For some reason the executorservice never gets shut down when this code is run in scala-cucumber, which makes test runs never complete.

I might have to remove this unless I can find a fix quickly

In UIAutomation.java. if the application is found, then the attached property is incorrectly set to false

As it says above

I'm struggling to find a way to send and read commands from terminal (CMD, PCOMM, etc).
The furthest I could get was getting the terminal window only, how can I send and read from there?

Thnaks

It was possible to open the firefox browser, but not to set the value of the adress line.
My fault or is there a problem with the setValue()-function?

AutomationEditBox editBox = window.getEditBox("Suche oder Adresse eingeben");
TimeUnit.SECONDS.sleep(2);
editBox.setValue("Suchbegriff");
System.out.println("Name: " + editBox.getValue());
AutomationButton automationButton = window.getButton("In der Adressleiste eingegebene Adresse laden");
automationButton.click();
 

Looks like there is a JNA version 4.5.0, so this project needs to use that version

Hello.

If I need implement something like this:
https://docs.microsoft.com/nl-nl/dotnet/framework/ui-automation/add-content-to-a-text-box-using-ui-automation

As I see I need method AutomationElement.TryGetCurrentPattern (with ValuePattern object) and method SendWait from System.Windows.Forms.SendKeys)

How can I do this?

When I use Inspect, I can see that the "Value.Value" under TitleBar of a Window is set to a text representing the text of the window. How do we get this with ui-automation? Not finding any corresponding getter to access this value.

Need to run some (or all) tests in travis-ci, currently these don't work as they need Windows. So either a way of running some tests, or a way of divorcing a Windows environment from the builds

Hello and greetings,

I am trying to start an application with multiple arguments, but it doesn't work as expected.
It looks all my arguments are put into a single String object like "--arguement1=value1 --argument2=value2", when my application starts, it use the whole String as a single argument but not two arguments.

Here is my code to start my application:
Application application =
new Application(
new ElementBuilder()
.automation(automation)
.applicationPath("PAHT\myapplication.exe").applicationArguments("--arg1=value1 --arg2=value2")

after I start my application, I tried to view it in Process Explorer from Microsoft, and I find it shows something like that my application.exe "--arg1=value1 --arg2=value2"

Is there any other ways to make this work?

Thanks

Apologies for posting here, my intent is to get the word out to those following this project. I tried sending a mail to the developers last week, but it looks like this project is not being actively maintained. But the admins here are welcome to delete this issue if needed.

I am the lead developer of Karate which is an open-source test-automation framework. I have a need for Windows automation, so very recently started exploring it. Some good progress was made, here is video proof: https://twitter.com/ptrthomas/status/1261183808985948160

The good thing about Karate is that it is a framework that already includes IDE support (including step-through debug), HTML reports and parallel execution. The Windows automation was achieved by the same JNA --> MS UIAutomationCore approach that this [ui-automation] project uses, but I think I was able to simplify the Java code a lot - which is good news for those who want to contribute and improve it. Here is the relevant code, and I think the reduction of code is at least 1:10 if not more: https://github.com/intuit/karate/tree/develop/karate-robot/src/main/java/com/intuit/karate/robot/win

Here's the draft documentation, but it will change a lot right now: https://github.com/intuit/karate/tree/develop/karate-robot

So if you like the sound of this, please get in touch and help test / contribute / improve. Thanks !

• Peter

Treewalker example doesn't seem to work as expected.

I have this code:

UIAutomation automation = UIAutomation.getInstance();
AutomationWindow window = automation.getDesktopWindow("ZAV - 1.0.05");
AutomationDocument ad = window.getDocument(0);
System.out.println(ad.getText());

But it throws this exception:

Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
	at java.util.ArrayList.rangeCheck(ArrayList.java:657)
	at java.util.ArrayList.get(ArrayList.java:433)
	at mmarquee.automation.controls.AutomationContainer.getElementByControlType(AutomationContainer.java:110)
	at mmarquee.automation.controls.AutomationContainer.getDocument(AutomationContainer.java:768)
	at cz.krasnyd.writer.Test.main(Test.java:20)

When I look to inspect.exe (screenshot below), I see there some Document elements and I want get their value. Other functionalies works well (e.g. function getButton) and for example this code:

List<AutomationBase> a = window.getChildren(true);
for(AutomationBase ab : a){
   if(ab.getClassName().equals("WindowsForms10.RichEdit20W.app.0.2004eee")) {
      ...
   }
}

...returns me some Document elements, but I cannot get their value. Don't you know where the problem might be or do I use the library wrong?

*some things are in my local language ("vlastní" = "Custom", "podokno" = "Pane", "skupina" = "Group")

Hi,

I started a few days ago to develop a framework for desktop applications using Java, TestNG and UI-automation. I managed to open the app and input data but i cannot get the name value of a label from a panel.

Panel Identification
    ClassName:	"testPanel"
    ControlType:	"ControlType.Pane"
    Culture:	"(null)"
    AutomationId:	"panel1"
    LocalizedControlType:	"pane"
    Name:	"testp"
    ProcessId:	"2134"
    RuntimeId:	"231133"
    IsPassword:	"False"
    IsControlElement:	"True"
    IsContentElement:	"True"


Label Identification
    ClassName:	"testLabel"
    ControlType:	"ControlType.Text"
    Culture:	"(null)"
    AutomationId:	"label8"
    LocalizedControlType:	"text"
    Name:	"abc"
    ProcessId:	"123"
    RuntimeId:	"123"
    IsPassword:	"False"
    IsControlElement:	"True"
    IsContentElement:	"False"

Thanks,

Hi,
When using elementFromPoint on x86 systems (windows 10) following exception is thrown:

java.lang.Error: Invalid memory access
	at com.sun.jna.Native.invokeInt(Native Method)
	at com.sun.jna.Function.invoke(Function.java:418)
	at com.sun.jna.Function.invoke(Function.java:353)
	at com.sun.jna.Function.invoke(Function.java:308)
	at com.sun.jna.Function.invoke(Function.java:299)
	at com.sun.jna.Function.invokeInt(Function.java:706)
	at mmarquee.automation.uiautomation.IUIAutomation$Converter$1.elementFromPoint(IUIAutomation.java:163)
	at mmarquee.automation.UIAutomation.getElementFromPoint(UIAutomation.java:551)

Does not reproduce on 64Bit os.

This happens with the latest version from master.