gaozhao1989 / pyjab Goto Github PK
View Code? Open in Web Editor NEWPython implementation for Java application UI automation with Java Access Bridge
License: GNU General Public License v2.0
Python implementation for Java application UI automation with Java Access Bridge
License: GNU General Public License v2.0
Hi, I have a table where all cells are elements type Panel. Unfortunately all of the cells have bounds {X=-1,Y=-1, Width=-1,Height=-1}. That shows "Access Bridge Explorer". I checked that NVDA also did not recognize cells in this table.
I checked that SilkTest is able to somehow get cells.
I know that probably silktest has their own magic but maybe you have idea
thanks in advance
一个由网站弹出的新窗口,例如OracleERP,通过title或handle初始化JABDriver会出现找不到相应的Java window和handle不是一个Java window的异常。这种情况下我应该如何初始化呢?
普通的Java应用程序例如intelliJ IDEA的连接是没有问题的
Xpath Parser enhancement.
Set Xpath Parser almost like this Xpath
Nodes
Predicates
Unkown Nodes
Several Paths
Axes(In future)
Operations
Hi Gary,
I am in the situation where I need to check live data from the table (i.e. status changing from "processing" to "done").
Currently, I need to reuse functions find_element_by_role("table")
and input_status = jqm_table.get_cell(0, 3)
in a while loop.
Is there a possibility to get current information from the element without a loop?
比如:_generate_childs_from_element/_get_accessible_selection_from_context 等,我们这边有场景需要直接调用这些内部方法
应用界面截图:
看得出这是个table, 于是我用下面的语句测试:
ele = driver1.find_element_by_xpath("//viewport[@ObjectDepth=10 @indexinparent=0]/table")
print(ele)
aa=ele.get_cell(0,2)
print(aa.get_element_information())
aa.click()
控制台输出为:
<pyjab.jabelement.JABElement object at 0x0000000002C235E0>
<pyjab.accessibleinfo.AccessibleTableCellInfo object at 0x0000000002C245C0>
1
{'name': '李一龙', 'description': '', 'role': 'label', 'role_en_us': 'label', 'states': ['启用', '可集中', '可见', '不透明'], 'states_en_us': ['enabled', 'focusable', 'visible', 'opaque'], 'bounds': {'x': -1, 'y': -1, 'height': -1, 'width': -1}, 'object_depth': 0, 'index_in_parent': -1, 'children_count': 0, 'accessible_component': True, 'accessible_action': False, 'accessible_selection': False, 'accessible_text': False, 'text': None, 'table': None}
Traceback (most recent call last):
File "D:/python/appPageObject/pyjab2.py", line 34, in
aa.click()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\site-packages\pyjab\jabelement.py", line 365, in click
self._do_accessible_action(action="click")
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\site-packages\pyjab\jabelement.py", line 315, in _do_accessible_action
raise JABException("JABElement does not support Accessible Action")
pyjab.common.exceptions.JABException: ('JABElement does not support Accessible Action', None)
从上面的信息可以看出,返回的是 'role': 'label' 的控件,但是为什么这个控件不支持 jab的点击事件呢?可不可以优化下,因为如果用模拟点击的话,注意它的 bounds: {'x': -1, 'y': -1, 'height': -1, 'width': -1} , 可以看出,模拟点击也是不行的,所以能不能给我解决这个问题,就算是专门为这个label单独写个click方法也行,比如click_in_label()之类的。
最近在研究公司的用友u8,有点类似electron,在应用frame内嵌入一个java,无法用[Access Bridge Explorer]捕获frame内的java元素,请问这种情况如何解决?
My application has a page tab list and inside it has 4 page tabs. Parameter Definition, Software, Language and Phonebook
Inside Parameter Definition and Language page tabs there is a label with the same name as of tab title
Code to select the Language tab:
element = driver.find_element_by_role("page tab list").select("Language")
print(element.role)
The above code throws timeout exception and prints element.role as label instead it should be page tab.
Code works fine for Software and Phonebook page as there are no labels matching with the Tab
<img width="239" alt="Capture" src="https://user-images.githubusercontent.com/115481757/194891783-61e4c383-b7de-4ad7-acb3-84
0ace162614.PNG">
Hi,
How I can correctly release java object?
See below error:
> jab.enter_textbox_details_by_label("Name Required", input_name)
ar_test.py:36:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
..\..\utils\jab_actions.py:21: in enter_textbox_details_by_label
element.send_text(input_text)
..\..\pyjab\jabelement.py:864: in send_text
self.clear()
..\..\pyjab\jabelement.py:301: in clear
self.click()
..\..\pyjab\jabelement.py:284: in click
self.set_element_information()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <pyjab.jabelement.JABElement object at 0x000001FD110C8BB0>
def set_element_information(self) -> None:
"""Set JABElement information(attributes) from internal jab funcs
Raises:
JABException: None value of vmid, hwnd or accessible_context
JABException: Get Accessible Context Info error(internal jab func error)
JABException: Get Object Depth error(internal jab func error)
"""
if not any([self.vmid, self.hwnd, self.accessible_context]):
raise JABException(
"JABElement attributes should have vmid, hwnd and accessible_context"
)
info = AccessibleContextInfo()
result = self.bridge.getAccessibleContextInfo(
self.vmid, self.accessible_context, byref(info)
)
if result == 0:
raise JABException(self.int_func_err_msg.format("GetAccessibleContextInfo"))
object_depth = self.bridge.getObjectDepth(self.vmid, self.accessible_context)
if object_depth == -1:
> raise JABException(self.int_func_err_msg.format("getObjectDepth"))
E pyjab.common.exceptions.JABException: ("Java Access Bridge func 'getObjectDepth' error", None)
..\..\pyjab\jabelement.py:954: JABException
I don't understand why we need this line of code in function "_set_window_foreground"
Hi,
Can you help me with my issue? I want to check if there are no (in most cases unexpected) elements visible during the test execution?
i.e. exists_element_by_name will return true or false rather than the exception
Missed action for call JAB API select for AccessibleTable
About BOOL GetAccessibleContextAt(long vmID, AccessibleContext acParent, jint x, jint y, AccessibleContext *ac)
This API can not work for every Java window. It is work in the demo(SwingSet2), it can be return corrent AccessibleContext.
but it can not work in Java Window Oracle EBS ERP, it return the top compoent AccessibleContext. Do you have any idea about it?
I have try to add these code into jabelement.py
def get_accessible_context_at(self, x, y) -> JABElement: target_accessible_context = self._get_accessible_context_at(x, y, None, None, None) return JABElement(self.bridge, self.hwnd, self.vmid, target_accessible_context)
def _get_accessible_context_at( self, x: c_int, y: c_int, vmid: c_long = None, parent_acc: JOBJECT64 = None, new_acc: JOBJECT64 = JOBJECT64(), ) -> JOBJECT64: """ Retrieves an AccessibleContext object of the window or object that is under the mouse pointer. BOOL GetAccessibleContextAt(long vmID, AccessibleContext acParent, jint x, jint y, AccessibleContext *ac) Args: x: position x of child accessible context y: position y of child accessible context vmid: java vmid of specific java window parent_acc: parent accessible context of specifc java object new_acc: new accessible context of specifc java object Returns: AccessibleContext: get Accessible Context success None: get Accessible Context failed """ vmid = self.vmid parent_acc = self.accessible_context new_acc = new_acc if isinstance(new_acc, JOBJECT64) else JOBJECT64() result_acc = self.bridge.getAccessibleContextAt(vmid, parent_acc, x, y, new_acc) if not result_acc or not new_acc: raise JABException(self.int_func_err_msg.format("GetAccessibleContextAt")) if not bool(self.bridge.isSameObject(self.vmid, parent_acc, new_acc)): return new_acc # elif new_acc != parent_acc: # self.release_jabelement(new_acc) return None
Hi,
can you check what is the CPU consumption for the java application under test during execution i.e. find_element_by_name or find_element_by_xpath?
Task Manager on my VM (hyper-v ws2019) shows about 24%, a pretty high, isn't it?
thanks in advance
Hello,
I have been able to retrieve elements but I am getting an error thrown when trying send_keys to an element.
I have attached the hs_err_pid report produced, which results in the app crashing.
Any pointers?
Cannot found new opened java window.
Sample:
Java Control Panel -> Open about
try to found new About window, but failed
Hi,
some time ago I wrote an email to you about a specific function which I called "scroll to view". The idea is to scroll to see elements from the list which are outside from the visible area. I know that you implemented the scroll function with press scrollbar time but it is hard to use it when you don't have full control over the list (the number of the elements is not fixed). Any chance to implement that function?
Hi Gary,
do you plan to add a function for hitting the "enter" key?
It would be useful for submitting some values in forms.
I did something like below and it works but maybe you will add some extra code:
def send_key(self, value: str) -> None:
value = str(value)
self.win32_utils._press_key(value)
aaaa
最近我用10个测试用例作为一组,不断循环跑。
大慨在跑了30分钟左右,发现被控制的窗体会变慢,比如说正常情况定位某个元素需要1秒钟,随着跑的时间越长,这个定位时间也会慢慢变长,大慨跑了不到1小时就跑不动了,虽然自动测试代码还在跑,但被控制的窗体已经被卡死,手动关都关不了,只能在任务管理器中杀掉,观察CPU和内存都正常,没有明显升高。
也让开发的人看过了,他们现在也没有什么好的解决方案,不知道有人是否碰到过这种情况,该如何解决呢?
SPIN_BOX value should be "spinbox", not "spin box"
Hi,
This project works wery well with my JAVA desktop GUI. I have one issue. I have panel with list of the clickable elements. Numer of elements is bigger that height of the panel, so scroll bars are visible to select elements from the bottm of the list. How can I scroll to view elements for selecting them by pyjab?
there is an option to single click but double click is not working.
这几天利用假期我又来学习pyjab,发现在控件多的窗体内定位控件比较慢,特别是有table的窗体,而且table里的内容还比较多,定位得特别慢,刚才定位一个控件需要40秒钟,这貌似也太久了,后来我把xpath改成下面的全路径模式:
driver.find_element_by_xpath(
"//root pane/layered pane/panel[@indexinparent=0]/panel[@indexinparent=0]/panel[@indexinparent=1]/panel[@indexinparent=1]/push button[@indexinparent=6]"
以为这样把所有路径都指定好,会快一些,谁知道并没有任何改善,还是需要约40秒才能成功定位,期间我观察这段时间的cpu利用率真,会上升一些,但是cpu还是很空闲,于是我想用多线程的方法提高CPU利用率,大慨是下面的方法:
from multiprocessing import Process
p = Process(target=, args=("bt",))
ps_list = []
for i in range(0, cpu_logical_count):
ps_list.append(Process(target=driver.find_element_by_xpath, args=(xpath,))
for p in ps_list:
p.start()
for p in ps_list:
p.join()
发现这样不行,会报错,现在也没什么好的办法了,不知道你那有什么好的建议没有?
Hi Gary,
can you add a function find_element_by_description?
I was trying to automate oracle r12 java applet and was able to navigate and find internal frame inside which need to set text and click on push button.
However, not able to do so. Every instance when trying to set text it goes to a fixed field and enter there and when try to click any button it is not able to click.
在一个窗体上有输入框,按钮有热键("alt","y")
1.输入框 send_text('aaaaa', simulate=False)
使用按钮热键的方式 _press_hold_release_key("alt","y"),正常
2.输入框 send_text('aaaaa', simulate=True)
使用按钮热键的方式 _press_hold_release_key("alt","y"),没有任何反应,就像没有这行代码一样,不会被阻塞。
另外这个窗体这个按键用 element.click(simulate=False)能正常点击,但此后弹出确认的窗体会被阻塞,完全控制不了,Access Bridge Explore 也被卡住,直到手动点击这个确认的窗体消失后才恢复正常
Hi,
Although the instruction is provided as below, I am not sure how to get this resolved because it says v1.0.0 thru 1.1.7 which means all of released version.
Do you have any ideas on this?
FYI, below is the version of python and pip. It looks all up to date.
Also, I tried to install pyjab in the mac but I got the same error too.
Regards,
Anthony
C:\Users\antho\projects>python --version
Python 3.10.6
C:\Users\antho\projects>pip --version
pip 22.3 from C:\msys64\mingw64\lib\python3.10\site-packages\pip (python 3.10)
C:\Users\antho\projects>
To fix this you could try to:
C:\Users\antho\projects>pip install pyjab
Collecting pyjab
Using cached pyjab-1.1.7-py2.py3-none-any.whl (40 kB)
Collecting Pillow>=8.3.2
Using cached Pillow-9.3.0.tar.gz (50.4 MB)
Preparing metadata (setup.py) ... done
Collecting pyjab
Using cached pyjab-1.1.6-py2.py3-none-any.whl (40 kB)
Using cached pyjab-1.1.5-py2.py3-none-any.whl (40 kB)
Using cached pyjab-1.1.4-py2.py3-none-any.whl (39 kB)
Using cached pyjab-1.1.3-py2.py3-none-any.whl (39 kB)
Using cached pyjab-1.1.2-py2.py3-none-any.whl (39 kB)
Using cached pyjab-1.1.1-py2.py3-none-any.whl (39 kB)
Using cached pyjab-1.1.0-py2.py3-none-any.whl (37 kB)
Collecting pypiwin32==223
Using cached pypiwin32-223-py3-none-any.whl (1.7 kB)
Collecting Pillow==8.3.2
Using cached Pillow-8.3.2.tar.gz (48.8 MB)
Preparing metadata (setup.py) ... done
Collecting pyjab
Using cached pyjab-1.0.9-py2.py3-none-any.whl (37 kB)
Using cached pyjab-1.0.8-py2.py3-none-any.whl (37 kB)
Using cached pyjab-1.0.7-py2.py3-none-any.whl (37 kB)
Using cached pyjab-1.0.6-py2.py3-none-any.whl (33 kB)
Collecting Pillow==8.2.0
Using cached Pillow-8.2.0.tar.gz (47.9 MB)
Preparing metadata (setup.py) ... done
Collecting pyjab
Using cached pyjab-1.0.5-py2.py3-none-any.whl (33 kB)
Using cached pyjab-1.0.4-py2.py3-none-any.whl (33 kB)
Using cached pyjab-1.0.3-py2.py3-none-any.whl (33 kB)
Using cached pyjab-1.0.1-py2.py3-none-any.whl (34 kB)
Using cached pyjab-1.0.0-py2.py3-none-any.whl (34 kB)
ERROR: Cannot install pyjab==1.0.0, pyjab==1.0.1, pyjab==1.0.3, pyjab==1.0.4, pyjab==1.0.5, pyjab==1.0.6, pyjab==1.0.7, pyjab==1.0.8, pyjab==1.0.9, pyjab==1.1.0, pyjab==1.1.1, pyjab==1.1.2, pyjab==1.1.3, pyjab==1.1.4, pyjab==1.1.5, pyjab==1.1.6 and pyjab==1.1.7 because these package versions have conflicting dependencies.
The conflict is caused by:
pyjab 1.1.7 depends on pywin32>=302
pyjab 1.1.6 depends on pywin32>=302
pyjab 1.1.5 depends on pywin32>=302
pyjab 1.1.4 depends on pywin32>=302
pyjab 1.1.3 depends on pywin32>=302
pyjab 1.1.2 depends on pywin32>=302
pyjab 1.1.1 depends on pywin32>=302
pyjab 1.1.0 depends on pywin32==302
pyjab 1.0.9 depends on pywin32==302
pyjab 1.0.8 depends on pywin32==302
pyjab 1.0.7 depends on pywin32==301
pyjab 1.0.6 depends on pywin32==300
pyjab 1.0.5 depends on pywin32==300
pyjab 1.0.4 depends on pywin32==300
pyjab 1.0.3 depends on pywin32==300
pyjab 1.0.1 depends on pywin32==300
pyjab 1.0.0 depends on pywin32==300
**To fix this you could try to:
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts**
感谢大佬开发了这个项目。
当我用AccessBridgeExplorer展开目标程序的节点时,到canvas这一级就无法继续展开了,显示没有更下级的child节点,但目标软件的主要功能全都集中在这个canvas中。
我JAVA写的很少,这是不是意味着这个软件的主要界面全是在canvas手搓出来的(但这个canvas的UI风格看上去非常JAVA),并不遵循swing或者awt的标准规则,导致JAB无法访问到其中的元素,我尝试按名称搜索了一些element,canvas外的element基本都能搜到,但canvas内的全都找不到。
请问是我获取元素的方法有问题,从而导致无法获取。还是JAB本身的限制导致其一开始就无法获取canvas内的元素,从而使得pyjab也无法获取元素。如果是后者的话是否有什么办法解决?
When I run the test locally from the machine , everything works fine but when I use Jenkins to run the testcases, I get exception on click(True) function (jabelement.py line 635)
File "C:\Test\Python38\lib\site-packages\pyjab\common\win32utils.py", line 271, in _set_window_foreground
win32gui.SetForegroundWindow(hwnd)
pywintypes.error: (0, 'SetForegroundWindow', 'No error message is available')
It seems like the application can't be taken to foreground
如题。。。。
最近我一有空便学习这个模块,遇到了些问题,不知能否解决:
在使用click(),send_text()时, 如果加上 simulate=True 这个参数,发现被控制的JAVA窗体会弹到前台来,这对我来说非常不友好, 因为我希望在整个使用过程中始终都会保持在后台, 后来我去掉这个参数, 发现是不会弹到前台来了, 但是我看到你的文档建议是加上这个参数, 是不是没有这个参数, 整个程序运行过程会不可靠呢?
如果当前有两个窗体
jabdriver_main
jabdriver_sub
对子窗体进行截图
jabdriver_sub.get_screenshot_as_file("./screenshot.png")
发现截出来的图片大小和位置是子窗体的,但是截的图是主窗体内容, 这样的话这个截图功能就没有意义了,不知能否改进, 并且截图时被控制的JAVA窗体也会弹到前台了, 我希望不要这样, 一定要保持在后台操作,这对我非常重要.
如何实现对element 的鼠标双击
初使化jabdriver时可以选择hwnd和title, 可不可以多加一个classname, 这是我在pinwinauto上看到的, 因为我这title 是动态的,不好控制,而且还有中文的, 是不是中文的title 对用jab来说会增加不稳定的因素呢? 不过这个可能换成hwnd, 倒不是很重要
目前想到的也就这么多.....谢谢.
Line 1474 in 3fe1e61
if find by from another JABElement by xpath, it should find JABElement from current node.
But it find JABElement from root node.
Hi @gaozhao1989 ,
Sometimes when display DPI changes, the logical coordinates returned by JAB API are different from physical coordinates. And the reason is that the java application is not "PROCESS_PER_MONITOR_DPI_AWARE".
When using simulate click, we need physical coordinates to locate the element, So I think we need to convert logical coordinates to physical coordinates by calling win32 function "LogicalToPhysicalPointForPerMonitorDPI".
My English is not good. First of all, I think this is a very good module.I hope you can keep doing it.
I have read your document carefully, and feel that it is too simple, there should be many functions that have not been written out.
I don't understand the Win32 API part.
your document:
Win32utils()._send_keys("abc")
Where did Win32utils() import from?
I don't understand that .I hope you can provide a complete example with JabDriver.
This is very important to me. Thank you again!
Locate root element failed with xpath
def _get_children_by_level(self, level: str = "root"):
if level in {"root", "level"}: # level in here should be "root" and "child"
return self._generate_all_childs if level == "root" else self._generate_childs_from_element
else:
raise ValueError("level should be in 'root' or 'child'")
Hi Gary,
is there possibility to look for a window title with wildcard i.e. JABDriver(title) as a parameter?
大佬你好:
1.我现在用javamonkey可以查看窗口树结构,但是定位不太方便,有没有类似可以拾取元素的工具
2.目前遇到个问题,窗口内有个动态元素,窗口激活才显示,不激活就消失了,不知道怎么定位到这个元素
今天我用这个方法:
element.click(simulate=True)
发现没有反映,于是我看了下你的源码,在上面加了打印,如下:
if simulate:
self.win32_utils._set_window_foreground(hwnd=self.hwnd.value)
x = self.bounds.get("x")
y = self.bounds.get("y")
print(self._acc_info)
print(x)
print(y)
width = self.bounds.get("width")
height = self.bounds.get("height")
print(width)
print(height)
if width == 0 or height == 0:
raise ValueError("element width or height is 0")
position_x = round(x + width / 2)
position_y = round(y + height / 2)
self.win32_utils._click_mouse(x=position_x, y=position_y)
最后的输出是这样:
<pyjab.accessibleinfo.AccessibleContextInfo object at 0x0000000004B99640>
-1
-1
-1
-1
按我的理解 self._acc_info这里应该是记录关控件的属性信息,然而它貌似是个对像,再往下去我就不好查了,反正最终获得的坐标不对,难怪没有反应,我觉得这个问题非常严重,希望快些修复。
Can you give an example, preferably in the driver_jab operation, thank you very much
I want to get the elements listed in Select Cells, but JABElement.get_cell method seems to get from Cells, which cannot be interacted with in my case.
Originally posted by @HuyNguyen7994 in #57 (comment)
Hi, I tried the following function:
desktop_pane = jabdrv.find_element_by_xpath("//desktop pane[@objectdepth=7]")
and I am getting this error:
("incorrect role set 'desktop pane'", None)
So I added to role.py this:
DESKTOP_PANE = "desktop pane"
After that I am getting:
'JABElement' object is not subscriptable
Could you tell me what is wrong?
Hi Gary
I'm able to click and read values of the cell elements in a table.But when the table has more number of records and some of the records not visible in UI and table displayed with a vertical scroll bar.At that time when I try to read the values of jabelement application crashes.
Is there any way to read the values of these invisible jab elements by scrolling automatically? I'm using below method to get all jab elements but limited to only visible elements.
def table_click(self, tabl, value):
info = tabl._get_visible_children()
# print(info)
# To get the row and column count
tableinfo = tabl.get_element_information()
# print(type(tableinfo))
# print(tableinfo["table"]["row_count"])
row = tableinfo["table"]["row_count"]
# print(tableinfo["table"]["column_count"])
col = tableinfo["table"]["column_count"]
result = tabl.bridge.getVisibleChildren(tabl.vmid, tabl.accessible_context, 0, byref(info))
cellcount = 0
values = []
elements = []
valuelist = [[]]
elementlist = [[]]
# To Store the value in a list of list
# print(JABElement(tabl.bridge, tabl.hwnd, tabl.vmid, info.children[1]).get_element_information()['name'])
for incr in range(0, ((row * col) - 1)):
values.insert(incr, JABElement(tabl.bridge, tabl.hwnd, tabl.vmid,
info.children[cellcount]).get_element_information()["name"])
elements.insert(incr, JABElement(tabl.bridge, tabl.hwnd, tabl.vmid,
info.children[cellcount]))
cellcount = cellcount + 1
# print(cellcount)
# print(values)
# print(elements)
d = dict(zip(elements, values))
for x in d:
if d[x] == value:
x.click(simulate=True)
time.sleep(3)
pyjab更多以类库的方式被使用,似乎LGPL协议比GPL协议更为适用。
虽然审视pyjab所采用的开源协议是使用者的义务,但当前的使用者可能没有意识到自己的项目也必须以GPL协议“公开”。
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.