Comments (6)
Hello @dubeyabhijeet! We do not officially support running AndroidEnv on real devices at the moment.
To do that, you'll need a new Simulator to grab screenshots and sending actions. If you don't care about performance that much, it's relatively straightforward to write one that uses adb screencap
(https://developer.android.com/studio/command-line/adb#screencap) and adb input tap
. Keep in mind that this solution will probably achieve only a handful of steps per second.
Cheers,
Daniel
from android_env.
Understood ! Thanks a lot....
Right now you are controlling it from PC to emulator. If i used real devices then performance will be trade off [PC to real devices].
What about deploy it through apk in phone? Like i use app traversal and on each screen use android_env to decide on actions to take? Like ndroid_env in wrapped in traversal apk and deployed on phone.
from android_env.
Yes, it's possible but at the moment there's no good way to do so. There are no APIs to fetch screenshots and execute actions from within Android, and that would also probably require root access, which is a security issue.
from android_env.
Got it ! Understood. Thanks a lot Daniel :)
from android_env.
Hi Daniel, What i see we need to add logs in apps source code. I have two queries:
- Not possible with 3rd party apps without source code?
- How deepmind trained agents for youtube? Have source code or youtube has those logs needed for rewards?
from android_env.
Hello @dubeyabhijeet!
Not possible with 3rd party apps without source code?
You can run anything you want in the emulator, but rewards are only available if something is shown in the log stream and captured by a regex.
How deepmind trained agents for youtube? Have source code or youtube has those logs needed for rewards?
We haven't trained explicitly for YouTube, it was just an example. However, we did train agents on apps for which we don't have the source code such as the Clock or System Settings by reusing accessibility events. Please see https://github.com/deepmind/android_env/blob/main/docs/example_tasks.md#accessibility-forwarder. At one point, we also extracted rewards from view hierarchies, but they're much slower to get (it needs a slow adb dumpsys
call + parsing) than simple Log.i()
calls. We also experimented with OCR, but though that works in synthetic text in static apps, it doesn't really work well for things like games or apps with unusal fonts and/or graphics. In theory, any event or message in Android could be used as a source of rewards, but we relied on logs because they're the easiest to implement (from a developer and task designer's point of views) and reliable.
Cheers,
Daniel
from android_env.
Related Issues (20)
- Issue with running the enviroment on new ubuntu devie HOT 5
- Crash on "run_random_agent.py" HOT 1
- Unable to run random agent HOT 3
- About launching android env with home screen HOT 3
- Cannot run "run_random_agent.py" HOT 8
- Crash on "run_acme_agent.py" HOT 13
- Running on Docker HOT 2
- Compatibility Error when using rich logging handler HOT 6
- 1V1 environmnet HOT 2
- Cannot run "run_random_agent.py" HOT 8
- launching multiple ends HOT 4
- Error connecting to simulator: "Missing the \'authorization\' header with security credentials" HOT 4
- Blocked after running loader.load HOT 5
- AndroidEnv quits because of an unexpected keyword argument 'use_poll' HOT 4
- .textproto log_regexp syntax HOT 2
- Migrate example Acme agents from TF to JAX
- Old problems in master HOT 1
- Extras not always returned . ( in case of floodit_basic). HOT 1
- Incompatible with Python3.10 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from android_env.