Git Product home page Git Product logo

mycroft-android's Introduction

License: GPL v3 CLA Team Status

PRs Welcome Join chat

Mycroft-Android

This is the Android companion app to Mycroft-core. It works by opening a websocket connection to the Mycroft-core messagebus and sending and receiving messages from there.

It implements voice recognition and Text To Speech (TTS) via Google API's at the moment, but that may change soon.

To Install

Import the repo into Android Studio, or your IDE of choice. Build and deploy to a device

Once the app is running on a device (Lollipop or later SDK 24), you will need to set the IP address of your Mycroft-core instance in the Settings -> General Options menu. That will then create a websocket connection to your Mycroft and off you go!

To help out

If you would like to help out on this project, please join Mattermost at https://chat.mycroft.ai/login and ask where you can contribute! Currently, design and UI/UX is most needed, but any and all help is greatly appreciated!

Submission Notes

Want to submit a fix, feature or...? Here is everything, we think you will need to know.

Mycroft.ai is a collaborative, open source project. That means we encourage and expect people to participate. But to make things a bit more clear here are some kind lines if you would like to submit a fix.

Passthrough (component app)

  1. Pull your own fork, work there
  2. make a branch of whatever you are working on, makes sure your fork is the latest.
  3. Test!!!!
  4. merge into your master.
  5. make pull request into project master
  6. assign a reviewer.
  7. check on it, if not reviewed after a week find a new reviewer, we are mostly volunteers so find one that has time.
  8. sit back and enjoy your handy work.

Coding style...

We have moved now to Kotlin and therefore will be following the standard coding practices. Also please use descriptive method/function names. And use comments to back up that name when complicated, like a calculation or similar. Remember, you want to come back 6 months from now and be able to read your code.

Most of all have fun. Ask questions and don't worry about breaking anything, that is why we have a versioning system.

mycroft-android's People

Contributors

cliabhach avatar doomsdayrs avatar joepoff avatar kalexmills avatar khasmek avatar kleo avatar krisgesling avatar mrbunkyman avatar nvarner avatar paulscott56 avatar penrods avatar seymour-bootay avatar skraynick avatar tree-ind avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mycroft-android's Issues

Don't depend too heavily on Google Play Services

I think we should wrap the Google Play Services calls in 28a46ae with the method from GooglePlayServicesUtil or GoogleApiAvailability that checks whether GMS is even installed. Otherwise we'll get a bunch of unhelpful exceptions on a device without google apps.

I can't pair my Mycroft app with my Mycroft installation

Hello everyone,

  1. I resulted to compile this Android code and I installed the apk on my smartphone
    and according to the official website: (https://mycroft.ai/get-started/#setup)
  2. I created a Mycroft Account
  3. I installed Mycroft on my Debian-based distribution (https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/get-mycroft/linux) basically after the download I just executed the "dev_setup.sh" script that also adviced me to remove two "jackd" packages that could conflit with Mycroft (done)
  4. I gave "./start-mycroft.sh all" from terminal, Mycroft registered a few seconds of my voice and I listened the audio, ok
  5. Now I should get my 6-digit Registration Code. This page (https://mycroft.ai/get-started/#setup) says that "during the device setup, your Mycroft should give you a 6-letter pairing code" , but I don't see/hear anything. I also run again the "dev_setup.sh" script but nothing...
  6. this other page (https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/get-mycroft/linux) says that I should pair the device speaking "Hey Mycroft, pair my device". I done this with Mycroft opened in terminal but nothing happens... what am I not understanding?
  7. If I try to speak anything from the Android app I always get "Connection to Mycroft has closed! Please re-connect and check connection settings!"
    (from the app I also set the "Mycroft IP adress" to the same IP of my linux distro and I didn't make any extra configuration for proxies to don't mess up things etc)

Thanks in advance

Running mycroft-core server on Android phones and tablets

I wonder why shouldn't Mycroft run directly in a standalone mode on an Android device instead of connecting to a mycroft-core server?

mycroft-core with all the dependencies could be compiled and just wrapped in an Android package to be able to run on all Android-based devices.
Then, an Android Wear device could communicate to the mycroft-core server running on an Android phone or tablet.

A few things have to change in the design of mycroft-core for it to be more easily portable to Android. Since Android is Linux, there shouldn't be much difficulty running mycroft-core on Android.

Solved: Mycroft Android will NOT connect to Mycroft Linux 18.08.1

Mycroft Android worked with an earlier version of Mycroft-core, but now it will NOT connect to this version of Mycroft-core. I think it may have something to do with the Internet Port 8181 now being used by Mycroft-core when it used to be Internet Port 8000 or 8080, I don't remember which. Is there a way to change the port EITHER on Mycroft-core OR Mycroft Android? Is that the real problem? What should I look for to find the cause of the non-connectivity?

Mycroft-core says it don't know how to play pandora when instructed via Android App

When instructed to play Pandora, Mycroft-Core says it don't know how to play Pandora. When instructed to play playlists, one out of two work.
I used the instruction Pandora Debug On, then tried to get Mycroft to play Pandora
Got the error, then I asked Mycroft to play a play list and he played it on Pandora.

Attached are the 5 log files that the debug should have updated
mycroft.tar.gz

Thanks in advance for your help.

Ability to interact with multiple Mycroft instances

Ability to interact with multiple Mycroft instances, like message app contacts. Will need to modify settings to support this. Initial roll out will support rules for talking to different Mycroft instances based on which network you are connected to or ability to select one manually. I have plans to handle this better with a bolt solution on the Mycroft instances (unless mycroft-core implements something to take care of this first).

Definable reply on mobile phone on/off

It would be nice if there was a setting (maybe a pill switch) to be able to turn the reply on the mobile phone off.

So that only the Mycroft RPI unit replies, not it and the phone.

Basically use the Mycroft unit on the phone/androidwear watch just as a wireless mic. Since its a bit tedious to listen to the same reply twice (once on the phone and the second time from the pi).

Cheers, thanks for the good work!

"Your device doesn't support speech input"

image
As you can see here I receive this error while trying to send audio, i use and LG G2 with The latest lineage OS 14.0 (Android 7.1) Can this depends from some missing service/app?

Mycroft-Android not sending messages to Mycroft

Looks like the ELSE block in MainActivity.java's sendMessage(String msg) function is attached to the wrong IF block so the json is not being sent. mWebSocketClient.send(json); can also be pulled out of the network status check so that it doesn't have to be there twice.

Failed to resolve: com.google.android.wearable:wearable:2.0.0

I am seeing the following error both on master and v18.09-beta when importing a new Android studio project and attempting a Gradle sync.

Failed to resolve: com.google.android.wearable:wearable:2.0.0

Working to resolve now, going to log progress here in case anyone else sees the same issue.

App Crashes at Startup

So, I compiled and installed the app on Android 5.1, and every time I compile it, it shows an error message stating that it stopped working. I do not really know how to debug / code an Android app so I have absolutely no idea what to do.

If there is any additional info I need to provide, just tell me...

MainActivity.connectWebSocket() is doing too much.

I'm going to split out functionality from that method into inner classes and use callbacks more judiciously. Also, killing that early return in favour of proper preventative action (the wsid should never be allowed to equal an invalid value)

How do i even build this?

I am used to installing deps and building in android studio. but in my case for this app, it litterally REFUSES to build,

I click make project (CTRL+F9) nothing

i click run (SHIFT+F10)
image

please provide more instructions on how to build this application, or provide prebuilt APKs

Incompatable Build Tools

Android studio is warning me that the build tools version is incompatible with the Android Gradle plugin. The app still compiles, but uses a newer version of build tools.

Android SDK Build Tools 28.0.3 will be used.
To suppress this warning, remove "buildToolsVersion '27.0.3'" from your build.gradle file, as each version of the Android Gradle Plugin now has a default version of the build tools.
Remove Build Tools version and sync project
Affected Modules: mobile, shared, wear```

I don't know enough about Android to be certain that this isn't my fault, and I don't know what the best solution is, so I don't want to submit a PR right now. Does anyone else have this problem?

Don't let card text take up so much space

Text added to card views take up more space than they need. Increase the right margins and use a simpler layout (if possible) so that it looks more appealing.

Also change that notifyDataSetChanged call to a notifyItemChanged/notifyItemInserted so we can take advantage of RecyclerView's builtin animations.

Long press on the speech bubble brings up a set of actions

Long press on the speech bubble (either Mycroft or user) will bring up a set of actions.

  • Mycroft speech bubble
    • Repeat
    • View Details
    • Copy text
    • Share
      • Text
      • Email
      • Other
  • User speech bubble
    • Resend
    • Send to different Mycroft instance
    • View Details
  • Need to flesh this out further before I start working on it

Run Mycroft Core on Device

This is a BIG addition, but it is important to make Mycroft on the phone powerful as a standalone experience.

Spruce up TTSManager

I want to bring the TTSManager class up a notch in quality. We'll need the following:

  • javadoc for class, non-obvious methods, and important fields
    • I just like javadoc, that's all.
  • some simple unit tests
    • Junit should be able to cover the existing logic with a couple simple mocks. I think.
    • never could get espresso to give useful test failure info, so I don't know how much I can do on the instrumentation side of things

@paulscott56 does that look good to you?

p.s. about that init method - any particular reason it's not a constructor?

Unable to build the app.

I cannot build the app. I have not done anything with the files. Please help!

To Reproduce

  1. Git clone
  2. Open Android studio
  3. Import the files
  4. Error!

P.S. If possible please send an .apk file.

Mute mycroft-core while speaking with the application

Hello and thank you for your awesome work after 30 minutes I have the working app on my phone now. I had to change a line of code in TTSManager from
val result = mTts.setLanguage(Locale.US)
to
val result = mTts.setLanguage(Locale.getDefault())
for TTS to be in my mother tongue but now it works. Only thing I am missing is the possibility to mute the mycroft-core voice feedback. If I use the app to talk to mycroft while I am in the same room as my Picroft instance I get voice feedback from my phone and at the same time from picroft. If there is a possibility to implement such a function I would really appreciate it. Anyway thank you for your work I am not really into android app writing but I got it running really quick. Hope I can help in the future with this project and other mycroft related stuff as well.

Android wear

Let users interact with Mycroft via Android wear

Add Data Layer

Need to store history of Mycroft interactions

  • Both the user’s utterances and messages from Mycroft
  • Need to manage when to archive/delete records

Possibility to move away from Android's builtin Google STT

Task: Offer the user an alternative to Android's builtin Google STT.

Motivation: My personal use case.
I started using Picroft to avoid using an Alexa or Google's home assistant. I then started using the android app because I did not want a microphone listening 24/7 at home.
I don't expect anyone else out there to need this, so I'll try to work on it myself as soon as I get rights to push a branch to the repo.

Expected bumps:

  • It is going to be slower and less precise, so I'll try to just hide the option a bit
  • If the STT is triggered from the phone (as it now works with Google), it will probably require that the user sets some API keys for that other service manually. For now I'll go the hard way, send the audio to the raspberry and make mycroft-core take care of the STT.
  • It will either need modifications on mycroft-core's messagebus, or a dedicated service to receive the audio from the app and pass it over to mycroft's STT. Meaning...even more difficult to test.
  • Security, I hope to get this right from the beginning but probably I will not
  • I have never programmed in Kotlin. I am not a developer and know little about tests (apart from the fact that I need to implement them).

How I'll try to make this work:

  • Add a switch to the app's configuration. Something like "Use Google STT?". Default to True.
  • If the switch is off, instead of opening Google's STT the app will record for x seconds (4?) and either stream it to the raspberry pi (which I don't know how to do) or save it in the phone and send it to the raspberry pi afterwards (which means using a compressed file and decompressing it in the raspberry, making it even slower)
  • At the Raspberry's side, once the audio file is right, I'll try to inject it as if it were a regular command spoken to the microphone, and let the process continue. I still haven't checked how to do this, but to avoid messing with the messagebus for now, I'll build another service that takes care of this.

Any ideas will be much appreciated.

Need to add `description` to repo

Can we get something like:
Android companion app, sends commands from your Android device to your Mycroft system and returns the output as speech or other medium to the Android device.
added to the description box of the repo.

Don't forcibly push the user to `SettingsActivity` from `MainActivity`

MainActivity::loadPreferences, when we don't have an IP address, automatically opens the settings screen (SettingsActivity). This means that if the user opens this app for the first time, they will be immediately pushed to SettingsActivity.

It also means that pressing back from SettingsActivity without having a valid stored IP address may cause this code to run, pushing the user forward into SettingsActivity. This occurs naturally on low-memory devices, and can be easily simulated by enabling the developer option Don't keep activities.

I suggest that we instead put a noticeable (but non-modal!) warning that the app is not fully configured, with a button to view app settings. Perhaps a SnackBar?

update repositories. No more maven.fabric.io!

I am using linux but not ubuntu and was able to get a successful build. I am sharing the files. Changes were with updating the repository because maven.fabric.io no longer works.
All changes were in the mobile, shared, wear build.gradle files, and :mobile/utils/MainActivity.kt.

Files Mentioned

I'm using android studio bumblebee.

In /wear/build.gradle:
[Might be better to use the 1.3.72, I was having problems with 1.5.30. You could probably mess with different repositories to update kotlin to 1.5.30.]

buildscript {} update ext.kotlin_version to "1.3.72"

New repository are:
maven { url 'https://mvnrepository.com/artifact' }
maven { url 'https://repo1.maven.org/maven2' }

These are the dependencies I put:
dependencies {
classpath 'com.android.tools.build:gradle:4.0.2'
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
classpath "org.jetbrains.kotlin:kotlin-android-extensions:$kotlin_version"
classpath 'com.android.tools.build:gradle:1.3.1'
classpath 'com.google.gms:google-services:4.3.10'
classpath 'com.google.firebase:firebase-crashlytics-gradle:2.7.1'
}
allprojects {
repositories {
maven { url 'https://mvnrepository.com/artifact' }
google()
jcenter() [jcenter no longer is supported but keep on because it prevents errors on MainActivity.kt]
Add mavenCentral()
}

At the very end:
ext {
minSdkVersion = 19
targetSdkVersion = 29
compileSdkVersion = 29
buildToolsVersion = "27.0.3"
kotlinVersion = "1.3.72"
supportVersion = "28.0.0"
}

/shared/build.gradle:
add apply plugin: 'org.jetbrains.kotlin.kapt'

in /mobile/build.gradle:
apply plugin: 'com.android.application'
//apply plugin: 'io.fabric'
apply plugin: 'com.google.gms.google-services'
apply plugin: 'com.google.firebase.crashlytics'
apply plugin: 'kotlin-android'
apply plugin: 'kotlin-android-extensions'
add apply plugin: 'org.jetbrains.kotlin.kapt' [maybe redundant or vice versa]

repositories {
google()
jcenter()
Add mavenCentral()
maven { url 'https://mvnrepository.com/artifact' }
}

[remove crashlytics and add two implementations below]
//implementation('com.crashlytics.sdk.android:crashlytics:2.10.1@aar') {
// transitive = true
Add
implementation 'com.google.firebase:firebase-crashlytics:18.2.1'
implementation 'com.google.firebase:firebase-analytics:19.0.1'

In the MainActivity.kt:
Remove import com.crashlytics.sdk.android and
import io.fabric.sdk.android.Fabric

Add import com.google.firebase.crashlytics.FirebaseCrashlytics;

Remove Fabric.with(this, Crashlytics())
and Add FirebaseCrashlytics.getInstance().setCrashlyticsCollectionEnabled(true)

Those are the major changes and I was able to get a successful build with no errors with only 5 warnings on new mapping. There are small upgrades to numbers that I didn't add but will be on the files.

Rick

Mycroft autodiscovery

Intent to scan networks for Mycroft devices and automatically set them up as app endpoints. Probably an avahi/zeroconf type of setup would do it

Git/github project import into Android Studio does not load modules properly

I suspect that the root cause is the .idea folder. This contains ide-specific information and really shouldn't be in the repo in the first place.

Errors seen:

3:56:52 AM Gradle sync started
3:56:59 AM Error Loading Project: Cannot load 2 modules Details...
3:57:03 AM Gradle sync failed: Failed to find target with hash string 'android-24' in: /opt/android/sdk

Note: The last line, about the gradle sync, is a different issue.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.