How do we want to start? about speech-to-text-in-tw5 HOT 27 CLOSED

Great! I just added the JS file containing the function I wrote for the speech transcription and output. Check the new message at the top of the README.md file for a more in-depth description.

from speech-to-text-in-tw5.

I have added a toggle button for recording on/off in the page-controls section and a base startup script that already gets the on/off state from the toggle button. Maybe you can start from that script to implement the speech transcription

from speech-to-text-in-tw5.

Great! Since I do not have much experience with TW plugins besides using the plugin store, I would appreciate it if you could explain how to run the plugin. Can I just drag and drop the plugins folder into an empty.html TW, or must I download and use the Node.JS TW?

from speech-to-text-in-tw5.

you'll need to clone the node.js TW
then you can

./tiddlywiki.js ~/path/to/your/dev-wiki --init server

then you can edit the tiddlywiki.info file in ~/path/to/your/dev-wiki/tiddlywiki.info and add the plugin there

to add the plugin, you may want to export the TIDDLYWIKI_PLUGIN_PATH in your shell to the location where the plugin is, like:

export TIDDLYWIKI_PLUGIN_PATH="$HOME/Code"

and in your tiddlywiki.info file you would add in the "plugins" section:

"tiddlywiki/some-plugin",
"tiddlywiki/another-plugin",
"Speech-To-Text-in-TW5/plugins/speech-to-text"

from speech-to-text-in-tw5.

Oh, I forgot to mention how you start your wiki when everything is set up:

./tiddlywiki.js ~/path/to/your/dev-wiki --listen

from speech-to-text-in-tw5.

Updated the Speech-to-text.js in the plugin startup folder. It now supports recording, as well as the transcript of the speech. Everything that is TW specific that is left is denoted with comments in all Caps in the JS startup file. Mainly, it is just asking for button colour changes, as well as a way to output the transcript of the speech (perhaps as the text of a new tiddler?). Could you please take a look, and add as much of this as you can?

from speech-to-text-in-tw5.

Whoops! accidentally closed!

from speech-to-text-in-tw5.

Updated the Speech-to-text.js in the plugin startup folder. It now supports recording, as well as the transcript of the speech. Everything that is TW specific that is left is denoted with comments in all Caps in the JS startup file. Mainly, it is just asking for button colour changes, as well as a way to output the transcript of the speech (perhaps as the text of a new tiddler?). Could you please take a look, and add as much of this as you can?

I've now implemented parts of it

from speech-to-text-in-tw5.

Perfect! It looks like all we have left to go is the creation of a new tiddler with the text transcription after it's done recording! Thanks for everything you've done so far!

from speech-to-text-in-tw5.

It already creates a tiddler with the text transcription in its text field, you can find them under the Recent tab

from speech-to-text-in-tw5.

from speech-to-text-in-tw5.

I don't know if we can do that from within the startup script. I'll have a look tomorrow, it's late here in Europe

from speech-to-text-in-tw5.

Love the changes!

Everything works as expected! I'll notify the Google Groups with a link to this repo.

Lastly, I was wondering if Node.js is required for the plugin. Is there a way to make it work for single-html file TW?

from speech-to-text-in-tw5.

https://groups.google.com/g/tiddlywiki/c/qhq4WyPeo0Q. Just started the thread in Google Groups presenting our plugin!

Now, all we have to do is respond to inquiries and keep it updated!

from speech-to-text-in-tw5.

Quick question: How to use drag-and-drop for the plugin in TW. This question has come up in the Groups forum, and I was hoping you might be able to explain it better than I. Thanks!

from speech-to-text-in-tw5.

Quick question: How to use drag-and-drop for the plugin in TW. This question has come up in the Groups forum, and I was hoping you might be able to explain it better than I. Thanks!

We would create a github page from where users can install the plugin

from speech-to-text-in-tw5.

Ok @flancast90 , I've now created the gh-pages branch which contains the index.html file. You would need to go to the Settings and set up a github page for that branch

from speech-to-text-in-tw5.

In the Google Groups, the idea came up to add an option in the edit tiddler bar for recording there. I think the plugin could also be significantly improved by this feature, where the transcript is added to the currently opened tiddler. If you will do the brunt of the TW work (as per usual), I could easily hook up the function call to have this work.

Don't feel rushed at all, I'm just very motivated at times, so its fine for you to take a few days on this.

from speech-to-text-in-tw5.

Yeah, I was thinking about this when writing the startup script but I felt the startup script would be an easier entry point for us.
I'll have to dig a bit in the TiddlyWiki internals to see how we can make this and if it's even possible for us

from speech-to-text-in-tw5.

Maybe @joshuafontany can help us here 👍

from speech-to-text-in-tw5.

Let's create a separate issue for this - "Speech to Text in editor toolbar"

from speech-to-text-in-tw5.

from speech-to-text-in-tw5.

@flancast90 @BurningTreeC I've worked a fair bit with the WebSpeech API, though more so on the speech synthesis side. If there is anything you need input on, feel free to ping me.

One gotcha to be aware of is that the embedded WebView used by applications that embed HTML pages, does not fully support all parts of the API that are available in Chrome. So mobile apps for TW as well as perhaps TiddlyDesktop may be impacted by this.

from speech-to-text-in-tw5.

Just released v1.0.4 and changed the drag-and-drop location to speech-to-text.finnsoftware.net (my domain) since GH pages was acting up. I would love for you to clean up some of the TW related stuff. I did the little bit I could with what I learned from your existing code, but there are still some annoyances such as the transcript being added to the tiddler when it is a command, etc.

Details on the release changes can be found in my post to the Google Groups at https://groups.google.com/g/tiddlywiki/c/3O0eT4K5mkI

from speech-to-text-in-tw5.

@saqimtiaz thanks for your kind offer! We'll ping you anytime we need some input :)
Yeah, we're aware that this API works best on chromium based browsers ... and not in embedded WebViews probably and neither in mobile apps for TW as well as TiddlyDesktop

from speech-to-text-in-tw5.

@BurningTreeC, is there a way to get the current language of TW, or change it using some code? If so, could you share that method with me? In response to some of the issues with the spoken commands, I've decided to add an auto-detect to the program to change the language to the language the wiki is in.

Lastly, without using some sort of setInterval, is there some TW function that will fire when the language changes, and not anytime else, it's fine if you can't find this last one, I can re-create something similar in vanilla JS with a loop.

from speech-to-text-in-tw5.

Let's close this issue since we now have a clearer picture how we're getting started

from speech-to-text-in-tw5.