GitLab
p/gitlab
Complete DevOps platform, delivered as a single application.
Daniel Ber
Jaco Assistant — Multi language offline and open source voice assistant
Featured
14
Jaco is an Offline and Open Source Voice Assistant for multiple languages following a privacy by design concept. Jaco was able to beat solutions from Amazon and Google in command recognition accuracy in different benchmarks.
Replies
Daniel Ber
Supporting Spanish too now:)
Daniel Ber
Hi Hunters, 👋 I want to share my Voice Assistant project Jaco with you. It's an Offline and Open Source Assistant for multiple languages following a privacy by design concept. Jaco was able to beat solutions from Amazon and Google in command recognition accuracy in different benchmarks. Currently Jaco understands English, German, French and since this evening also Spanish, but its extendable to further languages too. The Assistant uses a modular and container-based micro-service architecture for easy installation and to simplify development. It can run offline on almost any computer or a Raspberry Pi (tested with Pi4 and Pi3, but the later is a bit slow, you have to wait a few seconds until your request is processed). The installation does take about 30-60 min, depending on your hardware and internet speed. You can find the complete project and setup instructions here: https://gitlab.com/Jaco-Assistan... There’s also a skill store where you can share and download skills and I already did create some skills showing different possiblilities of what you can do with the Assistant. The skills are written in python, but you can use any language you like (just add a small python interface). You can give the skills full control over the hardware if you need to. Building a skill that does some heavy computations on a gpu? No problem. Want to control some gpio pins at the same time? You can run the skills distributed on multiple devices with almost no extra work. The skills are using a permission system (somewhat similar to android app permissions) so that the users downloading a skill from the store are always informed what resources it can access. You can find the skill store here: https://jaco-store.web.app/ And of course, the store can run offline as well:) To achieve high recognition accuracy Jaco uses the installed skills and automatically generates domain specific language models. This means Jaco does only understand words included in the installed skills, but you can add new words or skills within seconds (or minutes on a raspi). As long as you don't want to search special words on Wikipedia this shouldn't be a problem at all. If you have skills for light control and playing music, you just have to add a new weather skill and you can ask Jaco for tomorrows weather too. Why did I make this? Before creating Jaco I did use the proprietary Snips as offline Assistant. But it was bought by Sonos this winter and isn't usable anymore since then. So I was in need of a replacement. I wasn't satisfied with the alternatives existing at this time and started to try building an Assistant myself. While the program was working the performance was very poor and I did take a break from it. But then came the covid outbreak and my university was closed for some months and I had a lot of time to continue with the project. Now that the performance is quite good I wanted to share the project with others:) > What happens in your home should stay in your home < Check it out.
Daniel Ber
@cryogenicplanet Setup does take about 30-60 mins I would say, depending a bit on your hardware and internet speed
Daniel Ber
@cryogenicplanet Thanks for the link, I will look into it. The main installation is already done in the prebuilt containers the installation script will download. But running container in container doesn't work right now.
nicolas decoster
Something that I think would be cool, is to have the possibility to interact with the Jaco main node from any web browser, something like a pure web browser satellite. What do you think of that? Is it planned?
Daniel Ber
@ogadaki Cool idea. Want to open a feature request issue for this? I think it should be able to implement this as a skill, which handles the audio input and output. We just have to think about an elegant way to start another wake word container for this skill (I don't think this part has to run in the browser too).
Doris Ferguson
Are you going to "teach" the product more vocabulary?
Daniel Ber
@dorisferguson Not sure what you mean with "more". The vocabulary Jaco understands is depending on the skills. In those you write example sentences like "Turn on the light in the [kitchen](room)" and you also add a list of room names. If you install this skill then Jaco understands to turn on the lights in different rooms. If you install other or additional skills you get different vocabularies. So you should be able to teach whatever vocabulary you need. I don't think that Jaco will support the full vocabulary of a language (without the skill teaching) in the next time by default, because you will get much lower recognition rates then or you need a very powerful computer and a different speech-to-text module instead. But if you're still interested in this, it's possible (and not even complicated) to replace the generated language model file with the full vocabulary models I'm using for training the acoustic speech-to-text network, but I wouldn't recommend this.
Daniel Ber
@dorisferguson Did update the images to show how adding words and sentences to a skill works.
Doris Ferguson
@danber Thank you Daniel! You have answered my question :) And you gave me even more info! I wissh you luck!
Lis Louise
Is it possible to use it for profit? if I wanna include it in a product or is there a legal concern about this? Thank you
Daniel Ber
@lis_louise This is generally possible, the source code is released under LGPL licence. But for some languages (German for example) the Speech-to-Text network was trained using non-commercial-only datasets. I'm not sure how this has to be transferred to the trained network weights, which are subsequently used in Jaco.