Interaction

Components that give the robot the ability to communicate with someone or something or manipulate its environment.

Introduction

One of the 5 high-level components of the Autopilot Inference is the Interaction stack. The Interaction stack should ensure that the robot is able to interact with the environment, which means the artifacts within the environment as well as intelligent beings. Typical interactions of a robot are the manipulation of objects, such as pick and place, and conversation with human bystanders. Avular will develop these types of interactions in the future. For now, the scope of the Interaction stack is limited to providing an explanation to a user using vocal speech.

Capabilities

There is one capability that the Interaction stack provides, which is the behavior called InformUser that is still in development. It is designed to support the robot in informing a user on what the robot is planning to do next.

 

Informing a user is a scheduling capability that can be required within any task that the robot executes. Recall that tasks are executed by specific behaviors in the Ordination stack, where a behavior is a compilation of individual behavior subtrees that each are supported by one of the stacks within Autopilot Inference. The Interaction stack provides such a behavior subtree, called InformUser, that, when executed by the Ordination stack, is able to inform a user either by a terminal logging statement or via spoken text. How this behavior tree can be used in combination with the behavior trees of the other stacks is explained in the Ordination stack. Here, the illustration below is an abstract representation of the capability to inform a user.

 

Vocal server (unreleased) is a capability that can be build by a developer using our developing examples. The capability accepts a sentence, via either a service call or a topic, and then it will turn that sentence into a WAV file that is then played through the speaker. The WAV file is created locally using a tool called VOSK. Also, the vocal server will continuously check for sentences that were spoken by a user by turning the microphone stream to a text sentence. This sentence is than processed internally to check if it would match one of the known behaviors, but it is also published on the topic audiobridge/heard_sentence.

Info

Avular collaborators are authorized to read the details of the developments, which is found on the Development-Interaction