Multimodal Intuitive Communication Teleguidence of mobile robotic platforms (MICT)

Multimodal Intuitive Communication Teleguidence of mobile robotic platforms (MICT) will be developed in order to communicate and guide mobile robotic platforms on the base of scenarios using increasingly sophisticated modalities (gender, voice, gesture, other signals) in a telecommunication dialogue mode between the users and robotic platforms using various wireless networks (WiFi TCP/IP, ZigBee, etc.).
MICT should be capable of (i) taking into account the users’ preference in using kinds of modalities; (ii) capturing and tracking modalities, (iii) training to recognize chosen modalities, (iv) extracting features from modalities; (v) aggregating features and forming decision making criterions; (vi) selection and formation of the teleguidence commands transmitted to the robotic platforms appropriate to values of criterions in a telecommunication dialogue mode between the users and robotic platforms.

The envisaged outcome of this project is the development of theoretical and applied toolkit which is able to support intelligent multimodal intuitive communication with mobile robotic platforms to operate and handle these platforms in different type of environments. Such toolkit will be used in an environment where networked and wireless devices are much more likely to be transient parts of the system. The demands placed upon mobile robotic platforms will be different and they will be expected to be intelligent and multi- agential. It will be need to identify multimodal intuitive communication services depending on type of the environment, used sensors, abilities and problems which mobile robotic platforms have to decide.
This work will build on our previous work in these areas. The outcome of first phase will be a set of functional requirements for our toolkit. We can then decide how these services are supported depending mobile robotic platforms, environment in which the robotic platforms operate and problems which mobile robotic platforms decide. Having determined requirements and identified what support mechanisms are available, we can then prototype our toolkit and evaluate its operation in the teleguidence environment. Planned extension and growth will then be evaluated, probably by simulation, and we can then integrate the lessons learnt to reach the project goal - toolkit which is able to support multimodal intuitive communication with mobile robotic platforms to teleoperate and handle this platforms.

multimodal intuitive communication;
mobile robotic platforms;
multi- agential
teleguidence;
artificial intellect;
audio-visual feature extraction and analysis;
image and signal recognition;
feature fusing and aggregations;
wireless nets.

((i) research partners in (a) audio-visual feature extraction and analysis; (b) feature fusing and aggregations; (c) multimodal intuitive telecommunication; (d) wireless sensor nets; (e) mobile robotic platforms.
(ii) mobile robotic platforms consumer companies for embedded platform build-up to illustrate the intelligent multimodal intuitive communication and teleguidence;

(i) development self-calibration vision systems for telecommunication,
(ii) development standalone applications for concrete types of the mobile robotic platforms to teleguide with the help of the multimodal intuitive communication.
(iii) study and application new sensors expanding opportunities of the intuitive interface and improving robustness,
(iv) system integration and mobile robotic platform build up for the different applications.

Research Centers; Universities; Private Companies; Mobile robotic platform manufactures.

Yes