These CIFRE: Embedded machine learning solutions for vision-based autonomous navigation

Environment

This PhD is a collaboration between the ACENTAURI team at Inria and the Vision Technology Engineering Center (VTEC) at NXP Semiconductors.

ACENTAURI is a robotics team that studies and develops intelligent, autonomous and mobile robots that can help humans in their day-to-day lives at home, at work or during their travels. The team focuses on perception, decision and control problems for multi-robot collaboration by proposing an original hybrid

model-driven / data driven approach to artificial intelligence and by proposing efficient algorithms.

The team focuses on robotic applications in smart territories, smart cities and smart factories. In these applications several collaborating robots will help humans by using multi-sensor information eventually coming from infrastructure. The team demonstrates the effectiveness of the proposed approaches on real

robotic systems like cars AGVs and UAVs together with industrial partners. Innovation and the transfer of the research work towards industrial partners are a concern of ACENTAURI.

NXP Semiconductors is building the processors that will enable the future intelligent embedded applications. We aim to do so in a safe and secure way by offering complete, fast and reliable solutions to our clients. Thanks to NXP, autonomous vehicles, robots, drones and mobile devices of the future will be equipped with neural accelerators (such as our eIQ Neutron NPU) that allow fast edge inference.

However, the hardware is only the first part of the complete solution.

The NXP Vision Technology Engineering Center (VTEC), based in Sophia Antipolis, France, is in charge of the software enabling efficient video processing pipelines on NXP processors. In order to push the boundaries of what we can achieve with our hardware, we need to develop in-house efficient AI architectures specifically tailored to our processors and our customers’ use cases.

Motivations and general objectives

As part of a research collaboration between the ACENTAURI team at Inria Sophia Antipolis and NXP Semiconductors, we are interested in building autonomous devices such as robots, drones or vehicles that have to navigate through various dynamic indoor and outdoor environments, such as homes, factories or

cities.

In such context, it is necessary to design a specific and dedicated architecture able to handle uncertainties to guarantee safe autonomous navigation.

Since the introduction of occupancy grid OG [10] in 1989, many approaches have been proposed (i.e. Bayesian Occupancy Filter BOF [5], Bayesian Occupancy Filter Using Prior Map Knowledge BOFUM [3]) and dealing with dynamic environment (Probabilistic Velocity Obstacles PVO [14] or Conditional Monte Carlo Dense Occupancy Tracker CMCDOT [23, 2]).

It turns out that semantic maps [16, 20] have also been investigated making the awareness of the situation easier.

The object of the PhD thesis will be to setup a complete Perception system based on a generic spatio-temporal multi-level representation of the scene (geometrical, semantical, topological, …) that will provide information needed by an ontology of navigation task and directions originating from various

modalities (sound, text, images, other systems). The geometric representation will be provided by state of the art SLAM algorithm, while the PhD subject will focus on extracting semantic and topological information. Semantic and topology will be extracted using a Data based approach and an abstraction toolbox (Graphs based) will be developed to make the connection with ontologies on one side and with the task to be done on the other side. Previously developed work done in [7, 8, 9] will

be the starting point of the study.

Then, the PhD candidate will develop a Bayesian formalism able to handle dynamic contexts, uncertainties and inaccuracy (coming from measurements, modelling, estimations, and misunderstanding of erratic and unforeseen behaviour of humans) and occlusions coming from the dynamicity of the scene, in order to ensure the safe autonomous navigation task.

The PhD will address different contexts with increasing complexity, starting by defining a particular sensing system and a representation of the natural dynamic environment, and using state of the art algorithms to assess the situation at each time of evolution and to evaluate the different actions in a given horizon of time. The different contexts will concern various environments such as homes, factories, fields or cities.

The global system will be developed under ROS 2 and evaluated with different simulated scenarios.

Real experiments will be possible using the Living Lab that the ACENTAURI team is building in the European Projet Agrifood-TEF.

Onboard experiments on NXP hardware will be supported by the VTECteam at NXP.

More information about NXP in France…

Read Full Description