Publications

Prostate cancer is a major global health concern, requiring advancements in robotic surgery and diagnostics to improve patient outcomes. A phantom is a specially designed object that simulates human tissues or organs. It can be used for calibrating and testing a medical process, as well as for training and research purposes. Existing prostate phantoms fail to simulate dynamic scenarios. This paper presents a pneumatically actuated prostate phantom with multiple independently controlled chambers, allowing for precise volumetric adjustments to replicate asymmetric and symmetric benign prostatic hyperplasia (BPH). The phantom is designed based on shape analysis of magnetic resonance imaging (MRI) datasets, modeled with finite element method (FEM), and validated through 3D reconstruction.

Prostate cancer is one of the most common cancer globally, particularly among men aged over 50. Digital rectal examination (DRE) is one of the first-line method for diagnosing and screening for the prostate cancer. However, medical students often lack sufficient training in performing DRE effectively. In this regard, we are developing a digital tool that seamlessly integrates digital simulation with haptic feedback to transform DRE training. Our method relies on the Finite Element Method (FEM) to create a detailed model of the probing-finger, organs, and their interactions. The contact forces generated by the probing-finger's movements in simulation, are then fed back to the user via a haptic device. We aim to accurately replicate mechanical properties associated with prostate in different conditions and provide realistic feedback, facilitating the preparation of medical professionals for DRE procedures. This approach has the potential to improve the effectiveness and accessibility of DRE training, ultimately contributing to better patient care and outcomes in prostate cancer diagnosis and management.

Growing robots and their eversion principle have wide applications ranging from surgery to industrial inspection and archaeology. The eversion process involves deploying an inflatable device with a material located at the tip of the robot, which, when under pressure, elongates the robot's body. However, the simulation of this complex kinematic phenomenon is a significant challenge. Our approach proposes to use a combination of kinematics and quasi-static modeling to parameterize the starting conditions of the eversion process. This facilitates the understanding of the behavior of this complex kinematic phenomenon and help identify factors that have a significant impact on the eversion process and its response to external factors. The kinematic model uses the Cosserat rod models for local coordinates, while the quasi-static model is based on finite element analysis. The two models are combined to capture the behavior of the robot tip during eversion. This approach has been implemented and tested using the SOFA framework and has been evaluated on the deployment of a vine robot on a narrow passage. The results of our approach are encouraging to better understand the behaviour of soft growing robot during eversion.

Growing robots and their eversion principle have wide applications ranging from surgery to industrial inspection and archaeology. The eversion process involves deploying an inflatable device with a material located at the tip of the robot, which, when under pressure, elongates the robot's body. However, the simulation of this complex kinematic phenomenon is a significant challenge. Our approach proposes to use a combination of kinematics and quasi-static modeling to parameterize the starting conditions of the eversion process. This facilitates the understanding of the behavior of this complex kinematic phenomenon and help identify factors that have a significant impact on the eversion process and its response to external factors. The kinematic model uses the Cosserat rod models for local coordinates, while the quasi-static model is based on finite element analysis. The two models are combined to capture the behavior of the robot tip during eversion. This approach has been implemented and tested using the SOFA framework and has been evaluated on the deployment of a vine robot on a narrow passage. The results of our approach are encouraging to better understand the behaviour of soft growing robot during eversion.

Digital tools are increasingly used for intraoperative assistance. Initially designed for training and learning, finite element simulations are now used and considered essential in the operating room and can be used to display through augmented reality (AR) the internal structures (vessels, tumors, etc.) on top of the intraoperative images. An essential advantage of biomechanical models lies in their ability to predict structures' behavior, providing a physics-based extrapolation, not just geometric, in areas where few or no intraoperative data are available. Yet, a significant difficulty for training simulations and biomechanical AR for medical purposes concerns the need for real-time computing without sacrificing accuracy.

The research works briefly reviewed in this dissertation deal with questions related to the use of data in interactive simulations for applications in medicine and soft robotics. Geometrical modeling, (bio-)mechanical modeling as well as optimization techniques to reduce the computational footprint are presented. Using patient- specific or environment data with optimization techniques and simulation enables more realistic and predictive models and opens the way for new applications. Several examples are presented in the manuscript where our simulations were fed with data from medical images, videos streams or pre-operative data in order to provide more accurate medical simulations to provide better diagnoses or in the context of soft-robotics where visual servoing provides a more robust and stable control of soft-robots.

This plugin allows the modeling of deformable robots in the Sofa platform. It allows the modeling of different actuators, such as cable, pneumatic pressure, hydraulics and other simpler types of actuation. It also contains useful tools for animation design or communication with the robot. Coupled with the SoftRobots. Inverse plugin, it also allows the control of these robots. More information can be found on the dedicated website.

This paper presents the design method of a multi-resolution camera, named Visadapt. It is made of a conventional compact camera with a sensor and a lens pointed to a new deformable mirror so that the mirror in a flat state is parallel to the image plane. The main novelty of the latter mirror, to our knowledge, is the ability to control automatically strokes of several millimeters. This allows Visadapt to capture scenes with a spatially variable density of visual information. A grid of linear actuators, set underneath the mirror surface, deforms the mirror to reach the desired shape computed to capture several areas of different resolutions. Mechanical simulations are allowed to iterate on Visadapt's design, to reduce the geometrical distortions in the images. Evaluations made on an actual prototype of Visadapt show that, by adapting the mirror shape, this camera can magnify a scene object up to 20%, even off-centered in the field-of-view, while still perceiving the rest of the scene.

Soft continuum robots, and comparable instruments allow to perform some surgical procedures non-invasively. While safer, less morbid and more cost-effective, these medical interventions increase the complexity for the practitioners: the manipulation of anatomical structures is indirect through telescopic and flexible devices and the visual feedback is indirect through monitors. Interventional cardiology is an example of complex procedures where catheters and guidewires are manipulated to reach and treat remote areas of the vascular network. Such interventions may be assisted with a robot that will operate the tools but the planning (choice of tools and trajectories) remains a complex task. In this paper we use a simulation framework for flexible devices inside the vasculature and we propose a method to automatically control these devices to reach specific locations.

OpenAI Gym is one of the standard interfaces used to train Reinforcement Learning (RL) Algorithms. The Simulation Open Framework Architecture (SOFA) is a physics-based engine that is used for soft robotics simulation and control based on real-time models of deformation. The aim of this article is to present SofaGym, an open-source software to create OpenAI Gym interfaces, called environments, out of soft robot digital twins. The link between soft robotics and RL offers new challenges for both fields: representation of the soft robot in an RL context, complex interactions with the environment, use of specific mechanical tools to control soft robots, transfer of policies learned in simulation to the real world, etc. The article presents the large possible uses of SofaGym to tackle these challenges by using RL and planning algorithms.

Benefiting from the deformability of soft robots, calibration, and force sensing for soft robots are possible using an external vision-based system, instead of embedded mechatronic force sensors. In this letter, we first propose a calibration method to calibrate both the sensor-robot coordinate system and the actuator inputs. This task is addressed through a sequential optimization problem for both variables. We also introduce an external force sensing system based on a real-time finite element (FE) model with the assumption of static configurations, and which consists of two steps: force location detection and force intensity computation. The algorithm that estimates force location relies on the segmentation of the point cloud acquired by an RGB-D camera. Then, the force intensities can be computed by solving an inverse quasi-static problem based on matching the FE model with the point cloud of the soft robot. As for validation, the proposed strategies for calibration and force sensing have been tested using a parallel soft robot driven by four cables.

Benefiting from the deformability of soft robots, calibration, and force sensing for soft robots are possible using an external vision-based system, instead of embedded mechatronic force sensors. In this letter, we first propose a calibration method to calibrate both the sensor-robot coordinate system and the actuator inputs. This task is addressed through a sequential optimization problem for both variables. We also introduce an external force sensing system based on a real-time finite element (FE) model with the assumption of static configurations, and which consists of two steps: force location detection and force intensity computation. The algorithm that estimates force location relies on the segmentation of the point cloud acquired by an RGB-D camera. Then, the force intensities can be computed by solving an inverse quasi-static problem based on matching the FE model with the point cloud of the soft robot. As for validation, the proposed strategies for calibration and force sensing have been tested using a parallel soft robot driven by four cables.

Catheter-based intervention plays an important role in minimally invasive surgery. For the closed-loop control of catheter robot through contacts, the loss of contact sensing along the entire catheter might result in task failure. To deal with this problem, we propose a decoupled motion control strategy which allows to control insertion and bending independently. We model the catheter robot and the contacts using the Finite Element Method. Then, we combine the simulated system and the real system for the closed-loop motion control. The control inputs are computed by solving a quadratic programming (QP) problem with a linear complementarity problem. A simplified method is proposed to solve this optimization problem by converting it into a standard QP problem. Using the proposed strategy, not only the control inputs but also the contact forces along the entire catheter can be computed without using force sensors. Finally, we validate the proposed methods using both simulation and experiments on a cable-driven continuum catheter robot for the real-time motion control through contacts.

Catheter-based intervention plays an important role in minimally invasive surgery. For the closed-loop control of catheter robot through contacts, the loss of contact sensing along the entire catheter might result in task failure. To deal with this problem, we propose a decoupled motion control strategy which allows to control insertion and bending independently. We model the catheter robot and the contacts using the Finite Element Method. Then, we combine the simulated system and the real system for the closed-loop motion control. The control inputs are computed by solving a quadratic programming (QP) problem with a linear complementarity problem. A simplified method is proposed to solve this optimization problem by converting it into a standard QP problem. Using the proposed strategy, not only the control inputs but also the contact forces along the entire catheter can be computed without using force sensors. Finally, we validate the proposed methods using both simulation and experiments on a cable-driven continuum catheter robot for the real-time motion control through contacts.

Numerical simulation of pelvic system could lead to a better understanding of common pathology through objective and reliable analyses of pelvic mobility according to mechanical principles. In clinical context, patient‐specific simulation has the potential for a proper patient‐personalised cure. For this purpose, a simulable 3D geometrical model, well suited to patient anatomy, is required. However, the geometric modelling of pelvic system from medical images (MRI) is a complex operator‐dependent and time‐consuming process, not adapted to patient‐specific applications. This paper is addressing this challenging computational problem. The objective is to develop a technique, providing a smooth, consistent, and readily usable 3D geometrical model, seamlessly from image to simulation. In this paper, we use a generic topologically‐simplified B‐Spline model to represent pelvic organs. The presented paper develops a Virtual Image Correlation method to find the best correlation between the geometry and the image. The final reconstructed geometrical model is to be compatible with meshing and finite element simulation. Then, a variety of tests are performed to prove the concept, through both prototypical and pelvic models. Finally, since the pelvic system is complex, including structures hardly identifiable in MRI, some feasible solutions to introduce more complex pelvic models are also discussed.

In this letter, we propose a new framework of external force sensing for soft robots based on the fusion of vision-based measurements and finite element model (FEM) techniques. A precise mechanical model of the robot is built using real-time FEM to describe the relationship between the external forces acting on the robot and the displacement of the predefined feature points. The position of these feature points on the real robot is measured using a vision system and is compared with the equivalent feature points in the finite element model. Using the compared displacement, the intensities of the external forces are computed by solving an inverse problem. Based on the developed FEM equations, we show that not only the intensities but also the locations of the external forces can be estimated. A strategy is proposed to find the correct locations of external forces among several possible ones. The method is verified and validated using both simulation and experiments on a soft sheet and a parallel soft robot (both of them have nontrivial shapes). The good results obtained from the experimental study demonstrate the capability of our approach.

In this letter, we propose a new framework of external force sensing for soft robots based on the fusion of vision-based measurements and finite element model (FEM) techniques. A precise mechanical model of the robot is built using real-time FEM to describe the relationship between the external forces acting on the robot and the displacement of the predefined feature points. The position of these feature points on the real robot is measured using a vision system and is compared with the equivalent feature points in the finite element model. Using the compared displacement, the intensities of the external forces are computed by solving an inverse problem. Based on the developed FEM equations, we show that not only the intensities but also the locations of the external forces can be estimated. A strategy is proposed to find the correct locations of external forces among several possible ones. The method is verified and validated using both simulation and experiments on a soft sheet and a parallel soft robot (both of them have nontrivial shapes). The good results obtained from the experimental study demonstrate the capability of our approach.

Endovascular interventions can benefit from interactive simulation in their training phase but also during pre-operative and intra-operative phases if simulation scenarios are based on patient data. A key feature in this context is the ability to extract, from patient images, models of blood vessels that impede neither the realism nor the performance of simulation. This paper addresses both the segmentation and reconstruction of the vasculature from 3D Rotational Angiography data, and adapted to simulation: An original tracking algorithm is proposed to segment the vessel tree while filtering points extracted at the vessel surface in the vicinity of each point on the centerline; then an automatic procedure is described to reconstruct each local unstructured point set as a skeleton-based implicit surface (blobby model).

Female pelvic disorders have a large social impact; the diagnosis of which relies on a key indication: pelvic mobility. The normal mobility is present in a healthy patient, meanwhile the hypermobility can be a sign of female pelvic prolapse and the hypomobility for endometriosis. The evaluation of pelvic mobility is based on medical image analysis. However, the latter does not provide precise values of these indicators directly. Moreover, suspension devices play an important role in pelvic organ function but can hardly be observed on medical images. Our objective is to propose an image‐based analysis tool for the quantitative evaluation of pelvic mobility and the shear strain which has an impact on suspension devices.

The technological differences between traditional robotics and soft robotics have an impact on all of the modeling tools generally in use, including direct kinematics and inverse models, Jacobians, and dynamics. Due to the lack of precise modeling and control methods for soft robots, the promising concepts of using such design for complex applications (medicine, assistance, domestic robotics, etc.) cannot be practically implemented. This paper presents a first unified software framework dedicated to modeling, simulation, and control of soft robots. The framework relies on continuum mechanics for modeling the robotic parts and boundary conditions like actuators and contacts using a unified representation based on Lagrange multipliers. It enables the digital robot to be simulated in its environment using a direct model.

In this paper, we propose a strategy for the control of soft robots with visual tracking and simulation-based predictor. A kinematic model of soft robots is obtained thanks to the Finite Element Method (FEM) computed in real-time. The FEM allows to obtain a prediction of the Jacobian matrix of the robot. This allows a first control of the robot, in the actuator space. Then, a second control strategy based on the feedback of infrared cameras is developed to obtain a correction of the effector position. The robust stability of this closed-loop system is obtained based on Lyapunov stability theory. Otherwise, to deal with the problem of image features (the marker points placed on the end effector of soft robot) loss, a switched control strategy is proposed to combine both the open-loop controller and the closed-loop controller.

This paper aims at providing a novel approach to modeling and controlling soft robots. Based on real-time Finite Element Method (FEM), we obtain a globally defined discrete-time kinematic model in the workspace of soft robots. From the kinematic equations, we deduce the soft-robot Jacobian matrix and discuss the conditions to avoid singular configurations. Then, we propose a novel observer based control methodology where the observer is built by Finite Element Model in this paper to deal with the control problem of soft robots. A closed-loop controller for position control of soft robot is designed based on the discrete-time model with feedback signal being extracted by means of visual servoing. Finally, experimental results on a parallel soft robot show the efficiency and performance of our proposed controller.

During Cochlear Implant Surgery, the right placement of the implant and the minimization of the surgical trauma to the inner ear are an important issue with recurrent fails. In this study, we reproduced, using simulation, the mechanical insertion of the implant during the surgery. This simulation allows to have a better understanding of the failing cases: excessive contact force, buckling of the implant inside and outside the cochlea. Moreover, using a patient-specific geometric model of the cochlea in the simulation, we show that the insertion angle is a clinical parameter that has an influence on the forces endured by both the cochlea walls and the basilar membrane, and hence to post-operative trauma. The paper presents the mechanical models used for the implant, for the basilar membrane and the boundary conditions (contact, friction, insertion etc...) and discuss the obtained results in the perspective of using the …

n the context of the female pelvic medicine, non‐invasive magnetic resonance imaging is widely used for the diagnosis of pelvic floor disorders. Nowadays, in the clinical routine, diagnoses rely largely on human interpretation of medical images, on the experience of physicians, with sometimes subjective interpretations. Hence, image correlation methods would be an alternative way to assist physicians to provide more objective analyses with standard procedures and parametrisation for patient‐specific cases. Moreover, the main symptoms of pelvic system pathologies are abnormal mobilities. The finite element model simulation is a powerful tool for understanding such mobilities. Both the patient‐specific simulation and the image analysis require accurate and smooth geometries of the pelvic organs. This paper introduces a new method that can be classified as a model‐to‐image correlation approach.

This paper introduces a new methodology for semi-automatic registration of anatomical structure deformations. The contribution is to use an interactive inverse simulation of physics-based deformable model, computed in real time.

Virtual surgical simulators face many computational challenges: they need to provide biophysical accuracy, realistic feed-backs and high-rate responses. Better biophysical accuracy and more realistic feed-backs (be they visual, haptic.. .) induce more computational footprint. State-of-the-art approaches use high-performance hardware or find an acceptable trade-off between performance and accuracy to deliver interactive yet pedagogically relevant simulators. In this paper, we propose an interactive vascular neurosurgery simulator that provides bi-manual interaction with haptic feedback. The simulator is an original combination of states-of-the-art techniques that allows visual realism, bio-physical realism, complex interactions with the anatomical structures and the instruments and haptic feedback. Training exercises are also proposed to learn and to perform the different steps of intracranial aneurysm surgery (IAS). We assess the performance of our simulator with quantitative performance benchmarks and qualitative assessments of junior and senior clinicians.

A ce jour, en routine clinique, grâce aux technologies avancées d'imagerie par résonance magnétique (IRM), le diagnostic des troubles du système pelvien chez la femme dépend de l'analyse d'images IRM par les médecins avec un risque de subjectivités. La simulation éléments finis est un outil prometteur pour l'aide à la compréhension qui, entre autre basée sur l'IRM, peut réduire la subjectivité des analyses. Pour cela, nous introduisons une méthode permettant d'identifier semi-automatiquement les organes pelviens observables sur des images IRM. Ce travail permet de mettre en place des mesures objectives et quantitatives, qui aide à la modélisation géométrique du système pelvien et à l'analyse des mobilités pour les études plus approfondies. Un modèle paramétré de B-spline est utilisé comme descriptif de géométries dédiées. Ce modèle initial est recalé sur l'organe présenté dans l'image réelle par corrélation d'images virtuelles. Nous avons validé la détection (de la vessie, du vagin et du rectum) sur un jeu de données de 19 patientes, présentant des mobilités physiologiques ou pathologiques.