Shape Control of a Dual-Segment Soft Robot using Depth Vision

— Pneumatic soft robots outperform rigid robots in complex environments due to the high flexibility of their redundant configurations, and their shape control is considered a prerequisite for applications in unstructured environments. In this paper, we propose a depth vision-based shape control method for a two-segment soft robot, which uses a binocular camera to achieve 3D shape control of the soft robot. A closed-loop control algorithm based on depth vision is designed for shape compensation when subject to its own non-linear responsiveness and coupling by solving for the shape feature parameters used to describe the robot and analytically modeling the motion of curved feature points. Experimental results show that the position and angle errors are less than 2 mm and 1° respectively, the curvature error is less than 0.0001mm-1, and the algorithm has convergence performance for L-type and S-type shape reference 3D shapes. This work provides a general method for being able to adjust the shape of a soft robot without on-board sensors.


I. INTRODUCTION
In recent years, soft robots have attracted more and more attention due to their advantages in dexterous operation and safe interaction in complex unstructured environments, and have been widely used in exploration and rescue, medical surgery, seabed grasping and other fields [1].The unique infinite degree of freedom soft mechanism gives the soft robot good environmental adaptability, but also brings challenges to the precise control of its overall shape [2], especially when it navigates in unstructured environments.
To fully utilize the flexibility of soft robots and apply them in restricted environments, such as complex trajectory tracking [3], it is necessary to simultaneously control their configurations to meet the desired complex shape requirements.Parameterized curve models have been widely used in soft robots, such as the PH curve [4] or spline curve [5].In parameterized curves, Wiese et al. considered the thirdorder Hermite spline curve [6] for precise shape kinematics calculations of soft robots, while Gonthina et al. proposed a complex modeling method based on Euler parameterized curves [7], which has the advantages of high accuracy and computational efficiency but has not been used for control purposes.Existing methods calculate control points through curve arc length parameters [8] and optimize the positional errors numerically, but they are not suitable for solving feature point mapping problems and involve complex numerical calculations.
Currently, model-based methods can achieve shape control but require 3D position and virtual joint parameters as feedback information [9].In practical applications, due to the soft structure of soft robots, it is difficult to embed rigid sensors and high-cost limitations, making it sometimes impossible to directly measure 3D positions and shape parameters.In this case, vision-based shape control can provide a universal, economical, and feasible solution [10], [11], [12].
Vision-based feedback systems, due to their independence and small size, can provide visual information feedback for soft robots using cameras, and visual servo has been extensively studied [13], [14], [15].Wang et al. [16] first achieved hand-eye visual servo control of cable-driven soft robots based on kinematic modeling, and Greer et al. [17] solved the visual servo problem of soft robots by estimating the Jacobian matrix from image feedback.Considering that the endpoint position of a soft robot can be detected and calculated using a visual method, a feasible shape can be determined in 3D space, and then a visual shape control method can be designed to track the desired reference shape [18].Although parameterized curves have been proven to be feasible in 3D shape design tasks for continuum robots [19], it is still difficult to match the spatial position of currently measured feature points with the target position on the reference shape, making it difficult to control the feature points and therefore not suitable for designing the desired reference shape.
To address this issue, a constant-curvature-based threedimensional shape feature design method is proposed.Curvature-based shape description methods have been widely used in modeling soft robots [20], and this paper uses a constant-curvature-based method to describe the threedimensional shape of the robot.Then, the shape feature is determined using the solved curvature-based method to determine the desired three-dimensional shape curve.A visual shape control algorithm based on the constant-curvature feature is proposed.To perform simple and efficient visual shape control, markers are attached to the soft robot to capture its three-dimensional shape.Their spatial positions are then compared with the desired target positions to generate error feedback.Therefore, the feature matching problem is solved by constructing the shape feature.A visual control algorithm www.ijacsa.thesai.orgbased on inverse kinematics is designed to drive the soft robot to track the desired shape feature.
The main contributions of this paper are twofold.Firstly, in the case of reconstructing the three-dimensional coordinates of feature endpoints, the expected reference space shape feature is solved, and the shape curvature feature is introduced to solve the challenge of point feature correspondence, reconstructing the center curve of the soft robot.Second, using the solved shape feature, a three-dimensional visual shape algorithm is designed.In the remainder, the organization of this paper is as follows, Section II and Section III respectively explain the design of the soft robot's spatial shape feature and visual shape control algorithm, and Section IV performs experimental verification.The conclusion of this study is presented in Section V.

II. SHAPE FEATURE DESIGN
The prototype soft body robot is shown in Fig. 1(a).The two-segment soft robot is made of silicone, both with a crosssectional diameter of 30 mm and an overall length of 410 mm.The soft robot consists of two segments, each controlled individually by three air hoses, shown in Fig. 1(b).The robot can be made to perform specified movements by adjusting the air pressure variables, Overall view of the actuation system is shown in Fig. 1(c).In practical applications of soft robots in unstructured environments, it may sometimes be impossible to obtain global pose and shape information for executing control tasks.In light of this, previous research has demonstrated the feasibility of vision systems in measuring pose and sensing shape, inspiring the development and application of effective visual control algorithms in such scenarios.To address the shape design and vision-based shape control problems of soft robots, a solution is proposed using a method based on constant curvature for shape feature design, which utilizes the given three-dimensional endpoint conditions that are capable of estimating the shape curve.Curvature has been widely used in academia to study the shape of soft continuum robots [21].The objective of the curvature feature design task is to determine feasible reference shape features under the constraint of given endpoints.The method used is to establish a constant curvature motion model based on the given feature endpoints to solve the three-dimensional shape feature parameters of the curve and thus determine the shape curve of the soft robot.
Based on the geometric characteristics of soft robots, a series of continuous circular arcs are used to approximate smooth curves in this study.Therefore, the reference shape of the soft robot is considered to be approximated by a finite group of continuous constant curvature in free space [22], [23].As shown in Fig. 2, the expression of the robot shape feature is written as the following function:  , , , , , ]  [ , , , , , ]  [ , , , , , ] where K, L,  and  represent the vectors of curvature, arc length, deflection angle, and bending angle of the soft robot, respectively.The endpoint position can be represented using the chain rule of homogeneous transformation matrices, which has been extensively proven in the field of soft robotics [24].Therefore, the curvature, arc length, bending angle, and deflection angle of each segment can be solved by using the endpoint position, and the function relationship between the endpoint coordinates and K,  and  can be directly obtained.
where (x e , y e , z e ) and (x s , y s , z s ) denote the 3D spatial shape curve endpoints and start points.This section therefore solves for the curvature features and generates a soft robot reference shape curve by giving constraints on the spatial endpoints.

III. SHAPE CONTROL
This section discusses the design of a shape control algorithm based on deep vision, which can drive a soft robot to track the 3D reference shape solved in the first section.Typically, the main challenge faced by visual shape control www.ijacsa.thesai.orgtasks is that point feature-based visual shape control algorithms are unable to address feature correspondence issues [25].
Shape control based on deep vision requires matching the currently measured shape parameters with the given expected shape parameters, which generates shape parameter errors.When considering the expected reference shape given in space and the nonlinear response characteristics of the soft robot itself [26], the relationship between shape features and robot motion becomes complex, making it difficult to establish feature correspondence between the reference point features and the detected three-dimensional points of the robot shape.This hinders the design of visual shape controllers.To address this problem, a visual shape control algorithm based on constant curvature is proposed, which uses curvature, endpoints, and inflection points as reference shape features, overcoming the obstacle of finding point feature correspondences.

A. Shape Mapping Models
This section establishes a mapping model for shape features, laying the foundation for the design of visual shape control algorithms.A prototype of a two-segment soft robot driven by a six-pneumatic circuit is selected as the experimental platform.Three points are marked on the crosssection of each segment as features to adjust their motion in the spatial coordinate system.In the shape control scheme based on deep vision, the coordinates of these features should be compared with their expected reference coordinates to generate shape errors, which are then used to calculate the drive for closed-loop control.However, due to the difficulty in matching the reference position of the feature points with the currently measured position during the motion of the soft robot, visual shape control based on point features becomes challenging.This paper simplifies the solution of feature point correspondence by constructing shape curvature features.It is known that three feature points can uniquely determine the curvature, bending angle, and deflection angle of a circular arc, and these three feature points can be directly measured, as shown in Fig. 3, where and are the starting and ending points of the soft robot, respectively.Therefore, in general for the shape control task, we analyze the construction of the i th shape feature.
Curvature, bending angle, and deflection angle are introduced as spatial curve shape features, while the inflection points and endpoints of the curve also determine the unique pose of the soft robot.Therefore, the expected reference shape feature parameters contain the reference curvature, bending angle, deflection angle, inflection point position, and endpoint position.These three-dimensional shape feature parameters can be solved based on the shape feature design task in the previous section.The reference spatial shape feature vector s d is defined as follows: , where f d contains the combination of n curvatures, bending angles, and deflection angles calculated according to the constant curvature design algorithm, and p d contains n-1 inflection points p md and two visual detection starting points p s and ending points p e .Therefore, 2n+1 features are used in the visual shape control task.The currently measured spatial shape feature vector is given by the following equation.
The shape feature parameter error is defined as d E s s  , based on the given shape feature mapping, a visual-based three-dimensional shape control algorithm is developed, which uses the calibrated binocular camera to feedback the robot's 3D motion information, so that its actual shape finally converges to the reference shape.The schematic diagram of the control task is shown in Fig. 3, where represents the three marked points on the i th segment, corresponds to the starting point, and corresponds to the ending point, used to construct the i th segment's shape feature.

B. Control Scheme
The control objective is to drive the soft robot to track the desired reference shape generated in the first part.This paper selects curvature, bend angle, deflection angle, inflection point, endpoint, and start point as the shape features to solve the correspondence problem between measured point features and their reference values.The number of selected shape features is m=2n+1, which drives the soft robot to converge to the desired shape.In the control task, the desired shape features are time-invariant.Based on the method of describing soft robot shape features using constant curvature, a vision-based inverse kinematics shape control scheme has been developed as shown in Fig. 4. It should be noted that the endpoint of the soft robot is a key point used for shape control, and the measured endpoint position can be used for shape information feedback.( , , )  For soft robot, there is an unknown variable J which can be used to establish the mapping between shape parameters and actuation of soft robot () Given the expected shape parameter s dn , the corresponding expected induced pneumatic can be calculated as 1 () where, J values of different shapes are different.After iteration, J is updated as J n , which represents the unique value of iteration for the n th time, thus, it can be obtained A dn 1 () The challenge of this solution is how to accurately find J n of different shapes.To solve this problem, a nonlinear mapping function is designed to make ( , , ) According to the known shape characteristic information, the method based on the inverse kinematics model and the empirical fitting formula is implemented to find the required actuating input, by updating the iterative feedback shape information to close to the desired shape ( , , ) Under the non-extendable cavity hypothesis, as shown in Fig. 2, the relationship between a specific actuating length L n,i and the segment shape parameter ( , , ) ] Where {1, 2,3} i  denotes the number of the chamber ,, denotes the trichotomy Angle of the circle, and the cross-section radius of the chamber is d.Let's rewrite the above equation According to the fitting of several experiments, it can be seen that the induced pressure has an approximate linear relationship with the chamber elongation, that is nn AL  , the () Where 0.2

 
is the proportionality coefficient.

IV. CONTROL EXPERIMENTS
This section aims to verify the effectiveness of the proposed design of the soft robot shape features and the vision-based shape control algorithm.The experimental setup is shown in Fig. 6(a).A prototype of a two-segment soft robot driven by six pneumatic actuators is selected as the experimental platform for algorithm verification.Three different colored markers are labeled on each segment to construct the required shape features.A calibrated binocular camera is used to detect the motion and shape of the soft robot, and provide visual feedback to the controller by perceiving the spatial position of the marker feature points.The image processing framework process is shown in Fig 5 , and the 3D reconstruction of the two-segment soft robot is based on the detected feature point positions.First, the RGB image of the soft robot is binarized, and the feature points are identified using color reduction and median filtering algorithms.The corresponding labels are assigned to the shape feature endpoints, and the shape reconstruction is completed by combining stereo depth information.The shape control objective is to drive the dual-segment soft robot to converge to a specified expected shape, using the known endpoint and inflection points as shape features and controlling the robot with the control algorithm.In the verification process, the soft robot base and binocular camera are fixed, ensuring that the starting point of the detected curve is consistent.Given the endpoint positions of the dual-segment soft robot in spatial coordinates, the feature curvature K, bending angle θ, and deflection angle φ can be solved using Eq. ( 2), and the solved 3D shape features are used as the reference input for the shape controller, which can uniquely control the output robot reference shape.As shown in Fig. 6(b), L-shaped and S-shaped expected reference shapes can be tracked by detecting the three-dimensional positions of the feature endpoints through depth vision, allowing for the reconstruction of the soft robot's arbitrary omnidirectional motion configurations.Table I provides the calculated results of curvature K, bending angle θ, and deflection angle φ for the L-type and S-type shape features.The proposed shape control www.ijacsa.thesai.orgalgorithm is then used to verify the performance of the soft robot tracking the L-type and S-type shapes.L-type shape control: As shown in Fig. 7, the performance of L-type shape control was verified.Fig. 7(a) shows the captured images during the L-type shape control task.In these images, the curve labeled with orange represents the initial shape, the red curve represents the expected shape, and the blue curve represents the actual final shape, demonstrating the ability of the soft robot to achieve the expected 2D shape motion using the proposed control algorithm.To verify the accuracy and convergence ability of the proposed controller, the error curves of endpoint position, turning point position, bending angle, and curvature with respect to the iteration number are also displayed in Fig. 7(b) to (d).The error in shape feature parameters with respect to the expected shape is ΔK 1 =5.121e-5mm -1 , Δθ 1 =0.12,ΔK 2 =2.822e-5mm -1 , and Δθ 2 =0.64°.The shape feature error of the soft robot converged after the second iteration of feedback using visual feedback of the shape feature information.It is noteworthy that the iteration number refers to the finite number of times used to achieve the expected shape using visual feedback of the shape feature information.S-type shape control: The S-type shape means that the two segments deform in opposite directions.Fig. 8 shows the experimental results of the visual shape control performance of the S-type shape.Fig. 8(a) shows images captured from the camera and the captured three-dimensional shape curve during the control process.Compared to L-type shape control, the shape error is relatively significant, possibly due to the motion interference caused by the two segments with opposite deformations, where while the proximal part approaches the expected shape, the shape error of the distal part may increase.Fig. 8(b) to (e) show the errors of the endpoint positions, inflection point positions, bending angles, deflection angles, and curvatures of the two segments with iteration number.The final shape parameters of the soft robot also tend to be stable, with the errors of shape feature compared to the expected shape being ΔK 1 =0.0004mm -1 , Δφ 1 =0.69°,Δθ 1 =0.65°, and ΔK 2 =0.0004mm -1 , Δφ 2 =0.996°,Δθ 2 =0.39°.That is, the error of the soft robot shape curve also converges after the second iteration feedback.

V. CONCLUSION AND DISCUSSION
In this paper, a shape design algorithm and visual shape control scheme for a dual-segment soft robot in threedimensional space were proposed.By using marker features, the problem of reference shape design and control was solved solely through visual feedback.In the shape tracking experiment, the spatial shape deviation of the dual-segment soft robot was obtained using depth vision, and the visual perception of the endpoint and inflection point was used as closed-loop feedback for shape compensation.With this method, the overall configuration of the robot was adjusted by controlling its shape features without adding any other onboard sensors.It has good effect in shape servo control of L-type and S-type shapes.The motion information of the robot is directly captured by visual sensing to generate the threedimensional shape of the soft robot.In order to control the soft robot into the desired shape, the feedback and control of its shape feature parameters are transformed.The results showed that the position and angle errors of the shape feature were both less than 2mm and 1°, respectively, and the curvature error was less than 0.0001 mm -1 .This method exhibited good precision tracking and convergence performance for the required L-type and S-type shapes.The problem of reference shape design and servo for continuously deformed soft robot with only visual feedback is solved.This work demonstrates that soft robots have unlimited applications in future explorations in rescue and minimally invasive surgery fields, enabling safer human-machine interactions.In the future work, especially for the shape dynamic control task of soft robots, an efficient 3D curve tracking method and a model for real-time calculation of shape feature parameters are needed to conduct dynamic response analysis of the shape control system, so as to propose a better shape control strategy to cope with the challenges brought by coupled motion.It will also further promote the research of accurate shape modeling and algorithm design, and the extension of dynamic control of soft robot shape will also be used in more complex interactive environments and fast response control tasks.

Fig. 1 .
Fig. 1.Prototype of an air-driven soft-bodied robot.(a) Dual-segment soft robot.(b) Sketch of soft robot.(c) Overall view of the actuation system.


represents the expected shape feature parameters, and dn A represents the theoretical actuation output calculated by the controller based on the given expected shape parameter error ( , , ) KL     .www.ijacsa.thesai.org

Fig. 5 .
Fig. 5. Stereo vision tracking and 3D reconstruction of a soft robot.