In urban traffic, accurate prediction of pedestrian trajectory and advanced collision avoidance strategy can effectively reduce the collision risk between intelligent vehicles and pedestrians. In order to improve the prediction accuracy of pedestrian trajectory and the safety of collision avoidance, a longitudinal and lateral intelligent collision avoidance strategy based on pedestrian trajectory prediction is proposed. Firstly, the process of a pedestrian crossing the road is considered as a combination of free motion described by first-order Markov model and the constrained motion presented by improved social force model. The predicted pedestrian trajectory is obtained by weighted fusion of the trajectories of the two models with a multiple linear regression algorithm. Secondly, according to the predicted pedestrian trajectory and time to collision (TTC) the longitudinal and lateral collision avoidance strategy is designed. The improved artificial potential field method is used to plan the lateral collision avoidance path in real time based on the predicted pedestrian position, and a fuzzy controller is constructed to obtain the desired deceleration of the vehicle. Finally, the pedestrian motion fusion model and the longitudinal and lateral collision avoidance strategy are verified by Prescan and Simulink co-simulation. The results show that the average displacement error (ADE) and final displacement error (FDE) of pedestrian trajectory based on pedestrian motion fusion model are smaller compared with a Markov model and improved social force model, and the proposed pedestrian collision avoidance strategy can effectively achieve longitudinal and lateral collision avoidance.

In urban road scenes, pedestrians crossing roads are likely to have traffic accidents involving vehicles and usually suffer serious injuries. Reports show that, in recent years, the casualty rate of pedestrians in traffic accidents with motor vehicles is more than 50 % (Wang et al., 2019; Saito and Raksincharoensak, 2016). Therefore it is necessary to develop a pedestrian collision avoidance control strategy considering pedestrian trajectory prediction. This provides a reliable basis for intelligent vehicles to distinguish the intention and interaction of pedestrians crossing the road, and make a prejudgment to avoid potential collisions, which is of great significance to improve the safety of pedestrian crossing the road and traffic efficiency.

Recent research on pedestrian collision avoidance has mainly focused on two aspects: pedestrian trajectory prediction and collision avoidance control strategy (Narváez et al., 2019; Sighencea et al., 2021). Due to pedestrian collision risk, the accuracy of pedestrian trajectory prediction has an important impact on the safety of vehicle collision avoidance. Domestic and foreign research on pedestrian trajectory prediction is mainly classified as data-driven method or the model-based method. The data-driven method is mainly based on the improvement of the recurrent neural network to generate related variants (Eiffert et al., 2020; Song et al., 2020). Some researchers have optimized the network structure and loss function in the model to improve operational efficiency and accuracy. Hassan et al. (2021) introduced social attention mechanism and physical attention mechanism into generative confrontation networks and considered scene context information and historical trajectory information to realize trajectory prediction under the interaction of multiple intelligent bodies. Zhou et al. (2021) constructed the trajectory prediction model of a graph convolutional network to describe the interaction mode between pedestrians, so that the predicted trajectory conforms to the habits and behaviors of pedestrians. The model-based methods often generate future trajectories based on historical time series data according to the designed mathematical model (Keller and Gavrila, 2014). Compared with the data-driven method, the model-based method is more explanatory and simpler, which can describe pedestrian movement behavior in detail. Aiming at the interaction between a pedestrian and a vehicle at an intersection with mixed traffic flow, Zhang et al. (2020) established a pedestrian decision-making model by using logic regression and proposed an improved social force model to predict the trajectories of pedestrian. Based on the characteristics of Markov, Vasquez et al. (2009) proposed a hidden Markov model to predict trajectories of a pedestrian and a vehicle in the current environment by combining machine learning and semantic information. Wang et al. (2018) established a fuzzy logic system to estimate the transition probability between the pedestrian state and the motion model, and used a Kalman filter to predict the trajectory of pedestrian in a short time. Most of the studies mentioned above focus on intersection scenarios with signal lights and crosswalks, as well as the improvement of the basic model. However, the existing pedestrian motion models are not comprehensive enough to analyze interactive information and cannot be directly applied, which requires detailed modeling in combination with actual scenes to improve prediction accuracy. In terms of pedestrian collision avoidance control strategy, there are two main ways of collision avoidance for intelligent vehicles: longitudinal collision avoidance and lateral collision avoidance (Zhang et al., 2022; Gao et al., 2021). When there is collision risk, the collision avoidance system actively sends warning signals to the driver and controls the vehicle to brake or steer according to the risk indicators to avoid pedestrian injury. More and more researchers are focusing on how to improve the effectiveness and rationality of collision avoidance strategies. Based on velocity obstacle theory, Wu et al. (2019) proposed a real-time dynamic path planning collision avoidance method for pedestrians crossing the road in the environment of cooperative vehicle–infrastructure system to reduce the pedestrian collision risk. Chen and Zhang (2022) proposed a data-driven fusion model of attention mechanism long short-term memory (LSTM) network and modified social force model for pedestrian path prediction. According to the pedestrian safety area, the front wheel angle was calculated in real time to plan the collision avoidance path and tracked it based on model predictive control (MPC) theory. Yang et al. (2019) presented a hierarchical autonomous emergency braking pedestrian system, which built the upper-layer controller of a fuzzy neural network by introducing a genetic algorithm, and designed the lower-layer controller based on proportional–integral–derivative (PID) theory. Considering that road users were affected by perceived location and prediction errors, Themann et al. (2015) put forward a collision avoidance system that optimized the longitudinal and lateral trajectory. Most of the current studies only consider single lateral or longitudinal collision avoidance and few studies refer to a comprehensive longitudinal and lateral collision avoidance strategy according to the pedestrian motion state, which makes the pedestrian collision avoidance strategies less applicable and reliable.

On the basis of the previous analysis, a pedestrian collision avoidance control strategy is proposed in this paper based on pedestrian trajectory prediction in the scene of a pedestrian crossing the road without signal lights or crosswalks. Firstly, the pedestrian motion model based on the fusion of a Markov model and improved social force model is constructed in which the Markov model is applied to simulate random walking of a pedestrian in a free state and the improved social force model is established to represent the interaction between a pedestrian and the surrounding environment. Secondly, the parameters of pedestrian motion fusion model are calibrated based on the maximum likelihood estimation method. The pedestrian trajectories predicted by the two models are fused by a multiple linear regression algorithm. Then the risk of pedestrian–vehicle collision is evaluated based on the predicted pedestrian trajectory. Considering predicted pedestrian location and time to collision (TTC) as well as the road scene, a longitudinal and lateral collision avoidance strategy is designed, which can actively conduct lateral avoidance or longitudinal braking. With respect to lateral collision avoidance, the improved artificial potential field method is used to plan the collision avoidance path for the intelligent vehicle. In the aspect of longitudinal collision avoidance, the desired deceleration of the vehicle is obtained by a fuzzy control method. Finally, the accuracy of the predicted pedestrian trajectory based on the pedestrian motion fusion model and the feasibility of collision avoidance control strategy are verified by Prescan and MATLAB co-simulation.

Pedestrians usually tend to walk freely and attempt to identify a destination without disturbance from the external environment. During the procedure, the speed and direction of each step would vary with the state of the previous moment. Therefore, a first-order Markov chain is selected to describe the randomness of the pedestrian movement process (Yuan et al., 2023). According to the principle of Markov, the position and speed of the pedestrian in the following step can be determined by the current position and speed. The coordinate system in this paper is the global coordinate system, in which the

The speed and position of the pedestrian in the

Similarly, the speed and position of the pedestrian in the

Thus, the pedestrian position at

According to a social force model (Wu et al., 2022), the moving pedestrian is considered as a particle that confirms to the law of mechanics, and the relationship between the pedestrian and the surrounding traffic participants can be expressed by mechanics equations. The classical social force model includes three forces: pedestrian self-driving force, pedestrian interaction force, and interaction force with boundary or obstacle. Based on the basic theory, the social force model in this paper is improved for applications in urban scenes without crosswalks or signal lights. The resultant force of the pedestrian is shown in Fig. 1, and the improved social force model (I-SFM) is built as follows:

Schematic of pedestrian forces.

In Eqs. (9)–(12),

A pedestrian crossing the road is attracted to their destination and will take the shortest path to reach it at the desired speed. When the surrounding environment interferes in the pedestrian crossing process, the deviation between the actual speed and the desired speed occurs and the pedestrian's walking direction varies, which causes the self-driving force to automatically adjust the current speed to the desired speed. The expression of self-driving force is presented as follows:

Usually the crossing behavior of pedestrian is influenced by surrounding pedestrians. When surrounding pedestrians are closer to the pedestrian, the pedestrian will avoid collision, stop, and so on. In order to meet their own motion space, the virtual repulsive force will act on the pedestrian to maintain a certain distance from surrounding pedestrians. According to the classical social force model, the repulsive force between pedestrian

A pedestrian crossing the road will not only be disturbed by surrounding pedestrians but also by conflict with vehicles. The pedestrian usually tries their best to avoid collision with approaching vehicles. Since the speed of vehicles is much faster than that of the pedestrian, an elliptical potential field (Zeng et al., 2014) that describes the difference in speed is used to represent the repulsive force of the approaching vehicle on the pedestrian. The repulsive force is shown as follows:

When a pedestrian crosses the road, the uncertainty of the environment plays an important role in pedestrian motion. The Markov model can better predict the random and free movement of pedestrians without interference, while the social force model can better predict the interfered motion by the surrounding environment. To combine the advantages of the Markov model and social force model, a pedestrian motion fusion model is constructed to reflect the characteristics of pedestrian motion, as shown in Fig. 2.

In the first layer, the Markov model and the improved social force model are used to calculate the speed and position of the pedestrian at each time step respectively according to the conditions of the initial moment and obtain the pedestrian trajectory in future time through iteration.

The predicted pedestrian trajectory datasets are obtained by the Markov model and improved social force model and are recorded as

The optimization problem is established by defining the following loss function, which can be solved by the least squares method (Lenth, 2016) to obtain the weight coefficients

Since the calculated weights are positively correlated with the performance of each model, the weighted fusion formula is obtained by multiplying the weights with the predictions of each model.

Process of pedestrian trajectory prediction based on model fusion.

In order to calibrate parameters of the Markov model and improved social force model, and to verify the performance of the pedestrian motion fusion model established in this paper, real pedestrian walking data are collected. The data collection scene is a mixed pedestrian–vehicle road without signal lights or crosswalks. The test section is two-lane road with a width of 11 m. The test time is 08:30–10:30 and 14:30–16:30 LT (local time).

The main test device is an HD camera to collect the image data of the pedestrian crossing the road. The process of data collection and the results are shown in Figs. 3 and 4. The HD camera takes a video of the pedestrian crossing the road and then extract continuous frame images. The pedestrian in the frame image is detected by histogram of oriented gradients (HOG) and support vector machines (SVM), and the target position of each frame in the continuous image sequence is tracked by a Kalman filter. Finally, the coordinates of the position in the image coordinate system are converted into ground coordinates by a perspective transformation matrix method (Barone et al., 2020), so as to filter and analyze the real data.

Flow chart of pedestrian trajectory data collection.

Results of pedestrian trajectory data collection.

The parameter value in the model established in this paper determines the accuracy of the model. The measurable parameters can be obtained directly from real data, while the unmeasurable parameters need to be calculated by statistical methods. In this paper, the unmeasurable parameters are calibrated by the maximum likelihood estimation method (Ko et al., 2013; Mirabella et al., 2023). The measurable parameters include the following: pedestrian mass is

Assuming that the parameter set to be calibrated in the model is

Parameters calibration results.

To ensure the comfort and safety of vehicle collision avoidance, a longitudinal and lateral pedestrian collision avoidance strategy is designed considering the predicted trajectory of a pedestrian crossing the road. The collision avoidance strategy is as shown in Fig. 5.

The strategy for longitudinal and lateral collision avoidance.

In Fig. 5, the road and traffic information is perceived by sensors of the vehicle. If there are vehicles in the other lane, the lateral collision avoidance will pose a threat to the traffic participant on the road, so the longitudinal collision avoidance is implemented to ensure enough safety of the pedestrian in this lane and vehicles in other lanes. It is known that the position of the pedestrian crossing the road influences vehicle collision avoidance decision-making. The pedestrian crossing area is divided into a high-risk area in front of the vehicle, a potential-risk area from the right edge of the vehicle to the road edge, and a safe area, as shown in Fig. 6. The way of collision avoidance is adopted according to the predicted pedestrian location in high-risk or potential-risk areas.

When the pedestrian is in front of the vehicle, there is a large collision risk. The risk level is determined by the longitudinal collision avoidance time of vehicle which is relative to the vehicle speed. The longitudinal collision avoidance time is defined as

According to the pedestrian motion fusion model described in Sect. 2, the predicted pedestrian trajectory can be obtained. Substituting Eq. (32) into Eq. (25), the predicted position of the pedestrian in the longitudinal collision avoidance time is calculated as shown in Eq. (33).

Division of pedestrian crossing area.

The artificial potential field algorithm proposed by Khatib (1986) has been widely used in obstacle avoidance path planning. In this paper, considering that the longitudinal safety distance for intelligent vehicle is much longer than the lateral distance in the process of lateral obstacle avoidance as well as the structural parameters of the road, an improved artificial potential field is constructed for lateral collision avoidance path planning. The artificial potential field includes the gravitational potential field of the road centerline, the repulsive potential field of the road boundary, and the elliptical obstacle repulsion potential field. Combining the updated pedestrian position predicted by the pedestrian motion fusion model in the planning period, the position of the vehicle in the process of lateral obstacle avoidance can be obtained by solving the artificial potential field force balance equation to realize the dynamic path planning of the vehicle.

With respect to the traditional artificial potential field method, the point is usually taken as the gravitational target. The vehicle usually drives along the road centerline under conditions of normal driving. Therefore, the gravitational potential field of the road centerline is constructed with lane centerline as the gravitational target.

The repulsive force potential field of the road boundary is constructed to prevent the vehicle from leaving the road. The repulsion force of the road boundary on the vehicle is determined by the distance between the vehicle and the road boundary:

Regarding the traditional artificial potential field method, the obstacle repulsive force field is usually a circular virtual field with the obstacle and the influence distance as the center and the radius, respectively. During lateral collision avoidance, the longitudinal speed of the vehicle is much greater than the lateral speed, and the planned collision avoidance path should satisfy the condition that the longitudinal distance is greater than the lateral distance (Ji et al., 2017). Therefore an elliptical repulsive potential field is established as follows rather than circular repulsive potential field.

In this paper, a fuzzy controller is built based on fuzzy control theory (Wang et al., 2023) to realize longitudinal collision avoidance. The relative distance and relative speed between vehicle and pedestrian are taken as the inputs of the fuzzy controller, and the controller outputs the desired deceleration to ensure the safety of longitudinal collision avoidance. The universe of relative speed is defined as

Surface of fuzzy relationship.

In order to verify the feasibility of the pedestrian motion fusion model and the longitudinal and lateral pedestrian collision avoidance control strategy proposed in this paper, a simulation platform is built based on Prescan and MATLAB/Simulink. Prescan is used to build a virtual traffic scene and provide road and pedestrian information. The pedestrian trajectory prediction module and collision avoidance control module are built in MATLAB/Simulink. The simulations are carried out under different conditions.

In this study, a total of 258 groups of pedestrian trajectories are collected as the observation dataset, of which 116 groups are used for parameter calibration and 96 groups are used for model validation. The collected trajectory data can truly reflect pedestrian crossing behavior. The pedestrian crossing scene is set in two cases: slowing down for collision avoidance and keep crossing, as shown in Fig. 9. In the simulation, the pedestrian motion fusion model established on the MATLAB platform is used to generate the predicted trajectories, which are respectively compared with the real trajectories. The performance of the model is evaluated by the average displacement error (ADE) and the final displacement error (FDE), shown as follows:

Pedestrian crossing scene.

Case 1: the pedestrian crosses the road from the starting point at the edge of the road and slows down for collision avoidance with the approaching vehicle. The initial position of the vehicle is

Comparisons of trajectory prediction applying different models in case 1.

It can be seen from Fig. 10 that the position and speed of the pedestrian predicted by the fusion model is the closest to the real pedestrian trajectory. In the crossing process, the pedestrian maintains a certain speed at the beginning and then gradually slows down when the vehicle approaches. The results predicted by the pedestrian motion fusion model designed in this paper not only consider random behavior fluctuations but also reflect the pedestrian's movement behavior affected by the surrounding environment. By calculation, the ADE and FDE of the Markov pedestrian model are 0.1697 and 0.174, respectively, the ADE and FDE of the improved social force model are 0.2373 and 0.1864, respectively, and the ADE and FDE of fusion model are 0.1103 and 0.1294, respectively. The results indicate that the pedestrian motion fusion model can truly reflect the behavior and predict pedestrian trajectory more accurately.

Case 2: the pedestrian walks from the starting point at the edge of road and towards the target point. The vehicle is far away from the pedestrian and the driving speed is slow. The initial position of the vehicle is

Comparisons of trajectory prediction applying different models in case 2.

Figure 11 shows the pedestrian trajectories predicted by different models and the relationship between the lateral/longitudinal displacement and time. In Fig. 11b and c, the longitudinal and lateral displacement under the proposed pedestrian motion fusion model is roughly consistent with the real displacement. Figure 11a shows that the predicted trajectory of the Markov model is quite different from the actual trajectory, and the predicted trajectory of the improved social force model is similar to the actual trajectory at the beginning; however, the longitudinal displacement error between the predicted position and the actual position increases gradually. The predicted trajectory of the pedestrian motion fusion model is basically consistent with the real trajectory. In order to compare performance of three models intuitively, the ADE and FDE of the predicted trajectory are calculated. The results show that the ADE and FDE of the Markov model are 0.1628 and 0.1872, respectively, the ADE and FDE of the improved social force model are 0.1685 and 0.1457, while the ADE and FDE of the pedestrian motion fusion model are 0.1158 and 0.1081, respectively. The comparison results indicate that the pedestrian motion fusion model is more accurate in pedestrian trajectory prediction than the Markov model and improved social force model, and the predicted pedestrian trajectory is closer to the actual trajectory.

To verify the effectiveness of the proposed path planning method for lateral collision avoidance, the simulation conditions are set as follows: the road includes two lanes, the width of every lane is 3.5 m, the coordinate of the vehicle's center of gravity is

Comparisons of lateral collision avoidance path at different speeds.

In Fig. 12, the red solid line is the improved path considering pedestrian trajectory prediction based on improved artificial potential field algorithm and the blue dotted line is the unimproved path without taking the information of predicted pedestrian trajectory into account during the path planning process. It can be seen from Fig. 12 that the unimproved planned path has shorter lateral collision avoidance distance and is closer to the pedestrian, which results in potential collision risk. Based on the improved artificial potential field method, the safety distance from the vehicle to the pedestrian is adjusted in real time during the planning process. The lateral deviation of the planned path is longer than that of the unimproved artificial potential field method, and the steering collision avoidance operation can be taken earlier to ensure the safety of collision avoidance. As can be seen from Fig. 12, the vehicle can plan a smooth and continuous obstacle avoidance path at different speeds and always within the road boundary, which meets the requirement of lateral collision avoidance safety.

The simulation conditions for longitudinal collision avoidance are set as follows: the pedestrian crosses the road at a certain distance from the vehicle. The results of longitudinal collision avoidance of the vehicle at the speeds of 30, 45, and 60 km h

From Figs. 13a, 14a and 15a, it can be seen that the vehicle starts to brake when it receives the signal of longitudinal collision avoidance, and completes braking at the time of 5.2, 6, and 8.1 s, respectively, with small deceleration fluctuation, which demonstrates the designed fuzzy controller can meet the control demand of smooth deceleration. In Figs. 13c, 14c, and 15c, the vehicle starts braking at a distance of 25, 35, and 65 m from the pedestrian, and keeps a safe distance of about 5 m, which verifies that the designed longitudinal collision avoidance system can ensure collision avoidance safety. From the results in Figs. 13, 14 and 15, it can be found that the pedestrian collision avoidance system functions well at different speeds, and the minimum distance between the vehicle and the pedestrian when completing braking is in the range of 2–5 m, which can better meet the requirements of pedestrian protection.

Results of longitudinal collision avoidance at 30 km h

Results of longitudinal collision avoidance at 45 km h

Results of longitudinal collision avoidance at 60 km h

Aiming to solve the problems of insufficient prediction accuracy of pedestrian trajectory and the shortcomings of pedestrian collision avoidance control methods under conditions without signal lights or crosswalks, in this paper, a pedestrian motion fusion model is constructed to predict the pedestrian trajectory by the fusion of a Markov pedestrian model and improved social force model with regression algorithm. According to the predicted pedestrian trajectory, the longitudinal and lateral pedestrian collision avoidance control strategy is established. The main conclusions are as follows.

Based on the analysis of the dynamic behavior of a pedestrian crossing the road, the behavior of the pedestrian is considered as the combination of free movement without external influences and interference movement influenced by the surrounding environment. The pedestrian motion fusion model combines the advantages of two models, which better reflects the overall distribution of pedestrian trajectory.

Compared with the Markov model, the ADE and FDE of the pedestrian motion fusion model proposed in this paper are reduced by 35.00 % and 25.63 % in the collision avoidance scene, and the ADE and FDE are reduced by 28.86 % and 42.25 % in the keep crossing scene. Compared with the improved social force model, the ADE and FDE in the collision avoidance scene are reduced by 55.87 % and 30.58 %, and ADE and FDE in the keep crossing scene are reduced by 31.28 % and 25.81 %. The comparisons indicate the pedestrian motion fusion model is more accurate in pedestrian trajectory prediction.

According to the analysis of vehicle-to-pedestrian collision risk, a longitudinal and lateral collision avoidance control strategy is developed. The simulation results show that intelligent vehicles can conduct collision avoidance in time based on the driving environment, vehicle states, and risk level under various conditions to avoid collision accidents and improve safety.

In order to further improve the applicability and performance of the algorithm, future studies can be conducted considering the following aspects.

This study aims to explore the interaction between vehicles and pedestrians crossing the street. However, in real urban road traffic, there are often multiple motor vehicles, pedestrians, and non-motor vehicles. Therefore, it is necessary to further study the pedestrian model suitable for complex scenes.

The research on lateral collision avoidance planning algorithms in this paper is relatively idealized, and the collision avoidance does not consider the speed planning of vehicle. Therefore, it is necessary to comprehensively plan the path and speed of intelligent vehicles to realize stable and safe driving in a complex environment.

The code and data included in this article can be made available by the corresponding author upon reasonable request. Please note that the data and codes are confidential and cannot be made publicly available with respect to future applications.

BT and ZY designed and performed the research, analyzed data, and wrote the paper. ZY and ZH collected and analyzed data. HJ supervised this paper.

The contact author has declared that none of the authors has any competing interests.

Publisher’s note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this paper. While Copernicus Publications makes every effort to include appropriate place names, the final responsibility lies with the authors.

Bin Tang would like to express their heartfelt gratitude to everyone who helped throughout the process of preparing this paper, especially the editors, reviewers, and the academic leader.

This research has been supported by the National Natural Science Foundation of China (grant nos. 51605199, 52225212, U20A20333), the Six Talent Peaks Project in Jiangsu Province (grant no. 2019-GDZB-084), and the Key Science and Technology Support program in Taizhou (grant no. TG202307).

This paper was edited by Zi Bin and reviewed by four anonymous referees.