Portal de Periódicos da CAPES

Joint tracking and classification of extended object based on support functions

2018; Institution of Engineering and Technology; Volume: 12; Issue: 7 Linguagem: Inglês

10.1049/iet-rsn.2017.0499

1751-8792

Lifan Sun, Jian Lan, Xuezhen LI,

Fault Detection and Control Systems

IET Radar, Sonar & NavigationVolume 12, Issue 7 p. 685-693 Research ArticleFree Access Joint tracking and classification of extended object based on support functions Lifan Sun, Lifan Sun School of Information Engineering, Henan University of Science and Technology, Luoyang, 471023 People's Republic of China Henan Key Laboratory of Robot and Intelligent Systems, Henan University of Science and Technology, Luoyang, 471023 People's Republic of ChinaSearch for more papers by this authorJian Lan, Corresponding Author Jian Lan lanjian@mail.xjtu.edu.cn Center for Information Engineering Science Research (CIESR), School of Electronics and Information Engineering, Xi'an Jiaotong University, Xi'an, 710049 People's Republic of ChinaSearch for more papers by this authorX. Rong Li, X. Rong Li Department of Electrical Engineering, University of New Orleans, New Orleans, LA, 70148 USASearch for more papers by this author Lifan Sun, Lifan Sun School of Information Engineering, Henan University of Science and Technology, Luoyang, 471023 People's Republic of China Henan Key Laboratory of Robot and Intelligent Systems, Henan University of Science and Technology, Luoyang, 471023 People's Republic of ChinaSearch for more papers by this authorJian Lan, Corresponding Author Jian Lan lanjian@mail.xjtu.edu.cn Center for Information Engineering Science Research (CIESR), School of Electronics and Information Engineering, Xi'an Jiaotong University, Xi'an, 710049 People's Republic of ChinaSearch for more papers by this authorX. Rong Li, X. Rong Li Department of Electrical Engineering, University of New Orleans, New Orleans, LA, 70148 USASearch for more papers by this author First published: 01 July 2018 https://doi.org/10.1049/iet-rsn.2017.0499Citations: 5AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract This study considers joint tracking and classification (JTC) of an extended object using measurements of down-range and cross-range extent. Using such measurements, existing approaches handle only tracking, that is estimating the kinematic state and the extension. In many practical applications, tracking and classification (e.g. classifying the object by its size and shape) are highly coupled (i.e. they affect each other) but are handled separately. For JTC of extended objects, this study deals with this problem jointly by integrating class-related extension information (i.e. the size and shape characteristics distinguishing objects of different classes) into a support function model. This facilitates the derivation of their JTC algorithm for jointly estimating the kinematic state and object extension and obtaining the probabilities of the object classes. In the proposed JTC algorithm, the useful information between the tracker and the classifier is sufficiently exchanged to improve overall performance. Furthermore, they also propose an effective method to fuse object extension estimates. The benefit of what they proposed is illustrated by simulation results. 1 Introduction In the past several decades, target tracking techniques have been studied extensively with fruitful results [1]. Most classical approaches focus on estimating the kinematic state (e.g. position, velocity, acceleration) of a point target based on uncertain measurements without considering the object extension (i.e. size, shape and orientation) because of limited sensor resolution. With the increased resolution of modern sensors, treating an object as a point mass is not necessarily reasonable. An object should be considered as extended if its target extent is much larger than the sensor resolution [2]. Extended object tracking (EOT) aims to estimate the kinematic state and object extension jointly. EOT uses not only measurements of the target centroid but also high-resolution sensor measurements. For example, some modern sensors can resolve individual features or generate measurements from different scattering centres of an extended object. Such an object can be modelled as a rigid body or a semi-rigid set of points [3]. Several models and approaches have been proposed for [4-6], including the spatial probability distribution model [7], the random hypersurface model [8, 9], the random matrix approach [10-13], the multiple hypothesis tracking [14], and the probability hypothesis density filters [15-17]. In addition, some modern surveillance sensors are able to provide the target's down-range and cross-range extent measurements on a single extended object along the line of sight (LOS) [18]. Using range extent measurements benefits track retention and recognition of convoys [19, 20] in practical applications. In this context, several existing approaches differ mainly in extended object models for the object shape [19, 21, 22]. We have proposed two approaches based on support functions (SFs) and extended Gaussian images (EGI), respectively, to model smooth and non-smooth objects in [23]. These approaches seem more promising since they are capable of modelling a variety of objects and capture more detailed shape information. For extended objects, the above approaches handle only tracking. However, classification is a critical problem. It aims to identify the target allegiance, target class and so on [24]. Tracking and classification arise in many practical applications, which have received a great deal of attention in recent years. For point targets, tracking and classification are usually treated separately. This is not necessarily good because they are coupled in many cases (i.e. they affect each other), and classification may facilitate tracking and vice versa [25]. In this context, a joint approach to tracking and classification (JTC) is more promising. JTC involves both decision (decide the object class) and estimation (estimate the object state), and they are inter-dependent. Such joint decision and estimation (JDE) are usually solved by a two-stage strategy (decision-then-estimation or estimation-then-estimation). This suffers from several serious drawbacks. For example, for decision-then-estimation, the estimation is obtained without considering possible decision errors; for estimation-then-decision, it would not work well if estimation depends on decision significantly or estimation is not secondary [25]. Thus for a JTC problem, tracking and classification need to be handled jointly. Recently, several approaches have been proposed for JTC of point targets. For example, the authors in [25] proposed a new JDE approach based on a new Bayes risk as a generalisation of those for decision and estimation. This approach considers the risks of both decision and estimation in a unified framework. It outperforms the conventional two-stage strategies and separate decision and estimation, especially for problems where decision and estimation are highly correlated. This new JDE framework was applied to an important JTC problem [26], which is superior in joint performance to the conventional two-stage optimisation strategy. To reduce the computational complexity, the authors in [27] proposed a conditional JDE risk, which is a generalised Bayes risk conditioned on on-line data to solve JTC problems. In [28], two sequential Monte-Carlo algorithms were proposed for solving JTC of a manoeuvring target using kinematic measurements only. Using measurements of radar and electronic support measure, the authors in [29] proposed a JTC approach by using the dependence of the target state on the target class via class-dependent dynamic models. The authors in [30] introduced a Bayesian framework for JTC and proposed a robust and computationally efficient algorithm-based on particle filtering. Besides, the authors in [31] proposed a framework for multiple target tracking and classification in a collaborative sensor network. To sum up, JTC aims at a dual goal: tracking and classification, which has received more and more attention in recent years [32-38]. Unlike point targets, JTC of extended objects not only infers the kinematic state and the class of the object but also estimates the object extension. The authors in [39] proposed a random-matrix approach to JTC of extended object using multiple measurements of scattering centres, in which the size and shape characteristics distinguishing objects of different classes are treated as constraints and integrated into the random-matrix framework as pseudo-measurements. The authors in [40] deal with JTC of extended objects by combining recursive JDE based on a new Bayesian risk [25, 26, 41] with the random-matrix-based multiple models for EOT [12], in which extended objects differ in manoeuvrability. To our knowledge, there is no explicit consideration in the literature for JTC of extended objects using the down-range and cross-range extent measurements. The main difficulties are summarised as follows: (i) How do we describe extended objects in different classes while using the prior size and shape information (i.e. class-related information) in a concise mathematical form? (ii) How do we jointly track and classify extended objects based on the extension information in a unified framework? To solve these problems, this paper proposes an approach to JTC of extended objects using SFs. The proposed approach not only estimates the kinematic state and object extension but also determines the class to which the object belongs. Many practical extended objects can be classified by their sizes and shapes. When classifying extended objects, the size and shape of an extended object in a class is assumed known because only several classes of the extended objects (e.g. ships and aircraft) may exist in a practical application. In this work, extended objects in different classes are represented by different extension models based on SFs. The SF-based model can describe a variety of different object extensions, in which the class-related extension information (i.e. the size and shape of an extended object in a class) is utilised effectively. Based on SFs, many methods can be used to describe the object extension by selecting different suitable parametric representations, including in particular those of elliptical and rectangular objects. For JTC of extended objects, we consider integrating prior size and shape information into SF models to improve performance, because good use of such information can effectively improve object extension estimation and classification. Not only tracking results can assist classification, classification also helps tracking because appropriate class-dependent (kinematic and extension) models can be used for tracking. In this way, mutual exchange of information between the tracker and the classifier is sufficiently utilised to achieve better performance. By using an SF extension model, we propose an algorithm for JTC of extended objects to jointly estimate the kinematic state and object extension in a class and obtain the probabilities of the object classes. Here fusion of object extension estimates conditioned on the class is needed. Thus, we also propose such an effective method. Generally, our proposed JTC approach has the following merits: (i) By integrating the class-related extension information with the dynamic and the measurement models, different classes of extended objects can be described in the unified framework of SFs. (ii) When extended objects of multiple classes differing in sizes and shapes are modelled based on SF, the joint estimate of the centroid state and the extension is obtained easily, and the probability of each class is also calculated. (iii) The proposed extension fusion method has a concise mathematical form and favourable properties: the fused extension estimate is simply a weighted sum of SFs. This paper provides significant added value to the conference version [42], and it is an extended and revised version of the conference paper [42], in which the idea based on the SFs was introduced for JTC of extended object using the measurements of down-range and cross-range extent. Compared with [42], this paper provides new contributions, investigations and further details (not given in [42]), including mainly the following: (i) more explanations on our JTC formulation are added, and class-related extension models based on SFs are explicitly illustrated; (ii) more detailed derivations and discussions of the proposed extension estimation fusion method are provided; (iii) more complete simulation studies are given, i.e. the simulation part is extended and enriched to illustrate the benefits of the proposed JTC approach compared with some other tracking methods, and the performance comparison of tracking and classification in different measurement noise levels is also given. This paper is organised as follows: Section 2 first formulates the extended object JTC problem in which extended objects of different classes differ in extensions, and then elaborates on the class-related extension model based on SF. By incorporating the proposed extension model into our JTC formulation, we propose an algorithm for JTC of extended objects in a Bayesian framework. Furthermore, fusion of object extension estimates is also provided. In Section 3, simulation results are presented to demonstrate the effectiveness of what we proposed. Section 4 concludes the paper. 2 JTC of extended object based on support functions 2.1 Problem formulation The JTC of extended objects using the down-range and cross-range extent measurements infers the kinematic state and object extension in a class and the probability of . is time-invariant, and takes a value in a discrete set: (1) Most practical extended objects can be classified by their extension (e.g. size and shape), that is, extended objects of different classes differ in extensions. Support functions not only can model a wide spectrum of object extensions but also have very natural and intuitive ties with the down-range and cross-range extent of an object. Due to their favourable properties and concise mathematical forms, we consider integrating prior class-related extension information into SF models, which helps improve object extension estimation and classification. Consider the system: (2) where k is the time index, is the measurement vector with the measurement function , is the measurement noise, is the state vector with transition function , superscript i denotes quantities pertinent to the ith class of objects in , and is the process noise. Unlike a point target, the state vector of an extended object is given by , which consists of the centroid kinematic state and the vector characterising the object extension. Consider , where and are the position and velocity in the Cartesian plane, respectively. In this work, we assume that a high-resolution sensor provides measurements of down-range extent D and cross-range extent C along LOS, as well as range r and bearing measurements of the object centroid. Since the elements of measurements are obtained from different physical channels, the noise is generally assumed to be zero-mean white Gaussian with independent elements: (3) and are mutually independent white Gaussian noise. In this formulation, extended objects in different classes are represented by different class-related extension models based on SFs, which makes good use of prior size and shape information. The SF of an extended object is characterised by . By integrating the class-related extension information with the dynamic and the measurement models, different classes of extended objects can be described in a unified framework of SFs. Then we propose an algorithm for JTC of extended objects in a Bayesian framework. Before discussing JTC of extended objects using down-range and cross-range measurements, we review briefly the SF-based approach to EOT [23]. 2.2 Object extension modelling based on support functions The SF approach can model a variety of (smooth/non-smooth) object extensions, including in particular elliptical and rectangular objects. 2.2.1 Smooth object extension modelling Based on SFs, the authors in [23] proposed a general approach to modelling smooth object extensions. For example, an elliptical object extension (i.e. size, shape and orientation) is approximated by using a symmetric positive semi-definite matrix as a suitable parametric representation, where (4) By using this representation, the SF of this object at viewing angle is (5) where . Here entries of the matrix can be taken to form the extension parameters . Clearly from (5), is characterised by . In addition, an elliptical object extension can also be parameterised as follows. Factorise as (6) where is a rotation matrix and and are the lengths of the semi-major and semi-minor axes, respectively. Every rotation matrix has the form: (7) where is the orientation angle between the major axis and the x-axis of the Cartesian coordinate system. By using this parametric representation, we also obtain the SF of the elliptical object extension as (i.e. (5) becomes) (8) where , , and can be taken as the extension parameters. The SF not only can describe object shapes but also has natural ties with the down-range and cross-range extent of extended objects. and measure the length and breadth of an extended object along the LOS, respectively. For the elliptical object in Fig. 1, the down-range extent is the sum of SFs and [23]: (9) where and are the distances from the centroid to supporting lines and , respectively. Correspondingly, the cross-range extent can be written as (10) where and are the distances from the centroid to supporting lines and , respectively. This type of measurement can be directly expressed in terms of SFs. Since the elliptical object is centrosymmetric, we have (11) (12) Fig. 1Open in figure viewerPowerPoint Range extent and SFs Then from (5), (9), and (10), we have (13) (14) 2.2.2 Non-smooth object extension modelling The above approach cannot be applied directly to a rectangular object, so another object modelling approach based on EGI was proposed for non-smooth shapes in a concise form [23]. The EGI is particularly convenient to represent and parameterise the shape of a convex body K concisely [43]. If K is an N-sided polygon whose jth edge has length and outer unit normal vector , its EGI can be represented by vectors for counterclockwise (see Fig. 2), so a convenient set of parameters can be taken as EGI parameters. In particular, if we have the EGI parameters of a convex polygon K, the shape of K can be uniquely determined. Fig. 2Open in figure viewerPowerPoint EGI representation of a convex polygon At the viewing direction , the down-range and cross-range extent of object K can be written as [23] (15) (16) Clearly from (15) and (16), the down-range extent and cross-range extent can be calculated by the above equations using the EGI parameters . This provides a connection between the EGI and the SF. Some symmetric non-smooth objects can be modelled in the framework of SFs this way. For example, we can obtain the SF representations of rectangular objects by utilising this connection indirectly. For more details about EGI-based modelling, see [23]. Note that the down-range and the cross-range extent of an extended object are actually functions of the viewing angle along LOS for tracking. For a rectangular object, its EGI parameters can be easily obtained as . Thus (i.e. the length of minor and major axes) and the angle of orientation can be taken as the extension parameters. By (15), we have (17) Since the rectangular object is centrosymmetric, its SF is easily obtained as (9) (18) Remark 1.The class-related extension information (i.e. the size and shape of an extended object) is effectively integrated into the SF models. Remark 2.Based on SFs, many methods can be used to describe the object extension by selecting different suitable parametric representations. In this case, extended objects in different classes are represented by different SF extension models. 2.3 SF-Based algorithm for JTC of extended object By using the class-related extension model based on SFs, we propose an algorithm for JTC of extended objects in a Bayesian framework. It aims to jointly obtain the kinematics and extension estimates in a class and the probabilities of the object classes under the following two fundamental assumptions: : The object class is time invariant. : is in the set . For JTC of extended objects, both and are reasonable. The proposed algorithm runs a class-conditional filter for each class (i.e. a linear/non-linear filter for each class) to obtain the class-conditional state estimate and associated covariance . Suppose that at time , and are available for class . Let be the event that the object is in class i. ( is measurements through time k) is the minimum mean-square error estimate from the ith class-conditional filter assuming is true throughout the time. It can be obtained recursively in two steps (prediction and update). Here we assume that the dynamics equation in (2) is linear, given as (19) where is the state transition matrix. Thus the state prediction can be done exactly as in the Kalman filter: (20) (21) where and are the first two moments of . Since the measurement equation is non-linear, we use the unscented transformation (UT) [44] for the non-linear part of (2) and leave the remaining part to the Kalman filter. This is because it is easy to apply without the need to linearise the non-linear measurement equation. Note that this is not the only option – other non-linear filtering methods (e.g. EKF) can also be used to estimate position and velocity (with a suitable initialisation). For simplicity, this UT is denoted as (22) where and are the first two moments of . Here the UT is used to approximate mean and covariance of a non-linear function by a set of deterministic points with weights for the measurement prediction and its covariance : (23) (24) Note that the number M of sampled points is generally chosen as ( is the dimension of the estimated state) in the framework of UT. It contains the adequate number of sigma points needed to capture mean and covariance exactly. The filter update is (25) (26) (27) (28) where is the updated estimate of the kinematic state and extension parameter, and is its covariance. 2.3.1 Classification of extended object The object classification depends on the posterior probability of . Suppose that initial probabilities at time are available for class , . The posterior probabilities are obtained by Bayes' formula recursively as (29) where is measurements through time Note that is the likelihood of class at time k. We consider approximating this likelihood with a Gaussian distribution by moment matching, as in [1]: (30) where is the measurement residual. The classifier based on uses the information in the estimation (i.e. the estimate of the kinematics and object extension) through . The output of extended object classification (i.e. class probability) is used to affect the kinematics and extension fusion weight. 2.3.2 Kinematic state estimation fusion The fused kinematic state estimate is calculated as a sum of weighted by their corresponding class probabilities : (31) where is obtained in (27). 2.3.3 Extension estimation fusion The class-related kinematic state vector has explicit physical meaning – it characterises the position and velocity of the extended object. Thus the kinematic state can be fused in (31) directly. Similarly, the fused extension parameter vector may be calculated as a sum of the class-related extension parameter vectors for class i weighted by their corresponding class probabilities : (32) Correspondingly, the SF of the fused object extension is characterised by in (32). However, these extension vectors () may have different physical meanings. So (32) has no physical meaning. As an example of two extended objects in different classes, the elliptical object extension is parameterised by a matrix – the non-zero entries of the matrix are chosen as the extension parameters, but the rectangular object extension is directly parameterised by its major and minor axes and the angle of orientation. The weighted sum (see (32)) of the estimated elliptical and rectangular extension vectors parameterised in different mathematical forms with different units actually makes no sense. Besides, for different i may even have different dimensions. Our study shows that the extension estimation fusion mentioned above can be too sensitive to the parameterisation used, not to mention the fact that the mathematical form of is difficult to obtain based only on . Note that different parameterisations of the same elliptical object might also have different physical meanings. To deal with the above problems, we propose a method for fusing object extension estimates in terms of SFs. Since the extension of an extended object in class is represented by the SF , it is an embodiment of the size and shape characteristics that distinguish the classes. has the same physical meaning, and thus the actual fused extension estimate can be calculated as their sum weighted by the corresponding class probabilities : (33) where is characterised by the updated extension parameter vector Remark 3.The proposed extension fusion method is good. From (33) the weighted sum of with their corresponding class probabilities is also an SF. Thus our fused extension estimate is simply a linear combination of For extended objects in different classes with various size and shape characteristics, we can adopt different parametric representations to describe them based on SF. This method is simple, general, and flexible. It takes advantage of the general and unified framework of SFs that enables the flexibility of using different parametric representations for different classes of object extensions. Suppose we have two classes of extended objects (e.g. elliptical and rectangular ) and , the extension estimation fusion is illustrated in Fig. 3. Clearly from (33) (34) Fig. 3Open in figure viewerPowerPoint Fused object extension and its SF representation Remark 4.As mentioned above, the classifier relies on the estimation results (i.e. the estimate of kinematic state and extension), and the output of the classifier is helpful to the tracker as shown in (31). To achieve better performance, exchange of useful information between the tracker and the classifier is sufficiently utilised by the SF-based JTC algorithm. 2.4 Implementation of JTC algorithm By using the SF model, the proposed algorithm can be easily implemented for JTC of extended objects. One cycle of the algorithm is: Step 1: Class-conditional filtering for each class () with and : (35) For details, see (19)–(28). Step 2: Update the posterior probability of each class (for ): (36) where and . Step 3: Kinematic state estimate fusion: (37) Step 4: Object extension estimate fusion: (38) where is characterised by the extension parameter vector . Object classification depends on the posterior probability of class i. The flowchart of our JTC algorithm is given in Fig. 4. Fig. 4Open in figure viewerPowerPoint Flowchart of SF-based algorithm for JTC of extended object 3 Simulation study To illustrate the effectiveness of the proposed approach to JTC of extended objects, simulation examples are presented in this section. Suppose we know that two classes of an extended object (as an example of elliptical and rectangular objects in Fig. 3) may exist in the area of interest: : An ellipse with major axis 100 m and minor axis 10 m. : An 80 m × 10 m rectangle. For extended objects in and , size and shape information is known a priori. Consider a scenario in which the extended object of class moves at a nearly constant velocity [45] in the 2D Cartesian coordinate system with the initial kinematic state and the angle of orientation (as extension parameter ). Then the initial state vector of the extended object is , and the dynamics is (39) with where F describes both the centroid state and target extension transition. Given more information, we can design F more specifically for different scenarios because a different F leads to different centroid state and object extension transition. Note that the uncertainty of the target state is modelled by the process noise w. The sensor is fixed at the origin (0, 0) and it provides measurements of range, bearing, and the target range extent along LOS every . Each measurement is corrupted by independent, zero-mean Gaussian noise with standard deviations , , , and . The measurement equation is (40) This work is focused on the JTC of extended objects using the down-range and cross-range measurement. We evaluate the tracking and cla