Fast‐efficient shape error concealment technique based on block classification
2016; Institution of Engineering and Technology; Volume: 10; Issue: 10 Linguagem: Inglês
10.1049/iet-ipr.2016.0043
ISSN1751-9667
AutoresShih‐Chang Hsia, Cheng Hung Hsiao,
Tópico(s)Optical measurement and interference techniques
ResumoIET Image ProcessingVolume 10, Issue 10 p. 693-700 Research ArticleFree Access Fast-efficient shape error concealment technique based on block classification Shih-Chang Hsia, Corresponding Author Shih-Chang Hsia hsia@yuntech.edu.tw Department of Electronics, National Yunlin University of Science and Technology, Douliou, TaiwanSearch for more papers by this authorCheng Hung Hsiao, Cheng Hung Hsiao GeoVision Inc., Taipei, TaiwanSearch for more papers by this author Shih-Chang Hsia, Corresponding Author Shih-Chang Hsia hsia@yuntech.edu.tw Department of Electronics, National Yunlin University of Science and Technology, Douliou, TaiwanSearch for more papers by this authorCheng Hung Hsiao, Cheng Hung Hsiao GeoVision Inc., Taipei, TaiwanSearch for more papers by this author First published: 01 October 2016 https://doi.org/10.1049/iet-ipr.2016.0043Citations: 4AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract This study presents a fast-efficient error concealment method for recovering information related to shape. The proposed technique comprises block classification, edge direction interpolation and filtering interpolation. Missing blocks are classified into four categories: transparent, opaque, edge and isolated blocks. Most of the computation is spent on edge blocks and isolated blocks to maximise the cost and performance tradeoff. For the recovery of edge blocks, the edge slope is computed by referring to the nearest available block, from which the missing shape is interpolated parallel to the edge. Isolated blocks are dealt with using a cascade filter to approximate the actual shape. Experimental results show that the proposed method provides better cost performance in the restoration of shapes than that afforded by comparable algorithms, both in numerical parameters and the resulting shapes. The processing speed is approximately two to three times faster than previous methods and low computational load makes the proposed technique applicable to real-time MPEG-4 systems. 1 Introduction Recently, MPEG-4 has enabled content-based functionality and high coding efficiency in low bit-rate systems through the consideration of shape-related information [1, 2]. In object coding, each object is split between information related to shape and that related to texture, which is then independently encoded into a single bit-stream. Shape information provides the position and outline of the object, which is crucial to object-based coding. Shape information enables the decoder to restore objects one by one. In this manner, the MPEG-4 protocol provides flexible reconstruction and manipulation of content by the decoder. However, the coded bit-stream is highly sensitive to channel error, due to the variable length coder employed. If shape information cannot be reconstructed, the decoded object can be seriously distorted. The MPEG-4 codec provides both intra- and inter-shape coding. Inter-shape decoding depends heavily on the accuracy of shape information from the previous frame, such that decoding error is propagated to following frames. Hence, error concealment is required to correct shape information and improve the robustness of MPEG-4 decoding. There are two general approaches to overcome shape error: spatial processing and temporal processing [3-17]. In this paper, we study spatial interpolation to recover missing intra-shape blocks. Simple bilinear interpolation can reconstruct shape information using neighbouring data. This approach is generally adopted for computational simplicity rather than performance. Edge-directed interpolation was employed by Ma et al. [7], which estimated based on the edges in the neighbouring frames. Unfortunately, the computational load of this approach is very high and performance is strongly dependent on image features. Shirani et al. [8] presented a novel error concealment method for recovering shape information, which iteratively interpolates missing pixels using weights in eight directions. Iterative procedures enable the gradual recovery of shape error; however, the recovered shape does not appear smooth in the final results. Lee et al. [9] used fuzzy theory to recover the shape error. In this approach, the error block is first computed through bilinear interpolation. The results are then modified according to their fuzzy membership function to truncate the pixel to a binary form with a single threshold; the results are better than those using simple bilinear interpolation. Schuster et al. [10] and Tsiligianni et al. [11] proposed the spatial error concealment with Hermite and B-spline curve approaches, respectively. Generally, such approaches are capable of smoothing recovered shapes but require considerable computation. This paper presents a new approach to the recovery of object shape for the intra-frame and still images. The proposed algorithm comprises a block classification scheme in conjunction with edge interpolation to improve quality without increasing computational costs. This paper is organised as follows. The proposed algorithm for shape recovery is presented in Section 2. Simulation results and comparisons with other approaches are shown in Section 3. Concluding remarks are outlined in Section 4. 2 Proposed algorithm Fig. 1 illustrates the processing flow of the proposed algorithm. Receiving a decoded frame with an error message from the previous decoding stage enables the location of missing block to be identified. Processed blocks can be classified into four types, with the features of neighbouring blocks, as shown in Fig. 2. To improve performance, each type of block is processed independently using different methods. If neighbouring blocks are all dark or all bright, the current-block type will be either opaque or transparent. The kind of block is easily recovered because the data can simply be replicated using adjacent pixels. Edge blocks provide shape data that is partially dark or partially bright because the neighbouring blocks contain edge information. Edge direction can be determined using the top and bottom blocks to interpolate lost shapes with a high degree of accuracy. Isolated blocks can be classified with neighbouring blocks of various types. For example, the top and bottom blocks adjacent to the isolated block in Fig. 2 are opaque and transparent, respectively. This kind of block is always located at the top or bottom of an object; therefore, the spatial correlation between the processed block and neighbouring blocks is quite low. Isolated blocks are extremely difficult to recover since they lack the information required to restore the edge of the shape. Fig. 1Open in figure viewerPowerPoint Processing flow of the proposed shape recovery Fig. 2Open in figure viewerPowerPoint Block classification for the binary shape plane 2.1 Block classification Next, we explore the classification of blocks and recover missing blocks using the proposed method, which is fast and accurate. In video coding systems, a slice is the smallest synchronous unit. As the videos are scanned from left to right, the direction of the slices is horizontal. Assuming that data related to the current slice are destroyed, neighbouring blocks in the vertical direction may be available as they belong to different slices; however, adjacent horizontal blocks are lost since these blocks belong to the same slice. Thus, for the processing of missing blocks, we only employ the information related to BT, BB, BTL, BTR, BBL and BBR denoting the top, bottom, top-left, top-right, bottom-left, bottom-right blocks, respectively. Fig. 3 illustrates the positions of blocks neighbouring the processed block. Fig. 3Open in figure viewerPowerPoint Relative position for the data reconstruction of the lost block If the BT, BB, BTL, BTR, BBL and BBR blocks are all 0 (or 1), the current block belongs to an opaque (or transparent) block. We fill in 0 or 1 for missing pixels without computation. If the T-block and B-block provide edge information, then the lost block is an edge block, in which the T-block comprises blocks BTL, BT and BTR; and the B-block comprises blocks BBR, BB and BBR. According to the information provided by the T-block and B-block, the block can be grouped into four categories. To establish the classification scheme, we employed the simple logic criterion of AND and OR operations. The 'T_{and}' function states that the boundary pixels of the BTL, BT and BTR blocks (T-block) are performed with the AND function such that (1)where are the ith boundary pixels of blocks BTL, BT and BTR, respectively. If the block size is N × N, we must perform (3N − 1)-bit AND operations. By the same concept, 'B_{and},' 'T_{or}' and 'B_{or}' are defined to perform the AND function of B-blocks, the OR function of the T-blocks and the OR function of the B-blocks, respectively. In summary, blocks can be classified according to four values, T_{and}, B_{and}, T_{or} and B_{or}, as listed in Table 1. Obviously, if all operations result in 0 or 1, this will refer to an opaque or transparent block. When T-block and B-block have edge information, the partial data is 0 or 1. Hence, the T_{and} and B_{and} should be 0, but T_{or} and B_{or} are 1. Table 1. Block classification with the boundary pixels Opaque block Transparent block Edge block Isolated block Case 1 Case 2 Case 3 Case 4 Case 5 Case 6 T-and 0 1 0 0 0 0 1 0 1 B-and 0 1 0 0 0 1 0 1 0 T-or 0 1 1 0 1 1 1 0 1 B-or 0 1 1 1 0 1 1 1 0 We present some cases of estimating isolated blocks in Figs. 4a–d. In case 1, the background is dark and the missing block is at the top of the object, the result of T_{and}, B_{and}, T_{or} and B_{or} = {0001}. In case 2, the missing block is the bottom of the object, and the following result is {0010}. Conversely, if the background is higher than the foreground level, the results of the bottom and top of the object will be {0111} and {1011} in cases 3 and 4, respectively. In cases 1–4, the blocks are located at the boundary of a series of isolated blocks. Blocks located in the middle of a group of isolated blocks (in cases 5 and 6) have values {1010} and {0101}, respectively. In case 5, the bottom block of object B_{and} and B_{or} operations are high in the middle of continuous isolated-blocks with values {0101}. Case 6 results in {1010} from the top of the object because T_{and} and T_{or} are high. For the processing of isolated blocks, we propose local filtering approximated interpolation to recover the shape information. The details are described later in this paper. Fig. 4Open in figure viewerPowerPoint Isolated block classified into six cases (a) Case 1, (b) Case 2, (c) Cases 3 and 5, (d) Cases 4 and 6 2.2 Handling edge blocks Next, we describe the method of processing edge blocks. The XOR operation is used to estimate the edge direction. If the block size is N × N, the XOR function of the T-block and B-block is performed as follows: (2)where the symbol ⊕ is for XOR. The XOR function is performed for each of adjacent pixels. For example, in Fig. 5a, T_{Xor} = (0,…,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,…,0) and B_{Xor} = (0,…,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,…,0), where the symbol '0,…,0' includes 16 zeros. The output of (2) is a set of 47 logical values for checking the boundary vector for one 16 × 16 block processing. We define that vector (0) as being located at the first pixel of blocks BT and BB, and the first pixel of BTL and BBL will be vector (−16). Thus, the vectors of blocks BT and BB are 10 and 3, respectively. The edge direction is between the two vectors. If the adjacent top and bottom blocks adjacent to the missing block have the same edge direction, clearly the edge of the lost block is vertical and the T-blocks and B-blocks will have the same vector value. If the distance to the boundary vector of the T-block is greater than that of the B-block, the edge slope will be between 0° and 90°. Otherwise, it will be between 90° and 180°. Fig. 5Open in figure viewerPowerPoint Estimation of edge direction and interpolation along the edge direction (a) Estimation of edge direction for the missing block, (b) Interpolation along the edge direction The missing block is then interpolated along the edge direction of the T- and B-blocks. First, we can interpolate the edge line between the vector of the T- and B-blocks, where the result is shown in Fig. 5b. As shape information is binary, we can draw an edge line using 1 between the vector (10) of the T-block and the vector (3) of the B-block. Next, the other missing pixels can also be filled in using 0 or 1 by referring to the edge line. In this case, the missing pixels are interpolated using 1 from the first pixel of each row to the edge line, and are filled using 0 s from the edge line to the last pixel of each row. The edge line of the jth row can be determined for the lost block using two cases provided by (3)where the size of the lost block (N) is 16 × 16, and symbols VT and VB denote the vector of the top and bottom blocks, respectively. Rj is the location of the edge point in the jth row, and symbol 'int(b)' takes the integer value of b. For example, the 14th row edge, R14 = 3 + int((10−3)/16) × 15 = 9 in Fig. 5b. As mentioned above, the proposed algorithm is used for vertical slope-edge blocks. However, the object has horizontal edges, which are difficult to recover using spatial processing because the information related to horizontal edges is lost when slice data is corrupted. Horizontal edge blocks are classified as isolated blocks because the adjacent top and bottom blocks belong to different block types. 2.3 Handling isolated blocks Next, we turn our attention to the processing of isolated blocks. The shape information in cases 1–6 cannot be recovered efficiently using the edge information of the T- and B-blocks. We propose the use of a local approximate interpolation filter to recover the isolated block efficiently. Following the processing flow of Fig. 1, we first perform linear interpolation for the lost block with boundary pixels of the BT and BB blocks. As N = 16, the ith pixel of the first vertical line can be interpolated using (4)where is the interpolated result of the first vertical line for the missing block BM, is the first row and first column of the bottom block BB and is the last row and first column of the top block BT, as shown in Fig. 6a. The interpolation result from (4) will be a grey level because the top block BT is bright with a value of 255 in an 8-bit image and the value of the bottom block BB is 0. According to (4), we can interpolate all vertical lines of the lost block with a column-by-column approach. After interpolation, the missing block appears as a grey level because the levels of the boundary of the top and bottom block of the isolated block are inversed. Fig. 6Open in figure viewerPowerPoint Linear interpolation, interpolation for bottom block error and prediction for the shape with a 5 × 9 window (a) Linear interpolation for the corrupted block, (b) Interpolation for bottom block error along the edge direction of T-block, (c) Prediction for the shape with a 5 × 9 window The interpolated value is further processed using a low-pass filter to recover the error blocks as approximate real shapes using the approximation approach. The low-pass filter is only used for the error pixels of isolated blocks. To avoid violating our objective, the other block types and the normal pixels are not processed using the low-pass filter. The reason is that we wish only to use the low-pass filter to spread the information of normal shapes in neighbouring blocks to modify the grey level of the error blocks. The low-pass filter was designed using an N × M window, which may cover pixels of the true shape from the blocks adjacent to the boundary. Following recursive low-pass filtering, the grey level for error pixels is modified to nearly match the shape, which becomes increasingly close to the real shape with an increase in the number of processing stages. For example, if the processing window covers object pixels with a level of 255, the grey level of the error pixel after low-pass filtering will increase. Closer to the background, the level will decrease since the window may cover more black pixels. This approach can be used to efficiently modify the bilinear interpolation results for the lost block when moving toward the actual shape of the object. Considering complexity and quality tradeoffs, this study designed a simple, multiplierless low-pass filter using a 3 × 5 window to reduce computation with a cascading structure, which requires only three line-buffers. The value of the processed pixel becomes (5)where is the result of filtering at the (M, N) pixel. to pixels were processed in the previous stage, and to were not filtered. We employed a 4-bit right shifter rather than one division to reduce computational cost. Using pixel by pixel processing, the grey level of the lost block trends to the shape outline because the recursive low-pass filter spreads available data from neighbouring blocks to the lost block. The recursive low-pass filter uses a cascade structure, such that the results of the first low-pass filter are processed using subsequent low-pass filters. Finally, the grey level is truncated at a threshold to form binary shape data. The threshold value is determined according to the number of available neighbouring shape blocks, denoted by P. The threshold value is determined by 128−8 × (6−P). If the processed block is close to the object, the threshold will be higher, in which case all neighbouring blocks are available and P = 6. When two-slice error appears in continuous blocks in vertical direction, the threshold is decreased to encompass the information of the entire shape, where P is reduced. However, when the features of the processed block cannot be detected using Table 1, non-defined block types are also processed using the same processing flow as that used for isolated blocks. It is worth noticing when two or more vertically adjacent slices are lost in a real system. In such a case, continuous blocks in the vertical direction are not available; however, the proposed algorithm can handle this error. We employ the available slices above and below between adjacent vertical block errors to recover the shape information using the proposed algorithm for edge blocks. However, with an increase in the number of lost vertically adjacent slices, the correlation of available slices above and below the lost blocks drops. As a result, recovery performance is degraded. When slices either above or below are unavailable, shape information cannot be restored using edge direction interpolation. Instead, the missing blocks are treated as isolated blocks to be recovered using local filtering approximated interpolation. In the special case, if the missing blocks occur at the boundary of the frame, we cannot recover the above-mentioned block because either the adjacent T-blocks or B-blocks are unavailable. In the top slice, the B-blocks are still available. We can employ information of the B-block to classify the block type. If the results of B_{and} and B_{or} are all high (or low), clearly the B-block is transparent (or opaque), which enables us to fill the error block directly with 1 s (or 0 s). However, when the B-block contains the edge of the shape, the B_{or} operation results in a high value but B_{and} results in a low value. The missing shape can be restored from the edge direction of the B-block. Alternatively, when the missing block is in the last bottom slice of a frame, T-blocks are available, enabling the interpolation of shape information for the missing block along the edge direction of the T-block. Fig. 6b illustrates the error concealment using edge prediction for the bottom block along the edge slope of the shape. We employed a 5 × 9 window to predict the direction of the shape. Fig. 6c shows the processing window for the error concealment in the bottom blocks, in which the 'dash block' is processed. The first three rows are the available shape, and the last two rows need to be interpolated. First, we search the shape edge located in the last row of the T-blocks using the centre point of the window (j = 2, i = 4), and then check the shape edge in the first row (j = 0) and the second row (j = 1). The shape edge in the third and fourth rows is restored from (4−j, 8−i) as the shape location of the first and second rows. For example, the error concealment for the shape location in the fourth row at (4, 2) is found the shape of the first row is at (j = 0, i = 6) in Fig. 6c. In the same manner, shape recovery for the third row (j = 3) is estimated using the first row (j = 1). The recovered shape in the third row at (3,3) is found at (1,5). When an edge dot is found, we can fill '1' from the first column to the edge dot, and interpolate '0' from the edge dot to the last column. When the two rows are completely recovered, the window moves to the next row and uses the same procedures to interpolate the missing shape to the boundary. This approach enables the correction of the most shape information for the bottom block. By the same manner, we can recover the shape of the lost block at the top slice. 3 Simulations and comparisons In the following, we evaluate the performance of the proposed algorithm. First, three video sequences were selected, and shape information of each object was extracted, as shown in Figs. 7a–c. The frame size employed the common intermediate format with 352 × 288. The videos were encoded by referring to the shape information. The coded bit-stream will be transmitted to the decoder through an error-prone channel. Errors were generated randomly using a random generator function. 'Slice' is a minimum synchronous unit in video coding, which introduces the possibility of partial or complete slice errors. The block size used was 16 × 16, which accounts for 16 × 16 × m of corrupted data, where m is the number of error blocks. For simulations of shape error concealment, 16 × 16 × m data is corrupted by random pattern in image blocks. The shape images in the first frame are corrupted as shown in Figs. 8a–c, in which the error rate is about 26, 22 and 23% of the macroblocks lost, respectively, in which no error resiliency used. We compared the performance of the proposed system with pixel-level simulations using bilinear interpolation [1], the fuzzy theory method [9] and B-spline [11]. Each corrupted image was recovered with each of the methods. We first employed bilinear interpolation to fill the missing block and then truncated the interpolated results as a threshold of 127. The results are shown in Figs. 9a–c. The performance is poor, as indicated by the lack of smoothness resulting from distortion in the shape outline. Fuzzy theory was used to refine results of bilinear processing, which were also truncated to obtain binary shape information. The results from the fuzzy method were better than those obtained using the bilinear method. Results can be evaluated using an error point that represents the difference in the number of shape pixels between the original and reconstructed alpha planes. The error points can be checked by an XOR operation of the original and restored images. If the pixels in the restored image differ from pixels in the same position of the original image, the XOR operation returns a high value. When all pixels in a frame are computed using XOR, the high values are accumulated to denote the number of error points. The results are listed in Table 2. Missing pixel interpolated using B-spline [11], their result outperformed those obtained using the fuzzy approach, as evidenced by the reduced error point. Table 2. Comparisons of recovery performance by error points using various methods Methods Images TV Salesman Irene bilinear 401 518 252 fuzzy theory [9] 395 523 236 B-spline [11] 338 472 209 our method 352 496 220 Fig. 7Open in figure viewerPowerPoint Original pictures TV, Salesman and Irene (a) Original picture 'TV' and its shape, (b) Original picture 'Salesman' and its shape, (c) Original picture 'Irene' and its shape Fig. 8Open in figure viewerPowerPoint Corrupted pictures TV, Salesman and Irene (a) 'TV' shape 26% corrupted, (b) 'Salesman' shape 22% corrupted, (c) 'Irene' shape 23% corrupted Fig. 9Open in figure viewerPowerPoint TV, Salesman and Irene shape recovery with bilinear method (a) 'TV' shape recovery with bilinear method, (b) 'Salesman' shape recovery with bilinear method, (c) 'Irene' shape recovery with bilinear method In the following, we present the results of our proposed algorithm. The shape error image first is processed using bilinear interpolation for isolated blocks. The results were further processed using local filtering approximation through several stages of low-pass processing. As the number of low-pass stages increased, the grey level of the error block grew increasingly closer to the real shape. Finally, the grey level was truncated to binary with an adaptive threshold. We found that the results were satisfactory following three stages of low-pass cascaded filtering. In fact, this approach efficiently recovered the shape error not only for isolated blocks but also for edge blocks. To reduce computation time, we employed a simple logic operation to recover the edge block. The complete results are shown in Figs. 10a–c for error shape recovery, in which a three-stage low-pass filter is used for isolated blocks. The proposed method can efficiently restore shape information, outperforming the bilinear and fuzzy approaches with results close to those of the high-performance algorithm [11]. Fig. 10Open in figure viewerPowerPoint TV, Salesman and Irene shape recovery with the proposed method (a) 'TV' shape recovery with the proposed method, (b) 'Salesman' shape recovery with the proposed method, (c) 'Irene' shape recovery with the proposed method In a special case, the missing block at the bottom is simulated with Fig. 9a. Clearly, the B-blocks are not available. However, two-boundary points are required for the bilinear interpolation. In simulations, the non-available pixels use 0 s for bilinear computation and the results from other forms of interpolation are poor. The proposed algorithm recovers the shape information for the missing block at the bottom by the vector of the edge direction and shape error can be efficiently concealed by edge prediction, as shown in Fig. 10a. For numerical analysis, the parameters of the distortion and complexity are taken into account. The distortion rate (DR) can be evaluated as follows: (6)Computational complexity is evaluated according to CPU time. Table 3 lists the DR result and the computational time using various methods. With an increase in shape complexity, the performance of error concealment is reduced because the content related to details is difficult to recover. For example, the shape information in Fig. 7b is more complex than in Fig. 7c and the DR of Fig. 7b increases. These results demonstrate that the proposed algorithm outperforms bilinear interpolation and the fuzzy theory in terms of DR and CPU time. Computational complexity is an important factor in real-time applications. The computational load of the proposed algorithm is adaptively dependent on the kind of block being processed. Using the logic criterion of classifying the block type, we can save computational power for opaque and transparent blocks because these blocks can be restored without any distortion. As for the edge blocks, logic operations are employed only to determine the edge direction and to interpolate the shape data between the T-block and the B-blocks, such that the computational load is not high. Most of the computational cost goes into isolated blocks to improve quality. We employed a local filtering approximation approach for the processing of isolated blocks. In fact, the number of edges and the isolated blocks in the error sequence dominates both the complexity and reconstructed quality. According to our statistical analysis, the transparent and opaque blocks account for ∼60% of the missing data, while the edge blocks and isolated blocks make up 30 and 10%, respectively, according to the error patterns. As most of the blocks are processed without computation, the computational complexity of the overall system is low. In the software simulations, the processing speed of the proposed method is about two to four times faster than that of competing algorithms [9, 11], as shown in Table 3. If the detection of transparent and opaque blocks is used in all competing algorithms, the processing speed of the proposed method is very close to that of bilinear interpolation, while providing an improvement of 1.2 and 1.6 times over the results in [9, 11], respectively. Table 3. Comparisons of DR and computing time (CT) from shape information recovery TV Salesman Irene DR CTa, s DR CT, s DR CT, s bilinear 0.33 0.092 0.45 0.098 0.13 0.096 fuzzy theory [9] 0.23 0.123 0.47 0.129 0.11 0.126 B-spline [11] 0.21 0.122 0.39 0.120 0.1 0.198 our method 0.22 0.045 0.41 0.049 0.1 0.046 a CT is measured with Intel Pentium-4 2.4 GHz CPU for one frame. Next, we analyse the requirement of computational operators. For the computations of block classifications, we required 94 'AND' and 94 'OR' operators using (1). The computations can be completely saved for opaque or transparent blocks. For edge blocks, the vector of the edge is estimated using 94 'XOR' operators using (2). Then we interpolate the missing pixels along the edge direction with (3), which only requires three additions and one multiplier. For isolated block, first we perform the bilinear interpolation using (4), which required two multipliers and one addition. Then a low-pass filter with (5) only required 14 additions × 3 stages. As the percentage of edge blocks and isolated block is only 30 and 10% of the whole frames, the computational complexity can be greatly reduced. We also checked the error performance in continuous slices in the vertical direction. As the error region expands, the spatial correlation of the top and bottom blocks to the lost block drops and the error concealment are subsequently impaired. To simulate real networking conditions, we randomly generated 20 error patterns to damage the coded bit-stream of the slice data in various frames, producing an error rate of ∼20–30% with a loss of one to four adjacent slices. Clearly, the performance drops with an increase in the corruption of vertically adjacent slices, because the correlation between the lost blocks and neighbouring blocks drops. Table 4 lists the results using various methods from the average of 50 frames for each sequence. The parameters are estimated using false positive rate (FPR), false negative rate (FNR) and accuracy. The FPR and FNR are defined by the summation of the wrong interpolation with '1' and '0', respectively, which is the ratio of the total missing pixels. The accuracy is defined by the ratio of the summation of corrected interpolation pixel and the total missing pixels. The parameters shown that our performance was closer to [11] and better than [9]. Table 4. Comparisons of recovery performance in averaged parameters with 20 random error patterns for various methodsa Methods Images TV Salesman Irene FPR, % FNR, % Accuracy, % FPR, % FNR, % Accuracy, % FPR, % FNR, % Accuracy, % bilinear 0.81 0.71 98.48 1.24 1.08 97.68 0.53 0.55 98.92 fuzzy theory [9] 0.83 0.68 98.49 1.16 1.19 97.65 0.45 0.56 98.99 B-spline [11] 0.70 0.58 98.72 1.04 1.08 97.88 0.42 0.47 99.11 our method 0.74 0.6 98.66 1.10 1.12 97.78 0.48 0.46 99.06 a Each sequence used 50 frames as one to four adjacent slices are lost. 4 Conclusions This paper presents a fast-efficient method for the recovery of shape information. First, the processed blocks are classified into four categories. We then employ various techniques to process each type of block to restore the shape information and improve performance. Following block classification, no computational power needs to be spent processing opaque or transparent blocks since these blocks can be easily restored simply by copying neighbouring pixels. Much more computation is dedicated to edge blocks and isolated blocks because the quality of these blocks largely determines recovery performance. To restore edge blocks, we propose an edge-oriented algorithm using the boundary check method with simple logic operations rather than complex computation. Spatial interpolation is used to produce the pixel for the lost block parallel to the edge. Error concealment for the isolated block is the most difficult because it lacks available blocks in one direction. The filtering approximation approach is used to recover shape information for isolated blocks. Compared with competing algorithms, our approach achieves better cost performance based on image quality and computational requirements. The proposed algorithm is applicable to the recovery of shape error in low-cost MPEG-4 coded video surveillance systems. 5 References 1Coding of audio-visual objects: video, ISO/IECJTC/SC29/WG11, January 1999 2MPEG-4 overview. Available at http://drogo.cselt.stet.it/mpeg/standards/mpeg-4/mpeg-4.htm 3Sohel F., Bennamoun M., and Hahn M.: ' Spatial shape error concealment utilizing image texture'. IEEE Conf. on Industrial Electronics and Applications, 2011, pp. 265– 270 4Uhlíková I., Benešová W., and Polec J. et al: ' Texture aware image error concealment'. IEEE EUROCON Conf. on Computer as a Tool, 2015, pp. 1– 6 5Dun Y., and Liu G.: ' An extrapolation method for MDCT domain frame loss concealment'. IEEE Conf. on Signal and Information Processing, 2014, pp. 650– 654 6Ranjbari M., Sali A., and Karim H.A. et al: ' Depth error concealment based on decision making'. IEEE Conf. on Signal and Image Processing Applications, 2013, pp. 193– 196 7Ma M., Au O., and Chan S.-H.G. et al: 'Edge-directed error concealment', IEEE Trans. Circuits Syst. Video Technol., 2010, 20, (3), pp. 382– 395 8Shirani S., Erol B., and Kossentini F.: 'A concealment methods for shape information in MPEG-4 coded video sequences', IEEE Trans. Multimedia, 2000, 2, (3), pp. 185– 190 9Lee P.J., Chen L.G., and Wang W.J. et al: ' Robust error concealment algorithm for MPEG-4 with the aids of fuzzy theory'. IEEE Conf. on Consumer Electronics, 2001, pp. 154– 155 10Schuster G.M., Li X., and Kataggelos A.: 'Shape error concealment using hermit splines', IEEE Trans. Image Process., 2004, 13, (6), pp. 808– 820 11Tsiligianni E., Kondi L.P., and Katsaggelos A.s.K.: 'Shape error concealment based on a shape-preserving boundary approximation', IEEE Trans. Image Process., 2012, 21, (8), pp. 3573– 3585 12Tsiligianni E., Kondi L.P., and Katsaggelos A.K.: ' Shape error concealment based on a shape preserving boundary approximation'. IEEE Conf. on Image Processing, 2010, pp. 457– 460 13Zhang D., Li S., and Yang K.J. et al: ' An error concealment adaptive framework for intra-frames'. IET Conf. on Image Processing, 2013, pp. 1880– 1884 14Soares L.D., and Pereira F.: 'Temporal shape error concealment by global motion compensation with local refinement', IEEE Trans. Circuits Syst. Video Technol., 2006, 15, (6), pp. 1331– 1347 15Liu J., Zhai G., and Yang X. et al: 'Spatial error concealment with an adaptive linear predictor', IEEE Trans. Circuits Syst. Video Technol., 2015, 25, (3), pp. 353– 366 16Wu Z., and Gao S.: ' An adaptive multi-level matching interpolation algorithm for spatial error concealment'. IET Conf. on Information and Communications Technologies, 2013, pp. 539– 543 17Baxes G.A.: ' Digital image processing-principles and applications' ( Wiley, New York, 1994) Citing Literature Volume10, Issue10October 2016Pages 693-700 FiguresReferencesRelatedInformation
Referência(s)