Optical Flow
 |
 |
 |
|
|
|
|
The materials provided here have been published in [7].
IntroductionThe optical flow is the estimation of the 2D apparent motion field of two consecutive images in an image sequence, which can be presented as a 2D vector field on the image plane. It is an ill-posed problem as the motion is three dimensional while the images are projections of the 3D scene onto a 2D plane. The most common assumption used in optical flow estimation is the brightness constancy assumption. It states that the gray value of corresponding pixels in the two consecutive frames should be the same. The brightness constancy assumption fails in the case of changes in illumination or upon observation of transparent or non-Lambertian material [4]. Another source of ambiguity arises from the so-called aperture problem. The aperture problem arises as a consequence of motion ambiguity when an object is viewed through an aperture. If an untextured object moves within the image, the motion within the object area cannot be recovered without additional information. Even at object edges, where the gray value of the background differs from the gray value of the object (or object edge) itself, motion can only be recovered in one dimension without additional information. The 2D motion can only be calculated where more information is available, e.g. at object corners or if texture is available.Concept of Optical FlowDue to the ill-pose character of optical flow, it needs to be reformulated for numerical treatment. Additional assumption have to be considered such as the smoothness of the optical flow field which known as smoothness. Optical flow methods can be categorized into the two general classes feature-based and variational approaches based on the regularization type.Feature-based approaches compute the optical flow displacement for a pixel and its neighborhood independently for the optical flow solutions obtained at the other pixels in the image. These algorithms can again be categorized into pixel accurate optical flow and sub-pixel accurate optical flow approaches. Pixel accurate optical flow approaches assign pixels of the input image to pixels of the output image. This is usually done by evaluating a pixel matching score based on gray value of a pixel’s neighborhood. Algorithms of this class can be understood as a combinatorial task, because the number of possible flow vectors or displacement vectors for a certain pixel is bounded by the image size. Due to the combinatorial structure, pixel accurate optical flow algorithms can be parallelized yielding real-time efficiency on dedicated hardware. They have the nice property of robustness due to the restricted solution space (only discrete integer pixel positions are considered and outliers are less likely), but at the same time suffer from accuracy limitations, i.e. the displacements are only pixel-discrete.Assume that the gray value image sequence has a continuous domain, the change of a pixel’s gray value between two time instances can then be computed by evaluating the image gradient. The early work of Lucas and Kanade [1] is reviewed which employs the linearized version of the optical flow constraint. Variational approaches take into account the optical flow solutions of neighboring pixels and imply smoothness assumptions on the optical flow field. Horn and Schunck [2] proposed an approach to cope with the under-determined optical flow constraint. The authors reverted to regularization, or smoothness, for the resulting flow field. Such smoothness was introduced by penalizing the derivative of the optical flow field, yielding an energy which was minimized by means of variational approaches. Ongoing Research on Motion Estimation in GET LabIn the recent years, vast amount of approaches which estimate a dense flow field have been proposed. However, the accuracy and performance need to be improved. Also, there are challenge problems such as large scale or illumination change sequences. Our research focus on obtaining accurate large and small scale optical flow by using the combination of local and global approaches integrated with the point correspondences to refine the optical flow results, while persevering the motion boundaries. The second direction of our research is make use of stereo vision for estimating the motion in 3D (Scene flow).The Motion Estimation Model Local methods such as Lucas and Kanade are more robust under noise, while global techniques such as Horn and Schunck yield dense flow field. A. Bruhn[3] combines important advantages of local and global approaches into the so called CLG. It yields dense flow fields that are robust against noise. Our work based on the CLG approach. Our contribution is the integration of the local approche algorithm with the L1 total variational algorithm in the energy function: Optimizing this energy function w.r.t (u,v) is done by using dual approach by divide this energy function into two diffrent equations. The first equation depend on the data term. It is calculated point-wise by using the Euler-Lagrange equation, then optimized by using least square minimization, while the second equation is depend on the smoothness term, and is optimized using a fixed-point iteration scheme. We can say that the optimization of these two equations equal to finding the optimal solution of (u,v) from the first equation and doing TV-L1 denoising of (u,v) in the second one. The energy function we used is isotropic and propagates the flow in all directions regardless of local properties. In order to enhance the propagation, the weighted median filter has been used to solve the isotropic propagation problem. The weighted function of the median filter has been calculated based on spatio-temporal image segmentation approach. The optimization is done during the coarse to fine strategy, in which the optical flow values are propagate from the coarse levels to the finer one. The propagation of optical flow is done using interpolation which cause the lost of motion of small image details. A solution to this problem can be done using the correspondences points. in which accurate algorithm can be use to get the correspondences. Then these correspondences can be used to refine the optical flow values. In our work we used the modified census transform to get the correspondences.
New Topics
References
ContactDo you have any questions or comments? Please contact: |