Aware Motion Wavelets Download
Tandra Kanter
To deal with large motions, a layer-based video magnification approach was proposed in [19], with some help of a manually drawn mask by the user, outlining regions whose pixels are specified to be tracked and then magnified and yielding good magnification results. Whereas a mask indicates which pixels should be used, motion filter effects on the border of the mask cannot be ignored, leading to a certain spatial extent and eventually leaking across the mask edge. On the other hand, manual selection is time consuming and error prone; the selected region tracking is sensitive to occlusions and 3D object rotations. Furthermore, the alignment is based on a homography, which may generate wrong information for non-planar objects and a non-static camera. By using depth cameras and bilateral filters, the recent work in [20] proposed an alternative approach, making it possible for the amplification processing to be applied on pixels located at the same depth. In a sense, depth-aware motion processing extends the layer-based approach in replacing the manual selection mask by a weighted mask obtained from depth ranges, which avoids manual annotation to some extent. In addition, it also prevents the leaking problem in [19] by ignoring motion effects from different depth layers. However, this technique cannot cope with any moving objects; more importantly, the lack of depth knowledge will introduce inaccurate manual operations in processing. Based on the assumption that the large motion is typically linear at the scale of the small variations, the innovative processing framework [21] was proposed to magnify small deviations of linear motion by linking the response of a second-order Gaussian derivative to spatial acceleration. This work achieves impressive results for motion magnification; however, the downside is the inability to cope with nonlinear large motion. Inspired by the above approaches, this essay exploits time-frequency characteristics to automatically define the mask. In addition, based on the observation in [21,22], the significant differences are found in the frequency domain between these two kinds of variations, making our technique in principle suitable for large motion isolation.
Video spectrum-aware magnification pipeline. Our approach does not require manual region annotation nor additional depth information as done in conventional techniques; instead, by employing the proposed BE-SCT, the intrinsic frequency characteristics can be understood to achieve the goal of adaptive large motions isolation, meanwhile avoiding the nonlinear limitation in the Eulerian acceleration approach.
For easy readability of the temporal spectrum-aware filtering, a time-domain mathematics model is established for illustration. Consider the time domain of intensity changes denoted by Ix,y,t at position x,y and time t based on the significant anti-noise performance of complex steerable pyramid, small temporal variations in the spatial offset of edges can be converted to subtle temporal changes in polar coordinates of the complex filter responses in the pyramid. Therefore, in the temporal mathematics model, the temporal variations are first turned into the frequency domain by Fourier transform, which is Sω,ρ,t. Then, based on the observation that the large motions differ evidently from small ones in frequency property, the sophisticated time domain of intensity variations is reconstructed in the frequency domain as a combination of two components:
where ρl,ρh stands for the identified amplitude-frequency ranges in the time-domain spectrogram generated by BE-SCT. ρl is the minimum amplitude bound, which is not critical, since small noise can be negligible after arbitrary simple built-in spatial filtering algorithms. ρh is the maximum amplitude threshold used for eliminating the large motions, which can be experience-modifiable.
Figure 5 demonstrates various motion amplification results for a gun shooting video with magnification factor α=8. In this case, the recoil of the gun induces subtle movement in the arm muscles. To preform an in-depth and meticulous analysis, the movements of the bracelet, upper limb, and the forearm are recorded in the spatio-temporal slices indicated with three green lines over the original sequence. Due to the strong arm movement, the phase-based processing induces ripples and motion artifacts, which cover the subtle motion in the muscles. The Eulerian-acceleration method only magnifies the nonlinear motion, leading to the loss of linear subtle movement. Our proposed technique not only magnifies the intensity changes of the arm muscles but also magnifies clearly the intensity variations of the bracelet, which is caused by the reflection of the muscles, as shown in the plot on the bottom-right of Figure 5.
In the gun shooting sequence, the strong recoil causes small vibrations of the arm. The spatio-temporal slice is shown at different positions with three green lines over the sequence for each processing. (a) Original video frame. (b) Phase-based video magnification. (c) Eulerian-acceleration magnification. (d) Our proposed spectrum-aware magnification. The Eulerian-acceleration approach only magnifies the nonlinear motion by linking the response of a second-order Gaussian derivative, whereas the phase-based method results in large blurs and artifacts. Our proposed method magnifies the arm movements correctly without being affected by the background clutter [21].
Figure 7 shows magnification results for iris wobbling, combined with large-scale eye horizontal movements, and sets the magnification factor α=15. As demonstrated in the figure (top-right), when applied to the video with the phase-based technique, the small motion remains hard to be seen because it is overshadowed by the then-magnified large motions and its blurring artifacts. Our spectrum-aware magnification maintains the local motions of the iris wobbling. Eulerian acceleration does magnify segmental temporal variations; however, it kills more useful information than our approach.
The eye video and its magnification with the phase-based approach, the Eulerian-acceleration approach, and our spectrum-aware processing. The spatio-temporal slice is shown in each approach for the green stripe (top-left). This video demonstrates an eye moving along the horizontal direction, as shown in the original sequence; such wobbling is too subtle to be observed (top-left). The global motion of the eye generates significant blurring artifacts when processed with the phase-based approach. However, processing the sequence with Eulerian acceleration and our approach show clearly that the iris wobbles as the eye moves; through the in-depth comparison, more local details can be preserved in our approach [19].
With the increasing demand in using 3D mesh data over networks, supporting effective compression and efficient transmission of meshes has caught lots of attention in recent years. This article introduces a novel compression method for 3D mesh animation sequences, supporting user-defined and progressive transmissions over networks. Our motion-aware approach starts with clustering animation frames based on their motion similarities, dividing a mesh animation sequence into fragments of varying lengths. This is done by a novel temporal clustering algorithm, which measures motion similarity based on the curvature and torsion of a space curve formed by corresponding vertices along a series of animation frames. We further segment each cluster based on mesh vertex coherence, representing topological proximity within an object under certain motion. To produce a compact representation, we perform intra-cluster compression based on Graph Fourier Transform (GFT) and Set Partitioning In Hierarchical Trees (SPIHT) coding. Optimized compression results can be achieved by applying GFT due to the proximity in vertex position and motion. We adapt SPIHT to support progressive transmission and design a mechanism to transmit mesh animation sequences with user-defined quality. Experimental results show that our method can obtain a high compression ratio while maintaining a low reconstruction error.
That said, wavelets are not the best tool for making VHS tapes or 19th-generation MPEG-2 videos look great. Wavelets will usually perceive blocky borders as intentional detail and preserve them rather than eliminate them, hence the need for a decent-quality start point. Use NLMeans or HQDN3D for restoration purposes.
This paper implements a quantitative approach to detect pulse-like ground motions based on continues wavelet transform, which is able to clearly identify sudden jumps in time history of earthquake records by considering contribution of different levels of frequency. These analyses were performed on a set of time series records obtained in near-fault regions of Iran. Pulse-like ground motions frequently resulted from directivity effects in near-fault area and are of interest in the field of seismology and also earthquake engineering for seismic performance evaluation of structures. The results of this study basically help us to establish a suitable platform for selecting pulse-like records, while performance evaluation of structure in near-fault area will need to account. The period of velocity pulses as a key parameter that significantly affects structural response is simply determined by using a pseudo-period of the mother wavelets. In addition, the efficiency of different types of mother wavelets on classification performance and the features of detected pulse are investigated by applying seven different kinds of mother wavelets. The analyses indicate that the selection of most appropriate mother wavelet plays a significant role in effective extraction of ground motion features and consequently in estimation of velocity pulse period. As a result, the user should be aware of what is selected as a mother wavelet in the analysis. The comparisons given here among different mother wavelets also show the better performance of BiorSpline (bior1.3) basis from biorthognal wavelet families for the preferred purpose in this paper.
aa06259810