# statsWaveletFilt package¶

## `cusum` module¶

Wavelet Based in CUSUM control chart for filtering signals Project (module `statsWaveletFilt.cusum` ): Functions to analise data using Control Chart CUSUM. In this package the application of this chart is for filtration of wavelet coefficients.

Created by Tiarles Guterres, 2018

`cusum.``analysisCusum`(data, k=0.5, mean=None, std=None, SjBi_start=0, Sjsi_start=0)[source]

Calculates the Control Limits of CUSUM like in . This is a Control Chart defined in  and this type of tool serves to make a control of data who is called in statistic “process”.

For more details about the parameters see  Chapter 9: Cumulative Sum and Exponentially Weighted Moving Average Control Charts.

Parameters: data (list or array-like) – This is the data of “process” who CUSUM has to analize k (int or float) – Optional, 1/2 (or .5) by default. It’s a parameter of the CUSUM algorithm. Helps to the Control Chart acummulate the Control Limits of each element of data. mean (int or float) – Optional, is None by default, but turns the mean of data. Also an intern parameter of the algorithm for help to acumulate the control limits. std (int or float) – Optional, is None by default, but turns the standard deviation of data. The same function of mean in relation of control limits. SjBi_start (int or float) – Optional, is 0 by default. Is the start value for acumulation of superior control limit. Sjsi_start (int or float) – Optional, is 0 by default. Is the start value for acumulation of inferior control limit. A tuple of control limits. In  the superior limits and in  the inferior limits. tuple

`thresholdCusum()`
Function used to truncation of data using the control limits obtained in this function and a decision interval, called “H”.

References

  MONTGOMERY, D. C. Introduction to Statistical Quality Control. Sixth edition. United States: John Wiley & Sons, Inc., 2009. 733 p.
  PAGE, E. S. Continous Inspection Schemes. Biometrika, v. 41, p. 100-115, 1954
`cusum.``thresholdCusum`(data, SjB, Sjs, std=None, h=5)[source]

Makes the truncation of data accordyling with control limits SjB and Sjs and the interval of decision [H = h * data.std()]. The threshold method it’s showed in , more about cusum it’s showed in , Chapter 9.

Note

The size of data must be equal to size of SjB and Sjs.

Note

After the test (via pytest) the fuction was changed for better performance.

Parameters: data (list or array-like) – The data who corresponding to control limits. SjB (list or array-like) – The control superior limits who corresponding to data . Sjs (list or array-like) – The control inferior limits who corresponding to data . std (int or float) – Optional, is None by default, but turns the standard deviation of data. It’s an intern parameter of the algorithm for help to acumulate the control limits. h (int or float) – Optional, 5 by default. This variable multiply with standard deviation of data to obtain the interval of decision (H). An array with elements of data truncated or not, depending of the control limits and the interval of decision. numpy.array

`analysisCusum()`
Make the cusum analysis inthe data, return the control limits corresponding to data input.

References

  GUTERRES, T. D. R. M.; BAYER, F. M; KOZAKEVICIUS, A. D. J. (2018) Análise do gráfico de controle CUSUM para filtragem de coeficientes wavelet, Undergraduation Thesis, Universidade Federal de Santa Maria. In portuguese.
  MONTGOMERY, D. C. Introduction to Statistical Quality Control. Sixth edition. United States: John Wiley & Sons, Inc., 2009. 733 p.

## `filtration` module¶

Wavelet Based in CUSUM control chart for filtering signals Project (module `statsWaveletFilt.statisticFilter` ): Top level functions to filter wavelet coefficients using consagrated methods (`threshold`) and using Control Chart CUSUM (`cusum`) proposed in my Undergraduate Thesis, together with prof. Dr. Fábio Mariano Bayer and prof. Dr. Alice de Jesus Kozakevicius called Análise do gráfico de controle CUSUM para a filtragem de coeficientes wavelet, in portuguese for the Universidade Federal de Santa Maria (2°/2018).

Created by Tiarles Guterres, 2018

`filtration.``filtration`(coefficients, method='visu', p=3, mode='hard', dim_t=1024)[source]

Filters the wavelet coefficients returned by the pywt.wavedec function. All methods are implemented and showed in .

Parameters: coefficients (list of 1-D array-like) – The wavelet coefficients and the scale coefficients of the last level. The scale coefficients isn’t modify by the filtration. method (string) – Optional, is ‘visu’ by default. p (int or float) – Optional, is 3 by default. mode (string) – Optional, is ‘hard’ by default. A tuple with  A list if numpy.array. The wavelet coefficients truncated by the choiced method, with scale coefficients. Ready for pywt.waverec function. (a little ‘tip’) and  a list of float. The lambda value used for each wavelet coefficient level. tuple

`cusumFiltration()`
Function that use Cumulative Sum Control Chart and some variation for filter wavelet coefficients.

References

  KOZAKEVICIUS, A. D. J.; BAYER, F. M. Filtragem de sinais via limiarização de coeficientes wavelet. Ciência e Natura, v. 36, p. 37–51, 2014. In portuguese.
`filtration.``cusumFiltration`(coefficients, h=5, k=0.5, method='cusumTrad')[source]

Filters the wavelet coefficients returned by the pywt.wavedec function using the Cumulative Sum Control Chart (CUSUM) .

Parameters: wavCoeff (list of array-like.) – Wavelet coefficients h (int, float or array-like) – Optional, 5 by default . See “method” parameter. k (int, float or array-like) – Optional, 1/2 (or .5) by default . See “method” parameter. method (string) – Optional, “cusumTrad” by default. If “method” is “cusumTrad” the Control Chart considered to filter the wavalet coefficients is the same considered in . If the control limit (called SjB and Sjs here) is bigger than threshold limit the wavelet coefficient corresponded will be zero. In this “method” “k” and “h” will be constant and equals for all wavelet coefficients in all levels, the  recomend h = 5 and k = 1/2, but you can change or just not alterate. But if “method” is “cusumDecay” occurs the same in “cusumTrad” relative to truncation form but the “k” and “h” will be like is described in . Or if “method” is “cusumAdap” will be the same of the “cusumDecay” but the user can be choice who values of “h” and “k” will be for each wavelet level. A tuple with  A list if numpy.array. The wavelet coefficients truncated by the choiced method, with scale coefficients. Ready for pywt.waverec function. (a little ‘tip’),  a list of float. The “k” values used for each wavelet coefficient level and  a list of float. The “h” values used for each wavelet coefficient level. tuple

`filtration()`
Function that use this function to filter via wavelet coefficients
`pywt.wavedec()`
Function that decomposes the signal in wavelet and scale coefficients
`pywt.waverec()`
Function that recomposes the signal from wavelet and scale coefficients

References

  MONTGOMERY, D. C. Introduction to Statistical Quality Control. Sixth edition. United States: John Wiley & Sons, Inc., 2009. 733 p.
  GUTERRES, T. D. R. M.; BAYER, F. M; KOZAKEVICIUS, A. D. J. (2018) Análise do gráfico de controle CUSUM para filtragem de coeficientes wavelet, Undergraduation Thesis, Universidade Federal de Santa Maria. In portuguese.

## `miscellaneous` module¶

Wavelet Based in CUSUM control chart for filtering signals Project (module `statsWaveletFilt.miscellaneous` ): A Miscellaneous of functions for work with data and show wavelet coefficients

Created by Tiarles Guterres, 2018

`miscellaneous.``showWaveletCoeff`(coefficients, filename='tmp', format='pdf', threshold_value=0, color='black', color_threshold='black', figsize=(7, 8), title='')[source]

Show and save the wavelet and scale coefficients in a plot.

Parameters: coeff (list of numpy.array's) – With in ‘0’ position the scale coefficients. Equal to the `pywt.wavedec()` return. filename (string) – Optional, is ‘tmp’ by default. This is the first part of the name of the figure. format (string) – Optional, is ‘pdf’ by default. This is the last part of the name of the figure. Can be ‘png’, ‘ps’, ‘eps’ and ‘svg’ too. threshold_value (int, float or list.) – Optional, is 0 by default, this means that bothing new happens. Otherwise, a line in threshold value will be plotted in all wavelet coefficients plots. This value can be a list too, but they was to be the same size of wavelet coefficients (without the scale coefficient). Nothing is returned, the plots is show and save. void

`pywt.wavedec()`
Function that decomposes the signal in wavelet and scale coefficients
`pywt.waverec()`
Function that recomposes the signal from wavelet and scale coefficients
`filtration.filtration()`
Function that use this function to filter via wavelet coefficients
`filtration.filtrationCusum()`
Function that use Cumulative Sum Control Chart and some variation for filter wavelet coefficients.
`miscellaneous.``normalizeData`(data, min=0, max=1)[source]

Its almost a map function. This function normalize the data between a min and max values.

Parameters: data (list or array-like) – The values that desire normalize. min (int or float) – Optional, is -1 by default. The min value correspond, in the end, of the min value of data. max (int or float) – Optional, is 1 by default. The max value correspond, in the end, of the max value of data. The data normalized between min and max values. numpy.array
`miscellaneous.``generateData`(functions=['doppler', 'block', 'bump', 'heavsine'], varNoises=[0.001, 0.002, 0.003, 0.004, 0.005, 0.006, 0.007, 0.008, 0.009, 0.01], dim_signals=1024, n_samples_per_sig_per_noise=10000, folder='tmp')[source]

If you like to generate your dataset before run your test you can use this function to generate the data. With the 1) type of signal and 2) quantity of noise (in variance). Saves in `.npy`

## `signals` module¶

Wavelet Based in CUSUM control chart for filtering signals Project (module `statsWaveletFilt.signals` ): Functions to evaluate the dignal fitering process using the module `statisticFilter` or any kind of filtration.

Created by Tiarles Guterres, 2018

`signals.``for_dB_scale`(x)[source]

Converts x to dB scale using 10*log10(x)

Parameters: x (int or float) – The value for convertion The x value converted in dB scale. float
`signals.``for_real_scale`(x)[source]

Converts x to real scale using 10**(x/10)

Parameters: x (int or float) – The value for convertion The x value converted in eal scale. float
`signals.``snr_square_mean_error`(currentSignal, idealSignal)[source]

Calculate the SNR via the current signal and the ideal using the square mean error approach.

Parameters: currentSignal (1-D array-like) – The signal for compare with ideal. idealSignal (1-D array-like) – The ideal signal, based in the currentSignal. Mean of idealSignal by standard deviation of the noise. float
`signals.``snr_mean_standardNoise`(idealSignal, noiseSignal)[source]

Calculate the SNR via ideal signal mean and standard deviation of the noise.

Parameters: idealSignal (1-D array-like) – The ideal signal, based in the currentSignal. noiseSignal (1-D array-like) – Noise apply to ideal signal, could be a initial or residual noise. Mean of idealSignal by standard deviation of the noise. float
`signals.``snr_variances`(idealSignal, noiseSignal)[source]

Calculate the SNR via ratio of variances of ideal signal and noise.

Parameters: idealSignal (1-D array-like) – The ideal signal, based in the currentSignal. noiseSignal (1-D array-like) – Noise apply to ideal signal, could be a initial or residual noise. Variance ratio value between the ideal and noise signals. float
`signals.``cnr_amplitude_standardNoise`(idealSignal, noiseSignal)[source]

Calculate the CNR (contrast-to-noise ratio ) via the amplitude of idealSignal and standard deviation of the noise.

Parameters: idealSignal (1-D array-like) – The ideal signal, based in the currentSignal. noiseSignal (1-D array-like) – Noise apply to ideal signal, could be a initial or residual noise. Ratio of maximum distance of zero and standard deviation of the noise. float
`signals.``differential_snr_dB`(initialSignal, finalSignal, method='square_mean_error', idealSignal=None)[source]

Calculate the SNR or CNR difference between two signals: after and before filtering. Ideal signal may be used.

Parameters: initialSignal (1-D array-like) – Initial Signal, before the filtering process finalSignal (1-D array-like) – Final Signal, after the filtering process method (string, optional) – Is ‘square_mean_error’ by default, other forms of calculate the SNR differential is ‘mean_StandardNoise’, ‘variances’ and ‘amplitude_standardNoise’. idealSignal (1-D array-like or 0, optional) – Is 0 by default, is necessary in all methods except in ‘square_mean_error’ method. The SNR differential value in dB. float
`signals.``dopplerFunction`(dim=1024, normalize=True, fq=0)[source]

Generate the Doppler function in a range of 0 to 1, with dim points.

Parameters: dim (int) – Dimension of the signal. normalize (bool, optional) – It is True by default. This parameter normalize the data values in a range of 0 to 1 with a function present in `statsWaveletFilt.miscellaneous`. fq (int or float, optional) – It is 0 by default. With this default value the original doppler, shown by Donoho  will be used.  1-D array-like, coordinates in X axis and  1-D array-like, coordinates in Y axis tuple

References

  DONOHO, D. L.; JOHNSTONE, I. M. Ideal spatial adaptation via wavelet shrinkage. Biometrika, v. 81, p. 425–455, 1994.
`signals.``heavsineFunction`(dim=1024, normalize=True, heavs=0)[source]

Generate the Heavsine function in a range of 0 to 1, with dim points.

Parameters: dim (int) – Dimension of the signal. normalize (bool, optional) – It is True by default. This parameter normalize the data values in a range of 0 to 1 with a function present in `statsWaveletFilt.miscellaneous`. heavs (int or float, optional) – It is 0 by default. This parameter, called * heavs * is the number of discontinuities in the heavens characteristic signal shown by Donoho  with 0 the signal will be the original, used in .  1-D array-like, coordinates in X axis and  1-D array-like, coordinates in Y axis tuple

References

  DONOHO, D. L.; JOHNSTONE, I. M. Ideal spatial adaptation via wavelet shrinkage. Biometrika, v. 81, p. 425–455, 1994.
`signals.``blockFunction`(dim=1024, normalize=True, ht=0)[source]

Generate the Block function in a range of 0 to 1, with dim points.

Parameters: dim (int) – Dimension of the signal. normalize (bool, optional) – It is True by default. This parameter normalize the data values in a range of 0 to 1 with a function present in `statsWaveletFilt.miscellaneous`. ht (int, optional) – It is 0 by default. The parameter called ht is the commutation characteristic of block signal. The default parameter will generate the signal shown in .  1-D array-like, coordinates in X axis and  1-D array-like, coordinates in Y axis tuple

References

  DONOHO, D. L.; JOHNSTONE, I. M. Ideal spatial adaptation via wavelet shrinkage. Biometrika, v. 81, p. 425–455, 1994.
`signals.``bumpFunction`(dim=1024, normalize=True, wht=0)[source]

Generate the Bump function in a range of 0 to 1, with dim points. Take care to the representation limits of this function is blows infinity in Y axis.

Parameters: dim (int) – Dimension of the signal. normalize (bool, optional) – It is True by default. This parameter normalize the data values in a range of 0 to 1 with a function present in `statsWaveletFilt.miscellaneous`. wht (int, optional) – It is 0 by default. The parameter called wht is the number of peaks characteristic of bump signal. The default parameter will generate the signal shown in .  1-D array-like, coordinates in X axis and  1-D array-like, coordinates in Y axis tuple

References

  DONOHO, D. L.; JOHNSTONE, I. M. Ideal spatial adaptation via wavelet shrinkage. Biometrika, v. 81, p. 425–455, 1994.

## `threshold` module¶

Wavelet Based in CUSUM control chart for filtering signals Project (module `statsWaveletFilt.threshold` ): Statistic functions for obtain threshold values for wavelet coefficients based in some referenced works.

Created by Tiarles Guterres, 2018

`threshold.``lambdasVisuShrink`(wavCoeff)[source]

Computes the threshold value (lambda) by VisuShrink  method.

Parameters: wavCoeff (list of lists or array-like) – Wavelet coefficients The threshold values for each wavelet coefficients vector. list of float

`filtration()`
Function that use this function for filter via wavelet coefficients
`pywt.wavedec()`
Function that decomposes the signal in wavelet and scale coefficients
`pywt.waverec()`
Function that recomposes the signal from wavelet and scale coefficients

References

  DONOHO, D. L.; JOHNSTONE, I. M. Ideal spatial adaptation via wavelet shrinkage. Biometrika, v. 81, p. 425–455, 1994.
`threshold.``lambdasSureShrink`(wavCoeff, dim_t=1024)[source]

Computes the threshold value (lambda) by SureShrink  method. It’s showed also in .

Note

After the test (via pytest) the fuction was changed for better performance.

Parameters: wavCoeff (list of lists or array-like) – Wavelet coefficients dim_t (optional, 1024 by default. t-dimension. Input vector from) – internal function _sure(vector, dim_t). The threshold values for each wavelet coefficients vector. list of float

`filtration()`
Function that use this function to filter via wavelet coefficients
`pywt.wavedec()`
Function that decomposes the signal in wavelet and scale coefficients
`pywt.waverec()`
Function that recomposes the signal from wavelet and scale coefficients

References

  DONOHO, D. L.; JOHNSTONE, I. M. Ideal spatial adaptation via wavelet shrinkage. Biometrika, v. 81, p. 425–455, 1994.
  KOZAKEVICIUS, A. D. J.; BAYER, F. M. Filtragem de sinais via limiarização de coeficientes wavelet. Ciência e Natura, v. 36, p. 37–51, 2014. In portuguese.
`threshold.``lambdasBayesShrink`(wavCoeff)[source]

Computes the threshold value (lambda) by BayesShrink  method. It’s showed also in .

Parameters: wavCoeff (list of lists or array-like) – Wavelet coefficients The threshold values for each wavelet coefficients vector. list of float

`filtration()`
Function that use this function to filter via wavelet coefficients
`pywt.wavedec()`
Function that decomposes the signal in wavelet and scale coefficients
`pywt.waverec()`
Function that recomposes the signal from wavelet and scale coefficients

References

  CHANG, S. G.; YU, B.; VETTERLI, M. Adaptive wavelet thresholding for image denoising and compression. IEEE Transactions on Image Processing, v. 9, p. 1532–1546, 2000.
  KOZAKEVICIUS, A. D. J.; BAYER, F. M. Filtragem de sinais via limiarização de coeficientes wavelet. Ciência e Natura, v. 36, p. 37–51, 2014. In portuguese.
`threshold.``lambdasSPC_Threshold`(wavCoeff, p=3)[source]

Computes the threshold value (lambda) by SPC-Threshold ,  method

Note

After the test (via pytest) the fuction was changed for better performance.

Parameters: wavCoeff (list of lists or array-like) – Wavelet coefficients p (int or float) – Optional, 3 by default. Parameter for the algorithm , generally is used 2 or 3. The threshold values for each wavelet coefficients vector. list of float

`filtration()`
`pywt.wavedec()`
`pywt.waverec()`