statsWaveletFilt package

cusum module

Wavelet Based in CUSUM control chart for filtering signals Project (module statsWaveletFilt.cusum ): Functions to analise data using Control Chart CUSUM. In this package the application of this chart is for filtration of wavelet coefficients.

Created by Tiarles Guterres, 2018

cusum.analysisCusum(data, k=0.5, mean=None, std=None, SjBi_start=0, Sjsi_start=0)

Calculates the Control Limits of CUSUM like in [1]. This is a Control Chart defined in [2] and this type of tool serves to make a control of data who is called in statistic “process”.

For more details about the parameters see [1] Chapter 9: Cumulative Sum and Exponentially Weighted Moving Average Control Charts.

Parameters:

• data (list or array-like) – This is the data of “process” who CUSUM has to analize
• k (int or float) – Optional, 1/2 (or .5) by default. It’s a parameter of the CUSUM algorithm. Helps to the Control Chart acummulate the Control Limits of each element of data.
• mean (int or float) – Optional, is None by default, but turns the mean of data. Also an intern parameter of the algorithm for help to acumulate the control limits.
• std (int or float) – Optional, is None by default, but turns the standard deviation of data. The same function of mean in relation of control limits.
• SjBi_start (int or float) – Optional, is 0 by default. Is the start value for acumulation of superior control limit.
• Sjsi_start (int or float) – Optional, is 0 by default. Is the start value for acumulation of inferior control limit.

Returns:

A tuple of control limits. In [0] the superior limits and in [2] the inferior limits.

Return type:

tuple

See also

thresholdCusum()

Function used to truncation of data using the control limits obtained in this function and a decision interval, called “H”.

References

[1]	MONTGOMERY, D. C. Introduction to Statistical Quality Control. Sixth edition. United States: John Wiley & Sons, Inc., 2009. 733 p.

[2]	PAGE, E. S. Continous Inspection Schemes. Biometrika, v. 41, p. 100-115, 1954

cusum.thresholdCusum(data, SjB, Sjs, std=None, h=5)

Makes the truncation of data accordyling with control limits SjB and Sjs and the interval of decision [H = h * data.std()]. The threshold method it’s showed in [1], more about cusum it’s showed in [2], Chapter 9.

Note

The size of data must be equal to size of SjB and Sjs.

Note

After the test (via pytest) the fuction was changed for better performance.

Parameters:

• data (list or array-like) – The data who corresponding to control limits.
• SjB (list or array-like) – The control superior limits who corresponding to data .
• Sjs (list or array-like) – The control inferior limits who corresponding to data .
• std (int or float) – Optional, is None by default, but turns the standard deviation of data. It’s an intern parameter of the algorithm for help to acumulate the control limits.
• h (int or float) – Optional, 5 by default. This variable multiply with standard deviation of data to obtain the interval of decision (H).

Returns:

An array with elements of data truncated or not, depending of the control limits and the interval of decision.

Return type:

numpy.array

See also

analysisCusum()

Make the cusum analysis inthe data, return the control limits corresponding to data input.

References

[1]	GUTERRES, T. D. R. M.; BAYER, F. M; KOZAKEVICIUS, A. D. J. (2018) Análise do gráfico de controle CUSUM para filtragem de coeficientes wavelet, Undergraduation Thesis, Universidade Federal de Santa Maria. In portuguese.

[2]	MONTGOMERY, D. C. Introduction to Statistical Quality Control. Sixth edition. United States: John Wiley & Sons, Inc., 2009. 733 p.

filtration module

Wavelet Based in CUSUM control chart for filtering signals Project (module statsWaveletFilt.statisticFilter ): Top level functions to filter wavelet coefficients using consagrated methods (threshold) and using Control Chart CUSUM (cusum) proposed in my Undergraduate Thesis, together with prof. Dr. Fábio Mariano Bayer and prof. Dr. Alice de Jesus Kozakevicius called Análise do gráfico de controle CUSUM para a filtragem de coeficientes wavelet, in portuguese for the Universidade Federal de Santa Maria (2°/2018).

Created by Tiarles Guterres, 2018

filtration.filtration(coefficients, method='visu', p=3, mode='hard', dim_t=1024)

Filters the wavelet coefficients returned by the pywt.wavedec function. All methods are implemented and showed in [1].

Parameters:

• coefficients (list of 1-D array-like) – The wavelet coefficients and the scale coefficients of the last level. The scale coefficients isn’t modify by the filtration.
• method (string) – Optional, is ‘visu’ by default.
• p (int or float) – Optional, is 3 by default.
• mode (string) – Optional, is ‘hard’ by default.

Returns:

A tuple with [0] A list if numpy.array. The wavelet coefficients truncated by the choiced method, with scale coefficients. Ready for pywt.waverec function. (a little ‘tip’) and [1] a list of float. The lambda value used for each wavelet coefficient level.

Return type:

tuple

See also

cusumFiltration()

Function that use Cumulative Sum Control Chart and some variation for filter wavelet coefficients.

References

[1]	KOZAKEVICIUS, A. D. J.; BAYER, F. M. Filtragem de sinais via limiarização de coeficientes wavelet. Ciência e Natura, v. 36, p. 37–51, 2014. In portuguese.

filtration.cusumFiltration(coefficients, h=5, k=0.5, method='cusumTrad')

Filters the wavelet coefficients returned by the pywt.wavedec function using the Cumulative Sum Control Chart (CUSUM) [1].

Parameters:

• wavCoeff (list of array-like.) – Wavelet coefficients
• h (int, float or array-like) – Optional, 5 by default [1]. See “method” parameter.
• k (int, float or array-like) – Optional, 1/2 (or .5) by default [1]. See “method” parameter.
• method (string) –

  Optional, “cusumTrad” by default.

  If “method” is “cusumTrad” the Control Chart considered to filter the wavalet coefficients is the same considered in [1]. If the control limit (called SjB and Sjs here) is bigger than threshold limit the wavelet coefficient corresponded will be zero. In this “method” “k” and “h” will be constant and equals for all wavelet coefficients in all levels, the [1] recomend h = 5 and k = 1/2, but you can change or just not alterate.

  But if “method” is “cusumDecay” occurs the same in “cusumTrad” relative to truncation form but the “k” and “h” will be like is described in [2].

  Or if “method” is “cusumAdap” will be the same of the “cusumDecay” but the user can be choice who values of “h” and “k” will be for each wavelet level.

Returns:

A tuple with [0] A list if numpy.array. The wavelet coefficients truncated by the choiced method, with scale coefficients. Ready for pywt.waverec function. (a little ‘tip’), [1] a list of float. The “k” values used for each wavelet coefficient level and [2] a list of float. The “h” values used for each wavelet coefficient level.

Return type:

tuple

See also

filtration()

Function that use this function to filter via wavelet coefficients

pywt.wavedec()

Function that decomposes the signal in wavelet and scale coefficients

pywt.waverec()

Function that recomposes the signal from wavelet and scale coefficients

References

[1]	MONTGOMERY, D. C. Introduction to Statistical Quality Control. Sixth edition. United States: John Wiley & Sons, Inc., 2009. 733 p.

[2]	GUTERRES, T. D. R. M.; BAYER, F. M; KOZAKEVICIUS, A. D. J. (2018) Análise do gráfico de controle CUSUM para filtragem de coeficientes wavelet, Undergraduation Thesis, Universidade Federal de Santa Maria. In portuguese.

miscellaneous module

Wavelet Based in CUSUM control chart for filtering signals Project (module statsWaveletFilt.miscellaneous ): A Miscellaneous of functions for work with data and show wavelet coefficients

Created by Tiarles Guterres, 2018

miscellaneous.showWaveletCoeff(coefficients, filename='tmp', format='pdf', threshold_value=0, color='black', color_threshold='black', figsize=(7, 8), title='')

Show and save the wavelet and scale coefficients in a plot.

Parameters:

• coeff (list of numpy.array's) – With in ‘0’ position the scale coefficients. Equal to the pywt.wavedec() return.
• filename (string) – Optional, is ‘tmp’ by default. This is the first part of the name of the figure.
• format (string) – Optional, is ‘pdf’ by default. This is the last part of the name of the figure. Can be ‘png’, ‘ps’, ‘eps’ and ‘svg’ too.
• threshold_value (int, float or list.) – Optional, is 0 by default, this means that bothing new happens. Otherwise, a line in threshold value will be plotted in all wavelet coefficients plots. This value can be a list too, but they was to be the same size of wavelet coefficients (without the scale coefficient).

Returns:

Nothing is returned, the plots is show and save.

Return type:

void

See also

pywt.wavedec()

Function that decomposes the signal in wavelet and scale coefficients

pywt.waverec()

Function that recomposes the signal from wavelet and scale coefficients

filtration.filtration()

Function that use this function to filter via wavelet coefficients

filtration.filtrationCusum()

Function that use Cumulative Sum Control Chart and some variation for filter wavelet coefficients.

miscellaneous.normalizeData(data, min=0, max=1)

Its almost a map function. This function normalize the data between a min and max values.

Parameters:

• data (list or array-like) – The values that desire normalize.
• min (int or float) – Optional, is -1 by default. The min value correspond, in the end, of the min value of data.
• max (int or float) – Optional, is 1 by default. The max value correspond, in the end, of the max value of data.

Returns:

The data normalized between min and max values.

Return type:

numpy.array

miscellaneous.generateData(functions=['doppler', 'block', 'bump', 'heavsine'], varNoises=[0.001, 0.002, 0.003, 0.004, 0.005, 0.006, 0.007, 0.008, 0.009, 0.01], dim_signals=1024, n_samples_per_sig_per_noise=10000, folder='tmp')

If you like to generate your dataset before run your test you can use this function to generate the data. With the 1) type of signal and 2) quantity of noise (in variance). Saves in .npy

signals module

Wavelet Based in CUSUM control chart for filtering signals Project (module statsWaveletFilt.signals ): Functions to evaluate the dignal fitering process using the module statisticFilter or any kind of filtration.

Created by Tiarles Guterres, 2018

signals.for_dB_scale(x)

Converts x to dB scale using 10*log10(x)

Parameters:x (int or float) – The value for convertion Returns:The x value converted in dB scale. Return type:float

signals.for_real_scale(x)

Converts x to real scale using 10**(x/10)

Parameters:x (int or float) – The value for convertion Returns:The x value converted in eal scale. Return type:float

signals.snr_square_mean_error(currentSignal, idealSignal)

Calculate the SNR via the current signal and the ideal using the square mean error approach.

Parameters:

• currentSignal (1-D array-like) – The signal for compare with ideal.
• idealSignal (1-D array-like) – The ideal signal, based in the currentSignal.

Returns:

Mean of idealSignal by standard deviation of the noise.

Return type:

float

signals.snr_mean_standardNoise(idealSignal, noiseSignal)

Calculate the SNR via ideal signal mean and standard deviation of the noise.

Parameters:

• idealSignal (1-D array-like) – The ideal signal, based in the currentSignal.
• noiseSignal (1-D array-like) – Noise apply to ideal signal, could be a initial or residual noise.

Returns:

Mean of idealSignal by standard deviation of the noise.

Return type:

float

signals.snr_variances(idealSignal, noiseSignal)

Calculate the SNR via ratio of variances of ideal signal and noise.

Parameters:

• idealSignal (1-D array-like) – The ideal signal, based in the currentSignal.
• noiseSignal (1-D array-like) – Noise apply to ideal signal, could be a initial or residual noise.

Returns:

Variance ratio value between the ideal and noise signals.

Return type:

float

signals.cnr_amplitude_standardNoise(idealSignal, noiseSignal)

Calculate the CNR (contrast-to-noise ratio [1]) via the amplitude of idealSignal and standard deviation of the noise.

Parameters:

• idealSignal (1-D array-like) – The ideal signal, based in the currentSignal.
• noiseSignal (1-D array-like) – Noise apply to ideal signal, could be a initial or residual noise.

Returns:

Ratio of maximum distance of zero and standard deviation of the noise.

Return type:

float

signals.differential_snr_dB(initialSignal, finalSignal, method='square_mean_error', idealSignal=None)

Calculate the SNR or CNR difference between two signals: after and before filtering. Ideal signal may be used.

Parameters:

• initialSignal (1-D array-like) – Initial Signal, before the filtering process
• finalSignal (1-D array-like) – Final Signal, after the filtering process
• method (string, optional) – Is ‘square_mean_error’ by default, other forms of calculate the SNR differential is ‘mean_StandardNoise’, ‘variances’ and ‘amplitude_standardNoise’.
• idealSignal (1-D array-like or 0, optional) – Is 0 by default, is necessary in all methods except in ‘square_mean_error’ method.

Returns:

The SNR differential value in dB.

Return type:

float

signals.dopplerFunction(dim=1024, normalize=True, fq=0)

Generate the Doppler function in a range of 0 to 1, with dim points.

Parameters:

• dim (int) – Dimension of the signal.
• normalize (bool, optional) – It is True by default. This parameter normalize the data values in a range of 0 to 1 with a function present in statsWaveletFilt.miscellaneous.
• fq (int or float, optional) – It is 0 by default. With this default value the original doppler, shown by Donoho [1] will be used.

Returns:

[0] 1-D array-like, coordinates in X axis and [1] 1-D array-like, coordinates in Y axis

Return type:

tuple

References

[1]	DONOHO, D. L.; JOHNSTONE, I. M. Ideal spatial adaptation via wavelet shrinkage. Biometrika, v. 81, p. 425–455, 1994.

signals.heavsineFunction(dim=1024, normalize=True, heavs=0)

Generate the Heavsine function in a range of 0 to 1, with dim points.

Parameters:

• dim (int) – Dimension of the signal.
• normalize (bool, optional) – It is True by default. This parameter normalize the data values in a range of 0 to 1 with a function present in statsWaveletFilt.miscellaneous.
• heavs (int or float, optional) – It is 0 by default. This parameter, called * heavs * is the number of discontinuities in the heavens characteristic signal shown by Donoho [1] with 0 the signal will be the original, used in [1].

Returns:

[0] 1-D array-like, coordinates in X axis and [1] 1-D array-like, coordinates in Y axis

Return type:

tuple

References

[1]	DONOHO, D. L.; JOHNSTONE, I. M. Ideal spatial adaptation via wavelet shrinkage. Biometrika, v. 81, p. 425–455, 1994.

signals.blockFunction(dim=1024, normalize=True, ht=0)

Generate the Block function in a range of 0 to 1, with dim points.

Parameters:

• dim (int) – Dimension of the signal.
• normalize (bool, optional) – It is True by default. This parameter normalize the data values in a range of 0 to 1 with a function present in statsWaveletFilt.miscellaneous.
• ht (int, optional) – It is 0 by default. The parameter called ht is the commutation characteristic of block signal. The default parameter will generate the signal shown in [1].

Returns:

[0] 1-D array-like, coordinates in X axis and [1] 1-D array-like, coordinates in Y axis

Return type:

tuple

References

[1]	DONOHO, D. L.; JOHNSTONE, I. M. Ideal spatial adaptation via wavelet shrinkage. Biometrika, v. 81, p. 425–455, 1994.

signals.bumpFunction(dim=1024, normalize=True, wht=0)

Generate the Bump function in a range of 0 to 1, with dim points. Take care to the representation limits of this function is blows infinity in Y axis.

Parameters:

• dim (int) – Dimension of the signal.
• normalize (bool, optional) – It is True by default. This parameter normalize the data values in a range of 0 to 1 with a function present in statsWaveletFilt.miscellaneous.
• wht (int, optional) – It is 0 by default. The parameter called wht is the number of peaks characteristic of bump signal. The default parameter will generate the signal shown in [1].

Returns:

[0] 1-D array-like, coordinates in X axis and [1] 1-D array-like, coordinates in Y axis

Return type:

tuple

References

[1]	DONOHO, D. L.; JOHNSTONE, I. M. Ideal spatial adaptation via wavelet shrinkage. Biometrika, v. 81, p. 425–455, 1994.

threshold module

Wavelet Based in CUSUM control chart for filtering signals Project (module statsWaveletFilt.threshold ): Statistic functions for obtain threshold values for wavelet coefficients based in some referenced works.

Created by Tiarles Guterres, 2018

threshold.lambdasVisuShrink(wavCoeff)

Computes the threshold value (lambda) by VisuShrink [1] method.

Parameters:wavCoeff (list of lists or array-like) – Wavelet coefficients Returns:The threshold values for each wavelet coefficients vector. Return type:list of float

See also

filtration()

Function that use this function for filter via wavelet coefficients

pywt.wavedec()

Function that decomposes the signal in wavelet and scale coefficients

pywt.waverec()

Function that recomposes the signal from wavelet and scale coefficients

References

[1]	DONOHO, D. L.; JOHNSTONE, I. M. Ideal spatial adaptation via wavelet shrinkage. Biometrika, v. 81, p. 425–455, 1994.

threshold.lambdasSureShrink(wavCoeff, dim_t=1024)

Computes the threshold value (lambda) by SureShrink [1] method. It’s showed also in [2].

Note

After the test (via pytest) the fuction was changed for better performance.

Parameters:

• wavCoeff (list of lists or array-like) – Wavelet coefficients
• dim_t (optional, 1024 by default. t-dimension. Input vector from) – internal function _sure(vector, dim_t).

Returns:

The threshold values for each wavelet coefficients vector.

Return type:

list of float

See also

filtration()

Function that use this function to filter via wavelet coefficients

pywt.wavedec()

Function that decomposes the signal in wavelet and scale coefficients

pywt.waverec()

Function that recomposes the signal from wavelet and scale coefficients

References

[1]	DONOHO, D. L.; JOHNSTONE, I. M. Ideal spatial adaptation via wavelet shrinkage. Biometrika, v. 81, p. 425–455, 1994.

[2]	KOZAKEVICIUS, A. D. J.; BAYER, F. M. Filtragem de sinais via limiarização de coeficientes wavelet. Ciência e Natura, v. 36, p. 37–51, 2014. In portuguese.

threshold.lambdasBayesShrink(wavCoeff)

Computes the threshold value (lambda) by BayesShrink [1] method. It’s showed also in [2].

Parameters:wavCoeff (list of lists or array-like) – Wavelet coefficients Returns:The threshold values for each wavelet coefficients vector. Return type:list of float

See also

filtration()

Function that use this function to filter via wavelet coefficients

pywt.wavedec()

Function that decomposes the signal in wavelet and scale coefficients

pywt.waverec()

Function that recomposes the signal from wavelet and scale coefficients

References

[1]	CHANG, S. G.; YU, B.; VETTERLI, M. Adaptive wavelet thresholding for image denoising and compression. IEEE Transactions on Image Processing, v. 9, p. 1532–1546, 2000.

[2]	KOZAKEVICIUS, A. D. J.; BAYER, F. M. Filtragem de sinais via limiarização de coeficientes wavelet. Ciência e Natura, v. 36, p. 37–51, 2014. In portuguese.

threshold.lambdasSPC_Threshold(wavCoeff, p=3)

Computes the threshold value (lambda) by SPC-Threshold [1], [2] method

Note

After the test (via pytest) the fuction was changed for better performance.

Parameters:

• wavCoeff (list of lists or array-like) – Wavelet coefficients
• p (int or float) – Optional, 3 by default. Parameter for the algorithm [1], generally is used 2 or 3.

Returns:

The threshold values for each wavelet coefficients vector.

Return type:

list of float

See also

filtration()

Function that use this function to filter via wavelet coefficients

pywt.wavedec()

Function that decomposes the signal in wavelet and scale coefficients

pywt.waverec()

Function that recomposes the signal from wavelet and scale coefficients

References

[1]	BAYER, F. M.; KOZAKEVICIUS, A. J. SPC-threshold: uma proposta de limiarização para filtragem adaptativa de sinais. Tendências em Matemática Aplicada e Computacional, v. 11, n. 2, p. 121–132, 2010. In portuguese.

[2]	KOZAKEVICIUS, A. D. J.; BAYER, F. M. Filtragem de sinais via limiarização de coeficientes wavelet. Ciência e Natura, v. 36, p. 37–51, 2014. In portuguese.