mandag 13. mars 2017

Detect serial correlation in data with outliers

This is the sixth post in a series of six that describes mathematics for calculation of correlation and trend in data with outliers. The posts are numbered 1 to 6. They should be read consecutively.

Post 1  Introduction to Statistical analysis of data with outliers
Post 2  Correlation when outliers in the data.
Post 3  Trend when outliers in the data.
Post 4  Correlation and trend when an outlier is added. Example.
Post 5  Compare Kendall-Theil and OLS trends.            Simulations.
Post 6  Detect serial correlation when outliers.          Simulations.

The posts are gathered in this pdf document.

Start of post 6:    Detect serial correlation in data with outliers


This chapter deals with Monte Carlo simulations that calculate the serial correlation coefficients in noisy data. They are calculated with two different approaches. One uses the noise values, and the other uses the ranks of the noise values. Both approaches work well when the noise is white and when there is serial correlation in the noise. The approach that uses the ranks works much better than the other when there are outliers in the noise. The results are presented as probability density plots.

torsdag 9. mars 2017

Compare Kendall-Theil and OLS trends

This is the fifth post in a series of six that deals with mathematics for calculation of correlation and trend in data with outliers. The posts are numbered 1 to 6. They should be read consecutively.

Post 1  Introduction to Statistical analysis of data with outliers
Post 2  Correlation when outliers in the data.
Post 3  Trend when outliers in the data.
Post 4  Correlation and trend when an outlier is added. Example.
Post 5  Compare Kendall-Theil and OLS trends.        Simulations.
Post 6  Detect serial correlation when outliers.               Simulations.

The posts are gathered in this pdf document.

Start of post 5:  Compare Kendall-Theil and OLS trends


This blog post deals with Monte Carlo simulations that calculate trends. The calculations use both the Kendall-Theil (K-T) methodology and the Ordinary Least Squares (OLS) methodology. The post compares the results. Both methodologies work well when the noise in the dependent variable is white. They are about equal when there is serial correlation in the dependent variable. K-T is much more robust against outliers than OLS is. The results are presented in probability density plots.