TRAINING IN STATISTICS
SCI EAME
Hilversum
Speaker: Frank Sauvage
Mean & Variance estimators, notion of p-value
- The mean of all possible sample's means (X̄) is μX̄ = µ, the target population mean
- Variance of samples means decreases with sample size, n. In fact Var(X̄)=σ²/n in the normal case
- Range (X̄) decreases when n increases
- Different samplings bring slightly different results
- Central Limit Theorem: whatever the parent population's
distribution, X̄ follows a normal distribution of same mean µ if n is
large enough (>30)
Definition of α and β errors and the trade-off between them
We want to compare a snack's
saltiness to another one well known product.
In other words, is the snack B's saltiness different in average from the mean saltiness of snack A?
We first have to valid that the saltiness of snacks follows a normal distribution N(mu, sigma). Our example here is then a one-sample z-test for means comparisons (σ0 is known, unilateral test).
If the null hypothesis H0 is true, then saltiness measures from snacks A and snacks B follow the same law N(μ0, σ0) , known from a long history of snack A production.
We draw a sample from snack B production, measure its saltiness XB and compute the following statistic:
zobs=|XB-μ0|/σ0
Under H0, zobs follows the standard normal N(0, 1).
Then, 1-2*Phi(zobs)=p-value of the test (if bilateral) or 1-Phi(zobs)=p-value (if we know a priori the direction of the difference).
We conclude accordingly to the critical value associated to the chosen α and interpret the result.
In other words, is the snack B's saltiness different in average from the mean saltiness of snack A?
We first have to valid that the saltiness of snacks follows a normal distribution N(mu, sigma). Our example here is then a one-sample z-test for means comparisons (σ0 is known, unilateral test).
If the null hypothesis H0 is true, then saltiness measures from snacks A and snacks B follow the same law N(μ0, σ0) , known from a long history of snack A production.
We draw a sample from snack B production, measure its saltiness XB and compute the following statistic:
zobs=|XB-μ0|/σ0
Under H0, zobs follows the standard normal N(0, 1).
Then, 1-2*Phi(zobs)=p-value of the test (if bilateral) or 1-Phi(zobs)=p-value (if we know a priori the direction of the difference).
We conclude accordingly to the critical value associated to the chosen α and interpret the result.
| |
Test conclusion |
||
| H0 not rejected | H0 rejected | ||
|---|---|---|---|
| Unknown reality |
H0 true | Right decision, confidence level (1-α) | Type I error, α |
| H0 false | Type II error, β | Right decision, test power (1-β) | |
The heart of experimental design!
Table of sampling results:
Simulation of treatment comparison & t-test outcome
Linear model, variances & homoscedasticity importance
Signal Detection Theory principles, language and graphic notations
Starting point for signal detection theory: decision making takes place in the presence of some uncertainty.
Simple Forced Choice
A yes/no task: i) signal trials, which present one or more signals, and ii) noise trials, which present one or more noise stimuli. After each trial, the subjects indicate whether a signal was presented ("yes" or "no").
Two kinds of noise factors limit the subject’s performance: i) internal noise and ii) external noise.
Simple Forced Choice
A yes/no task: i) signal trials, which present one or more signals, and ii) noise trials, which present one or more noise stimuli. After each trial, the subjects indicate whether a signal was presented ("yes" or "no").
Two kinds of noise factors limit the subject’s performance: i) internal noise and ii) external noise.
- External noise: the signal will slightly vary from trial to trial.
- Internal noise: neural responses would be noisy, even if the stimulus was exactly the same on each trial.
- either there was a signal (plus noise) or there was no signal (noise alone).
- either the subjects perceived the signal ("yes") or they did not
(“no”).
- hit (signal present and subject says “yes”),
- miss (signal present and subject says “no”),
- false alarm (signal absent and subject says “yes”), and
- correct rejection (signal absent and subject says “no”).
- the stimulus strength, and
- the criterion, the subjective value of the decision variable above wich the subject answer "yes" (influenced by the personnal feeling about errors and the recommandations).
Decision making in the presence of uncertainty.
Table of sensory trials:
Summary of sensory trials:
The full range of the subjectâs options for criterion in a single curve:
A measure of sensitivity independant from the response bias under some assumptions: d'.
General statistics:
Arsham Hossein, Statistical Thinking for Managerial Decisions.
http://home.ubalt.edu/ntsbarsh/business-stat/opre504.htm (Lecture web site)
Park Hun Myoung, 2010, Hypothesis Testing and Statistical Power of a Test. The Trustees of Indiana University.
http://www.indiana.edu/~statmath/stat/all/power/power.pdf
Thurstonian models and Signal Detection Theory:
Christensen Rune H. B., 2012, Sensometrics: Thurstonian and Statistical Models. PhD thesis, Technical University of Denmark.
http://orbit.dtu.dk/fedora/objects/orbit:111007/datastreams/file_b3b75800-dc2a-4489-ad98-b2363833ec8a/content
This thesis includes the following seven methodological research papers:
Christensen, R. H. B. and P. B. Brockhoff (2009) Estimation and Inference in the Same-Different test. Food Quality and Preference, 20, 514-524.
Brockhoff, P. B. and R. H. B. Christensen (2010) Thurstonian models for sensory discrimination tests as generalized linear models. Food Quality and Preference, 21, 330-338.
Christensen, R. H. B., G. Cleaver and P. B. Brockhoff (2011) Statistical and Thurstonian models for the A-not A protocol with and without sureness. Food Quality and Preference, 22, 542-549.
Christensen, R. H. B., H.-S. Lee and P. B. Brockhoff (2012) Estimation of the Thurstonian model for the 2-AC protocol. Food Quality and Preference, 24, 119-128.
Christensen, R. H. B. and P. B. Brockhoff (2013) Analysis of sensory ratings data with cumulative link models. Journal of the French Statistical Society, 154 (3).
Christensen, R. H. B., J. M. Ennis, D. M. Ennis and P. B. Brockhoff (2014) Paired preference data with a no-preference option - statistical tests for comparison with placebo data. Food Quality and Preference, 32 (A), pp. 48-55.
Mortensen, S. B. and R. H. B. Christensen (2010) Flexible estimation of nonlinear mixed models via the multivariate Laplace approximation. Computational Statistics and Data Analysis, working paper.
Christensen Rune H. B., 2014, Statistical methodology for sensory discrimination tests and its implementation in sensR. sensR Package methodology presentation.
http://cran.r-project.org/web/packages/sensR/vignettes/methodology.pdf
Cleaver Graham, 2008, Discrimination Tests With Sureness: Thurstonian and R-Index Analysis. Sensometrics
http://www.sensometric.org/Resources/Documents/2008/C1-Cleaver.pdf
Hacker Michael & Ratcliff Roger, 1979, A revised table of d-prime for M-alternative forced choice. Perception & Psychophysics, 26 (2), pp. 168-170.
http://star.psy.ohio-state.edu/coglab/People/roger/pdf/Papers/perpsyph79.pdf
Heeger David, 1997, Signal Detection Theory.
http://www.cns.nyu.edu/~david/handouts/sdt-advanced.pdf
http://www.cns.nyu.edu/~david/courses/perceptionGrad/Lectures/Landy/sdt.pdf (lecture ppt)
Keating Pat, 2005, D-prime (signal detection) analysis. Web page.
http://www.linguistics.ucla.edu/faciliti/facilities/statistics/dprime.htm
Klein Stanley A., 2001, Measuring, estimating, and understanding the psychometric function: A commentary. Perception & Psychophysics, 63 (8), 1421-1455.
http://cornea.berkeley.edu/pubs/148.pdf
Lee Hye-Seong & Van Hout Danielle,2009, Quantification of Sensory and Food Quality: The R-Index Analysis. Journal of Food Science, 74 (6): R57-R64
http://onlinelibrary.wiley.com/doi/10.1111/j.1750-3841.2009.01204.x/pdf
Naes, T., P. B. Brockhoff, and O. Tomic, 2010, Statistics for sensory and consumer science. John Wiley & sons Ltd, 301 pp.
http://api.ning.com/files/z-Cs3-aWqh-pdtI06a0iu58ZnYYcDxW0xdHh2rIHKXouR1*s0h-9gz0KbTu9Zl9AVB5BnzB82wAbBs0scNQtcziOnwJyK4P1/2010.Statisticforsensoryandconsumer.pdf
Rousseau, Benoit, 2006, Indices of Sensory Difference: R-Index and d'. IFPress, 9(3) 2-3.
http://ifpress.com/publications-cat/technical-reports/ifpress-93-indices-of-sensory-difference-r-index-and-d/
Stanislaw Harold & Todorov Natasha, 1999, Calculation of signal detection theory measures. Behavior Research Methods, Instruments, & Computers, 31 (I), pp. 137-149.
http://people.brandeis.edu/~sekuler/stanislawTodorov1999.pdf
Arsham Hossein, Statistical Thinking for Managerial Decisions.
http://home.ubalt.edu/ntsbarsh/business-stat/opre504.htm (Lecture web site)
Park Hun Myoung, 2010, Hypothesis Testing and Statistical Power of a Test. The Trustees of Indiana University.
http://www.indiana.edu/~statmath/stat/all/power/power.pdf
Thurstonian models and Signal Detection Theory:
Christensen Rune H. B., 2012, Sensometrics: Thurstonian and Statistical Models. PhD thesis, Technical University of Denmark.
http://orbit.dtu.dk/fedora/objects/orbit:111007/datastreams/file_b3b75800-dc2a-4489-ad98-b2363833ec8a/content
This thesis includes the following seven methodological research papers:
Christensen, R. H. B. and P. B. Brockhoff (2009) Estimation and Inference in the Same-Different test. Food Quality and Preference, 20, 514-524.
Brockhoff, P. B. and R. H. B. Christensen (2010) Thurstonian models for sensory discrimination tests as generalized linear models. Food Quality and Preference, 21, 330-338.
Christensen, R. H. B., G. Cleaver and P. B. Brockhoff (2011) Statistical and Thurstonian models for the A-not A protocol with and without sureness. Food Quality and Preference, 22, 542-549.
Christensen, R. H. B., H.-S. Lee and P. B. Brockhoff (2012) Estimation of the Thurstonian model for the 2-AC protocol. Food Quality and Preference, 24, 119-128.
Christensen, R. H. B. and P. B. Brockhoff (2013) Analysis of sensory ratings data with cumulative link models. Journal of the French Statistical Society, 154 (3).
Christensen, R. H. B., J. M. Ennis, D. M. Ennis and P. B. Brockhoff (2014) Paired preference data with a no-preference option - statistical tests for comparison with placebo data. Food Quality and Preference, 32 (A), pp. 48-55.
Mortensen, S. B. and R. H. B. Christensen (2010) Flexible estimation of nonlinear mixed models via the multivariate Laplace approximation. Computational Statistics and Data Analysis, working paper.
Christensen Rune H. B., 2014, Statistical methodology for sensory discrimination tests and its implementation in sensR. sensR Package methodology presentation.
http://cran.r-project.org/web/packages/sensR/vignettes/methodology.pdf
Cleaver Graham, 2008, Discrimination Tests With Sureness: Thurstonian and R-Index Analysis. Sensometrics
http://www.sensometric.org/Resources/Documents/2008/C1-Cleaver.pdf
Hacker Michael & Ratcliff Roger, 1979, A revised table of d-prime for M-alternative forced choice. Perception & Psychophysics, 26 (2), pp. 168-170.
http://star.psy.ohio-state.edu/coglab/People/roger/pdf/Papers/perpsyph79.pdf
Heeger David, 1997, Signal Detection Theory.
http://www.cns.nyu.edu/~david/handouts/sdt-advanced.pdf
http://www.cns.nyu.edu/~david/courses/perceptionGrad/Lectures/Landy/sdt.pdf (lecture ppt)
Keating Pat, 2005, D-prime (signal detection) analysis. Web page.
http://www.linguistics.ucla.edu/faciliti/facilities/statistics/dprime.htm
Klein Stanley A., 2001, Measuring, estimating, and understanding the psychometric function: A commentary. Perception & Psychophysics, 63 (8), 1421-1455.
http://cornea.berkeley.edu/pubs/148.pdf
Lee Hye-Seong & Van Hout Danielle,2009, Quantification of Sensory and Food Quality: The R-Index Analysis. Journal of Food Science, 74 (6): R57-R64
http://onlinelibrary.wiley.com/doi/10.1111/j.1750-3841.2009.01204.x/pdf
Naes, T., P. B. Brockhoff, and O. Tomic, 2010, Statistics for sensory and consumer science. John Wiley & sons Ltd, 301 pp.
http://api.ning.com/files/z-Cs3-aWqh-pdtI06a0iu58ZnYYcDxW0xdHh2rIHKXouR1*s0h-9gz0KbTu9Zl9AVB5BnzB82wAbBs0scNQtcziOnwJyK4P1/2010.Statisticforsensoryandconsumer.pdf
Rousseau, Benoit, 2006, Indices of Sensory Difference: R-Index and d'. IFPress, 9(3) 2-3.
http://ifpress.com/publications-cat/technical-reports/ifpress-93-indices-of-sensory-difference-r-index-and-d/
Stanislaw Harold & Todorov Natasha, 1999, Calculation of signal detection theory measures. Behavior Research Methods, Instruments, & Computers, 31 (I), pp. 137-149.
http://people.brandeis.edu/~sekuler/stanislawTodorov1999.pdf