Résumés
Abstract
Parameter estimation and model fitting underlie many statistical procedures. Whether the objective is to examine central tendency or the slope of a regression line, an estimation method must be used. Likelihood is the basis for parameter estimation, for determining the best relative fit among several statistical models, and for significance testing. In this review, the concept of Likelihood is explained and applied computation examples are given. The examples provided serve to illustrate how likelihood is relevant, and related to, the most frequently applied test statistics (Student’s t-test, ANOVA). Additional examples illustrate the computation of Likelihood(s) using common population model assumptions (e.g., normality) and alternative assumptions for cases where data are non-normal. To further describe the interconnectedness of Likelihood and the Likelihood Ratio with modern test statistics, the relationship between Likelihood, Least Squares Modeling, and Bayesian Inference are discussed. Finally, the advantages and limitations of Likelihood methods are listed, alternatives to Likelihood are briefly reviewed, and R code to compute each of the examples in the text is provided.
Keywords:
- parameter estimation,
- modeling Likelihood,
- Likelihood ratio,
- R script
Résumé
L’estimation de paramètres et l’ajustement de modèles est au coeur de toutes procédures statistiques. Que l’objectif soit d’examiner la tendance centrale ou une pente de régression, une méthode d’estimation est nécessaire. La fonction de vraisemblance est la pierre angulaire sur laquelle repose l’estimation de paramètres, les tests d’hypothèses et la comparaison de modèles. Cet article présente le concept de vraisemblance et les tests statistiques communément utilisés (tests t, ANOVA). Certains exemples présentent le calcul de la fonction de vraisemblance lorsque le postulat de normalité est présent et lorsqu’il n’est pas adéquat. Les liens entre vraisemblance, rapport de vraisemblance, méthodes des moindres carrés et bayésienne sont discutés. Finalement, les forces et les faiblesses des méthodes basées sur la vraisemblance sont énumérées et des méthodes alternatives sont mentionnées. Des instructions en R sont données pour tester les exemples du texte.
Mots-clés :
- estimation de paramètres,
- modélisation,
- vraisemblance,
- rapport de vraisemblance,
- programme R
Resumo
A estimativa de parâmetros e o ajustamento de modelos está no cerne de todos os procedimentos estatísticos. Se o objetivo é analisar a tendência central ou uma inclinação de regressão, é necessário um método de estimativa. A função de verossimilhança é a pedra angular sobre a qual assentam a estimativa de parâmetros, os testes de hipóteses e a comparação de modelos. Este artigo introduz o conceito de verosimilhança e os testes estatísticos vulgarmente utilizados (testes t, ANOVA). Alguns exemplos mostram o cálculo da função de verossimilhança quando o pressuposto de normalidade está presente e sempre que não é adequado. Discutem-se as ligações entre a verosimilhança, razão de verossimilhança, os métodos dos mínimos quadrados e o bayesianismo. Por fim, são enumeradas as forças e as fraquezas dos métodos baseados na verosimilhança e são mencionados os métodos alternativos. As instruções em R são dadas para testar os exemplos do texto.
Palavras chaves:
- estimativa de parâmetros,
- modelização,
- verosimilhança,
- razão de verossimilhança,
- o programa R
Parties annexes
References
- Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716–723. doi: 10.1109/TAC.1974.1100705
- Birnbaum, A. (1969). Statistical theory for logistic mental test models with a prior distribution of ability. Journal of Mathematical Psychology, 6, 258-276. doi: 10. 1016/ 0022-2496(69)90005-4
- Bozdogan, H. (1987). Model selection and Akaike’s information criterion (AIC): The general theory and its analytical extensions. Psychometrika, 52(3), 345-370. doi: 10.1007/BF02294361
- Brown, S., & Heathcote, A. (2003). QMLE: Fast, robust and efficient estimation of distribution functions based on quantiles. Behavior Research Methods, Instruments, & Computers, 35, 485-492. doi: 10.3758/BF03195527
- Burnham, K., & Anderson, D. R. (2004). Multimodel interference, understanding AIC and BIC in model selection. Sociological Methods & Research, 33(2), 261–304. doi: 10.1177/0049124104268644
- Cheng, R. C. H., & Amin, N. A. K. (1983). Estimating parameters in continuous univariate distributions with a shifted origin. Journal of the Royal Statistical Society B, 45, 394–403. doi: 10.2307/2345411
- Chernoff, H. (1954). On the distribution of the likelihood ratio. The Annals of Mathematical Statistics, 25(3). 573-578. doi: 10.1214/aoms/1177728725
- Cohen, J., & Cohen, P. (1975). Applied Multiple Regression/correlation analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum Associates.
- Cousineau, D. (2009). Nearly unbiased estimates of the three-parameter Weibull distribution with greater efficiency than the iterative likelihood method. British Journal of Mathematical and Statistical Psychology, 62, 167–191. doi: 10.1348/000711007X270843
- Cousineau, D. (2010). Panorama des statistiques pour psychologues. Bruxelles, Belgique: Les éditions de Boeck Université.
- Cousineau, D., Brown, S., & Heathcote, A. (2004). Fitting distributions using maximum likelihood: Methods and packages. Behavior Research Methods, Instruments, & Computers, 36, 742–756. doi: 10.3758/BF03206555
- Cousineau, D., & Hélie, S. (2013). Improving maximum likelihood estimation using prior probabilities: Application to the 3-parameter Weibull distribution. Tutorials in Quantitative Methods for Psychology, 9, 61–71.
- Daszykowksi, M., Kaczmarek, K., Vander Heyden, Y., & Walczak, B. (2007). Robust statistics in concept analysis - A review: Basic concepts. Chemometrics and Intelligent Laboratory Systems, 85, 203–219. doi: 10.1016/j.chemolab.2006.06.016
- Fisher, R. A. (1925). Statistical Methods for Research Workers. Edinburgh, Scotland: Oliver and Boyd.
- Forbes, C., Evans, M., Hastings, N., & Peacock, B. (2010). Statistical Distributions. New York, NY: Wiley.
- Glover, S., & Dixon, P. (2004). Likelihood ratios: A simple and flexible statistic for emprical psychologists. Psychonomic Bulletin & Review, 11, 791–806. doi: 10.3758/BF03196706
- Grünwald, P. (2000). Model Selection based on minimum description length. Journal of Mathematical Psychology, 44, 133–152. doi: 10.1006/jmps.1999.1280
- Hadfield, J. D. (2012). MasterBayes: Maximum Likelihood and Markov chain Monte Carlo methods for pedigree reconstruction, analysis and simulation. Retrieved from: http://cran.r-project.org/web/packages/.
- Hays, W. L. (1973). Statistics for the social sciences. New York, NY: Holt, Rinehart and Winston, Inc.
- Heathcote, A., Brown, S., & Cousineau, D. (2004). QMPE: Estimating lognormal, Wald and Weibull RT distributions with a parameter dependent lower bound. Behavior Research Methods, Instruments, & Computers, 36, 277–290. doi: 10.3758/BF03195574
- Heathcote, A., Brown, S., & Mewhort, D. J. K. (2000). The power law repealed: The case for an exponential law of practice. Psychonomic Bulletin & Review, 7, 185–207. doi: 10.3758/BF03212979
- Hélie, S. (2006). An introduction to model selections: Tools and algorithms. Tutorials in Quantitative Methods for Psychology, 2, 1–10.
- Hurvich, C. M., & Tsai, C. L. (1989). Regression and time series model selection in small samples. Biometrika, 76(2), 297–307. doi: 10.2307/2336663
- Kiefer, N. M. (2005). Maximum likelihood estimation (MLE), Retrieved from: http://instruct1.cit.cornell.edu/courses/econ620/reviewm5.pdf
- Myung, I. J. (2000). The importance of complexity in model selection. Journal of Mathematical Psychology, 44, 190–204. doi: 10.1006/jmps.1999.1283
- Nachmias, J. (1981). On the psychometric function for contrast detection. Vision Research, 21, 215–223. doi: 10.1016/0042-6989(81)90115-2
- Nagatsuka, H., Kamakura, T., & Balakrishnan, N. (2013). A consistent method of estimation for the three-parameter Weibull distribution. Computational Statistics and Data Analysis, 58, 210–226. doi: 10.1016/j.csda.2012.09.005
- Nelder, J. A., & Mead, R. (1965). A simplex method for function minimization. The Computer Journal, 7, 308–313. doi: 10.1080/00401706.1975.10489269
- Neyman, J., & Pearson, E. S. (1933). On the problem of the most efficient tests of statistical hypotheses. Philisophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical of Physical Character, 231, 289–337. doi: 10.1098/rsta.1933.0009
- Ng, H. K. T., Luo, L., & Duan, F. (2011). Parameter estimation of three-parameter Weibull distribution based on progressively type-II censored samples. Journal of Statistical Computation and Simulation, 10, 1-18. doi: 10.1080/00949655.2011. 591797
- Rose, C., & Smith, M. D. (2001). Mathematical Statistics with Mathematica. New York, NY: Springer-Verlag.
- Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6(2), 461–464. doi: 10.1214/aos/1176344136
- Smith, J. D., & Minda, J. P. (2002). Distinguishing prototype-based and exemplar-based processes in dot-pattern category learning. Journal of Experimental Psychology: Learning, Memory and Cognition, 28, 800–811. doi: 10.1037/0278-7393.28.4.800
- Weston, R., & Gore, P. A. Jr. (2006). A brief guide to structural equation modeling, The Counseiling Psychologist, 34, 719–751. doi: 10.1177/0011000006286345
- Woltman, H., Feldstain, A., MacKay, J. C., & Rocchi, M. (2012) An introduction to hierarchical linear modeling, Tutorials in Quantitative Methods for Psychology, 8, 52–69.
- Wu, T.-J., Chen, P., & Yan, Y. (2013). The weighted average information criterion for multivariate regression model selection. Signal Processing, 93, 49-55. doi: 10.1016/S0167-7152(98)00003-0