On penalised likelihood and bias reduction

We propose a likelihood function endowed with a penalisation that reduces the bias of the maximum likelihood estimator in regular parametric models. The penalisation hinges on the first two derivatives of the log likelihood and can be computed numerically. The asymptotic properties and the sensitivity to nuisance parameters of the penalised likelihood and derived quantities are addressed. In models for stratfied data in a two-index asymptotic setting, the bias of the penalised profile score function is found to be equivalent to the bias of a modified profile score function.