 | :: Bias-Reduced Logistic Regression - Free Statistics Software (Calculator) :: | |
|
All rights reserved. The non-commercial (academic) use of this software is free of charge. The only thing that is asked in return is to cite this software when results are used in publications.
This free online software (calculator) computes the Bias-Reduced Logistic Regression (maximum penalized likelihood) as proposed by David Firth.
The penalty function is the Jeffreys invariant prior which removes the O(1/n) term from the asymptotic bias of estimated coefficients (Firth, 1993). It always yields finite estimates and standard errors (unlike the Maximum Likelihood Estimation in situations of (quasi-)complete discrimination).
Note: the first column (=endogenous series) must be a binary variable.
Click here to edit the underlying code of this R Module.Enter (or paste) a matrix (table) containing all data (time) series. Every column represents a different variable and must be delimited by a space or Tab. Every row represents a period in time (or category) and must be delimited by hard returns. The easiest way to enter data is to copy and paste a block of spreadsheet cells. Please, do not use commas or spaces to seperate groups of digits! Click here to edit the underlying code of this R Module.
| Cite this software as: | | Wessa P., (2008), Bias Reduced Logistic Regression (v1.0.2) in Free Statistics Software (v1.1.22-r6), Office for Research Development and Education, URL http://www.wessa.net/rwasp_logisticregression.wasp/ | | | The R code is based on : | | | Firth, D. (1993) Bias reduction of maximum likelihood estimates., Biometrika, 80, 27-38. | | | Firth, D. (1992) Bias reduction, the Jeffreys prior and GLIM. In Advances in GLIM and Statistical Modelling, Eds. L Fahrmeir, B J Francis, R Gilchrist and G Tutz, pp91-100., New York: Springer.
| | | Heinze, G. and Schemper, M. (2002) A solution to the problem of separation in logistic regression., Statistics in Medicine, 21, 2409-2419.
| | | Zorn, C (2005). A solution to separation in binary response models., Political Analysis, 13, 157-170. | | | P.D.M. Macdonald, R Functions for ROC Curves and the Hosmer-Lemeshow Test, URL http://www.math.mcmaster.ca/peter/s4f03/s4f03_0607/rochl.html, retrieved 2008/01/28 | | Viv Bewick, Liz Cheek, and Jonathan Ball, Statistics review 14: Logistic regression, Crit Care. 2005; 9(1): 112–118.
Published online 2005 January 13. doi: 10.1186/cc3045.
URL http://ccforum.com/content/9/1/112 | | | | |
|
| Source code of R module | | 1 | library(brlr) | | 2 | roc.plot <- function (sd, sdc, newplot = TRUE, ...) | | 3 | { | | 4 | sall <- sort(c(sd, sdc)) | | 5 | sens <- 0 | | 6 | specc <- 0 | | 7 | for (i in length(sall):1) { | | 8 | sens <- c(sens, mean(sd >= sall[i], na.rm = T)) | | 9 | specc <- c(specc, mean(sdc >= sall[i], na.rm = T)) | | 10 | } | | 11 | if (newplot) { | | 12 | plot(specc, sens, xlim = c(0, 1), ylim = c(0, 1), type = "l", | | 13 | xlab = "1-specificity", ylab = "sensitivity", main = "ROC plot", ...) | | 14 | abline(0, 1) | | 15 | } | | 16 | else lines(specc, sens, ...) | | 17 | npoints <- length(sens) | | 18 | area <- sum(0.5 * (sens[-1] + sens[-npoints]) * (specc[-1] - | | 19 | specc[-npoints])) | | 20 | lift <- (sens - specc)[-1] | | 21 | cutoff <- sall[lift == max(lift)][1] | | 22 | sensopt <- sens[-1][lift == max(lift)][1] | | 23 | specopt <- 1 - specc[-1][lift == max(lift)][1] | | 24 | list(area = area, cutoff = cutoff, sensopt = sensopt, specopt = specopt) | | 25 | } | | 26 | roc.analysis <- function (object, newdata = NULL, newplot = TRUE, ...) | | 27 | { | | 28 | if (is.null(newdata)) { | | 29 | sd <- object$fitted[object$y == 1] | | 30 | sdc <- object$fitted[object$y == 0] | | 31 | } | | 32 | else { | | 33 | sd <- predict(object, newdata, type = "response")[newdata$y == | | 34 | 1] | | 35 | sdc <- predict(object, newdata, type = "response")[newdata$y == | | 36 | 0] | | 37 | } | | 38 | roc.plot(sd, sdc, newplot, ...) | | 39 | } | | 40 | hosmerlem <- function (y, yhat, g = 10) | | 41 | { | | 42 | cutyhat <- cut(yhat, breaks = quantile(yhat, probs = seq(0, | | 43 | 1, 1/g)), include.lowest = T) | | 44 | obs <- xtabs(cbind(1 - y, y) ~ cutyhat) | | 45 | expect <- xtabs(cbind(1 - yhat, yhat) ~ cutyhat) | | 46 | chisq <- sum((obs - expect)^2/expect) | | 47 | P <- 1 - pchisq(chisq, g - 2) | | 48 | c("X^2" = chisq, Df = g - 2, "P(>Chi)" = P) | | 49 | } | | 50 | x <- as.data.frame(t(y)) | | 51 | r <- brlr(x) | | 52 | summary(r) | | 53 | rc <- summary(r)$coeff | | 54 | hm <- hosmerlem(y[1,],r$fitted.values) | | 55 | hm | | 56 | bitmap(file="test0.png") | | 57 | ra <- roc.analysis(r) | | 58 | dev.off() | | 59 | te <- array(0,dim=c(2,99)) | | 60 | for (i in 1:99) { | | 61 | threshold <- i / 100 | | 62 | numcorr1 <- 0 | | 63 | numfaul1 <- 0 | | 64 | numcorr0 <- 0 | | 65 | numfaul0 <- 0 | | 66 | for (j in 1:length(r$fitted.values)) { | | 67 | if (y[1,j] > 0.99) { | | 68 | if (r$fitted.values[j] >= threshold) numcorr1 = numcorr1 + 1 else numfaul1 = numfaul1 + 1 | | 69 | } else { | | 70 | if (r$fitted.values[j] < threshold) numcorr0 = numcorr0 + 1 else numfaul0 = numfaul0 + 1 | | 71 | } | | 72 | } | | 73 | te[1,i] <- numfaul1 / (numfaul1 + numcorr1) | | 74 | te[2,i] <- numfaul0 / (numfaul0 + numcorr0) | | 75 | } | | 76 | bitmap(file="test1.png") | | 77 | op <- par(mfrow=c(2,2)) | | 78 | plot((1:99)/100,te[1,],xlab="Threshold",ylab="Type I error", main="1 - Specificity") | | 79 | plot((1:99)/100,te[2,],xlab="Threshold",ylab="Type II error", main="1 - Sensitivity") | | 80 | plot(te[1,],te[2,],xlab="Type I error",ylab="Type II error", main="(1-Sens.) vs (1-Spec.)") | | 81 | plot((1:99)/100,te[1,]+te[2,],xlab="Threshold",ylab="Sum of Type I & II error", main="(1-Sens.) + (1-Spec.)") | | 82 | par(op) | | 83 | dev.off() | | 84 | load(file="createtable") | | 85 | a<-table.start() | | 86 | a<-table.row.start(a) | | 87 | a<-table.element(a,"Coefficients of Bias-Reduced Logistic Regression",5,TRUE) | | 88 | a<-table.row.end(a) | | 89 | a<-table.row.start(a) | | 90 | a<-table.element(a,"Variable",header=TRUE) | | 91 | a<-table.element(a,"Parameter",header=TRUE) | | 92 | a<-table.element(a,"S.E.",header=TRUE) | | 93 | a<-table.element(a,"t-stat",header=TRUE) | | 94 | a<-table.element(a,"2-sided p-value",header=TRUE) | | 95 | a<-table.row.end(a) | | 96 | for (i in 1:length(rc[,1])) { | | 97 | a<-table.row.start(a) | | 98 | a<-table.element(a,labels(rc)[[1]][i],header=TRUE) | | 99 | a<-table.element(a,rc[i,1]) | | 100 | a<-table.element(a,rc[i,2]) | | 101 | a<-table.element(a,rc[i,3]) | | 102 | a<-table.element(a,2*(1-pt(abs(rc[i,3]),r$df.residual))) | | 103 | a<-table.row.end(a) | | 104 | } | | 105 | a<-table.end(a) | | 106 | table.save(a,file="mytable.tab") | | 107 | a<-table.start() | | 108 | a<-table.row.start(a) | | 109 | a<-table.element(a,"Summary of Bias-Reduced Logistic Regression",2,TRUE) | | 110 | a<-table.row.end(a) | | 111 | a<-table.row.start(a) | | 112 | a<-table.element(a,"Deviance",1,TRUE) | | 113 | a<-table.element(a,r$deviance) | | 114 | a<-table.row.end(a) | | 115 | a<-table.row.start(a) | | 116 | a<-table.element(a,"Penalized deviance",1,TRUE) | | 117 | a<-table.element(a,r$penalized.deviance) | | 118 | a<-table.row.end(a) | | 119 | a<-table.row.start(a) | | 120 | a<-table.element(a,"Residual Degrees of Freedom",1,TRUE) | | 121 | a<-table.element(a,r$df.residual) | | 122 | a<-table.row.end(a) | | 123 | a<-table.row.start(a) | | 124 | a<-table.element(a,"ROC Area",1,TRUE) | | 125 | a<-table.element(a,ra$area) | | 126 | a<-table.row.end(a) | | 127 | a<-table.row.start(a) | | 128 | a<-table.element(a,"Hosmer–Lemeshow test",2,TRUE) | | 129 | a<-table.row.end(a) | | 130 | a<-table.row.start(a) | | 131 | a<-table.element(a,"Chi-square",1,TRUE) | | 132 | a<-table.element(a,hm[1]) | | 133 | a<-table.row.end(a) | | 134 | a<-table.row.start(a) | | 135 | a<-table.element(a,"Degrees of Freedom",1,TRUE) | | 136 | a<-table.element(a,hm[2]) | | 137 | a<-table.row.end(a) | | 138 | a<-table.row.start(a) | | 139 | a<-table.element(a,"P(>Chi)",1,TRUE) | | 140 | a<-table.element(a,hm[3]) | | 141 | a<-table.row.end(a) | | 142 | a<-table.end(a) | | 143 | table.save(a,file="mytable1.tab") | | 144 | a<-table.start() | | 145 | a<-table.row.start(a) | | 146 | a<-table.element(a,"Fit of Logistic Regression",4,TRUE) | | 147 | a<-table.row.end(a) | | 148 | a<-table.row.start(a) | | 149 | a<-table.element(a,"Index",1,TRUE) | | 150 | a<-table.element(a,"Actual",1,TRUE) | | 151 | a<-table.element(a,"Fitted",1,TRUE) | | 152 | a<-table.element(a,"Error",1,TRUE) | | 153 | a<-table.row.end(a) | | 154 | for (i in 1:length(r$fitted.values)) { | | 155 | a<-table.row.start(a) | | 156 | a<-table.element(a,i,1,TRUE) | | 157 | a<-table.element(a,y[1,i]) | | 158 | a<-table.element(a,r$fitted.values[i]) | | 159 | a<-table.element(a,y[1,i]-r$fitted.values[i]) | | 160 | a<-table.row.end(a) | | 161 | } | | 162 | a<-table.end(a) | | 163 | table.save(a,file="mytable2.tab") | | 164 | a<-table.start() | | 165 | a<-table.row.start(a) | | 166 | a<-table.element(a,"Type I & II errors for various threshold values",3,TRUE) | | 167 | a<-table.row.end(a) | | 168 | a<-table.row.start(a) | | 169 | a<-table.element(a,"Threshold",1,TRUE) | | 170 | a<-table.element(a,"Type I",1,TRUE) | | 171 | a<-table.element(a,"Type II",1,TRUE) | | 172 | a<-table.row.end(a) | | 173 | for (i in 1:99) { | | 174 | a<-table.row.start(a) | | 175 | a<-table.element(a,i/100,1,TRUE) | | 176 | a<-table.element(a,te[1,i]) | | 177 | a<-table.element(a,te[2,i]) | | 178 | a<-table.row.end(a) | | 179 | } | | 180 | a<-table.end(a) | | 181 | table.save(a,file="mytable3.tab") |
|
To cite Wessa.net in publications use:
Wessa, P. (2008), Free Statistics Software, Office for Research Development and Education,
version 1.1.22-r6, URL http://www.wessa.net/
© Resa
R&D, and the 'Office for Research, Development, and Education' - All rights reserved. Academic license for non-commercial use only.
This website is an intellectual
and physical property of Resa R&D. This includes: html content,
graphical illustrations (gif files), computer software, online or electronic
documentation, associated media, and printed materials. The free use of the scientific content, services, and applications in this website is
granted for non commercial use only. In any case,
the source (url) should always be clearly displayed. Under no circumstances are
you allowed to reproduce, copy or redistribute the design, layout, or any
content of this website (for commercial use) including any materials contained
herein without the express written permission of Resa R&D or the Office for Research, Development, and Education.
Information provided
on this web site is provided "AS IS" without warranty of any kind, either
express or implied, including, without limitation, warranties of
merchantability, fitness for a particular purpose, and noninfringement. We use reasonable efforts to include accurate and timely information
and periodically update the information, and software without notice. However, Resa
R&D makes no warranties or representations
as to the accuracy or completeness of such information (or software), and it assumes no
liability or responsibility for errors or omissions in the content of this web
site, or any software bugs in online applications. Your use of this web site is AT YOUR OWN RISK. Under no circumstances and
under no legal theory shall Resa R&D be liable to you or any other
person for any direct, indirect, special, incidental, exemplary, or
consequential damages arising from your access to, or use of, this web site.
Software Version : 1.1.22-r6
Algorithms & Software : Prof. dr. P. Wessa
Facilities : Resa R&D - Office for Research, Development, and Education
Server : www.wessa.net
Comments, Feedback & Errors | Privacy Policy | Statistics Resources | Wessa.net Home
|
|
|
