Skip to main content

Table 5 Penalty functions and estimators for some group regularized regression methods used in this study

From: Genomic prediction using machine learning: a comparison of the performance of regularized regression, ensemble, instance-based and deep learning methods on synthetic and empirical data

Method

Penalty

Estimator

 

gbridge

\(p_{\lambda ,\gamma }(\varvec{\beta })= \lambda \sum \limits _{l=1}^L c_l\Vert \varvec{\beta }_{A_l}\Vert _1^{\gamma }\)

\(\widehat{\varvec{\beta }}_{\texttt {g}bridge}=\underset{\varvec{\beta }}{\textit{argmin}} \ \Big \{\text {RSS} + \lambda \sum \limits _{l=1}^L c_l\Vert \varvec{\beta }_{A_l}\Vert _1^{\gamma }\Big \}\)

(11)

gLASSO

\(\texttt{p}_{\lambda }(\varvec{\beta })=\lambda \sum \limits _{l=1}^L \sqrt{p_l}\Vert \varvec{\beta }_{A_l}\Vert _2\)

\(\widehat{\varvec{\beta }}_{\texttt {g}lasso}=\underset{\varvec{\beta }}{\textit{argmin}} \ \Big \{\text {RSS} + \lambda \sum \limits _{l=1}^L \sqrt{p_l}\Vert \varvec{\beta }_{A_l}\Vert _2\Big \}\)

(12)

sgLASSO

\(\texttt{p}_{\lambda ,\alpha }(\varvec{\beta })= \alpha \lambda ||\varvec{\beta }||_1 + (1-\alpha ) \lambda \sum \limits _{l=1}^L \sqrt{g_l}||\varvec{\beta }_l||_2\)

\(\widehat{\varvec{\beta }}_{\texttt {sg}lasso}\underset{\varvec{\beta }}{\textit{argmin}} \ \Big \{\text {RSS} + \alpha \lambda ||\varvec{\beta }||_1 + (1-\alpha ) \lambda \sum \limits _{l=1}^L \sqrt{g_l}||\varvec{\beta }_l||_2 \Big \}\)

(13)