Why Cross-Entropy Beats Regression for [0, 1] Bounded Targets in LightGBM

TL;DR: If you are predicting a continuous target strictly bounded in \([0, 1]\) (such as probabilities, crop disease damage fractions, conversion rates, or pixel land-cover proportions), do not use standard regression (MSE / L2). Mean squared error produces unphysical out-of-bounds predictions (\(< 0\) or \(> 1\)), suffers from constant curvature, and treats catastrophic \(5\times\) relative errors near zero with negligible penalty. By switching to objective="cross_entropy" in LightGBM, predictions are passed through an internal sigmoid link \(\sigma(z) \in (0, 1)\), gradients automatically scale with relative odds (\(g_i = p_i - y_i\)), and the adaptive Hessian \(h_i = p_i(1 - p_i)\) stabilizes Newton-Raphson tree splits.

---

1. The Trap of Fractional & Probability Targets

In applied machine learning, we often need to predict continuous values that are inherently constrained to the unit interval \([0, 1]\):

• Fractional Crop Damage / Yield Loss: Proportion of field ruined by pest or hail (\(0.00 \to 1.00\)).
• Click-Through & Conversion Rates (CTR / CVR): Historical cohort probabilities.
• Remote Sensing Fractional Vegetation Cover: Percentage of pixel covered by canopy (\(0\% \to 100\%\)).
• Financial Default Probabilities: Calibrated risk scores.

What Happens When You Use Default objective="regression" (MSE)?

                         Standard MSE Regression on [0, 1] Target
      y ↑
  1.0 ┼───────────────────────────┬──────────────  Upper Physical Bound
      │                          / ◄── Prediction Overshoot (e.g. 1.28)
      │                         /
      │                        /
      │                       /
  0.0 ┼───────\──────────────┴───────────────────  Lower Physical Bound
      │        \ ◄── Negative Prediction (e.g. -0.15)
      └────────────────────────────────────────► Features x

1. Out-of-Bounds Predictions: Tree leaves simply sum additive scalar residuals. A leaf can easily output \(-0.18\) or \(+1.35\). Clamping predictions post-hoc (\([0, 1]\) clipping) destroys calibration and creates artificial spike artifacts at \(0\) and \(1\).
2. Homoskedasticity Assumption Failure: MSE assumes constant variance \(\sigma^2\) across the whole range. But \([0, 1]\) data is intrinsically heteroskedastic—its variance shrinks toward zero as \(y \to 0\) or \(y \to 1\) (just like a binomial variance \(\text{Var}(y) \propto y(1-y)\)).
3. The Relative Error Blind Spot:
4. If true \(y = 0.50\) and \(\hat{y} = 0.54\), the absolute error is \(0.04\) (a minor \(8\%\) deviation).
5. If true \(y = 0.01\) and \(\hat{y} = 0.05\), the absolute error is also \(0.04\). But this is a \(500\%\) relative overestimation!
6. To MSE, both errors yield the exact same tiny loss penalty: \((0.04)^2 = 0.0016\).

---

2. How cross_entropy for Continuous Targets Works

Most data scientists think of Cross-Entropy as a classification loss. However, Papke & Wooldridge (1996)[^1] established the Fractional Logit / Quasi-Likelihood framework: the Bernoulli log-likelihood is mathematically valid for any continuous fractional target \(y_i \in [0, 1]\).

LightGBM natively supports continuous \([0, 1]\) targets with objective="cross_entropy".

1. The Logit Parameterization (Sigmoid Link)

Instead of predicting \(\hat{y}\) directly, the decision trees output an unconstrained real-valued log-odds (logit) score \(z_i \in (-\infty, +\infty)\):

\[p_i = \sigma(z_i) = \frac{1}{1 + e^{-z_i}} \in (0, 1)\]

Because the sigmoid function asymptotically approaches \(0\) and \(1\), the model can never produce out-of-bounds predictions.

flowchart LR
    TreeEnsemble["Tree Leaves Output Raw Logits z_i ∈ (-∞, +∞)"] --> Sigmoid["Sigmoid Link Function: p_i = 1 / (1 + e^-z_i)"]
    Sigmoid --> BoundedPred["Guaranteed Bounded Output p_i ∈ (0, 1)"]
    BoundedPred --> CrossEntropyLoss["Continuous Cross-Entropy Loss: -[y ln(p) + (1-y) ln(1-p)]"]

    style TreeEnsemble fill:#1e293b,stroke:#475569,stroke-width:2px,color:#fff
    style Sigmoid fill:#0284c7,stroke:#0369a1,stroke-width:2px,color:#fff
    style BoundedPred fill:#059669,stroke:#10b981,stroke-width:2px,color:#fff
    style CrossEntropyLoss fill:#b91c1c,stroke:#ef4444,stroke-width:2px,color:#fff

2. The Continuous Cross-Entropy Loss Function

For a continuous target \(y_i \in [0, 1]\) and predicted probability \(p_i = \sigma(z_i)\):

\[\mathcal{L}(y_i, p_i) = - \left[ y_i \ln p_i + (1 - y_i) \ln(1 - p_i) \right]\]

Substitute \(p_i = \frac{1}{1 + e^{-z_i}}\) and \(1 - p_i = \frac{e^{-z_i}}{1 + e^{-z_i}} = \frac{1}{1 + e^{z_i}}\):

\[\mathcal{L}(y_i, z_i) = \ln\left(1 + e^{z_i}\right) - y_i z_i\]

3. First Derivative (Gradient \(g_i\))

To find how the loss changes with respect to the tree output logit \(z_i\):

\[g_i = \frac{\partial \mathcal{L}}{\partial z_i} = \frac{e^{z_i}}{1 + e^{z_i}} - y_i = \sigma(z_i) - y_i = p_i - y_i\]

The gradient is elegantly simple: the difference between predicted probability and true fractional target (\(p_i - y_i\)).

4. Second Derivative (Hessian \(h_i\))

Differentiating the gradient with respect to \(z_i\):

\[h_i = \frac{\partial^2 \mathcal{L}}{\partial z_i^2} = \frac{\partial}{\partial z_i} \left( \sigma(z_i) - y_i \right) = \sigma(z_i) \left( 1 - \sigma(z_i) \right) = p_i (1 - p_i)\]

The Hessian is self-adaptive: - When predictions are uncertain (\(p_i \approx 0.5\)), \(h_i = 0.25\) (maximum curvature, taking bold steps). - When predictions approach boundaries (\(p_i \to 0\) or \(p_i \to 1\)), \(h_i \to 0\), naturally preventing numerical divergence!

---

3. Tree Split Calculation: MSE vs. Cross-Entropy

In gradient boosting (LightGBM / XGBoost), when a decision tree creates a leaf node containing a subset of samples \(I\), the optimal leaf value \(w^*\) is computed via Newton-Raphson optimization[^2]:

\[w^* = - \frac{\sum_{i \in I} g_i}{\sum_{i \in I} h_i + \lambda}\]

Property	Standard Regression (MSE / L2)	Continuous Cross-Entropy (cross_entropy)
Output Space	Unbounded: \(\hat{y} \in (-\infty, +\infty)\)	Strictly Bounded: \(p \in (0, 1)\) via \(\sigma(z)\)
Gradient (\(g_i\))	\(2(\hat{y}_i - y_i)\)	\(p_i - y_i\)
Hessian (\(h_i\))	Constant: \(2.0\)	Adaptive: \(p_i(1 - p_i)\)
Optimal Leaf Value (\(w^*\))	Mean of residuals: $\frac{\sum (y_i - \hat{y}_i)}{	I
Loss Near Boundary (\(y=0.01, \hat{y}=0.05\))	Negligible penalty: \(0.0016\)	Heavy penalty: Steep log divergence
Probability Calibration	Poor (distorted by clamping)	Theoretically optimal (KL-divergence)

---

4. Concrete Numerical Walkthrough

Let's look at 4 real data points to see why cross-entropy behaves so much better near boundaries:

Sample 1: Rare event / low damage    ->  y₁ = 0.01
Sample 2: Low-moderate value         ->  y₂ = 0.20
Sample 3: High value                 ->  y₃ = 0.80
Sample 4: Near-saturation event      ->  y₄ = 0.99

Suppose a baseline model predicts \(p_i = 0.10\) for all samples (\(z_i = \ln(0.10 / 0.90) \approx -2.197\)):

Sample	True \(y_i\)	Prediction \(p_i\)	MSE Error \((\hat{y}_i - y_i)^2\)	Cross-Entropy Loss \(-[y\ln p + (1-y)\ln(1-p)]\)	Gradient \(g_i = p_i - y_i\)	Hessian \(h_i = p_i(1-p_i)\)
Sample 1	\(0.01\)	\(0.10\)	\(0.0081\) (tiny)	\(0.1274\) (sharp penalty)	\(+0.090\)	\(0.090\)
Sample 2	\(0.20\)	\(0.10\)	\(0.0100\)	\(0.5241\)	\(-0.100\)	\(0.090\)
Sample 3	\(0.80\)	\(0.10\)	\(0.4900\)	\(1.8631\)	\(-0.700\)	\(0.090\)
Sample 4	\(0.99\)	\(0.10\)	\(0.7921\)	\(2.2882\) (severe)	\(-0.890\)	\(0.090\)

Why This Matters for Tree Splits:

Notice Sample 1: The model predicted \(0.10\) when true was \(0.01\) (\(10\times\) over-prediction). - MSE gives this sample a loss of \(0.0081\)—virtually ignoring it during tree split finding. - Cross-Entropy computes a loss of \(0.1274\), forcing the gradient booster to split and correct the tail error!

---

5. Interactive Loss & Gradient Comparison

Compare how MSE and Cross-Entropy penalize predictions as you drag the slider across the unit interval:

⚡ Interactive Loss & Gradient Simulator

True Target Value ($y \in [0, 1]$): 0.05

Model Prediction ($\hat{y} \text{ or } p$): 0.25

Standard MSE Loss

0.0400

Hessian: 2.0 (Constant)

Cross-Entropy Loss

0.3421

Hessian: 0.1875

When the true value is near 0 (y=0.05), an overprediction of 0.25 is penalized 8.5x more aggressively under Cross-Entropy than MSE!

---

6. Complete Python Implementation with LightGBM

Here is a ready-to-run benchmark demonstrating how to train LightGBM with objective="cross_entropy" versus objective="regression" on a synthetic \([0, 1]\) fractional dataset:

View Full Python Benchmark Script (LightGBM vs MSE)

import numpy as np
import pandas as pd
from lightgbm import LGBMRegressor
from sklearn.metrics import mean_absolute_error, mean_squared_error
from sklearn.model_selection import train_test_split

# 1. Generate Synthetic [0, 1] Bounded Data (e.g. Beta-distributed crop damage)
np.random.seed(42)
N = 10_000
X = np.random.randn(N, 5)

# Latent log-odds response function
latent_z = 0.8 * X[:, 0] - 1.2 * X[:, 1] + 0.5 * (X[:, 2] ** 2) - 1.5
true_prob = 1.0 / (1.0 + np.exp(-latent_z))

# Add beta-distributed fractional noise
y = np.clip(np.random.beta(true_prob * 10, (1 - true_prob) * 10), 0.001, 0.999)

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

# =======================================================
# Model 1: Standard Regression (MSE / L2)
# =======================================================
model_mse = LGBMRegressor(
    objective="regression",
    n_estimators=150,
    learning_rate=0.05,
    random_state=42,
    verbose=-1
)
model_mse.fit(X_train, y_train)
pred_mse = model_mse.predict(X_test)

# Check for out-of-bounds predictions
out_of_bounds = np.sum((pred_mse < 0) | (pred_mse > 1))
print(f"MSE Model Out-of-Bounds Predictions: {out_of_bounds} / {len(y_test)}")

# =======================================================
# Model 2: Cross-Entropy for Continuous Targets
# =======================================================
model_ce = LGBMRegressor(
    objective="cross_entropy",
    n_estimators=150,
    learning_rate=0.05,
    random_state=42,
    verbose=-1
)
model_ce.fit(X_train, y_train)
pred_ce = model_ce.predict(X_test)

# =======================================================
# Model Evaluation
# =======================================================
def log_loss_continuous(y_true, y_pred, eps=1e-15):
    p = np.clip(y_pred, eps, 1 - eps)
    return -np.mean(y_true * np.log(p) + (1 - y_true) * np.log(1 - p))

print("\n--- BENCHMARK RESULTS ---")
print(f"MSE Model:      MAE = {mean_absolute_error(y_test, np.clip(pred_mse, 0, 1)):.4f} | LogLoss = {log_loss_continuous(y_test, np.clip(pred_mse, 0, 1)):.4f}")
print(f"Cross-Entropy:  MAE = {mean_absolute_error(y_test, pred_ce):.4f} | LogLoss = {log_loss_continuous(y_test, pred_ce):.4f}")

Key Implementation Takeaways:

1. No Extra Scaling Needed: You do not need to convert your continuous \([0, 1]\) target into binary \(0/1\) classes. Pass raw floats directly into LGBMRegressor(objective="cross_entropy").
2. Output Predictions Are Already Probabilities: When calling model.predict(X_test), LightGBM automatically applies the sigmoid transform \(\sigma(z)\), returning values strictly in \((0, 1)\).
3. Alternative in XGBoost / CatBoost:
4. In XGBoost: Use objective="reg:logistic" (logistic regression for continuous target).
5. In CatBoost: Use loss_function="Logloss" with continuous float targets.

---

7. When to Use Which Objective?

Target Characteristic	Recommended LightGBM Objective	Why?
Strictly Bounded in \([0, 1]\) (Fractions, Rates, Probabilities)	objective="cross_entropy"	Enforces \((0, 1)\) bounds, adaptive Hessian, and high tail sensitivity.
Unbounded Normal Targets (Temperature, Revenue)	objective="regression" (MSE)	Direct linear projection in \(\mathbb{R}\).
Skewed Positive Counts / Claims (\(y \ge 0\))	objective="poisson" or "tweedie"	Accounts for zero-inflation and Poisson dispersion.
Heavy-Tailed Outliers	objective="regression_l1" (MAE) or "huber"	Robust median-oriented loss.

---

References

[^1]: Papke, L. E. & Wooldridge, J. M. Econometric methods for fractional response variables with an application to 401(k) plan participation rates. J. Appl. Econom. 11, 619–632 (1996). https://doi.org/10.1002/(SICI)1099-1255(199611)11:6<619::AID-JAE418>3.0.CO;2-1 [^2]: Ke, G. et al. LightGBM: A highly efficient gradient boosting decision tree. Adv. Neural Inf. Process. Syst. 30, 3146–3154 (2017). https://papers.nips.cc/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html