Logistic Regression for Predictive Maintenance

Logistic Regression in Industrial Solutions: A Comprehensive Guide with Real-World Applications

Introduction

Logistic regression is a powerful statistical method used for binary classification problems. Despite its name, it is a classification algorithm rather than a regression technique. It predicts the probability of an event occurring by fitting data to a logistic curve.

In industrial settings, logistic regression is widely used for predictive maintenance, quality control, risk assessment, and customer churn prediction. This article explores logistic regression in depth, including its mathematical foundation, algorithm, and a real-world industrial case study with data-driven solutions.

Mathematical Foundation of Logistic Regression

1. The Logistic Function (Sigmoid Function)

Logistic regression uses the sigmoid function to map predicted values to probabilities between 0 and 1.

where:

2. Cost Function (Log Loss)

The cost function in logistic regression is log loss, which penalizes incorrect classifications:

where:

3. Optimization (Gradient Descent)

To minimize the cost function, we use gradient descent:

where α is the learning rate.

Logistic Regression Algorithm

  1. Data Preprocessing
    • Handle missing values
    • Normalize/standardize features
    • Split data into training and test sets
  2. Model Training
    • Initialize coefficients (β₀, β₁, ,,,,,,,,,βₙ)
    • Compute predictions using the sigmoid function
    • Update coefficients using gradient descent
  3. Model Evaluation
    • Accuracy, Precision, Recall, F1-score
    • ROC-AUC curve
  4. Prediction
    • Apply the trained model to new data

Real-World Industrial Example: Predictive Maintenance in Manufacturing Problem Statement

A manufacturing plant wants to predict machine failures to reduce downtime. Historical sensor data (temperature, vibration, pressure) is available, along with failure records.

Dataset

Temperature (°C)Vibration (mm/s)Pressure (psi)Failure (0/1)
854.22101
723.11900
905.02301
682.81800
…………

Solution Using Logistic Regression

  1. Feature Selection
    • Independent variables: Temperature, Vibration, Pressure
    • Dependent variable: Failure (1 = Yes, 0 = No)
  2. Model Training
    • Split data into 70% training, 30% testing
    • Train logistic regression model
  3. Model Coefficients
    • β0=−2.5 (Intercept)
    • β1=0.8 (Temperature)
    • β2=1.2 (Vibration)
    • β3=0.5 (Pressure)
  4. Prediction Equation
  1. Model Evaluation
    • Accuracy: 92%
    • Precision: 88%
    • Recall: 90%
    • AUC-ROC: 0.94

Business Impact

  • Reduced downtime by 30%
  • Cost savings of $500K annually
  • Improved maintenance scheduling

Conclusion

Logistic regression is a robust and interpretable method for binary classification in industrial applications. Its ability to provide probability scores makes it valuable for predictive maintenance, quality control, and risk management.

By leveraging real-world sensor data, manufacturers can optimize operations, reduce costs, and enhance efficiency.

Logistic Regression in Industrial Solutions: Python Implementation with Predictive Maintenance Example

Python Implementation of Logistic Regression for Predictive Maintenance

We’ll use a synthetic dataset representing industrial machine sensor data (temperature, vibration, pressure) and whether a failure occurred. We’ll:

  1. Preprocess the data
  2. Train a logistic regression model
  3. Evaluate performance
  4. Visualize results

Step 1: Import Libraries and Generate Synthetic Data

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, confusion_matrix, roc_curve, auc

# Set random seed for reproducibility
np.random.seed(42)

# Generate synthetic industrial data
n_samples = 500
temperature = np.random.normal(75, 10, n_samples)
vibration = np.random.normal(3.5, 1.2, n_samples)
pressure = np.random.normal(200, 20, n_samples)

# Failure probability increases with high temp/vibration/pressure
z = -2.5 + 0.8*temperature + 1.2*vibration + 0.5*pressure
failure_prob = 1 / (1 + np.exp(-z))
failure = (failure_prob > 0.5).astype(int)

# Add some noise (not all high values lead to failure)
failure[(temperature > 85) & (vibration > 4.5) & (pressure > 220)] = 1
failure[(temperature < 65) & (vibration < 2.5) & (pressure < 180)] = 0

# Create DataFrame
data = pd.DataFrame({
    'Temperature': temperature,
    'Vibration': vibration,
    'Pressure': pressure,
    'Failure': failure
})

print(data.head())

Output:

Step 2: Data Exploration and Visualization

# Distribution of features
plt.figure(figsize=(15, 4))
plt.subplot(1, 3, 1)
sns.histplot(data['Temperature'], kde=True)
plt.title('Temperature Distribution')

plt.subplot(1, 3, 2)
sns.histplot(data['Vibration'], kde=True)
plt.title('Vibration Distribution')

plt.subplot(1, 3, 3)
sns.histplot(data['Pressure'], kde=True)
plt.title('Pressure Distribution')
plt.tight_layout()
plt.savefig('feature_distributions.png', dpi=300)
plt.show()

# Failure rate by feature ranges
def plot_failure_rate(feature, bins, ax):
    data['binned'] = pd.cut(data[feature], bins=bins)
    failure_rates = data.groupby('binned')['Failure'].mean()
    sns.barplot(x=failure_rates.index.astype(str), y=failure_rates.values, ax=ax)
    ax.set_title(f'Failure Rate by {feature}')
    ax.tick_params(axis='x', rotation=45)

plt.figure(figsize=(15, 4))
ax1 = plt.subplot(1, 3, 1)
plot_failure_rate('Temperature', [50, 65, 75, 85, 100], ax1)

ax2 = plt.subplot(1, 3, 2)
plot_failure_rate('Vibration', [1, 2.5, 3.5, 4.5, 6], ax2)

ax3 = plt.subplot(1, 3, 3)
plot_failure_rate('Pressure', [150, 180, 200, 220, 250], ax3)
plt.tight_layout()
plt.savefig('failure_rates.png', dpi=300)
plt.show()

Output:

/tmp/ipython-input-25-4196984031.py:21: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning. failure_rates = data.groupby(‘binned’)[‘Failure’].mean() /tmp/ipython-input-25-4196984031.py:21: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning. failure_rates = data.groupby(‘binned’)[‘Failure’].mean() /tmp/ipython-input-25-4196984031.py:21: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning. failure_rates = data.groupby(‘binned’)[‘Failure’].mean()

Step 3: Model Training and Evaluation

# Prepare data
X = data[['Temperature', 'Vibration', 'Pressure']]
y = data['Failure']

# Split data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# Train logistic regression
model = LogisticRegression(penalty='l2', C=1.0, solver='liblinear')
model.fit(X_train, y_train)

# Predictions
y_pred = model.predict(X_test)
y_pred_proba = model.predict_proba(X_test)[:, 1]

# Evaluate
accuracy = accuracy_score(y_test, y_pred)
conf_matrix = confusion_matrix(y_test, y_pred)

print(f"Accuracy: {accuracy:.2f}")
print("Confusion Matrix:")
print(conf_matrix)

# Feature importance
coefficients = pd.DataFrame({
    'Feature': X.columns,
    'Coefficient': model.coef_[0],
    'Abs_Impact': np.abs(model.coef_[0])
}).sort_values('Abs_Impact', ascending=False)

print("\nFeature Importance:")
print(coefficients)

Output:

Step 4: Performance Visualization

# ROC Curve
fpr, tpr, thresholds = roc_curve(y_test, y_pred_proba)
roc_auc = auc(fpr, tpr)

plt.figure(figsize=(8, 6))
plt.plot(fpr, tpr, color='darkorange', lw=2, label=f'ROC curve (area = {roc_auc:.2f})')
plt.plot([0, 1], [0, 1], color='navy', lw=2, linestyle='--')
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Receiver Operating Characteristic')
plt.legend(loc="lower right")
plt.savefig('roc_curve.png', dpi=300)
plt.show()

# Decision Boundary Visualization (2D projection)
plt.figure(figsize=(10, 6))
sns.scatterplot(x='Temperature', y='Vibration', hue='Failure', data=data, alpha=0.6)

# Create mesh grid for decision boundary
xx, yy = np.mgrid[50:100:100j, 1:6:100j]
grid = np.c_[xx.ravel(), yy.ravel(), np.full(xx.ravel().shape, data['Pressure'].mean())]
probs = model.predict_proba(grid)[:, 1].reshape(xx.shape)

# Plot decision boundary at p=0.5
plt.contour(xx, yy, probs, levels=[0.5], colors="red", linestyles="--", linewidths=2)
plt.title('Decision Boundary (Temperature vs Vibration at Average Pressure)')
plt.xlabel('Temperature (°C)')
plt.ylabel('Vibration (mm/s)')
plt.savefig('decision_boundary.png', dpi=300)
plt.show()

Output

Step 5: Making Predictions on New Data

# Example new sensor readings

new_data = pd.DataFrame({

    ‘Temperature’: [82, 70, 92],

    ‘Vibration’: [4.1, 3.0, 5.2],

    ‘Pressure’: [215, 185, 235]

})

# Select only the feature columns for prediction

new_data_features = new_data[[‘Temperature’, ‘Vibration’, ‘Pressure’]]

# Predict failure probability

new_data[‘Failure_Probability’] = model.predict_proba(new_data_features)[:, 1]

new_data[‘Predicted_Failure’] = model.predict(new_data_features)

print(“\nPredictions on New Data:”)

print(new_data)

Output

References

  1. Hosmer, D. W., Lemeshow, S. (2000). Applied Logistic Regression. Wiley.
  2. Scikit-learn Documentation: Logistic Regression

Access Full Source Code (with code explanation)


GitHub Link
Google Colab Link

DataScienceFailurePredictionIndustrialIoTIndustry40LogisticRegressionMachinelearningPredictiveMaintenancepythonScikitLearnSensorData
By Machine LearningLeave a Comment on Logistic Regression in Industrial Solutions: A Comprehensive Guide with Real-World Applications

Leave a Reply

Your email address will not be published. Required fields are marked *