Machine Learning in IIoT: Using K-means Clustering for Predictive Maintenance and OEE

Manufacturing process control relies on a loop of data collection, analysis, feedback, and feedback control. Data quality is essential for proper operations. There are many ways data quality is influenced and managed through manufacturing processes. Data is used in planning, architecture, process control during the actual production, data collection, and post-manufacturing analyses.   

In this article, we’ll explore how a relatively simple yet powerful machine learning (ML) technique called K-means clustering can be used to rapidly ingest and analyze data to gain powerful insights into the operation of your devices, machinery, and sensors in IoT and IIoT applications. By applying this technique, organizations can not only identify patterns in equipment performance but also predict potential failures, enabling effective predictive maintenance strategies. Furthermore, these insights can help optimize Overall Equipment Effectiveness (OEE) by reducing unplanned downtime, improving production efficiency, and enhancing the overall operational performance of industrial systems.

Background: The Reality of Data Quality in IIoT

In IIoT applications, sensors in the real world rarely behave in the predictable and consistent manner one might expect. Data quality and trust in data validity is absolutely essential to operating IIoT and other advanced industrial applications. As the level of criticality rises, the need for data assurance increases. Factors include safety, production fidelity, energy efficiency, and real-time control scenarios.

There are many factors that plague data quality and stand in the way of reliable machine and factory operations, including electromagnetic (EM) noise, sensor faults, operating conditions, temperature and humidity, mechanical resonance, hysteresis, and other factors that cause problems for data and signal reliability.

Optimized Signal Filtering in Industrial Systems: A Common Need

Signals have thresholds and ranges for what is considered acceptable, too low, too high, as well as other classifications of data points. In many cases, these thresholds are not fixed and cannot be determined a priori. That is, many real-world signals need to be measured first. Then these signals need to be processed, removing noise, and finally filtered and classified. While doing this is possible and perhaps easy after a signal or data stream has been collected, doing so in real-time or near-real time can be very challenging. The following diagram depicts the general classification problem:

Often, in both OT and IT spaces, we want to have the ability to deal with challenges of real-world signals - noise, errors, sensor faults, missing data, and more. All the while, people and systems need to react to real-world conditions and take action based on the current state of sensors, robots, and manufacturing equipment with high levels of confidence.

Traditional statistical methods have their limitations. Machine learning techniques build on traditional statistical methods, offering a good balance between ease-of-use and reliability. The following section summarizes main approaches to solving the data classification problem.

Data Analysis Methods

There are several main groupings of techniques to analyze data. These include statistical methods, machine learning methods, and more advanced techniques such as Artificial Intelligence (AI) models. The following table summarizes several analysis methods, their relative benefits and drawbacks, and lists a few examples of techniques, tools, and measures. 

Summary of Several Data Analysis Techniques

What is K-means Clustering?

K-means clustering is an unsupervised machine learning algorithm with the goal of assigning data points to a number of clusters. This assignment to clusters effectively partitions the data. The algorithm works to find these centers, called centroids, by first randomly assigning centers for a selected number of clusters. Then it repeatedly recalculates those centers based on the proximity of the data points around it by minimizing the variances within each cluster. The final partitions are called Voronoi cells which are a type of tessellation – a tiling to cover a plane completely. We’ll explore these details in future articles in this series. For now, let's get started with practical applications of K-means clustering.

Applications of K-means Cluster Data Analysis for IIoT

In this article and the series that follows, we touch briefly on a few common applications of K-means cluster analysis for:

• Equipment Baselining

• Overall Equipment Effectiveness (OEE) 

• Predictive Maintenance (PM)

Sample Use case: Predictive Maintenance and OEE for Pizza Dough Mixers

Overall Equipment Effectiveness (OEE) is a fundamental measurement used in manufacturing. OEE is calculated:

OEE = Availability x Performance x Quality

K-means cluster analysis can help manufacturers directly improve both availability and performance to drive OEE higher. Higher OEE results in lower costs, greater profit margins.

Example: Motors - a Central Part of Industrial Applications

It goes without saying that motors are essential to nearly every industrial operation - conveyors, pumps, stepper motors, compressors, autonomous mobile robots (AMRs), printers, laser engraving machines, CNC machines, welding robots, environmental controls, fans, and much more.

Being electromechanical physical devices, motors are dependent on and subject to many factors that affect their operation. Factors affecting motor operation and performance include:

• Brush wear

• Induction factors - hysteresis

• Environmental - temperature and humidity

• Coil winding insulation degradation

• Line-power quality - harmonics, noise, power fluctuations - brown-out, spikes. 

The example shown here is predictive maintenance for a fleet of pizza dough mixers at a fictional pizza dough manufacturer. Motor load in the form of the current draw is sampled once per hour at a frequency of 100 samples per second for 100 seconds, totaling 10,000 samples. The data analyzed in this article is synthesized by the gen_ts_data.py code. Any real-time data can be substituted and tested for suitability to the K-means method described in this article.

In a real-world scenario, the data is collected from sensors on each mixer and sent for analysis via HiveMQ Edge, which performs protocol conversion and transmits the data to a HiveMQ plant broker. The PM/OEE client subscribes to the data topic of interest, mixers/+/motor/+/current and performs the data analysis.

Upon completion of analysis, cluster assignments, centroid calculations, and related data are stored in the database of choice. Later, a manufacturing execution system (MES) or other analysis and control software client component reads the cluster analysis to take action. This action could be trending of data all the way up to isolation, triage, and replacement of potentially faulty equipment,

In this article, we focus more on the analysis and classification of the data. In future articles we will detail the HiveMQ Edge and MQTT transports along with storage in relational databases or time-series databases via HiveMQ’s rich Extensions system.

Overall Schematic:

Component functions:

PM/OEE MQTT client 

• Samples data from all motors in the pizza mixer fleet

• Calculates the centroids of the sampled current using K-means clustering

• Stores sampled data and centroids in time-series database like TimescaleDB or similar

MES or other control clients

• Processes stored data and centroids to detect which mixers may have under or over current conditions

• For those mixers that are indicated aberrant behavior, the MES Client sends a stop command to the affected mixer(s)

• Notifies the people in the bakery floor that a mixer is in danger of failing and needs servicing

HiveMQ Cluster

• Coordinates the receipt of data from HiveMQ Edge clients via MQTT bridge that read the state and operating parameters of the pizza dough mixers

• Coordinates publishing of the data to interested subscribers, the PM/OEE clients in this example

HiveMQ Edge

• Connects at the edge-of-network to translate various machine protocols coming from mixers and related dough and bakery systems into a standardized MQTT format for uniform data ingestion

• Converts data from disparate protocols in a Unified Namespace (UNS) to ease integration and actioning of data at the OT and IT levels within the bakery

Notes: 

• This processing and K-means clustering could be done at the edge, however it is out of scope for this article

• A later article in this series will examine the edge processing application for this use case

Code and Data

The following snippets of code are written in Python (>3.11) using Pandas, Scikit-Learn, Matplotlib. The full code with example data is available at my github repository: https://github.com/williamsommers-hmq/kmeans-demo/tree/main 

The data included here are synthetic. The purpose of this sample data is to show simply how the technique of K-means clustering can be applied to applications of baselining, OEE, and predictive maintenance.

Data 

In this example, the data is synthesized to simulate base current, noise, and fault injections representative of real-world motor current data. The code can easily be adapted to accommodate using real data measured from actual motors and other equipment.

Code 

Code Snippet 1 - Core K-means clustering

The following code shows the heart of the code to classify the data into 3 clusters - under, nominal, over current conditions:

In the above code snippet 1, we see that there are 3 clusters (k=3). Next, the code creates a KMeans object, calls the .fit() method. Note that K-means is typically used for multivariate clustering. In this case, the code reshapes the data into a 1-dimensional set (X.reshape(-1,1)) before calling the fit() method. Next, the code invokes the .predict() method, again reshaping data into one dimension. Lastly, the code stores the centroids of the clusters.

These centroids represent the center of the cluster and can be thought of as similar to the average value of that cluster of data. Computed centroids are used to compare measurements values for classification, answering the question, “Which cluster does the current measure belong to?” Or, in the case of this experimental setup, “Is the motor current too low, too high, or just right?”. Then action can be taken in the forms of alarms, alerting, or long-term analysis of equipment. 

The publisher code (include sample and reference) produces synthetic data, publishes to the MQTT topic:

on the HiveMQ Broker.  

Next, the subscriber client, the one doing the K-means clustering analysis, picks up the data and executes the core K-means analysis code, that will republish the results on an MQTT topic:

Results of K-means Clustering on the Data 

The code snippet included in the Github repository summarizes the data using Pandas dataframe description() method. Here time and the composite signal columns are of interest. The other columns are used to synthesize the sample data. In the case of real-world measured data, only time and signal would be relevant.

 With traditional statistical methods only we would have to rely on various five-number summary statistics like quartiles, upper or lower hinges, mean and standard deviation of the collected data to make inferences about the data and act upon that. While traditional methods are data-driven, they can be inflexible and hide data that is trending toward aberrant conditions. This type of blindspot prevents or severely limits one’s ability to predict failures or act on problems quickly.

The following chart shows a small, 200 data-point sample of the collected motor current:

In contrast, the K-means algorithm makes predictions about the data and assigns the data to clusters. After invoking the K-means clustering algorithm, the data is assigned to one of three (3) clusters:

We can see how the K-means cluster analysis assigns each data point to a cluster in the following 1-dimensional plot. The color assignments are as follows:

• Green represents potential over-current data

• Blue is potentially good or acceptable current values while

• Red-colored data points are potentially too low, or drop-out conditions

This prediction gives more information about potential problems such as under-current, over-current, or other data quality issues. Staff can potentially see a problem coming and act to remediate failures, preserve up-time, and save money while preserving operational efficiency.

Usage of Analysis Results for Optimizing Operations in IIoT

In the sample code, we can use the cluster predictions given by the scikit-learn KMeans algorithm to test data in near-realtime and provide immediate feedback about potential under, over, or good measurement. This cluster labeling or direct prediction of datapoints with the kmeans.predict() method can be applied generally or selectively to test future measured data against previous observations.   

Manufacturing staff can use these predictions to gain instant and ongoing insights into the behavior, baselining, and tracking of overall equipment effectiveness (OEE). 

Conclusion

K-means clustering is a powerful unsupervised learning technique that can be used at the Edge, Core or anywhere along the data pipeline to analyze, classify, categorize, baseline IIoT device and sensor data. With its relatively low computational overhead, K-means clustering is an attractive method to use at the Edge to enhance your IoT, IIoT, Smart Manufacturing, and Industry 4.0 processes.

In Part 2 of this series, we’ll explore how Kernel Density Estimation (KDE) can be used as another method for analyzing and classifying data. Stay tuned!