Smartic: A smart tool for Big Data analytics and IoT

Introduction

The Internet of Things (IoT) is reshaping communication with technologies and is becoming a vital part of the development of a smart environment dedicated to make our lives convenient and comfortable.¹ Several IoT application sectors like smart homes, smart cities,² smart healthcare, assisted driving, smart retail, and consumer goods like wearables and smartphones are already available.^3–5 IoT is built with electronics hardware, software, and connectivity, which enables device interaction and transfer of data. The IoT ecosystem generates massive amounts of data. This data could be analyzed to make business decisions,⁶ predict consumer behavior, or to bring solutions to problems that might exist.⁷ Big Data offers the solutions to handle various types of data on a large scale.

Big Data extends the possibility to conduct extensive and rich analyses utilizing a vast amount of data.^4,8 Standard data processing tools are limited in data management capacity, where Big Data goes beyond the capabilities of traditional database management systems (DBMS).⁹ Big Data comprises a large volume of information that is complex (structured and unstructured) in nature. Data are often being generated in real-time and can be of uncertain provenance.¹⁰ New Big Data technologies are being developed to meet the demands for processing massive amounts of heterogeneous data. Big Data management benefits are significant and sometimes far-reaching, and many companies have started operating with Big Data to translate a large amount of data into valuable insights.¹¹

The bulky and heterogeneous nature of Big Data requires investigation using Big Data Analytics (BDA). These data will yield meaningful outcomes by using methods of dissection in BDA,⁹ which help to discover concealed patterns, anonymous relationships, trends of the current market situation, consumer preferences and other aspects of data that can assist institutes and companies to make up to date, faster and better decisions for their business. However, the biggest issue with available datasets is the data quality itself. The data quality issues differ depending on the data source; they could be duplicated records, spelling errors or more complex issues relating to unit misuse. A mixture of clean and dirty records in data can mislead to the well-known Simpsons Paradox^12,13 in which a pattern appears in a particular dataset but disappears or reverses when datasets are combined. A mixture of dirty and clean data could poorly fit an ML model; Figure 1 shows the different ML models fitting with different sets of mixed data. This would lead to unreliable analysis results. Hence, data pre-processing is an important factor in the data analysis process.

Figure 1. Simpsons Paradox.

To make the data suitable for analysis we must clean it. Identifying dirty records and cleaning data sometimes require manual data inspection, which is time-consuming and can be costly. Pre-processing includes several steps, for example, (1) loading the data from the file, (2) cleaning it to fix inconsistencies or errors, (3) encoding the numeric and categorical data types, and finally, (4) the missing value imputation. Missing values can be handled in different ways. Columns or rows containing missing values can be dropped or a value can be imputed in each cell with a missing or improper value. Sometimes, crowdsourcing is used to correct some types of errors, which costs a significant amount of time and human-level works. Some researchers have used statistical computing such as mean, median, sum, among others, and approximate query processing to pre-process data. Some researchers have used sample-based cleaning techniques which can gradually improve data quality. Machine learning is an expanding research area, and is being used in some cases of data cleaning. We propose a hybrid model of the data pre-processing technique called Smartic, which is the combination of sample-based statistical techniques and ML. While sample-based statistical techniques lead to faster execution, ML models provide great accuracy. Our research contribution on Smartic will mitigate challenges related to dirty data cleaning and imputing missing values with better performance accuracy, within a reasonable time frame.

In this paper we present a tool for IoT data preparation and BDA with ML. After data pre-processing, we carried out some feature engineering. This consists in checking which features are highly informative and which are less informative, and then considering features for the analytic purpose. Highly informative features will usually have the most benefits during feature development, while uninformative features can lead to overfitting. The main sections of this study are listed below:

Related work

Ahmad et al.¹⁴ reviewed the recent literature on IoT and BDA. Massive data production in IoT environments, and the versatile nature of the data, make Big Data a suitable solution for IoT systems. They discussed the opportunities for organizations to get valuable insights about their customers and help predict upcoming trends. BDA and ML¹⁵ tools like classification, clustering and predictive modeling, provide data mining solutions that create many more opportunities to expose variability, improve decision-making habits and boost performance.¹⁶ Cross-domain data gathered from different IoT appliances can be fed into BDA that can provide efficient solutions for different domains.

To overcome the challenges of collecting, processing, and examining the massive-scale, real-time data produced by smart homes, Bashir and Gill¹⁷ offered an analytical framework composed of IoT, Big Data administration, and data analytics. The purpose of data analytics in their study was to automatically maintain the oxygen level consistency, detect hazardous gases or smoke, and control light conditions or quality. The work scheme was executed in the Cloudera Hadoop distribution platform, where PySpark¹⁸ was used for big data analysis. The outcomes revealed that the proposed scheme could be used for smart building management with BDA.

Idrees et al.¹⁹ proposed a two-step data cleaning method, using Big Data on a network of IoT wireless sensor devices. They attempted to minimize communication cost, save energy, and expand the lifespan of sensors by cleaning and reducing the redundant data. Their proposed two-level data reduction and cleaning approach in IoT wireless sensor networks includes a sensor level and an aggregator level. The aggregator level merged a near- similar data sets by implementing a divide and conquer technique. The reduced data sets were retransmitted to the sink, then the leader cluster algorithm-based cleaning method was applied to remove redundant data.

Salloum et al.²⁰ proposed a Random Sample Partition (RSP) Explore technique, to explore Big Data iteratively on small computing clusters. Their work included three main tasks: statistical estimation, error detection, and data cleaning. They partitioned the entire data into ready-to-use RSP blocks using an RSP-distributed data model. To get samples of clean data, they used block-level samples to understand the data and detect any potential value errors. Their experimental results showed that a sample RSP block cleaning is enough to get an estimation of the statistical properties of any dataset, and the approximate results from RSP-Explore can rapidly converge toward the true values.

García-Gil et al.²¹ worked on data pre-processing to transform raw data into high-quality, clean data. The quality of the data used in any knowledge discovery process directly impacts the output. They experimented with classification problems due to the presence of noise affecting data quality, particularly a very disruptive feature of data known as incorrect labelling of training dataset. They proposed two Big Data pre-processing techniques with a special emphasis on their scalability and performance traits. The filters they used to remove noisy data were a homogeneous ensemble and a heterogeneous ensemble filter. The results from their experiments show that anyone can retain a smart dataset efficiently from any Big Data classification problem using these proposed filters.

Snineh et al.²² proposed a solution that can be performed in real time to handle the frequent errors of Big Data flows. They proposed a repository for each given domain in their two-step model to store the metadata, cleaning and correction algorithms, and an error log. An advisor was appointed to supervise the system for the first step. The advisor could estimate the algorithm corresponding to error cleaning for a given context. At the second step, the system became autonomous in the selection algorithm procedure based on its learning module. That capability was obtained by using a strategy pattern-based approach. The pattern allowed the building of a family of algorithms, which are interchangeable and evolve independently of the context of use.

Jesmeen et al.²³ presented a comparison between currently used algorithms and their proposed tool, Auto-CDD, to handle missing values. The developed system improved overall data processing and guaranteed to overcome processing unwanted outcomes in data analysis. Their intelligent tool used Gini index values of random forest for feature selection. Experimental evaluation results showed that the random forest classifier led to a high accuracy on a diabetes dataset from UCI.²⁴ They also imputed the missing values on a student database and performed logistic regression analysis on students’ performance.

Shah et al.²⁵ investigated the research gaps in understanding Big Data characteristics generated by industrial IoT sensors, and studied the challenges to processing data analytics. They studied the characteristics of the Big Data generated from an in-house developed IoT-enabled manufacturing testbed. They explored the role of feature engineering for predicting the key process variables in effective machine learning models. The comparison with different levels or extent of feature engineering in between simple statistical learning approaches and complex deep learning approaches, shows potential for industrial IoT-enabled manufacturing applications.

El-Hasnony et al.²⁶ presented challenges in building an optimal feature selection model for Big Data applications, due to the complexity and high dimensionality of the data sets. They used particle swarm optimization and grey wolf optimization to build a new binary variant of a wrapper feature selection. The optimal solution was found with the help of the K-nearest neighbour classifier and Euclidean separation matrices. The overfitting issue was checked using K-fold cross-validation, and the performance and the effectiveness of the model were validated by conducting statistical analyses.

Big data analytics

BDA follows some steps towards getting meaningful insights. Data analytics start with a non-trivial step of problem definition and evaluation. Research on expected gains and costs for reasonable solutions is needed. Generally, a data analytics framework is defined by five main steps:

Proposed method

Data cleaning

For this step, we tried to fix any errors and remove inconsistency. We fixed typos and different representations for any values in a common representation. We used $\text{fuzzy matching}$ or $\text{edit distance}$ algorithms to remove inconsistency. Outlier detection and removal help to get better accuracy. Figure 2 shows summary statistics (number summary) to represent data, such as, minimum, maximum, median, quartiles (Q1, Q3). The first quartile (Q1) is the middle value between the smallest value and the median (or the 50^th percentile, or Q2) of the dataset. A 25% portion of values in the dataset resides below the first quartile.

$$\text{Interquartile Range},\mathrm{IQR}=\mathrm{Q}3-\mathrm{Q}1$$

$$\text{minimum}=\mathrm{Q}1-1.5\ast \mathrm{IQR}$$

$$\text{maximum}=\mathrm{Q}3+1.5\ast \mathrm{IQR}$$

Figure 2. Outlier detection.²⁸

IQR, or midspread, or middle 50%, is the statistical dispersion equal to the range from lower quartile (25th percentile) to the upper quartile (75th percentile). The values that do not reside within the range of the minimum and maximum value are defined as outliers (Figure 2).

Feature engineering

We used mutual information (Figure 3) to determine the importance of a feature. For this step, we created new features as well. We used target encoding for categorical features with higher cardinality. Target encoding involves replacing a categorical feature with the average target value of all data points for that category. We used several other techniques of feature engineering for this purpose.

Figure 3. Feature utility scores of house prices dataset.²⁷

Evaluation criteria

We adopted the K-fold cross-validation for performance evaluation. Cross-validation divided the data set into training and a testing data set to train the model and test its performance using two distinct data sets. Training and testing on the same data create overfitting issues. To avoid this, we used K-fold cross-validation with a value of 5 for K (Figure 4). All our experimental results were five-fold cross-validated.

Figure 4. K-fold cross-validation (K=5).

The XGBoost model performance was evaluated using the root mean squared logarithmic error (RMSLE) metric. The formula for RMSLE is represented as follows:

$$\text{RMSLE}=\sqrt{\frac{1}{n}{\sum }_{i=1}^n{\left(log\left(p_i+1\right)-log\left(a_i+1\right)\right)}^2}$$

Where:

n is the number of observations in the dataset

$p_i$ is the prediction of target

$a_i$ is the actual target for i.

log(x) is the natural logarithm of x (${log}_e\left(x\right)$).

Experiments

Hyperparameter tuning

Hyperparameter tuning gave a performance boost in the final performance evaluation. Figure 5 shows the performance improvements after feature engineering and hyperparameter tuning. After fine-tuning some parameters, we obtained our highest RMSLE score with0.12426.

Figure 5. RMSLE metric score in different stage of analytics.

Results and discussion

We calculated the mean RMSLE value of five trials of train/test splits while varying the training dataset size from 0.1 to 0.9 (10% to 90%). In Figure 6(b), we notice that the ML-based missing value imputation technique outperformed all other traditional imputation methods. Imputing 0 in place of the missing value performed worst in our experiment (see Figure 6a). Replacing missing values of any feature column with the median of that column performed slightly better than imputing the mean of that feature column.

Figure 6. (a) Performance measurement of different techniques of missing value imputation. (b) Closer view of the best imputation techniques.

As the ML-based imputation technique outperformed state-of-the-art baseline methods, we evaluated missing value imputation performance of different ML models, such as LinearRegression, DecisionTreeRegressor, LinearSVR, GaussianNB, BaggingRegressor, KNeighborsRegressor, AdaBoostRegressor, XGBRegressor, among others. Although all ML models delivered higher accuracy with the continuous increment of training dataset size, we saw (Figure 7) more uniform and sheer increasing patterns in XGBRegressor and BaggingRegressor models. It proves that with sufficiently large datasets, the XGBRegressor model can outperform the other ML methods. In addition, we noticed that the XGBRegressor model showed a more stable performance with the varying training data size.

Figure 7. Mean squared error (MSE) of different machine learning (ML) models.

Conclusion

Almost every data set available may contain missing values, which are essential to analyze and understand the data. Dealing with these types of dirty data is difficult, and getting a robust analytical ML models is more challenging. We used statistical methods equipped to fix the datasets, and the sample-based approximate query processing was integrated to alleviate errors in analysis and predictions. The data fixed using different imputation techniques were fed into ML analytical models, and accuracy was compared against different data preparation techniques. Smartic’s data value imputation was faster than the ML-based missing value imputation model. The ML model, trained on data cleaned using a sample-based technique, showed a significantly better and more stable performance. In the future, evaluation can be done with data collected directly from IoT environments in real time.

Data availability