It’s All About Data: Understanding Predictive Analytics

Real-life examples of predictive analysis applications

	Waze The popular traffic app Waze uses predictive analytics to calculate your estimated time of arrival at your destination. The app uses data such as current and past traffic conditions in a given timeframe to make a prediction on how long your drive there will take.
	Amazon Amazon uses an algorithm to predict what their customers would likely want to see or buy based on your past purchases. They even monitor items that you check out but do not ultimately purchase, as well as items that you just view but never add to your cart. They take this information to show you similar items that you might be interested in.
	Netflix Similar to Amazon, Netflix’s predictive analytics takes into account shows and movies you’ve watched and uses that data to predict what types of entertainment you would most likely want to watch next.
	Weather forecasts Perhaps the most well-known real-world example is in meteorology, where historical weather and climate conditions for a particular area is studied and analyzed to make predictions about future conditions.

Artificial intelligence and machine learning in predictive analytics

To many people, the relationship between artificial intelligence, machine learning, and predictive analytics can be quite confusing. While all three are related, these terms are not interchangeable, and each of them refers to something specific.

Artificial intelligence is a broad term in computer science that emphasizes the development and creation of computers with particular skills, knowledge, and intelligence. It encompasses many things, among them machine learning.  Machine learning, meanwhile, refers to a computer’s ability to learn with little to no human intervention, such as programming. It’s a technique used in predictive analytics, notably in data mining.

AI is used in predictive analytics. While you can perform “traditional” predictive analytics based on old technology, AI is more autonomous and generates decisions instead of merely providing insights. 

AI has tremendously impacted the field of predictive analytics through its ability to analyze vast amounts of data, run simulations to predict the most likely outcome, and self-adjust according to new data, all without human intervention.

Predictive analytics, meanwhile, falls under the broader study of data analytics. Generally, there are four types of data analytics:

• Descriptive analytics – this simply describes the historical data in an organized form

• Diagnostic analytics – answers the question, “Why did X happen to Y data?”

• Predictive analytics – makes predictions about likely outcomes in the near future

• Prescriptive analytics – makes suggestions about what to do next, given the information from predictive analytics

Process of predictive analytics

The process of predictive analytics consists of several steps:

1. Project Definition

To ensure that the end product will address what is needed, the first step in the predictive analysis process is to define the project’s end goals and outcomes. This includes deliverables, scope, and particular data sets required for the project.

For example, your end goal might be to determine customers’ future behavior toward a certain brand, or perhaps it’s to assess the likelihood of a market crash in the next six months. Different objectives will require different data sets and methodologies, so knowing your goals will make the process more efficient. It will also help engineers in monitoring and evaluating the predictive models that are generated—whether or not they’re able to accurately and validly predict future outcomes.

2. Data Collection

Data is then collected from several different sources. The more varied and heterogeneous the sources, the more reliable the predictions will be—granted that each source is legitimate and reliable in itself.

These data sets can be massive; in order to have predictions accurate enough, the analysis needs big data from which to draw conclusions. 

Data sources can include databases, spreadsheets, web archives, and other files. In businesses, for example, these may include customer information databases gleaned from marketing efforts.

3. Data Analysis and Statistics

The process of data analysis, also known as data mining, involves extracting value from the data collected to produce meaningful information. It includes identifying outliers in the data, certain spikes, or pinpoint missing data. These are removed from the data since it might skew the predictions unfairly.

The organized information is then further analyzed using standard statistical analysis models in order to test the assumptions and hypotheses drawn from the data analysis. Trends and patterns are identified through various algorithms.

4. Modeling

In this part of the process, predictive models are generated from the analyzed information. These models are tested, validated, and evaluated to check whether they can accurately predict future outcomes from the historical data provided.

Several iterations might be required before a model performs according to what is expected. Depending on the objectives of the project determined in the first step, it might take a while before an accurate predictive model is generated. Some projects have a number of variables that make it more difficult to train the model, while others might be a little more straightforward. In any case, different approaches in training the model through various techniques are recommended to ensure that the final predictive model performs well against new data.

5. Deployment/Integration

When a predictive model performs accurately, we can then move on to deployment. Here, the model is integrated into a system that can now use it to make predictions. Basically, this is where you now use the predictive model to make decisions based on the results and reports that it generates about possible future outcomes.

6. Model Monitoring

This step in the process ensures that the performance of the model produces results that are reliable, valid, and aligned with the project objectives. The various models are consistently managed and monitored so that the appropriate interventions can be performed in case of unforeseen circumstances or errors that were not caught during the modeling iteration process.

Methodologies used in predictive analytics

Algorithms are a key part of the predictive analytics process. During the data analysis and statistics process, algorithms are used to mine data and identify trends and patterns that would help train the model in predicting future outcomes.

The predictive analytics software in use among businesses and other industries today utilize several different methods and techniques, including the following:

1. Logistic Regression

Logistic regression is a statistical technique used when there are one or more independent variables, and the dependent variable is dichotomous—meaning there are only two options for the result of the dependent variable. Logistic regression can describe the relationship between the independent variables (such as age, gender, etc.) and the dichotomous dependent variable (pass/fail, true/false, present/absent, etc.)

2. Decision Trees

Also called C4.5, decision trees as a predictive analytics tool are powerful and helpful in managing large data sets. Decision trees allow you to classify new information based on historical data, leading you to a final decision. An advantage of using decision trees for machine learning is its information clarity and explainability; the hierarchical structure of decision trees allows users to understand why a particular decision has been made.

Decision trees are particularly beneficial for the healthcare industry, given the large amount of diagnostic data and criteria they handle. This type of analysis, for example, can help in classifying symptoms based on existing data, leading healthcare workers to a more definite diagnosis of a patient’s illness.