Dealing With 5 Practical Issues in Machine Learning and Their Business Implications
Author | Dr. Shou-De Lin, Chief Machine Learning Scientist, Appier
Businesses today deal with more data and it’s arriving faster than ever before. At the same time, the competitive landscape is changing rapidly so the ability to make fast decisions is critical.
As Jason Jennings and Laurence Haughton put it “It’s not the big that eat the small… It’s the fast that eat the slow”.
Business success comes from making fast decisions using the best possible information.
Machine learning (ML) is powering that evolution. Whether a business is trying to make recommendations to customers, hone their manufacturing processes or anticipate changes to a market, ML can assist by processing large volumes of data to better support companies as they seek a competitive advantage.
However, while machine learning offers great opportunities, there are some challenges. ML systems rely on lots of data and the ability to execute complex computations. External factors, such as shifting customer expectations or unexpected market fluctuations, mean ML models need to be monitored and maintained.
In addition, there are a number of practical issues in machine learning to be solved. Here we will take a close look at five of the key practical issues and their business implications.
1. Data Quality
Machine learning systems rely on data. That data can be broadly classified into two groups: features and labels.
Features are the inputs to the ML model. For example, this could be data from sensors, customer questionnaires, website cookies or historical information.
The quality of these features can be variable. For example, customers may not fill questionnaires correctly or omit responses. Sensors can malfunction and deliver erroneous data, and website cookies may give incomplete information about a user’s precise actions on a website. The quality of datasets is important so that models can be correctly trained.
Data can also be noisy, filled with unwanted information that can mislead a machine learning model to make incorrect predictions.
The outputs of a ML model are labels. The sparsity of labels, where we know the inputs to a system but are unsure of what outputs have occurred, is also an issue. In such cases, it can be extremely challenging to detect the relationships between features and the labels of a model. In many cases, this can be labor intensive as it requires human intervention to associate labels to inputs.
Without accurate mapping of inputs to outputs, the model might not be able to learn the correct relationship between the inputs and outputs.
Machine learning relies on the relationships between input and output data in order to create generalizations that can be used to make predictions and provide recommendations for future actions. When the input data is noisy, incomplete or erroneous, it can be extremely difficult to understand why a particular output, or label, occurred.
2. The Complexity and Quality Trade-Off
Building robust machine learning models requires substantial computational resources to process the features and labels. Coding a complex model requires significant effort from data scientists and software engineers. Complex models can require substantial computing power to execute and can take longer to derive a usable result.
This represents a trade-off for businesses. They can choose a faster response but a potentially less accurate outcome. Or they can accept a slower response but receive a more accurate result from the model. But these compromises aren’t all bad news. The decision of whether to go for a higher cost and more accurate model over a faster response comes down to the use case.
For example, making recommendations to shoppers on a retail shopping site requires real-time responses, but can accept some unpredictability in the result. On the other hand, a stock trading system requires a more robust result. So, a model that uses more data and performs more computations is likely to deliver a better outcome when a real-time result is not needed.
As Machine Learning as a Service (MLaaS) offerings enter the market, the complexity and quality trade-off will come to greater attention. Researchers from the University of Chicago looked at the effectiveness of MLaaS and found that “they can achieve results comparable to standalone classifiers if they have sufficient insight into key decisions like classifiers and feature selection”.
3. Sampling Bias in Data
Many companies use machine learning algorithms to assist them in recruitment. For example, Amazon discovered that the algorithm they used to assist with selecting candidates to work in the business was biased. And researchers from Princeton found that European names were favored by other systems, mimicking some human biases.
The problem here isn’t the model specifically. The problem is that the data used to train the model comes with its own biases. However, when we know the data is biased, there are ways to debias or to reduce the weighting given to that data.
The first challenge is in determining if there is inherent bias in the data. That means conducting some pre-processing. And while it may not be possible to remove all bias from the data, its impact can be minimized by injecting human knowledge.
In some cases, it may also be necessary to limit the number of features in the data. For example, omitting traits such as race or skin color can help limit the impact of biased data on the results from a model.
4. Changing Expectations and Concept Drift
Machine learning models operate within specific contexts. For example, ML models that power recommendation engines for retailers operate at a specific time when customers are looking at certain products. However, customer needs change over time, and that means the ML model can drift away from what it’s designed to deliver.
Models can decay for a number of reasons. Drift can occur when new data is introduced to the model. This is called data drift. Or it can occur when our interpretation of the data changes. This is concept drift.
In order to accommodate this drift, you need a model that continuously updates and improves itself using data that comes in. That means you need to keep checking the model.
That requires the collection of features and labels and to react to changes so the model can be updated and retrained. While some aspects of the retraining can be conducted automatically, some human intervention is needed. It’s critical to recognize that the deployment of a machine learning tool is not a one-off activity.
Machine learning tools require regular review and update in order to remain relevant and continue to deliver value.
5. Monitoring and Maintenance
Creating a model is easy. Building a model can be automatic. However, maintaining and updating the models requires a plan and resources.
Machine learning models are part of a longer pipeline that starts with the features that are used to train the model. Then there is the model itself, which is a piece of software that can require modification and updates. That model requires labels so that the results of an input can be recognized and used by the model. And there may be a disconnect between the model and the final signal in a system.
In many cases when an unexpected outcome is delivered, it’s not the machine learning that has broken down but some other part of the chain. For example, a recommendation engine may have offered a product to a customer, but sometimes the connection between the sales system and the recommendation could be broken, and it takes time to find the bug. In this case, it would be hard to tell the model if the recommendation was successful. Troubleshooting issues like this can be quite labor intensive.
Machine learning offers significant benefits to businesses. The ability to predict future outcomes in order to anticipate and influence customer behavior and to support business operations are substantial. However, ML introduces a number of challenges to businesses. By recognizing these challenges and developing strategies to address them, companies can ensure they are prepared and equipped to handle them.
* Want to know more about how you can leverage machine learning to elevate your business and improve your marketing effectiveness? We are here to help! Get in touch with our team today for an exclusive consultation.
WE ARE HERE TO HELP
YOU MIGHT ALSO LIKE
As marketers grapple with the problem of ad fraud and its mounting losses, artificial intelligence (AI) is proving to be an effective weapon that can reverse the tide. Marketers in Asia Pacific continue to throw money at advertising, as ad spending is expected to increase 10.7 percent to US$210.43 billion in 2018, according to eMarketer. However, the ever-growing problem of ad fraud is skewing their reporting and standing in their way of showing better returns. Even mobile marketers who expected more safety with app installs, faced 30 percent more fraud during the first quarter of 2018 compared to the same period last year, according to AppsFlyer’s “The State of Mobile Fraud: Q1 2018” study. Mobile app marketers were exposed to US$700-US$800 million in ad fraud losses worldwide. What makes ad fraud such a challenging problem today? More Sophisticated Ad Fraud Methods Today In the early days of ad fraud, the methods adopted by fraudsters were relatively simple. They used bots focused on driving large volumes of traffic to websites, bought cheap traffic through auto redirects or employed people to install apps in click farms. Once a click was made or an app was installed, their job was done. However, this
Imagine that you are a marketer looking to run a targeted marketing campaign. What if you had a tool that could easily segment your market on the basis of factors like economic status, purchasing preferences, online shopping behavior, etc. so that you could customize your approach and messaging to each segment for maximum impact and conversion? These are the kind of insights that deep learning (DL)* can offer. DL refers to a family of advanced neural networks that mimic the way the brain processes information and extract goal-oriented models from scattered and abstract data. What differentiates it from traditional machine learning is the use of multiple layers of neurons to digest the information. A DL program trains a computer to perform human-like tasks, such as speech recognition or predicting consumer behaviors. It is fed large amounts of data and taught what the desired output should be. The more data it’s fed, the better performance. The program then applies calculations to achieve that output, modifying calculations and repeating the cycle until the desired outcome is achieved. The ‘deep’, hence refers to the number of processing layers that the data must pass through to achieve the outcome, and how the learning
Machine learning continues to make great strides in optimizing data collection and analysis tools for use across a wide range of industries, but now a new strand is emerging: automated machine learning, or AutoML for short. This promises to be less reliant on data scientists, who are in short supply because of their highly specialized skill set, and can be expensive to hire. However, there are a number of misconceptions about AutoML, the chief one being that it can do away with such data scientists altogether. Machine learning is an extraordinarily powerful general purpose technology that has a staggering number of applications. It is therefore no wonder people are excited about an evolution of the technology known as auto machine learning. To understand what AutoML can do, it is important to understand how standard machine learning works. Machine learning involves several steps: first you need to collect the relevant data, and then clean it in order to learn what you want to learn from it. You then define the feature representation of your data, and put it into a model which is trained to optimize the accuracy in order to achieve your predefined goal. It is a complicated task that