A framed picture of a pineapple with real pineapples either side to symbolize problem framing for machine learning

Problem Framing for Machine Learning

Problem Framing is a method used to understand, define and prioritize business problems. It is one of the most important phases in Machine Learning that will determine if all the work that is to be done subsequently is perceived to be of use and provides business value. Framing determines what will be observed and what will be predicted. It also identifies the metrics that need to be optimised to monitor performance and errors.

This Study Guide is for sub-domain: 3.1, Frame business problems as machine learning problems, of the Modeling domain of the AWS Machine Learning Speciality exam. A description of all the knowledge domains in the exam is in: AWS Machine Learning exam syllabus

Questions

Scroll to the bottom of the page for questions and answers.

Overview

Problem Framing is preceded by Business Goal Identification. This will provide a clear idea of the problem to be solved and the business value to be derived. This is not a unique ML process, it has been adopted by ML to enable business problems to be phrased so they are suitable for Machine Learning solutions. Framing presents a problem in a way to win support from stakeholders, whilst maximising the chances of a ML solution being successful.

In How To Frame A Problem To Find The Right Solution Paloma Cantero-Gomez offers techniques to help in Problem Framing:

Forbes: How To Frame A Problem To Find The Right Solution

  • The 40-20-10-5 rule to sequentially cut out words to reach a concise description.
  • Rephrase and focus this mostly reframes in a more strategic way.
  • Challenge assumptions – what do we know to be true and what have we assumed to be true.
  • Change the perspective of the problem by using people from outside of the business area to comment on the problem and so gain a new perspective.

The Google course Introduction to Machine Learning Problem Framing provides a suggested approach to Problem Framing:

Formulate Your Problem as an ML Problem bookmark_border

  1. Articulate your problem.
  2. Start simple.
  3. Identify Your Data Sources.
  4. Design your data for the model.
  5. Determine where data comes from.
  6. Determine easily obtained inputs.
  7. Ability to Learn.
  8. Think About Potential Bias.

Video: Coursera: Lesson 3: Framing Design Problems: 

Lesson 3: Framing Design Problems – Introduction to the Design Process

You may have to register with Coursera to view this video. Registering is free and you are not asked for payment information.

Problem framing best practice

infographic with image of pineapples in a frame and a list of best practice for problem framing for Machine Learning
Pin this infographic to your pinterest account

The Machine Learning Lens AWS Well-Architected Framework white paper lists six points of best practice:

  1. Define success criteria
  2. Establish a performance metric
  3. Define inputs, outputs and metrics
  4. Is Machine Learning suitable?
  5. Data sourcing and annotation objectives
  6. Select a simple model

Define success criteria

Define criteria for a successful outcome of the project. This comes from defining the Business goal.

Establish a performance metric

Establish an observable and quantifiable performance metric for the project, for example: 

  • accuracy
  • prediction latency
  • minimizing inventory value

This will be used to assess if the training was successful and, when in production, allow the model to be monitored.

Inputs, outputs, and performance metrics

Formulate the ML question in terms of inputs, desired outputs, and the performance metric to be optimized.

Is Machine Learning suitable?

Evaluate whether ML is a feasible and appropriate approach. Machine Learning inference works best with large amounts of data that cannot be easily understood, or valuable data points to be identified. The analysis performed so far may indicate that the objectives can be achieved without a Machine Learning process.

Create a data sourcing and annotation objective

Create a data sourcing and data annotation objective, and a strategy to achieve it. Identify where the data will come from and how clean it will be. This will allow you to estimate how much effort will be required to set up a ML Pipeline. You may be able to identify time and cost savings by leaving data out. Issues that cause increased effort to process are:

  • Data cleansing
  • Data conversion
  • Complex data ingestion, or transport
  • Personally identifiable information (PII)
  • Data requiring enhanced security
  • Massive data volumes
  • The cost to label

PII and enhanced security are expensive because of regulatory requirements. For labeling costs, some data may already be labelled, or easily labeled. The most expensive labeling is manual labeling.

Select a simple model

Start with a simple model that is easy to interpret and implement. More complex models are harder and slow to train than simple models. So by using a simpler model training iterations can be faster which makes debugging more manageable. A more complex model can be introduced later if it is justified.

Summary

Problem framing is one of the most important phases of the Machine Learning process and one that you have to get right. Everything that comes after relies on the information that comes out of this phase. If you get Framing wrong you can end up doing a lot of work that nobody wants and does not deliver business value.

I recommend you study the Google course Introduction to Machine Learning Problem Framing. It is very good, free and only takes about an hour to complete.

Introduction to Machine Learning Problem Framing

Credits

Photo by Pineapple Supply Co. on Unsplash


AWS Certified Machine Learning Study Guide: Specialty (MLS-C01) Exam

This study guide provides the domain-by-domain specific knowledge you need to build, train, tune, and deploy machine learning models with the AWS Cloud. The online resources that accompany this Study Guide include practice exams and assessments, electronic flashcards, and supplementary online resources. It is available in both paper and kindle version for immediate access. (Vist Amazon books)


Questions and answers

90
Created on By Michael Stainsbury

3.1 Problem Framing for Machine Learning

This quiz is for sub-domain 3.1, Frame business problems as machine learning problems, of the Modeling knowledge domain.

1 / 5

<–?–> Identification is performed before Problem Framing.

2 words left

2 / 5

3 / 5

Problem Framing is a method used to understand, define and <–?–> business problems.

4 / 5

A <–?–> is used to assess if the training was successful and, when in production, allow the model to be monitored.

2 words left

5 / 5

What project items does Problem Framing determine?

Your score is

The average score is 37%

0%


Amazon Study Guide for the AWS Machine Learning Speciality exam
Reviews
Amazon Study Guide review – AWS Certified Machine Learning Specialty

This Amazon Study Guide review is a review of the official Amazon study guide to accompany the exam. The study guide provides the domain-by-domain specific knowledge you need to build, train, tune, and deploy machine learning models with the AWS Cloud. The online resources that accompany this Study Guide include practice exams and assessments, electronic…

Pluralsight AWS Certified Machine Learning web page screen shot
Reviews
Pluralsight review – AWS Certified Machine Learning Specialty

Contains affiliate links. If you go to Pluralsight’s website and make a purchase I may receive a small payment. The purchase price to you will be unchanged. Thank you for your support. The AWS Certified Machine Learning Specialty learning path from Pluralsight has six high quality video courses taught by expert instructors. Two are introductory…


Similar Posts