Thread by @svpino

This is how I split machine learning projects:

1. Project scoping
2. Data definition and preparation
3. Model training and error analysis
4. Deployment, monitoring, and maintenance

Here are 33 questions that most people forget to ask.

"Project scoping":

• What problem are we trying to solve?
• Why does it need to be solved?
• Do we truly need machine learning for this?
• What constraints do we have?
• What are the risks?
• What's the best approach to solving this?
• How do we measure progress?

Still under "Project scoping":

• What does success look like?
• How is our solution going to impact people?
• What could go wrong with our solution?
• What's the simplest version we could build?

"Data definition and preparation":

• What data do we need?
• How are we going to get it?
• How frequently does it change?
• Do we trust the source?
• How is this data biased?
• Can we improve it somehow?
• How are we going to clean it?
• How are we going to augment it?

"Model training and error analysis":

• What's a good baseline?
• What's a good starting point?
• Has anyone solved this before?
• How are we going to test the model?
• Are the results good enough?
• Are we solving the problem?
• How can we improve the results?

"Deployment, monitoring, and maintenance":

• Where do we host this?
• How much do we need to scale?
• What metrics should we monitor?
• What results do we expect?
• How is the model doing?
• How do we keep the model up to date?
• What's our rollback strategy?

Every question opens a new set of possibilities and discoveries.

The more you ask, the better your system will be.

Follow me @svpino, and I'll help you stay curious, one thread at a time, right on your Twitter timeline.

If you build machine learning systems professionally, what questions would you recommend others to start asking?

What questions lead to interesting discoveries with the potential to change the project's outcome?