Thread by @mfgrp

How to pick a problem. Today in @CellCellPress. 1/53
cell.com/cell/fulltext/…

Your career = a finite number of weeks. Most would agree that time is precious & should be used for maximum impact, but acknowledging this and practicing it are two different things. Especially important when choosing a problem, which can impact time allocation for years. 2/53

Having observed that grad students are taught almost everything about science and engineering except how to pick a problem, me and Chris Walsh started a class at Stanford on the topic in 2019 (BIOE 395). What follows are eight lessons from the course. 3/53

1) Spend more time on problem choice. In science and (non-software) engineering, a typical project for a grad student might involve 1-2 weeks of planning and 2-5 years of execution. That is way out of balance, especially considering that… 4/53

…once you choose a project, you are confined to a relatively narrow ‘impact band’. It is hard to make the solution to a mediocre problem impactful. So the problem you choose will influence the impact of your work just as much as your execution. 5/53

If you’re writing software, cycle time is short. Just try it & see if it works; if not, you only lose a couple weeks. Doesn’t work in science & non-software engineering, where go/no-go = months and whole project = years. Inertia takes over; sunk cost fallacy hard to avoid. 6/53

It can help to reverse the polarity of our relationship with new ideas. Don’t treat them with reverence; confirmation bias will set in. Definitely don’t jump on the first idea and get started—that is the worst thing you can do. 7/53

Instead, think of new ideas as leeches trying to make a meal of your time. Treat them with skepticism and evaluate several of them in parallel—comparison shopping leads to better decision making. 8/53

2) Exercise intuition pumps and avoid common traps. There is no single way to generate new ideas, but certain prompts can help jumpstart the ideation process… 9/53

…and there are common traps to avoid (below). In addition, focus on areas where you will have a competitive advantage from your expertise, access to data, or love of a problem and exuberance for solving it (this will help you dig deeper than others). 10/53

3) Don’t avoid risk; befriend it. A useful starting point for evaluating a new idea is to place it on the graph below. We will consider the axes one at a time. 11/53

To score a project’s likelihood of success, we suggest an ‘assumption analysis’. List every assumption you are making from the project’s inception through its conclusion. Assign two scores to each one: likelihood of success and duration of effort. 12/53

Then look critically at the list. In the example above, there are two high-risk assumptions that do not read out until two years have passed. This kind of risk profile is problematic. There are a few ways to fix it; suggestions are shown below. 13/53

Importantly, the idea here is not to *eliminate* risk—risk-free projects tend to be incremental. Instead: name, quantify, and work steadily to chip away at risk. And when presenting a new idea, be candid about risk—it makes your case more convincing. 14/53

Finally, an important rule of thumb is to perform the go/no-go experiment at the earliest feasible moment. This is true even if it requires some compromise; build a clunky prototype and see if it works, even a little. 15/53

4) Pick the right optimization function. The y-axis of our graph—how much impact will it have if it succeeds?—is equally important. In general, it is more challenging to assess potential impact than likelihood of success. But there are two points worth considering. 16/53

First, articulate the criteria by which you hope to be evaluated. These are different for basic science vs. technology development. For basic science, we suggest How much did we learn? vs. How general is the object of study? 17/53

For technology development, a better alternative is: How widely will it be used? vs. How critical is it for the application? 18/53

Finally, an assessment of impact doesn’t have to be perfect to be useful—something is better than nothing. Even if an *absolute* estimate is challenging, one can still compare a few options. Much better to make an educated guess than to ignore altogether. 19/53

5) Fix one parameter; let the others float. In generating a new idea, one of the most common failure modes is to fix too many parameters. (Here, parameters = e.g., the system you will study or the methods you will use) 20/53

Consider a project that aims to provide a continuous supply of GLP-1 by engineering a T cell to produce it. This is an interesting idea with some merit, but too many parameters are fixed. 21/53

If the most important parameter is improving GLP-1 delivery, we should consider any reasonable solution: for example, peptide engineering to extend half-life or enable oral bioavailability. 22/53

If instead the fixed parameter is the use of an engineered T cell, then we should be open to any logical use case. Better alternatives exist: e.g., the production of smaller quantities of peptides that act locally (cytokines, chemokines, and growth factors). 23/53

One can have too *few* fixed parameters as well (i.e., too much freedom to think). “I want to do impactful work in cell engineering” is so broad a statement it can lead to paralysis. Constraints engender creativity; If you feel stuck, try fixing one parameter at a time. 24/53

6) Learn the altitude dance. Projects rarely unfold in a linear fashion; they require frequent course correction. Most people should spend more time on a project’s decision tree than they currently do. 25/53

Once you get into a project, you will have learned from initial experiments, new papers will have been published, & tech will have advanced. At any decision point, you should update your plan from 1-2 years ago; there will likely be a better alternative. 26/53

The key to navigating the decision tree is to move back and forth frequently between two types of work: getting stuff done (Level 1) and evaluating it critically (Level 2). These cannot be done at the same time. 27/53

Getting stuff done requires full immersion in the details of experimentation or coding. Critical evaluation demands that you clear your head, step away from the work, and evaluate it as though it were performed by someone else. 28/53

You can draw concrete conclusions (what did we learn?) and then decide what the next step should be, using the tools described in previous sections – e.g., re-evaluating fixed/floating parameters. 29/53

Troubleshooting is critical. But too much of it, coupled to a failure to seek alternative solutions = being stuck in a rut. **Remember: you are never stuck. There is always a way out. 30/53

At the same time, don’t ignore the decision tree if you succeed! A victory of any size can be an opportune moment to pivot to a more interesting path or speedier plan. 31/53

Some people gather tons data or write reams of code, but rarely stop to consider its implications. Others are brilliant strategists but have trouble reducing plans to practice. The most successful folks move back and forth frequently between planning and doing. 32/53

7) Capitalize on the ‘adversity feature’. Adversity in a project is inevitable and opportune. Almost every project suffers an existential crisis or takes a sharp turn. Odds are that yours will too. It will take you by surprise, but it shouldn’t! 33/53

There is a silver lining. A crisis is an opportunity to do two things: 1) Upgrade your project—make it better than it was before. 2) Realize you are backed into a corner and reason your way out. People who can do this are unstoppable. 34/53

At the start of a project, consider that a singularly mapped path has virtually no chance of coming true. In reality you are picking an *ensemble* of possible projects. When (inevitably) you hit a roadblock, you will likely end up on another path within this ensemble. 35/53

8) Turn a problem on its head. There are several ways to navigate around a problem. Three are particularly notable. The first goes back to the idea of fixed parameters. Earlier, we encouraged against fixing too many to avoid poor technology-application fit. 36/53

But as a project launches, additional parameters naturally get fixed. To troubleshoot, make a list of the fixed parameters and then let each of them float, one at a time, to explore alternative paths around a roadblock. 37/53

This is especially important when you are in a deep rut; the solution is often to let a “sacred” fixed parameter float. 38/53

Second, turn a problem on its head. An example: my colleague Nathanael Gray tried to make small-molecule degraders of HER3 and c-Raf by conjugating known ligands for each protein to pomalidomide. But it did not work, even after repeated attempts. 39/53

Instead of trying to force a solution, Nat reasoned that the project’s greatest liability was that any individual kinase may resist chemically induced degradation. He turned this problem on its head by asking instead: which kinases can be degraded? 40/53 pubmed.ncbi.nlm.nih.gov/29129717/

To answer this question, they fused pomalidomide to a promiscuous kinase binder, TL13-87, creating a promiscuous kinase degrader. After treating cells with this compound, they used proteomics to determine the complete list of kinases that could be degraded. 41/53

They found 28 kinases that were degradable, including important targets! Although they didn’t accomplish their original goal, their new project arguably had more impact than if their original plan had succeeded. 42/53

The final strategy can be used when you carry out a project, find the answer, and realize it no longer applies to the question you started with (‘I have the answer; what is the question?’). 43/53

A handy example of this is NVIDIA, who originally designed GPUs to enable gaming. But once they realized that their ‘answer’ (GPUs) could be applied to a more general ‘question’ (AI), things really picked up. 44/53

Wrapping up: This is an imperfect introduction to a difficult topic. I have no doubt folks will disagree on some of this! There is much to improve -- suggestions are welcome. 45/53

Most of all, we’re hoping to convince people to spend more time thinking about problem choice. Perhaps we might even inspire some of you to teach a similar course in your shop. We will happily offer course materials to anyone who is interested. 46/53

A few notes of gratitude: First, I developed and taught this course with Chris Walsh, who passed away as we began writing this piece. All of these ideas were developed together. I consider him a co-author & owe him an immeasurable debt of gratitude. 47/53

.@UriAlonWeizmann wrote an earlier piece that inspired us to discuss and ultimately design a course on this topic. Definitely worth reading. 48/53 pubmed.ncbi.nlm.nih.gov/19782018/

@UriAlonWeizmann Claudia Willmes and our referees were kind, reasonable, and constructive. Our paper was greatly improved by their suggestions. 49/53

@UriAlonWeizmann Third, I'm deeply grateful to @BRobertsVC, @RMedzhitov, John Stuelpnagel, and @ElliotHershberg for ideas that have become central tenets of BIOE 395, and to @JuliaBauman2, @SynBioGaoLab, @zgitai, Wendell Lim, & Wes Sundquist for helpful feedback on the manuscript. 50/53

@UriAlonWeizmann @BRobertsVC @RMedzhitov @ElliotHershberg @JuliaBauman2 @SynBioGaoLab @ZGitai .@JuliaBauman2 wrote an earlier thread on the course that is much clearer (and more concise!) than this one. 51/53

https://twitter.com/JuliaBauman2/status/1671582296045125632

@UriAlonWeizmann @BRobertsVC @RMedzhitov @ElliotHershberg @JuliaBauman2 @SynBioGaoLab @ZGitai Finally, we are grateful to the many students who have taken 395 and to our colleagues who participated in the first PI-focused round of the course (@fordycelab, @BintuLacra, @Sattely_lab, @hawa_racine, @RogelioHdezL, @jennbrophy7… 52/53

@UriAlonWeizmann @BRobertsVC @RMedzhitov @ElliotHershberg @JuliaBauman2 @SynBioGaoLab @ZGitai @fordycelab @BintuLacra @Sattely_lab @hawa_racine @RogelioHdezL @jennbrophy7 …@StevenMBanik, @nicolemmart, @Frutag33, and @yerinchen); their feedback has been immeasurably helpful in shaping our thinking and improving the course. 53/53