Thread by @_rsc

@sdboyer I listened to your talk. You are clearly still quite frustrated and upset about all this generally and with me in particular despite our many hours of conversation. I am truly sorry about that. I know how that feels, and it sucks.

@sdboyer One recurring theme in your talk is the inability to write down incompatibilities, but that's not off the table for Go modules, as we've discussed.

@sdboyer - You *can* write down "use this with X 1.5 or later", which implicitly says "this is incompatible with X 1.4 and earlier".

@sdboyer - As you noted in the talk, I can't predict an incompatibility with a not-yet-released version of a dependency, so the constraints in my tagged Gopkg.toml are necessarily incomplete.

@sdboyer - If the constraints fed into a SAT search are incomplete (and they are), then even a perfect SAT search can pick bad configurations (and Dep does). I walked through the details in my Singapore talk.

@sdboyer - Minimal version selection's avoidance of automatic updates of dependencies makes it do the right thing - even with no incompatibility declarations - more often than SAT-based systems with bad inputs (e.g., all of them). Again, see the Singapore talk.

@sdboyer - Even if the SAT constraints are perfect, there may not be one best choice. The right choice depends on user preference - do I care more about having a new X or a new Y? - and should not be silently guessed by a tool. The Singapore talk shows a similar ambiguity.

@sdboyer - Also, the SAT constraints are likely to be not only incomplete but also plain wrong. Minimums that aren't enforced go stale quickly, and then SAT spends its time walking a search space full of broken combinations.

blog.illicitonion.com/2018/06/rust-m…

@sdboyer - All this points to using a simple, predictable selection algorithm, and then simply notifying the user when that algorithm leads to a known-incompatible combination, so the user can apply the right preference.

@sdboyer - As we discussed on 4/17 and as I noted in #24301 on 5/25, I *do* want to find a place to record incompatibilities, but one that allows recording them after the fact instead of requiring accurate prediction of the future.

github.com/golang/go/issu…

@sdboyer - More generally we need a place to record other known problems with already-released versions, like security bugs. That's #24031. Maybe the answer there applies to incompatibilities too.

github.com/golang/go/issu…

@sdboyer - Incompatibilities are a mine field. I don't trust a SAT solver with an incomplete, incorrect map to self-drive through that mine field. Better to detect and report a mine at its preferred location and let the user navigate from there.

@sdboyer - Also note that minimal version selection's avoidance of aggressive forced updates means its default choice is far less likely to be atop a mine than a newest-preferring SAT solver's. This makes incompatibility reporting far less critical to the overall system.

@sdboyer Another recurring theme is what you called "servitude hell" in the talk and "the design goal of invisibly extracting work from unpaid people" in your tweet. I must admit I don't understand the logic here.

@sdboyer - The scenario seems to be that you are the author of C and import D, and D issues a new release that breaks C. Now what?

@sdboyer - In a newest-preferring SAT solver like Bundler/Cargo/Dep, users who pull in C for the first time automatically get the latest D. C doesn't work, they report the bug to you.

research.swtch.com/cargo-newest.h…

@sdboyer - Maybe you fix it and release a new C. If so, great, problem solved.

@sdboyer - But it sounds like you don't have time to fix C and just want the bug reports to stop. You issue a new C marked as incompatible with the new D. The SAT solver avoids the new D, and the bug reports stop for a while.

@sdboyer - Until something else in people's builds brings in the new D. Then Dep searches around and finds the older C release, which has an unmarked incompatibility. A new wave of bug reports begins, from people with SAT solvers finding the older C.

@sdboyer - The only available fix for this new wave of bug reports is to actually fix C.

@sdboyer - Minimal version selection reduces the pain here. Users of C get the version of D you used even after the new D is available, eliminating the first wave of bug reports. The second wave happens the same as in Dep. Again, see the Singapore talk.

@sdboyer - Regardless, you are hardly beholden to users filing bug reports. If C helps them, great. If C doesn't work for them, great, they don't have to use it, or they can send a fix. Either way, they're not "invisibly extracting work" from you.

@sdboyer - The same with D. You presumably wrote C using D because doing so was lower cost than directly implementing that functionality or finding a different source for it.

@sdboyer - If D changes to be less useful for writing C, the balance shifts, perhaps to the point where it doesn't make sense to use D anymore. But D's author is not "invisibly extracting work" from you. If anything, the opposite was true and is now a little less true.

@sdboyer - It is true that writing C using D means that now D's future changes might require you (eventually) to adjust your programs. You and C literally depend on D. Hence, dependency.

@sdboyer - To be completely clear, "invisibly extracting work from unpaid people" is not one of the design goals of Go modules. Not even close.

@sdboyer - I've been contributing to open source projects for over 20 years. I love doing it, and my primary goal is to make Go support those collaborations as well and as smoothly as possible, especially as Go scales to ever-larger code bases and developer efforts.

@sdboyer You and I have had some very useful, productive technical conversations, such as the one about finding a place to record pairwise incompatibilities discovered after the fact. I'm grateful to you and to everyone else who has helped refine the design of Go modules. Thank you.

@sdboyer Anyone is always welcome to file feedback about specific problems at golang.org/issue/new. The conversations work best with specific, real examples. Thanks again.