Thread by @JoelChan86

Some thoughts on search as "table stakes" in #ToolsForThought

Started as a "short thread", turned into a longer thing. Hope you find it helpful!

There's been some good discussion on effective search as a core primitive for tools for thought, and the limitations of existing tools, such as @RoamResearch wrt search.

Example:

https://twitter.com/TfTHacker/status/1493684090234720257

@RoamResearch And here:

https://twitter.com/RobertHaisfield/status/1492656306565500930?s=20&t=ty8j_tqi4oC-QFpG_U2N6A

@RoamResearch First, let me say that I respect these thinkers+doers a LOT, and I think *in the general case*, for a variety of use cases, search is a powerful and essential primitive.

@RoamResearch HOWEVER. There is something interesting about constraining search in favor of other routes of rediscovery.

Similar to what @calhistorian hints at here:

https://twitter.com/calhistorian/status/1493799303277469699?s=20&t=_nRR-3KSPiOQCOICkFLsmg

I want to riff on that idea a little bit.

@RoamResearch @calhistorian I suspect @RoamResearch is all in on *associative trails* (as in Bush "As We May Think") as the primary way to navigate, and begrudgingly supports structured directed search (querying).

See: theatlantic.com/magazine/archi…

@RoamResearch @calhistorian In this form of navigation, I think you are not trying to get to a precise query as fast as possible. Instead, you are associating, orienteering around, trying different entry points, following trails and paths, because the map isn't set yet.

@RoamResearch @calhistorian At a simple level, you "find your way" through exploration, by following associations.

@RoamResearch @calhistorian One (personal) example: I think I talked about this with [[Rob Haisfield]], and maybe it was related to [[Ted Nelson]]'s stuff? Ah, it's this block! (side note: it's the seed for this thread :D)

@RoamResearch @calhistorian Other examples: it's related to this one paper, check linked refs, find another term, follow that trail, explore block refs, find other people, literature, terms along the way, zettels, etc.

@RoamResearch @calhistorian Because each trail is editable in place, as a view of a subgraph, as opposed to a static list of results, I often make connections between the past blocks and new language or claims or ideas I'm thinking about now.

@RoamResearch @calhistorian You can also iteratively develop and curate higher level "index" entry points.

@RoamResearch @calhistorian Example: @NickMilo's Maps of Content

See: medium.com/@nickmilo22/in…

@RoamResearch @calhistorian @NickMilo Example: @maartenvandoorn's "structure notes" (adapted from the original zettelkasten)

See: roambrain.com/the-complete-g…

@RoamResearch @calhistorian @NickMilo @maartenvandoorn Example: @beauhann's global zettelkasten index, which emphasizes associative trails through a simple lightweight structure and navigation through block references.

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann In these cases also, the construction of these entry points and indices is not *for the sake of efficient retrieval*, but generative intellectual work in itself.

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann Thus, recall in @RoamResearch is a bit more like memory as reconstruction (like human memory, where the network changes with every recall) and less like memory as efficient retrieval. And the journey is as, if not more, important than, the destination (yes, cliche I know!)

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann I'm pretty sure this navigation approach is substantially less efficient than powerful search, and makes no sense if your goal is to get the right materials as quickly as possible: a worthy goal in many circumstances!

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann But for knowledge work that is characterized by grappling with "the complex, the changing and the indeterminate" (h/t @TheTedNelson), I have found that this recall process is wonderfully generative and playful, and stimulative of complex creative synthesis.

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann @TheTedNelson To return to search, at least in @RoamResearch, I think this means that trying to hack metadata and blocks to support precise querying might actually be an anti-pattern in Roam, fighting against its strengths.

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann @TheTedNelson Would better search improve things? Would it help me find things faster? Yeah, probably!

But I do wonder a bit about the opportunity cost: would I tend to lean less on associative trails and more on search if it were available? Maybe. Would this hurt my research process? Maybe!

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann @TheTedNelson Is semi-functional search but amazing affordances for associative trails an intentional design decision from Roam?

I suspect it is!

I know they have deep roots in Bush and Nelson and Engelbart.

And they sort of say as much in their white paper: roamresearch.com/#/app/help/pag…

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann @TheTedNelson See also:

https://twitter.com/Conaw/status/1099181050045952006?s=20&t=J9poSABhHEhLMILIn8iT2Q

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann @TheTedNelson Is this a good choice *overall*?

I'm pretty sure it isn't.

You only get gains from this sort of recall under certain conditions. And you always pay for it with efficiency.

That's just not a good tradeoff in most situations.

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann @TheTedNelson And often you're mixing the messy, complex indeterminate stuff with more structured, high certainty work where efficient retrieval is both possible and desirable.

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann @TheTedNelson So I think better search is a good idea.

However, I think @RoamResearch's needs sth. different from keyword/metadata-based search to build on its associative strengths to enable thinking further up the rigor ladder of synthesis, I think we need sth more like graph-based search.

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann @TheTedNelson I think this requires a little more structure, but more in terms of typed nodes and edges, rather than precise metadata.

For background on this, see

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann @TheTedNelson And this:

https://twitter.com/JoelChan86/status/1343422075743166464?s=20&t=drlL3K95-E8L56iD3dGCpQ

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann @TheTedNelson This can be quite annoying to author!

But I think it is solvable with some thoughtful extensions to the tool that enable us to write and link as naturally as we normally would, while adding typed graph-like structure for enhanced synthesis as a byproduct.

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann @TheTedNelson This is what I am trying to accomplish with the @RoamResearch discourse-graph extension, with @dvargas92495: roamjs.com/extensions/dis…

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann @TheTedNelson @dvargas92495 The extension smooths the process of specifying typed discourse nodes alongside my informal notes (e.g., claims, questions, evidence), and recognizes particular patterns of writing that relate them in typed ways (e.g., support/oppose), reifying this into a formal discourse graph

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann @TheTedNelson @dvargas92495 This enables targeted graph-based queries like "evd that informs q but opposes clm"

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann @TheTedNelson @dvargas92495 And enhanced discourse-graph-based navigation like "what other evd are consistent with this evd?"

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann @TheTedNelson @dvargas92495 And in-line overlays of the discourse context of particular nodes that complement the associative trails of block and page references, enabling more structured synthesis to emerge alongside my speculations:

https://twitter.com/JoelChan86/status/1478784924694032384

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann @TheTedNelson @dvargas92495 A somewhat current description of the extension is written up here: commonplace.knowledgefutures.org/pub/m76tk163/r…

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann @TheTedNelson @dvargas92495 One last thing: I hope more of our conversations around #ToolsForThought are deeply contextualized to specific use cases. We might gain inspiration from this template/diddy about design heuristics from Jan Van den Akker (h/t @andy_matuschak)

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann @TheTedNelson @dvargas92495 @andy_matuschak In this spirit, I think the umbrella term "tools for thought" can cause harm (diluting inherently use-case-varying discussion) that outweighs its benefits (bringing more voices and approaches into conversation). On this, I think I agree with @jessmartin:

https://twitter.com/jessmartin/status/1491565447480811527?s=20&t=faJ6V_uwLJgtspFRDRlm5g

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann @TheTedNelson @dvargas92495 @andy_matuschak @jessmartin And of course, this connects back to @andy_matuschak's more general claim that we need to ground our thinking about tools for thought in serious contexts of use: notes.andymatuschak.org/Effective_syst…

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann @TheTedNelson @dvargas92495 @andy_matuschak @jessmartin As a positive example of this, being very precise about the use case, I recommend reading this from @ejames_c:

https://twitter.com/ejames_c/status/1491364721399934980

@RoamResearch @calhistorian @NickMilo @maartenvandoorn @beauhann @TheTedNelson @dvargas92495 @andy_matuschak @jessmartin @ejames_c /FIN