Some live notes from #Urbit Volcano Summit (no spoilers). Mistakes if any are mine.
You should be excited for the future of Mars.
You should be excited for the future of Mars.
First up, @rovnys, CTO of @urbitfoundation.
Core dev has an open-door policy. The roadmap and grants are public, devs regularly work and meet in gather.town, and core dev has a commitment to merge all good contributions.
Core dev has an open-door policy. The roadmap and grants are public, devs regularly work and meet in gather.town, and core dev has a commitment to merge all good contributions.
Zero tolerance for show-stopper bugs.
Daily PR blitz with triage, reviewer assigned w/i 24 hours and sometimes PR merged w/i 24 hours.
Daily PR blitz with triage, reviewer assigned w/i 24 hours and sometimes PR merged w/i 24 hours.
Scry maximalism: read from a peer's Clay, read from a peer's agent, and sticky scry requests.
Loom maximalism coming: 8 GB as of last week, but New Mars aims to store arbitrary amounts.
Other specs could increase the loom faster but undecided as of now.
Demand paging (implemented multiple times) will be decided on and merged, will enable a ship to run with smaller RAM.
Other specs could increase the loom faster but undecided as of now.
Demand paging (implemented multiple times) will be decided on and merged, will enable a ship to run with smaller RAM.
External atom storage is one way to bridge towards that world of storing as much as you want on-ship. E.g. large matrices, media, etc. could have a separate memory arena.
Improvements in developer experience planned: simplify developer workflow, operations, and maybe application model.
%pyro (improvement on %aqua) allows you to run other ships on one ship. This will be great for testing distributed systems and interagent operations.
Quickboot: getting the 2 minute plus boot sequence down to under a second.
JSON autoconversions
Operations: stop the glob. (how we distribute JS now, but it's clunky for devs to use & is a lot of load on the publisher ship pushing large data)
This was necessary before Clay tombstoning, but it's no longer required.
This was necessary before Clay tombstoning, but it's no longer required.
Stretch goal: logging. Text logging messages you can aggregate for telemetry.
The current application model with Gall works but has nontrivial elements. You should be able to write "dumb code" declaratively with at least okay scaling performance.
.@mdfang working on userspace permissions
.@wereness has written solid-state subscriptions, a new model for our pubsub. Currently in userspace.
github.com/wicrum-wicrun/…
github.com/wicrum-wicrun/…
"Shrub" architecture experiment will be rolled out for devs to experiment with
notified when scry subscription comes in (@sigilante aside: Project Xanadu?—unclear on ramifications of this in passing)
notified when scry subscription comes in (@sigilante aside: Project Xanadu?—unclear on ramifications of this in passing)
Urbit-native UI.
Hoon School works to teach even new devs, but JavaScript tends to be a barrier for them.
The people want an Urbit-feeling experience even when building a client.
(analogy to how JS devs like Node for uniformity of DX)
Hoon School works to teach even new devs, but JavaScript tends to be a barrier for them.
The people want an Urbit-feeling experience even when building a client.
(analogy to how JS devs like Node for uniformity of DX)
Anyway, should at least be a quick-and-dirty deployment instead of a full UI replacement.
"Be the core devs you want to see in the world."
If you see a problem, fix it or tell us. Don't see Urbit as a black box.
The other thing you can do is complain. No learned helplessness. Our bar needs to be the best development platform in the world BY FAR.
If you see a problem, fix it or tell us. Don't see Urbit as a black box.
The other thing you can do is complain. No learned helplessness. Our bar needs to be the best development platform in the world BY FAR.
TERMINAL STACK ARCHITECTURE by @mdfang
The Urbit terminal stack has runtime/kernel/userspace architecture.
Right now runtime deals with old-school terminal issues. Kernel has a noun architecture to tell runtime how to deal. Userspace shouldn't need to know details.
The Urbit terminal stack has runtime/kernel/userspace architecture.
Right now runtime deals with old-school terminal issues. Kernel has a noun architecture to tell runtime how to deal. Userspace shouldn't need to know details.
Kernel's Dill terminal vane communicates with term.c in the runtime.
Userspace agent associated with a particular terminal session for Dill: input pokes, output facts.
Userspace agent associated with a particular terminal session for Dill: input pokes, output facts.
Two kinds of output aren't associated with sessions: formal and informal outputs.
Formal: system logging
E.g. |pass [%d %text 'hello summit']
Not associated w/ session so how does the system know? Dill exposes subscription endpoint for vanes/agents to listen.
Formal: system logging
E.g. |pass [%d %text 'hello summit']
Not associated w/ session so how does the system know? Dill exposes subscription endpoint for vanes/agents to listen.
Informal: debug printfs, slogs
Not part of the terminal output loop but a Nock hint the interpreter encounters. Direct to runtime but Vere may not know semantics of on-screen display.
Not part of the terminal output loop but a Nock hint the interpreter encounters. Direct to runtime but Vere may not know semantics of on-screen display.
Thus debug output goes to default session & default session always looks like prompt at bottom w/ upwards-rolling scrollback
`%webterm` wants to be `term.js` sibling to runtime's `term.c`, but can't talk the same way to the runtime. Thus `%herm` hooks webterm to Dill (to runtime).
Primary userspace agent is `%drum`, part of `%hood`.
This is userspace analogue to Dill. This handles details of input and session management.
This is userspace analogue to Dill. This handles details of input and session management.
(@mdfang's slides are in the terminal.)
Book, the Urbit `tmux` is a Dill multiplexer. Full panels etc. like tmux incl. scrolling—I'd love to have a pic of this but it'll come out eventually.
Book, the Urbit `tmux` is a Dill multiplexer. Full panels etc. like tmux incl. scrolling—I'd love to have a pic of this but it'll come out eventually.
He demos:
%minesweeper
%snek
on-Urbit text editor
terminal color changes
%minesweeper
%snek
on-Urbit text editor
terminal color changes
mouse clicks
modifier keys
text block pastes
modifier keys
text block pastes
outputs: styling text, pasting at cursor, etc.
lots of low-level control
lots of low-level control
Higher-level agent building is based on %sole as always. %sole doesn't give you a way to draw on the screen directly, just to add new lines to scrollback.
We don't have full libcurses YET but it's possible to build such now. @mdfang's slide rendering code provides a basis for starting to build composable text user interfaces, and he'll share this as %etui, Everyday Text UI Toolkit.
(much of the rest of his talk discusses this)
(much of the rest of his talk discusses this)
New Mars by Edward Amsden
Nock codegen is the current priority b/c biggest technical risk in the New Mars project.
Nock codegen is the current priority b/c biggest technical risk in the New Mars project.
Subject knowledge analysis is a difficulty of executing Nock on some other substrate. Nock does not have a procedure called FORM to execute code known ahead of time. It has an evaluation FORM.
Nock 2 lets us take a noun and evaluate subject–formula pairs (and Nock 9 in terms of 2 to evaluate one subformula/axis).
This creates difficulties because to generate code we don't want to put codegen at every step. We don't want to wait until we have evaluated Nock to generate next Nock code—we want to write in one go so we can have JMPs etc.
(lots of very technical Nock discussion that I can't see where I'm sitting)
What do we know about the subject so that we can generate code not only for this expression but for the whole gate etc.?
What do we know, how do we know? Subject knowledge analysis.
What do we know, how do we know? Subject knowledge analysis.
What assumptions should SKA start with?
1. Everything—but the next time we eval Arvo has changed and we have to regenerate.
2. Nothing—but no info about bundled code in Arvo (hoon, zuse, arvo, vanes) so these end up dynamically generated.
3. So which part of subject do we want?
1. Everything—but the next time we eval Arvo has changed and we have to regenerate.
2. Nothing—but no info about bundled code in Arvo (hoon, zuse, arvo, vanes) so these end up dynamically generated.
3. So which part of subject do we want?
Three levels to work with:
1. Hints. Tell evaluator what to include or leave out. (W/ Arvo all state in axis 6.)
2. Fixed point. E.g. dynamic cores like Gall agents. EA has avoided thus far but semantically good here.
3. Types. Use +slap and vase logic to handle nouns.
1. Hints. Tell evaluator what to include or leave out. (W/ Arvo all state in axis 6.)
2. Fixed point. E.g. dynamic cores like Gall agents. EA has avoided thus far but semantically good here.
3. Types. Use +slap and vase logic to handle nouns.
-:!> type spear
!> makes vase, pair of type and noun, - takes the head
!> makes vase, pair of type and noun, - takes the head
So if you have Hoon types available (jetted +slap) then you can get subject knowledge.
This problem (Nock codegen) is the blocker until we have it working for Arvo and vanes. After that then a lot of other problems are unblocked.
The major improvement is codegen, but the loom issue has multiple off-the-shelf solutions so it's not a big deal for New Mars as a new code base.
Codegen is the *open problem*. This means Urbit execution performance.
Codegen is the *open problem*. This means Urbit execution performance.
You should be able to bring a currently-running Vere ship to New Mars, but not necessarily the other way.
Related: not needed to dynamically search for jets every time we invoke an arm, and being able to build Nock code closer to machine code without dynamic code invocation at every point.
Demonstration that Nock could actually become "the only legal form of computing."
Demonstration that Nock could actually become "the only legal form of computing."
Given SKA and knowledge of formulas we'll be invoking at Nock 2/Nock 9, we can move towards turning Nock into something like low-level machine code.
Walk the tree in some fashion, use this shape to stretch out the tree into a line of code.
Walk the tree in some fashion, use this shape to stretch out the tree into a line of code.
"will there be a new new mars?" lol
From tree→lines of code is destination-driven codegen. Emit new register, test new register with Nock 3, put a label on that code, and then generate subformula of Nock 3 w/ info to put result in given register and jump to the label.
Which parts of tree go in registers? Which parts don't matter? Which are stored on the heap? DDCG lets us figure this out.
How: when we generate primitive ops (Nock 3 or 5 or a cons &c) assert expect input in registers & output in register. So keep map for where subject is.
How: when we generate primitive ops (Nock 3 or 5 or a cons &c) assert expect input in registers & output in register. So keep map for where subject is.
Then subformulas can put results in particular registers. Map axes of subject to intermediate result registers = fast & neat.
We pattern on info about destination.
We pattern on info about destination.
(about to run out of battery & we have insufficient outlets, may be offline for a bit)
(saved by the cord but now I can't see screen)
(it's okay, this is pretty finely diced register-level and tree-traversal-level discussion)
(it's okay, this is pretty finely diced register-level and tree-traversal-level discussion)
(discussion of jet ~~dashboard~~ and memory arena implementation)
Soon he'll have something to run code through then so will we!
Soon he'll have something to run code through then so will we!
Hoon compiler doesn't change—this is all runtime-side ops. But it's written in Hoon for convenience—
Small bootstrapping step will run codegen code on itself then it's available to the runtime.
Small bootstrapping step will run codegen code on itself then it's available to the runtime.
Jet integration for certain parts of compiler but this is normal.
(write jets in hoon wen)
(write jets in hoon wen)
"A sufficiently smart compiler" could recognize the +add formula and turn it into a CPU-level add. How does this differ from pattern-matching and jetting? Not clear that it does in a meaningful way.
Jets aren't because Nock is slow, but because on atoms Nock isn't capable of algorithmically expressing asymptotically optimal form.
Hearkening back to the Precepts urbit.org/blog/precepts
these are the principles from which those could be derived.
these are the principles from which those could be derived.
We have to take seriously the possibility that Urbit could fail. We can't think of it as too big to fail. It's super valuable and we need to protect it.
We have to steward what we have but not get distracted from the bold bet on the future.
We have to steward what we have but not get distracted from the bold bet on the future.
It's important to make Urbit real, to ship code etc., but also don't get distracted from the problem we haven't yet solved: how can Urbit truly succeed in the marketplace?
Until we're the scale of Linux we haven't succeeded. We use to think this way daily but there are trees instead of forest sometimes now.
Why is Tlon focusing on Groups/Talk? Tlon thinks it's the best way to make Urbit succeed. Core dev is critical but it's difficult to also have product mindset. Attention is divided, and in that conflict core always won—a good choice then but now needs to be grappled.
@urbitfoundation ready for core now so that's one problem solved.
If Urbit fails, it's because we didn't build it fast enough. It's important to build prototype as fast as you can, end to end, then iterate on top of it. This places problems in their proper context.
More projects are released too late than too early.
More projects are released too late than too early.
It's easy to update Hoon code.
Examples:
%eyre (HTTP server) was galaxy-brained w/ novel ideas about webapp design. Anton rewrote w/ even more ideas, more generic & complex. Used for several years but hard to use, slow, memory leaks, hard to reason about for performance, hard to build products on as result.
%eyre (HTTP server) was galaxy-brained w/ novel ideas about webapp design. Anton rewrote w/ even more ideas, more generic & complex. Used for several years but hard to use, slow, memory leaks, hard to reason about for performance, hard to build products on as result.
In 2018 Elliot rewrote dumb-as-bricks and it was much easier to fix. Problems became engineering problems instead of protocols.
Lesson: build something simple and small and get it used.
%ford (build vane) first real userspace for Urbit. Very large, bigger than 2× next vane by size. Many opinions about userspace app construction.
No one else was in a position to understand & rewrite %ford yet.
No one else was in a position to understand & rewrite %ford yet.
Eventually %ford rewrite threw away loftiest goals in generics and async, 1000s of lines of monadic code to 600 lines of pure code w/ all daily features. Simpler, smaller, more reliable, no memory leaks, etc.
Lesson: you learn so much by using it that thinking about it ahead of time leads you down the wrong path.
%gall came out same time as %ford. %gall interface was invented language (dynamic %gall) hard to reason with & about, dependent system reasoning about arms etc. (I learnt Urbit on dyn. %gall, was hard.)
Then static %gall rewrite was much simpler, 10 std arms.
Then static %gall rewrite was much simpler, 10 std arms.
Gave up some functionality, but immediate uptick in people actually writing apps in %gall.
Lesson: take out all the opinions, ship internals.
Lesson: take out all the opinions, ship internals.
"When you don't know how to build something, build a simple version and be humble." @pcmonk
If you can find someone w/ experience in that area, ask them for advice, what's applicable to your problem. But be wary b/c experts may be too general in examination, & make dependent judgment calls (overkill etc.).
Experts may have pet solution, but may be for irrational reasons or reasons that just don't apply to your case.
Esp. b/c of where they went to school.
Esp. b/c of where they went to school.
Advice may conflict. This doesn't mean all answers equally correct or false, but can distract. Prefer the simplest option.
The most important thing is to not get paralyzed.
Don't spend too much time researching. Don't aim for the perfect solution because you won't find it until you iterate.
Don't spend too much time researching. Don't aim for the perfect solution because you won't find it until you iterate.
People will criticize you for an "uninformed choice". Don't let it bother you. It's better for them to criticize you for something you built.
Solve the small problems first. 1 hard problem, 5 small—solve the small ones first. Counterintuitive advice, but @pcmonk says Urbit dev more often fell into the first problem.
The small problem solutions provide a scaffolding for attacking the big problem.
Once you decide what to go with for big problem, you'll find out about heuristic edge cases etc. Risk isn't that it takes a bit longer, it's that you have to be *really good* to ñ get bogged down.
Once you decide what to go with for big problem, you'll find out about heuristic edge cases etc. Risk isn't that it takes a bit longer, it's that you have to be *really good* to ñ get bogged down.
Build the glue code first. This lets you put the problem in its proper context: is it actually slow with actual data.
Complaint could be that you leave a hack in. But give no weight: if you never come back to fix the hack, then it didn't need to be solved, and you didn't need to spend time solving it.
"Never solve a hard problem if you can get away with it."—@pcmonk
Urbit is about solving all the hard problems that we have to, but no more.
Every hard problem is a risk.
Urbit is about solving all the hard problems that we have to, but no more.
Every hard problem is a risk.
"Overthinking hard problems is far more likely to build fragile complexity than building something that ships."—@pcmonk
"Don't let your decisions get held up by wordy arguments appealing to vague principles."—@pcmonk
Why not write `|pass` a long time ago? In 2018, you had to add a command in five or six places for it to work: registrations, generator, invocations, etc.
Nowadays you just use `|pass` once the command is in the vane.
Nowadays you just use `|pass` once the command is in the vane.
The dev cycle is a lot tighter now. It was too much of a hassle to PR and ship.
There's no technical reason why the other steps were necessary, but they chose to do them b/c they valued first-principles thinking (to make it all explicit).
There's no technical reason why the other steps were necessary, but they chose to do them b/c they valued first-principles thinking (to make it all explicit).
Not w/o logic, esp. w/ kernel, but benefits of being explicit never materialized esp. vis-à-vis tradeoffs.
"Don't make decisions that make your life painful today just because you think it will be necessary at some point in the future. Your future self knows more about what he'll need than you do."—@pcmonk
Super important to release well.
TDD/automated tests are good to a point. %aqua was written for managing personal breaches.
TDD/automated tests are good to a point. %aqua was written for managing personal breaches.
Manual testing important.
But either way you have to come up with useful tests.
Mechanical systems for this, but none of them replace proper mindset. Thorough and unrelenting. Repeat tests for every code change. If anything looks off, understand why.
But either way you have to come up with useful tests.
Mechanical systems for this, but none of them replace proper mindset. Thorough and unrelenting. Repeat tests for every code change. If anything looks off, understand why.
"If you don't understand code, you shouldn't be shipping code."—@pcmonk
Testing is the process of observing what you already know. Don't let your mind get involved. Test each thing.
Debugging: be wary of code diffs only. Either you spot the problem or you don't. If you don't spot the problem, you haven't made progress. So you start trimming the mental model but "as far as you know none of the code has bugs else you wouldn't have been trying to ship it."
The problem may not be in the diff anyway—old code that is now being called differently &c.
The worst part is that this strategy is not guaranteed to make progress at all.
The worst part is that this strategy is not guaranteed to make progress at all.
Instead, find two points (known good, known broken), and print state. Then trim towards the middle until you locate the bug.
Few bugs can stand up to this if you have a reproduction.
Few bugs can stand up to this if you have a reproduction.
When you release code, release to some real ships before you release to all everybody. Not comets, actual ships in use—your own ship. That way if something got past your entire process, it doesn't sink everyone at once.
"I don't want to risk my ship." Well you're risking everyone's ships.
Other objection is this slows down releases: narrowly true but cheaper than support for a novel bug. Someone has to pay that price even if not you.
The point is to increase the quality of releases and this will do it.
The point is to increase the quality of releases and this will do it.
Collaborate well. Most/all of Urbit companies are primarily remote. Many advantages esp. personal, but historically unusual and unnatural. It's easy to be ineffective at it b/c we don't have norms.
One failure pattern is too much asynchronous communication.
Duplicating the ability to quickly ask questions etc. online is super important. Regular checkins are less useful than irregular interruptions.
Duplicating the ability to quickly ask questions etc. online is super important. Regular checkins are less useful than irregular interruptions.
Meetings are worse than collaborative working sessions, esp. at the scale of a startup.
The productivity difference is immediate and dramatic: gather.town, long voice calls, something.
(Phew. Imma take a break. Enjoy memes.) 
.@AssemblyCap Building the Future
high-level view of what's going on the network, encouraging their portfolio companies (Holium, Zorp, Uqbar)
tectonic shifts:
- Core dev moving to the @urbitfoundation as a public good w/ accelerating pace of technical development
- crypto bear market, scarce capital, as forcing function
- Core dev moving to the @urbitfoundation as a public good w/ accelerating pace of technical development
- crypto bear market, scarce capital, as forcing function
@ajlamarc's TomeDB, Urbit apps w/o knowing Hoon
Urbit now multipolar. Multiple assets, competitive commercialization phase after a decade of being a research project.
Bear market forces recognizing what works, what doesn't. Status quo in question. (cap/acc energy here)
Bear market forces recognizing what works, what doesn't. Status quo in question. (cap/acc energy here)
Sea change means opportunity. For devs and potential leaders there is massive opportunity on network. New leaders, new companies, new rewards.
The companies that succeed will do so by building in public. Build powerful customer communities by building & responding in public. Build engagement, flex your product, be public.
Building in public requires leadership. There are few people publicly evangelizing the product. Big difference from Ethereum or Bitcoin.
We need leaders and evangelists. (Evocative of @rivatez talking about Urbit being priests needing merchants.)
We need leaders and evangelists. (Evocative of @rivatez talking about Urbit being priests needing merchants.)
As a permissionless network, no one needs to give you permission to be a leader.
@rovnys delivers a roadmap for core.
Not understanding core meant disempowerment—this exposure means empowerment. We're Ted's customers (as devs).
Not understanding core meant disempowerment—this exposure means empowerment. We're Ted's customers (as devs).
Devs debating in public is very good.
The smartest engineers can't have convos unless this is happening.
The smartest engineers can't have convos unless this is happening.
@hanfel_dovned and others having fun on Urbit in public.
"Urbit will not resonate with people until there's something fun to do there."
"Urbit will not resonate with people until there's something fun to do there."
WebSockets v. SSE. Just doing it, without permission.
%blog vibeshift w/ @AlephDao and @dachus_tiprel
@HoliumCorp public engagement
"Today is always the best day to start an Urbit company."—@AssemblyCap
(I'm eliding the secrets of @AssemblyCap here. Gotta save something for the volcano attendees.)
@jlehman_ on Urbit Developer Metrics via @urbitfoundation
To build Urbit, you need builders.
Hoon School Live, App School Live, App Workshop Live, Core School Live.
Grants.
Hoon School Live, App School Live, App Workshop Live, Core School Live.
Grants.
@ElectricCapital (hope I have right @!) aggregated and made a chart of hockey-stick developer growth.
Josh scraped all Urbit developer repos and reproduced for us to see here.
Josh scraped all Urbit developer repos and reproduced for us to see here.
Are people using Urbit? Are they staying or churning?
Enter growth accounting. (@jlehman_ it's on you to post these someday.)
Enter growth accounting. (@jlehman_ it's on you to post these someday.)
Good metrics on details since 2010.
Some developer loss we need to address, but good retention and growth overall.
Some developer loss we need to address, but good retention and growth overall.
Statistics for any repo that had anything to do with Hoon.
(math, but not the exciting physics kind, the boring business kind)
(math, but not the exciting physics kind, the boring business kind)
What causes churn? (It will never be zero, but we need to make sure we're not doing anything dumb.)
(We also don't know anything about private repos.)
Growth targets? "Exponential"
Not yet, pending rumination on these numbers (freshly acquired), but we'll put something together @urbitfoundation; retention also v. important.
Emergence companies to hire devs is also very important in framing targets.
Not yet, pending rumination on these numbers (freshly acquired), but we'll put something together @urbitfoundation; retention also v. important.
Emergence companies to hire devs is also very important in framing targets.
ąçḑęģḩįķļņǫŗşţų
ȧḃċḋėḟġḣıṁṅȯṗṙṡṫẇẋẏż
āđēī£ñōū¥
ȧḃċḋėḟġḣıṁṅȯṗṙṡṫẇẋẏż
āđēī£ñōū¥
(sorry, distracted by Compose key)
Questions about cohort effects, why developers leave v. stay, comparison to other ecosystems.
Growth in developers is a leading indicator for user growth, we believe.
That's wrap for Trewsday. Hope to see you tomorrow.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
