Since leaving Google / TensorBoard, I've been working on #sourcecred. SourceCred's goal is to recognize open-source developers and maintainers by assigning 'cred' to their contributions. Here's a development update from the past month.
1. @wchargin and built a new prototype, which ingests data from GitHub and Git into a large graph and runs PageRank on it. We wanted to check that we can scale in-browser PageRank to graphs with >100k nodes. The answer: we can! It's even quite fast.
2. Having de-risked one of our core technical challenges, we took a step back to refactor the architecture. At the start, we put all data - Git metadata, GitHub issue text, etc - into one data structure: the Graph. This proved unwieldy, and would make SourceCred hard to scale.
3. We decided to pay down this technical debt early, and re-write our core abstraction. We stopped storing arbitrary data in the graph. Now plugins (like the Git & GitHub plugins) store connections in the graph, but bring their own data stores for other info.
4. We're just polishing that up now. It's taken a few weeks to get right, but I'm proud that we are taking our time to make sure SourceCred is built on a good foundation. Since SourceCred will be distributed & involve many plugins, giant monolith data structure = bad.
5. Once we finish the graph re-write, we're planning to:
- make the cred attribution tunable, by setting prior weights on node and edge types
- make the cred attribution improvable, by adding the 'spotlight' plugin for directing cred towards important contributions
- make the cred attribution tunable, by setting prior weights on node and edge types
- make the cred attribution improvable, by adding the 'spotlight' plugin for directing cred towards important contributions
6. We've also been having more conversation going in our Discord - discussing how to get the incentives right so we reward development and not gaming the system; how not to mess with working social dynamics, etc. Feel free to join the convo! discordapp.com/invite/tsBTgc9
