it’s 2020, let’s start the new year with a long thread:
one like = one fact about or feature of c-lightning dev (max 100)
one like = one fact about or feature of c-lightning dev (max 100)
1/ a handy way to see all the configuration options for c-lightning is by running `lightningd --help`
2/ for all the available commands, try `lightning-cli help`
3/ as for seeing all of the currently configured options, you'll want the command `lightning-cli listconfigs`
4/ v0.8 remaps where the default lightning directory is -- from ~/.lightning to ~/.lightning/bitcoin (or ~/.lightning/testnet, if you're running on testnet)
5/ the c-lightning RPC works by talking through a socket file called lightning-rpc. it's created by lightningd and lives in the lightning directory
(as an aside, there's a small bug in 0.8 where it can get confused about where the rpc-file is located if you attempt to pass in the location via the config flag, this commit by @darosior makes it such that we use the rpc-file config if it's absolute github.com/ElementsProjec…)
@darosior 6/ c-lightning is a collection of dameons that alk to one another via file descriptors. Unix is the only OS (that i know of) that allows file descriptors to be passed to a new process at start (i.e. via fork)
@darosior 7/ this is why c-lightning hasn't been ported to Windows
@darosior 8/ Generally speaking, there's one daemon or process run per peer; every active channel runs in its own process
(this is a lot like how chrome uses one process per browser tab)
(this is a lot like how chrome uses one process per browser tab)
@darosior 9/ connectd manages new incoming and outgoing connections; it's basically the owner of all the 'networking stack' logic
@darosior 10/ it's the daemon that will take a net address and translate that into a working file descriptor connection to a remote peer (which can then be handed off to the next process, openingd, to handle a channel establishment)
@darosior 11/ openingd is the daemon that handles "opening a channel" (it's the one i've been reworking to get dual-funding up and running)
@darosior 12/ because of how openingd failures are handled, if a channel open fails you also lose the connection to the peer (so it'll disappear from `listpeers` and you'll need to reconnect to it to retry funding a channel with it again)
@darosior 13/ an elegant way to handle this would require passing the file descriptor for the peer back to connectd, which we don't do currently
@darosior 14/ You have to run `connect` to a peer before you can open a channel with them, because we need a file descriptor from connectd to pass to openingd to kick off a fundchannel attempt
@darosior 15/ Recently we changed fundchannel to attempt connecting first; this works if connectd is able to find the peer without explicit network addressing information (see github.com/ElementsProjec…)
@darosior 16/ There is no "abort a channel open" message so it's possible to get into weird states if something goes haywire during an open channel attempt
@darosior 17/ One example of this is if (for some reason) your funding tx is invalid (so bitcoind rejects it) your channel will end up in AWAITING_UNILATERAL state
@darosior 18/ (one way to get bitcoind to reject your transaction is by attempting to chain more than 25 opening funding txs together, the mempool will reject tx chains that deep. you can only get c-lightning to attempt this if you set minconf=0 on fundchannel)
@darosior 19/ This state (AWAITING_UNILATERAL) is incorrect because the funding transaction never made it (and never will make it) to into the blockchain
@darosior 20/ You can't 'unilaterally close' a channel that doesn't have a funding transaction!
(As an aside, I'm currently working on cleaning this up a bit so hopefully this won't be true for much longer)
(As an aside, I'm currently working on cleaning this up a bit so hopefully this won't be true for much longer)
@darosior 21/ One way to tell if a channel open made it into the blockchain is if you've got a short channel id associated with it (you can see this in the listpeers' channel object. if there's no short channel id, the funding transaction hasn't been mined yet)
@darosior 22/ The short channel id is the blockheight, txindex, and output index of the funding output for a channel, so 10000x10x5 would mean that your output for the channel was the 6th (index 5) output on the 11th transaction (index 10) of the 10.000th block
@darosior 23/ One way to roughly estimate the number of lightning channels that exist (including private, non-gossiped channels) is to count the number of P2WSH outputs that exist in the bitcoin UTXO set
@darosior 24/ This is increasingly less reliable as more wallets start using native segwit addresses (particularly script hash outputs)
@darosior 25/ Lightning was the first project to use Segwit only addresses
@darosior 26/ This is because funding transactions need a strong guarantee that the transaction id (txid) of any signed transaction will not change
@darosior (if the txid of the funding transaction change, it would invalidate the pre-commitments that you sign for the shared 2-of-2 output that forms the channel balance, making your funds basically non-recoverable. segwit fixed this though)
@darosior 27/ c-lightning's plugin system is absolutely incredible in the level of control and interaction it gives you over a node; check out the PLUGINS doc for more great info github.com/ElementsProjec…
@darosior 28/ that being said, one reason c-lightning is harder to write 3rd party apps for than LND is that we don't offer a standardized HTTP accessible interface out of the box, nor an authentication scheme
@darosior 29/ LND on the other hand shipped a gRPC interface and implemented macaroon authentication for its endpoints *very* early on. you can build out the same with a plugin, but it hasn't been done yet (and you'd need the node to run that plugin for 3rd party apps to work with it)
@darosior 30/ c-lightning uses a "csv" file to define the interfaces for wire messages nodes exchange
@darosior 31/ these csv files are generated directly from the spec itself, as defined in the github rfc repo
@darosior 32/ the python script that generates the csvs lives in the rfc repo, you can find it here github.com/lightningnetwo…
@darosior 33/ on the other side, c-lightning has a python script that will generate C source files from these csvs. the build process involves downloading a copy of the rfc from github, generating the csvs and then generating the C from that. then building it. github.com/ElementsProjec…
34/ c-lightning also uses this same csv format to define interfaces for messages between the different processes/daemons that make up c-lightning. e.g. here’s the hsm csv file github.com/ElementsProjec…
35/ sometimes the generated files from these csvs can get out of date (esp if you’re hopping between branches). you can clean them up with `make wire-clean`
36/ other generated files include the manpages and the database SQL statements
37/ if you want to edit the manpage for a c-lightning command, you'll edit the lightning-*.7.md version of the file (these are all in doc/ in the github repo)
38/ you'll need to run `make doc-all` before committing the changes, since you need to re-generate the corresponding lightning-*.7 file
39/ the SQL statements are generated because we now support a variety of database backends instead of just SQLite3
40/ there are only two implemented drivers however, one for SQLite3 and one for Postgres
41/ The reason that they need to be generated is that SQL dialects are not standardized; *every single* SQL database engine has a slightly different format for their query language
42/ @Snyke wrote a translator that will output inline SQLite3 queries into postgres compatible outputs
43/ these generated files are the reason why there are python dependencies in the c-lightning build
44/ there’s a bunch of handy tools in devtools/
45/ all of the `mk...` scripts were originally written to generate test cases for the protocol test suite @rusty_twit wrote
46/ one nice thing to do would be to have the protocol tests run as a part of our CI builds; they dont right now and rot from time to time because of it
47/ one if the more “idiotic” things I’ve done recently was put up an issue to grind c-lightning sigs (to match bitcoind); someone submitted a patch, i merged it and broke almost every test that had a signature
(the commit can be found here github.com/ElementsProjec…)
48/ to fix it, i moved all the signature verification over to requiring a secret key and a hash and dynamically verifying the returned sigs
49/ if a message you send c-lightning results in an error, we’ll return a copy of the originating message back in the error
50/ you can decode any message from hex using the handy `decodemsg` command line tool in devtools
51/ speaking of test suites, c-lightning has 2 ways of writing tests (protocol tests would be a 3rd)
52/ one suite is “unit tests” written in C; you can run them with `make check-units`
53/ these test files are located in with the other C files, usually in a subdirectory called `test`. so the wire tests are in `wire/tests`
54/ it’s a good idea to have Valgrind enabled when you run these, to find memory errors
55/ you can check that Valgrind is enabled with `./configure —enable-valgrind` (then run make to recompile)
56/ one cool thing that the unit tests for our wire parsers do is run a truncation and bit flipping check. we take a valid message and make sure that the parser fails if any bit is flipped or the message is truncated at any point
57/ the truncation tests started failing when we added TLVs to the wire messages, as TLVs allow a subset of 'valid' junk
58/ Here's the commit where @rusty_twit fixed up the truncation check to make our TLV truncation checking work as expected github.com/ElementsProjec…
@rusty_twit 59/ c-lightning has a check that will verify that comments in the code base that begin with 'BOLT-X' prefix exist, verbatim, in the rfc repository
@rusty_twit 60/ what commit, exactly, to use as the comparison point in the RFCs is the commit sha value in BOLTVERSION
@rusty_twit (it's specified in Makefile)
@rusty_twit 61/ This is also the commit that we use to generate the CSVs that our wire parsing code is based on; when new wire changes are added to the RFC we have to update c-lightning so that it passes any BOLT check so we can also pull in the new wire formats that are now in spec
@rusty_twit 62/ Both of these checks are run (along with some others) with `make check-source`. it's a good idea to run this before committing any code
@rusty_twit 63/ the other test suite in c-lightning is a Python based set of "integration tests"
@rusty_twit 64/ These actually spin up lightning nodes and have them interact with each other; as opposed to the unit tests that just run select parts of code in isolation
@rusty_twit 65/ The test driver relies heavily on pyln-client, the Python RPC and plugin client for c-lightning
@rusty_twit 66/ You can use pyln-client to write / script c-lightning interaction for non test purposes too. in fact, pyln-client stays up to date with c-lightning changes the best because the library gets updated every time we need to test some new functionality
@rusty_twit (a new version is published every time we make a release)
@rusty_twit 67/ if a python test fails, it'll print the directory where the node logs are; i use these a lot to figure out what's wrong
@rusty_twit 68/ a note though, sometimes Valgrind being enabled will eat/truncate backtraces in the logs, so i'll turn Valgrind off to get a decent backlog for a crash. (there's probably also a way to use the crash log file but 🤷♀️🤷♀️)
@rusty_twit 69/ speaking of errors, one common Valgrind error I make is allocating an object to something that gets freed before i use the object i actually needed. usually the stacks for these talk about "accessed here, but freed here"
@rusty_twit 70/ another common Valgrind error i make is re-assigning /allocating into a pointer that's already populated by a thing. this removes the reference to the previous item without freeing it (a memory leak)
@rusty_twit 71/ the solution here is usually to free the thing before re-populating. or to check that the reference is empty before populating it. you know, whatever makes sense for your particular case
@rusty_twit 72/ speaking of debugging things, i use `gdb` quite a bit for various crash things. it's particularly useful for the unit tests, i've found
@rusty_twit 73/ i've been told there's a way to get the python tests to run in debugger mode but have yet to actually figure out how to do that
@rusty_twit 74/ A not well known feature/tool that you can use for c-lightning integration tests is the disconnect 'toolset'
@rusty_twit 75/ This lets you disconnect a node at various points of receiving a message from a peer (before it receives the message, while it's receiving the message, after it's received a wire message)
@rusty_twit 76/ here's an example of it being used in the connection tests github.com/ElementsProjec… 
@rusty_twit 77/ the list of available options (and to give you an idea of what the wire name prefix symbol means) is in this header file github.com/ElementsProjec…
@rusty_twit 78/ the source for it is in this corresponding C file. it's not the most readable part of the c-lightning codebase but kind of cool how it takes advantage of the fact that peer connections are file descriptors github.com/ElementsProjec…
@rusty_twit 79/ note that to use this (and a lot of other small things) you'll need to have enabled developer mode at compile time
@rusty_twit 80/ as with most 'compile time' flags, you turn it on with ./configure --enable-developer
@rusty_twit 81/ in fact, you can see a complete list of available 'compile time flags' using the help flag on configure `./configure --help`
@rusty_twit 82/ some of the developer-only options are hard forgetting a channel (dev-forget-channel) or rescanning the blockchain for updates for wallet utxos (dev-rescan-outputs) or not gossiping (dev-suppress-gossip)
@rusty_twit 83/ recently we had some confusion around rescans; c-lightning, at start, will 'reprocess' the last 100 blocks from bitcoind
@rusty_twit 84/ we call this a 'rescan'. we actually forget the last 100 blocks from the database and refetch them from bitcoind, replaying them as we go. you can change how many blocks we 'rescan' at start with the `rescan` config flag
@rusty_twit 85/ yes, c-lightning does this every time you restart it. note that node functionality is limited while we catch up to the blockchain, so there's a bit of time at start when your node isn't fully operational
@rusty_twit 86/ by contrast, the dev-rescan-outputs only scans the blockchain for outputs your wallet *already* knows about
87/ so if your wallet is missing outputs, dev-rescan-outputs won't help you, but the rescan option will; if you've got outputs that are in the wrong state, dev-rescan-outputs is your friend
88/ note that *both* of these assume that your bitcoind provider is in a good state and is itself up to date with the latest blocks
89/ you can check the state of a sync with `getinfo` -- it'll print a notice if it's still syncing
90/ `getinfo` also prints the current blockheight that c-lightning knows about, if you want to know how far off bitcoind it is (which you can check with bitcoin-cli getchaintips)
91/ one cool thing c-lightning uses internally is the ccan C library called 'tal', which is short for "tree allocations"
92/ @rusty_twit wrote this library, it's basically a utility for managing memory allocations in tree-like structures. very very handy for making sure that the lifecycles of things are correctly scoped
93/ i can't imagine writing C code without tal 😱
94/ it's also got destructors, which let you do cleanup before freeing an object. very necessary for removing references to about-to-be-dead objects from tables or hashmaps or lists etc
95/ basically how it works is you always specify a context from which to allocate things off of (this is the ctx parameter a lot of c-lightning methods begin with)
96/ when you want to clean up, you free the context object `tal_free(ctx);` -- this will free any items allocated off of items allocated off of context, etc. because it's a tree!
97/ running out of things here. my favorite plugin is the summary plugin, it's super useful for getting a high level view of what's going on with your node github.com/lightningd/plu…
98/ if you want to contribute to c-lightning but don't know where to start, we love spelling corrections and formatting fixes and additions to documentation
100/ @snyke, @niftynei, @rusty_twit and a bunch of other contributors also hang out on #c-lightning on IRC (freenode), come say hi or ask questions!
finally, the three of us *just* gave presentations on developing different parts of c-lighting at a conference in early December here's our talks:
@Snyke on protocol development tools in c-lightning
@niftynei on getting started with c-lightning dev (basically some of this tweet thread, but in video form)
@rusty_twit covered more advanced plugin development, specifically using hooks for interactivity
