Thread by @matloff

#rstats for YOU: Ever encountered an R error message like "Couldn't create memory segment of size 3.2G"? Also, more subtly, ever had some code run slowly in spite of little apparent reason? And ever wonder why data.table is so blindingly fast? This post will be on MEMORY. 1/n

So here goes Memory 101A. Memory (meaning RAM) is broken down into "words," typically 8 bytes long. One R numeric quantity will occupy one word. So, e.g. 1 gig of memory will hold a numeric vector of length only 125 million. 2/n

Each word has an "address," an ID number. If your installation of R includes the tracemem() function, you can use it to determine where in memory your R object is, e.g.

> x <- 1:500000
> tracemem(x)
[1] "<0x7fb1eab6e010>"

3/n

That 0x means this is a hex, i.e. base 16, address. That could be converted to the familiar base 10, but it's not needed here. What is important, though, is that consecutive elements in a vector are stored in words with consec. addresses. (Which differ by 8, the word size.) 4/n

That consec. aspect can have major code speed impact. Your CPU has a cache (or several), in which there are copies of parts of memory that are likely to be accessed soon. Cache access is much faster than RAM. 5/n

OTOH, a cache "miss" -- failure of the desired quantity to be in the cache -- can really slow things down, bc many new words are brought in. The order in which your code accesses data can impact cache behavior, thus impact code speed. 6/n

R uses "column major" storage of matrices/dfs -- all of column 1 will stored contiguously, followed by col. 2 etc. @MattDowle noticed that R was not exploiting that fact, so he revolutionized R performance by tailoring data.table to that storage pattern. 7/n

On most hardware, a function call f(x) is performed internally using a section of memory known as the "stack." Here the address of x is put in (we say "on") the stack, where it is picked up by the internal workings of f(). 8/n

Note that typically it is the address of x, not x itself, that is placed on the stack, so this op takes the same time whether x is large or not. We say the address of x is a "pointer," since it "points to" x. 9/n

In my tweet tinyurl.com/yxtfzl5h I considered the code y <- f(x); g(y). Say the return value of f(x) is quite big. Since only pointers are involved, there is really no performance penalty for first saving f(x) in y, then calling g() on y. see the above tweet. 10/n

However, R will place the output of f(x) somewhere in memory. If we were to do the more direct g(f(x)), that area would be released once g() is through with it, but in our above case, y would still be pointing to it and it won't be released until we do rm(y) (or sim.). 11/n

Note that writing to a vector may trigger reallocation of memory:>

z <- 1:100000000
> tracemem(z)
[1] "<0x7f232de61010>"
> z[3] <- 8
tracemem[0x7f232de61010 -> 0x7f22fe370010]:

This can be very costly in code speed. R tries to avoid it, but keep this in mind. 12/n

Finally, if your data is too big for your RAM, consider using the bigmemory package. It exploits an OS feature that allows one to place data on disk (with huge storage capacity) but trick the CPU into thinking the data is in RAM! 13/13

PS: Here is a little more to our look at #rstats internally:

> w <- 1:500000
> object.size(w)
2000048 bytes
> w <- c(1:499999,8)
> object.size(w)
4000048 bytes

14/n

Both vectors are of length 500000. What happened? Recall, word size on today's machines is 8 bytes, So a vector of length 500000 should occupy 4000000 bytes (plus 48 bytes of "meta-information" here). That was true in the second case but not the first. 15/n

In the first case, the ':' apparently signaled to R that no element larger than 500000 would be used, and since 4-byte chunks can store numbers sized up to about 2 billion, only 4 bytes were used for each element of w. 16/n

And this apparently is what triggered the reallocation in my earlier tweet today, tinyurl.com/y52hp8c4; R no longer had a guarantee that z would contain numbers in the range 1 to 1000000000, so it switched to 8-byte numbers, necessitating reallocation. 17/17