Thread by @theburningmonk

Done more performance analysis of the new Lambda-EFS feat. Some interesting findings, longer write-up to follow.

1st: mounting an EFS volume has no impact on Cold Start duration (as measured by InitDuration)

2nd: it is much FASTER and more predictable than S3

#serverless

/1

On the initDuration, on a relatively small sample set (~100) I didn't detect any noticeable difference between the control and EFS ("efs-test") functions.

/2

For read/write performance, it's clear to see that EFS is both faster and more predictable (i.e. lower SD) compared reading & writing from/to S3.

But something was weird - notice those big "jumps" between the tail latencies for EFS read latencies?

/3

So it seems the first read on a new container (as you can see in the change in the logStream path) is much higher.

Same for diff file sizes.

Is there some caching in place? The Node.js code is straight forward enough, so don't think the caching is happening there.

/4

So, what if the file is changed frequently?

I updated the test to read the '-1' suffix'd file on each iteration to see what happens to the read latencies.

And sure enough, the numbers quite different, a lot more "high" read latencies.

/5

But did you notice that the smaller the file, the more frequently we see a "high" (relative) read latency on the same container?

I theorize that the longer the write ops took (bigger file) the more likely the cache hit on next read. Maybe writes are streams to cache?

/6

But is the caching happening at the container level? Is that why the 1st read (during cold start) is always higher?

Still much I don't understand about how it works. Maybe someone else can shed some light? @hichaelmart @heitor_lessa @em__shea @sthulb

7/