This week we helped a customer who was facing an issue with timezones.
Sit tight.
It's fun.
1/n
Customer reported that when they started using MS Build of OpenJDK, date/time data was being inserted with the wrong time. A close look showed that the timezone was GMT-0.
Weird. But not difficult to solve. Add the following to the Dockerfile:
ENV TZ "America/Sao_Paulo"
2/n
Now the date/time was *almost* correct: there was still a shift of one hour. Fuck you, Daylight Saving Time policies.
We checked IANA and it was up to date.
What was going on???
3/n
Looking at the source code of Spring Data, we see that it sets the timezone when starts the database session with Oracle. ALTER SESSION SET TIME_ZONE "America/Sao_Paulo".
Fine. But why the one hour shift? You guessed it right. The Oracle DB IANA info was outdated.
4/n
Customer didn't want to fix that right away, and still insisted the previous JDK base image they were using didn't have this problem. They were just given that image, and it was working fine.
Only when they changed base image to use MS JDK.
Let's dig.
5/n
We asked customer to boot up previous version, and boom. There it was in the log:
-Duser.timezone=GMT-3
A lazy workaround that was just waiting for the next DST change to blow up again. And also hide the fact Oracle DB IANA was outdated.
6/n
So here are lessons learned:
1) have *full* control of your Docker images. Avoid using base images from 3rd parties (including MS JDK) if possible. Build your own. 2) Timezones are devil's work. Have IANA up to date *everywhere*. Call your Sysadmins and DBAs. 3) Don't use GMT-*
Clarification: the previous base JDK image the customer was using wasn't put together by the customer themselves, it was just handed to them by 3rd party, and had user.timezone set.
(nor was it using MS JDK; but that's particularly irrelevant)
ActiveProcessorCount currently matches cpu_quota: up to 1000m, 1 proc. 1001-2000m, 2 procs, and so on.
This sounds reasonable, but as we've learned, CFS control does not limit CPU count, only CPU time. Multiple native threads may execute in parallel, until the quota is reached.
JVM Ergonomics also selects G1GC by default only when the JVM sees more than 1792 MB of available memory, and 2 available processors (2000m+ in k8s). Otherwise it picks SerialGC.
Thus why so many devs force -XX:+UseG1GC in well constrained containers.
Not only #Java developers are deploying their microservices to #Kubernetes with 1 vCPU only (1000m), there may be multiple pods on the same nodes since podAntiAffinity is unlikely to be common practice when scaling, to get an even distribution.
So, instead of having a bigger pod (2000m-4000m) on a node, there are two or more 1000m pods on a node.
For web apps, I think this setup is very unlikely to be reasonable.
I hope I am wrong and that my lame knowledge of Kubernetes followed by my hate towards YAML is as bad as this hypothesis.
This site was supposed to be the answer, but it only raises even more questions. The information here is mostly marketing-related, and has pointers to other online locations, outside go.java.
The Java source, no doubt. Yet, devs who want to just learn/code Java apps will only find binaries downloads. Nothing else serves the average Java software developer. "Developer's Guide" is for contributors.