M29 Milestone Updates

Here are the updates for Milestone M29 (Jan 27 - Feb 09).

NAT Gateway usage cost reduction.

We were incurring about 40$ a day for NAT Gateway usage. Once we optimized our EC2 cost, the NAT Gateway cost immediately became significant, and for the first time, we went about understanding and optimizing it.

A NAT (Network Address Translation) Gateway in AWS is a managed service that allows instances in a private subnet to connect to the Internet or other AWS services while preventing the Internet from initiating connections to these instances. For security, we run the EC2 machines that host the customer's applications in a private subnet (without direct internet exposure).

These machines need internet access for the following:

i. Serve web requests - This happens through the Load Balancer.

ii. Container Image pulls from ECR.

iii. Application log upload to S3.

iv. Package install and image push to S3. (During builds)

v. External API calls made by the hosted applications.

The ii, iii, and iv and v were going through the NAT gateway, which cost us money - image pulls, logs, and package installs all happen in the order of Giga bytes. Since S3 and ECR are AWS services, we set up Private Links so traffic remains inside the AWS network without going through the Internet. Also, we moved our Build machines to a public subnet. Build machines are short-lived, so direct exposure does not cause security concerns. Being on a public subnet allows package installs to happen via the internet gateway, not the NAT gateway.

The above measures have reduced our daily NAT Gateway cost from over 40$ to under 10$.

Application and addon Metrics revamp is underway.

We used the commonly used open source exporters to export metrics data from addons like Postgresql, Redis and our general purpose dynos to Prometheus. These exporters run as a side-car container to each of the add-on containers, and they also scrape and upload lots of metrics that are irrelevant to us. We wrote custom exporters for the add-ons.

A single exporter that gathers only relevant data points for all our Postgresql addons. Similarly, one for Redis. We will be optimizing the exporters for our general-purpose dynos in the same fashion.

Build system revamp

NeetoDeploy uses CloudNative Buildpacks to generate general OCI container images from the application source code. We were using the buildpacks that Heroku open-sourced. These build packs had Heroku references, and their license may not allow commercial use.

More importantly, it was like a black box, and we did not have the control over the build we needed while building a platform like NeetoDeploy to optimize the final image, leverage build caching, support custom dependencies and fine-tune the build process. So, we revamped our build system after forking and customizing Paketo buildpacks. We will roll this out in the coming days and gradually phasing out our Heroku-based build system gradually and completely.