How to save cost for your non production EC2 instances in AWS

Many companies pay a good amount of money to run non production workload on AWS. With increased number of developers, cloud cost also increases exponentially.

It is important to have the flexibility of launching and trying out new features required for your business case while maintaining a low cost footprint.

Lightwing Autostopping is a perfect fit for such cases. From our experience, we were able to save about 70% of the non-production cost using lightwing.

For this example, let us consider the following setup for a QA cluster, https://qa.example.com:

• 1 x t2.small: Load balancer instance running HAProxy
• 2 x t2.medium: API servers running RubyOnRails. These servers talk to Elastic search and postgres over private IP
• 1 x t2.xlarge: Elastic search server
• 1 x t2.xlarge: Postgres DB server

Total estimated cost: $357

Estimated cost with LW: $108. $249 saved

Lightwing autostopping is a convenient way to manage non-production environment without loosing developer productivity & flexibility.

Let’s onboard the above servers to autostopping and see how it helps to save cost.

Preparing environment

Before we onboard these servers, it is recommended to add enough metadata like tags to the servers so that lightwing can find the instances. Lightwing can also find instances by id. But it will be easier to configure with tags.

Open AWS console and add a type tag:

Note: You can add any arbitrary tags here.

We need to also ensure that your application services are autostarted when machine is rebooted. If not, add them to the init/systemd/windows script.

Configuring dependencies

In the above example, API servers are dependent on Elastic search and Postgres. Let us configure autostopping for ES and Postgres first.

• On lightwing console, click on Autostopping
• Click on Add a gateway
• Choose Instance based
• Use the following filter text to select ES machines:

[tags]
type = qa-es

• Save the gateway once done

Repeat the above for Postgres also.

Configuring API servers

• On lightwing console, click on Autostopping
• Click on Add a gateway
• Choose Loadbalanced
• Use the following filter text to select load balancer instances:

[tags]
type = qa-lbs

• Use the following filter text to select the API servers:

[tags]
type = qa-apiservers

• Click on Add a dependency
• Choose qa-postgres from the list of dependencies
• Add another dependency
• Choose qa-es from the list of dependencies
• Enter qa.<yourdomain> to the custom domain field
• Save the gateway

This will setup a load balanced gateway which has dependencies on Postgres and ES. When lightwing starts instances, it respects the dependency order and dependencies will be started first.

Finishing up

You can add a CNAME entry to the lightwing generated URL so that there is no change in the URL. Adding CNAME entry is different for each service provider. Refer to the documentation provided by the service provider.

Read more about setting up custom domain.

Summary

With the above setup, when someone visits qa.example.com, lightwing intercepts the traffic and start the servers if they are not running. If they are running, lightwing will just proxy the traffic. When lightwing detects idle time, it will shut down the servers.

Let us look at the cost saving:

Typically, a QA server which is used only during working days will be idle for about 70% of the time. Lightwing will provide granular data about idle time and savings in the analytics page.

In this example, considering 70% idle time, our initial cost of $357 will now become $108! This is higher than what reserved instances gives without worrying about paying upfront or committing to a period.