Even though AWS provides you with automation tools like CloudFormation, Beanstalk and OpsWorks to run your infrastructure there are still a lot of system maintenance tasks you must do on an ongoing basis: running backups, synchronizing data or up-/downgrading instances according to traffic patterns.
Today I want to show you five easy automation tips you can immediately put into practice that keep your infrastructure fresh and reduce your monthly hosting costs.
All tips can be implemented using AWS command line interface (CLI) and your own cron logic but for ease-of-use I highly recommend third-party service Skeddly. Skeddly allows you to define actions based on rules, all within a simple-to-use web interface. According to your schedule or parameters it automatically starts/stops instances, creates snapshots, deletes s3 objects, etc. The charming part is that there is no monthly subscription fee as you only pay for used actions.
1. Automate & Schedule Your Instance Backups Using Snapshots
AWS provides you with an easy way to create snapshots of EBS volumes. We are using this feature to manually create snapshots before scaling up/down. With Skeddly’s managed instances we also run an automated daily backup of important disks and have a rule for deleting backups older than two weeks. You could also use a homegrown solution with duplicity and S3 but with EBS snapshots it is easier to recover lost files as you can just attach the EBS to another instance. No worries about functioning backup jobs and no problems with provisioning storage. It just works.
2. Automatically Backup Your RDS Instance Or MySQL Database
Just like snapshots of EBS volumes you can create an action that takes a nightly snapshot of your RDS instance. You then not only have the binlogs to jump back in time but also a full snapshot of your current data profile.
At the same time you can create a more custom action that connects to any MySQL server, selects the specific databases you need then dumps and gzips them before pushing the package onto a S3 bucket. You can choose to separate and also to omit databases you might not need in the backup.
3. Reduce Costs By Switching To A Smaller Instance Type Or Put It To Sleep
My favorite 🙂 If you happen to run your development or staging servers in the AWS cloud and you want to save money there you go: create an action that automatically turns your EC2 instance off or switches to a less powerful thus less costly instance type. The advantage with keeping the instance ON is that you can still run your nightly cron and system administration jobs. For ex. we are running an m1.large instance during the day and changing it to m1.small from 11pm to 7am the next day. We reduced our costs with one simple action that doesn’t take more than 10mins of your time.
Savings potential
| Without change: | 1x m1.large running 720 hours (one full month) = $ 187,20 |
| With change: | 1x m1.large running 656 hours + 1x m1.small during 11pm and 7am, i.e. 64 hours + 8x Skeddly Change Instance Type = $ 170,56 + $ 4,16 + $ 1,20 = $ 175,92 |
Considering you have less traffic during evening hours you can save around $ 11 per month for a single server. Imagine the savings with more than one server or any other configuration you might have.
4. Delete S3 Objects Based On Age Or Other Metadata Criteria
If you are using a S3 bucket for temporarily stored user pictures that get resized by another script, you can use Skeddly to automate the deletion of objects based on a set of metadata or a timeframe, for ex. delete s3 objects older than 14 days. This is especially useful when you are dealing with temporary data and don’t want to build the logic into the application yourself.
5. Regularly Create AMI Images
System software (for ex. CentOS, Ubuntu Linux) is frequently updated with security patches, performance tweaks or other software libraries you need. In order to be able to spin up another machine with the most recent system configuration it is very useful to have a current AMI image at hand. Exactly this can be done by automating the daily or even weekly creation of an AMI image.
Keep It Secure By Using A Specific IAM User
You should follow best practices and give Skeddly only the permission it needs thus I highly recommend creating one specific account for your automation tasks. You can use Skeddly’s automatic policy generator to create an IAM policy document you then just need to copy and paste into your IAM.
Do you know more tips and tricks to automate AWS? Let me know in the comments!
PS: You should read Alestic by Eric Hammond, a great resource with tons of tutorials and help around Amazon AWS.
Thanks Nader, interesting. Will try this service out.
Cool! Let me know if you have any questions.