Skip to main content


Ubuntu Error Rebooting

Today we noticed a weird error where our CMS software was unable to write files to the server. Upon investigating, it appears the /tmp directory filled up the hard drive, which prevented file writes to the server.

We rebooted the server and it took 20-30 minutes to reboot. The AWS boot logs were saying:

"A start job is running for Create Volatile files and directories"

As it turns out, the /tmp directory is only cleared on reboot. Since there were so many temporary files on the disk, it took that long for the server to delete the files before coming back online.

GlusterFS Network Issues

Not sure what happened with GlusterFS tonight. Around 19:07 EST, the AWS alarm sounded. After rebooting the GlusterFS servers and the web servers, nothing was bringing the sites back online. There was a terrible lag when accessing the GlusterFS file systems and there weren't any glaring errors in the log files.

AWS - IAM Permissions - Setup permissions for one user to access only one S3 bucket

While using Amazon's AWS system, we needed to give access to one S3 bucket. Todo so, we ended up using Amazon's IAM. Once we created the user in the AWS system, we created the following policy to allow this new user to only be able to read/write to their bucket:

Restoring EBS Snapshots for Backup Restore

Today I had the unfortunate need to get a restored database from an Amazon EBS snapshot. To do this I logged into the Amazon AWS management console and determined which snapshot I needed to restore from. Once I highlighted the correct snapshot, I clicked the 'Create Volume' button. Next, I chose which availability zone the snapshot should reside in and click 'Create'

Next, I spun up a new instance of the production server, using the same AMI (ami-294aa340). Once the new instance was created, I attached the volume I created above to the instance and logged into the server.