Announcing ec2-extra-tools
by Carl Mercier
Hola! I just started a new project on GitHub, I’m calling it ec2-extra-tools. It’s basically a set of extra command line tools that I find useful on Amazon EC2. Right now, it only has 2, but I will add more in the future and encourage other people to collaborate on the project.
Here’s what’s included so far:
- ec2-identify-instance: returns the current EC2 instance id
- ec2-identify-volume: returns the EBS volume id of a device
I needed ec2-identify-volume to automate EBS snapshots, so I thought I’d release my work on GitHub.
Here’s what I added in /etc/crontab to automate EBS snapshots:
0 2 * * * root ec2-create-snapshot `ec2-identify-volume /dev/sdj` >> /var/log/ebs_snapshots
The great thing about this line is that I don’t need to know the volume id beforehands. If a new volume is attached at /dev/sdj, I don’t have to change my crontab file either. In my situation, this is very convenient.
Hope you find ec2-extra-tools useful!
The tools sounds great.
But when I look at your snapshot code I get a bit worried. A snapshot taken at a random point in time of a mounted EBS volume can be very corrupted. The volume may or may not re-mount without errors, the data in files may or may not have been written.
I would (at the least) start with a “sync” of the volume as a small start to making the volume consistent. But for best results the volume needs to be “frozen” for a moment.
If the content is all read only and changes are only made to whole files, for example a collection of .php files, this may just work. But for random access files you will see partial updates and broken links inside data files.
Caution, you have a great idea here just adding a backup to the cron. But I would add a few more steps, to get a good snapshot that would remount the next time you need it for a volume.
Edward M. Goldberg
e.m.g.
Ed,
Indeed. I wouldn’t do this on something like a database storage device, but for my purpose it was enough. On our MySQL master, we use XFS, which is much better than ext3 for this purpose.
Ed, might want to look at ec2-xfs-snapshot in my ec2-extra-tools: http://github.com/cmer/ec2-extra-tools
It does the freezing and plenty of nifty tricks to automate snapshots of XFS volumes. Works great in production too
Hi, Carl
> I wouldn’t do this on something like a database storage device,
> but for my purpose it was enough
Then, you should say so in the script.
People, use your script, are all newbies so they cannot tell it’s safe or not.
>On our MySQL master, we use XFS, which is much better than ext3 for this purpose.
Ouch !! yes, u can take a snapshot of the volume, but from MySQL’s perspective it could be inconsistent.
> It does the freezing and plenty of nifty tricks to automate snapshots of XFS volumes
I took a look at ur code but it doesn’t seem to do “plenty of nifty tricks”.
> Works great in production too
You’re just lucky.
I’m so sure you will get your data corrupted someday.
jack: we’ve been using these scripts in production for 6 months on our MySQL servers (which are very high traffic) and we haven’t had corrupted data. It’s just a matter of knowing how to use them, which, of course, I can’t take the responsibility of.
Hi Carl, great job!
For European instances you have to add –region eu-west-1 (or –region eu-east-1) on line 12 of ec2-identify-volume otherwise it will return nil.
Maybe you should add a parameter on your ruby script…
Carl,
I would love to add a few of my “tricks of the trade” to your project.
1) Next free device for attaching a new EBS Volume.
2) Format (only if needed) then mount a Volume.
3) Patch the “Whole File Tree” from a mounted Volume, with a “sparse” tree.
I have been working on these programs for a while. The mount script is a very careful mounter for Volumes. Never damages volumes that have data. Formats as needed.
We should talk more….
Will you be at the MySQL show next week?
Call me: Skype: EdwardMGoldberg
Edward M. Goldberg
http://myCloudWatcher.com/
Ed,
You are more than welcome to fork the project on GitHub and make the changes you think are valuable. I’ll be happy to merge back!
Won’t be at MySQL Conf this year. Cheers!
Hi, Carl > I wouldn’t do this on something like a database storage device, > but for my purpose it was enough Then, you should say so in the script. People, use your script, are all newbies so they cannot tell it’s safe or not. >On our MySQL master, we use XFS, which is much better than ext3 for this purpose. Ouch !! yes, u can take a snapshot of the volume, but from MySQL’s perspective it could be inconsistent. > It does the freezing and plenty of nifty tricks to automate snapshots of XFS volumes I took a look at ur code but it doesn’t seem to do “plenty of nifty tricks”. > Works great in production too You’re just lucky. I’m so sure you will get your data corrupted someday.
We’ve been using ec2-xfs-snapshot in production on our MySQL master (Defensio) and we never had a problem.