Proof of concept image
18 views
Skip to first unread message
doughyjoey5
Sep 8, 2008, 6:28:03 PM9/8/08
to 3rd&Urban Development Discussion
It would be SO awesome if you made a public image that ran this system
as a sort of proof of concept. I would love to play around with it
before I try getting it to work on my systems.
Great documentation!
as a sort of proof of concept. I would love to play around with it
before I try getting it to work on my systems.
Great documentation!
M. David Peterson
Sep 8, 2008, 6:38:22 PM9/8/08
to 3rdurban-develo...@googlegroups.com
Hello,
On Mon, Sep 8, 2008 at 4:28 PM, doughyjoey5 <jose...@gmail.com> wrote:
It would be SO awesome if you made a public image that ran this system
as a sort of proof of concept.
What's funny is that I actually created one but it never occurred to me to make this fact know. Yikes! :D
I actually have my own custom Linux distro that is the focus of my personal development (http://nuxleus.com), so I will need to dig around and find the original Fedora Core 8 AMI I created and/or create a new one. That said, is Fedora the preferred distro or is there another that you all would prefer to see as the focus.
As a matter of interest, I have the code finished up that adds EBS into the mix, and just need to update the white paper appropriately. If you can give me a feel for what distro you'd prefer to have as the base AMI I'll try to spend some time this week getting the EBS capabilities baked in and a baseline AMI to extend from mixed down and made available.
Great documentation!
Thanks!
/M:D
M. David Peterson
Co-Founder & Chief Architect, 3rd&Urban, LLC
Email: m.d...@3rdandUrban.com | m.d...@amp.fm
Mobile: (206) 999-0588
http://3rdandUrban.com | http://amp.fm | http://www.oreillynet.com/pub/au/2354 | http://news.oreilly.com/m-david-peterson/
doughyjoey5
Sep 9, 2008, 12:28:16 PM9/9/08
to 3rd&Urban Development Discussion
Wow. You are too kind. I admire your commitment to open development!
I have been using Ubuntu 8.04 Server for my development as requested
by my developer. However, if you are familiar with Fedora use that. I
can easily digest what you do there and translate it to Ubuntu.
Adding EBS support to your system would be very nice. When I was
reading about having a seperate drive for volume snapshots I began to
see how cool it would be if you integrated that in.
I've been developing a mysql cluster through replication on ec2. what
do you think about running mysql on the persistent filesystem you have
developed?
Thanks again!
Joe
> Mobile: (206) 999-0588http://3rdandUrban.com|http://amp.fm|http://www.oreillynet.com/pub/au/2354|http://news.oreilly.com/m-david-peterson/
I have been using Ubuntu 8.04 Server for my development as requested
by my developer. However, if you are familiar with Fedora use that. I
can easily digest what you do there and translate it to Ubuntu.
Adding EBS support to your system would be very nice. When I was
reading about having a seperate drive for volume snapshots I began to
see how cool it would be if you integrated that in.
I've been developing a mysql cluster through replication on ec2. what
do you think about running mysql on the persistent filesystem you have
developed?
Thanks again!
Joe
> On Mon, Sep 8, 2008 at 4:28 PM, doughyjoey5 <joseph...@gmail.com> wrote:
>
> > It would be SO awesome if you made a public image that ran this system
> > as a sort of proof of concept.
>
> What's funny is that I actually created one but it never occurred to me to
> make this fact know. Yikes! :D
>
> I actually have my own custom Linux distro that is the focus of my personal
> development (http://nuxleus.com), so I will need to dig around and find the
> original Fedora Core 8 AMI I created and/or create a new one. That said, is
> Fedora the preferred distro or is there another that you all would prefer to
> see as the focus.
>
> As a matter of interest, I have the code finished up that adds EBS into the
> mix, and just need to update the white paper appropriately. If you can give
> me a feel for what distro you'd prefer to have as the base AMI I'll try to
> spend some time this week getting the EBS capabilities baked in and a
> baseline AMI to extend from mixed down and made available.
>
> > Great documentation!
>
> Thanks!
>
> --
> /M:D
>
> M. David Peterson
> Co-Founder & Chief Architect, 3rd&Urban, LLC
> Email: m.da...@3rdandUrban.com | m.da...@amp.fm
>
> > It would be SO awesome if you made a public image that ran this system
> > as a sort of proof of concept.
>
> What's funny is that I actually created one but it never occurred to me to
> make this fact know. Yikes! :D
>
> I actually have my own custom Linux distro that is the focus of my personal
> development (http://nuxleus.com), so I will need to dig around and find the
> original Fedora Core 8 AMI I created and/or create a new one. That said, is
> Fedora the preferred distro or is there another that you all would prefer to
> see as the focus.
>
> As a matter of interest, I have the code finished up that adds EBS into the
> mix, and just need to update the white paper appropriately. If you can give
> me a feel for what distro you'd prefer to have as the base AMI I'll try to
> spend some time this week getting the EBS capabilities baked in and a
> baseline AMI to extend from mixed down and made available.
>
> > Great documentation!
>
> Thanks!
>
> --
> /M:D
>
> M. David Peterson
> Co-Founder & Chief Architect, 3rd&Urban, LLC
> Mobile: (206) 999-0588http://3rdandUrban.com|http://amp.fm|http://www.oreillynet.com/pub/au/2354|http://news.oreilly.com/m-david-peterson/
M. David Peterson
Sep 9, 2008, 1:17:55 PM9/9/08
to 3rdurban-develo...@googlegroups.com
Not at all. It's a reciprocating thing. 99.100% of what I know comes from somebody else providing a resource for me to learn from.
I admire your commitment to open development!
Thanks! :D
I have been using Ubuntu 8.04 Server for my development as requested
by my developer. However, if you are familiar with Fedora use that. I
can easily digest what you do there and translate it to Ubuntu.
Oh, I'm familiar with most distros. It's simple enough to adapt, and in fact I have an Ubuntu image that I started over the weekend for another purpose which I can extend from, so Ubuntu it is :)
Adding EBS support to your system would be very nice. When I was
reading about having a seperate drive for volume snapshots I began to
see how cool it would be if you integrated that in.
Oh, I agree. It's *VERY* cool! /So/ many possibilities!
I've been developing a mysql cluster through replication on ec2. what
do you think about running mysql on the persistent filesystem you have
developed?
It works quite well, though with EBS you gain the persistence by default. So adapting the script to use EBS instead of DRBD + the ephemeral devices from two instances seems to be a better overall solution. Of course, there's an additional financial cost with EBS that you don't have with local ephemeral devices, but that can easily be offset by the fact that you only need one instance running with EBS whereas with DRBD + ephemeral devices you are required to have two.
That said, to ensure near instant failover from one node to another you need to have two instances running at all times anyway, so it really comes down to a simple matter of requirements: Do you need instant failover if a node dies or can you get away with the lead time required to start a new instance and remount the EBS drive on that instance?
Thanks again!
No problem! Will update this thread again once I have the base AMI ready.
--
/M:D
M. David Peterson
Co-Founder & Chief Architect, 3rd&Urban, LLC
/M:D
M. David Peterson
Co-Founder & Chief Architect, 3rd&Urban, LLC
doughyjoey5
Sep 11, 2008, 2:32:32 PM9/11/08
to 3rd&Urban Development Discussion
I appreciate you developing on Ubuntu. That will help me quite a bit.
>That said, to ensure near instant failover from one node to another you need
>to have two instances running at all times anyway, so it really comes down
other so if it goes down I can just point a DNS entry to the backup.
Therefore the storage needs to be constantly in sync with each other
and be able to recover when the downed instance is brought back up.
One thing I've wondered is how you were able to use heartbeat in EC2.
I was under the impression that wouldn't work because the cloud
doesn't have static ips....or is that why you are using vtun?
Joe
On Sep 9, 12:17 pm, "M. David Peterson" <xmlhac...@gmail.com> wrote:
> Email: m.da...@3rdandUrban.com | m.da...@amp.fm
> Mobile: (206) 999-0588http://3rdandUrban.com|http://amp.fm|http://www.oreillynet.com/pub/au/2354|http://news.oreilly.com/m-david-peterson/
>That said, to ensure near instant failover from one node to another you need
>to have two instances running at all times anyway, so it really comes down
>o a simple matter of requirements: Do you need instant failover if a node
>dies or can you get away with the lead time required to start a new instance
>and remount the EBS drive on that instance?
What they want is a hot backup. An instance that is a duplicate of the
>dies or can you get away with the lead time required to start a new instance
>and remount the EBS drive on that instance?
other so if it goes down I can just point a DNS entry to the backup.
Therefore the storage needs to be constantly in sync with each other
and be able to recover when the downed instance is brought back up.
One thing I've wondered is how you were able to use heartbeat in EC2.
I was under the impression that wouldn't work because the cloud
doesn't have static ips....or is that why you are using vtun?
Joe
On Sep 9, 12:17 pm, "M. David Peterson" <xmlhac...@gmail.com> wrote:
> Mobile: (206) 999-0588http://3rdandUrban.com|http://amp.fm|http://www.oreillynet.com/pub/au/2354|http://news.oreilly.com/m-david-peterson/
M. David Peterson
Sep 12, 2008, 2:29:14 AM9/12/08
to 3rdurban-develo...@googlegroups.com
On Thu, Sep 11, 2008 at 12:32 PM, doughyjoey5 <jose...@gmail.com> wrote:
I appreciate you developing on Ubuntu. That will help me quite a bit.
No problem :)
What they want is a hot backup. An instance that is a duplicate of the
other so if it goes down I can just point a DNS entry to the backup.
Why not just use an Elastic IP and remap that IP to the backup node if/when the primary node fails? I realize an Elastic IP can take a bit to remap to the backup instance, but the alternative would be to use a TTL of less than five minutes in your DNS settings which -- dependent on your use case -- could bring down performance fairly dramatically if each client is forced to refresh the DNS cache every five minutes or less. Or is this less of a big deal in your particular case?
Therefore the storage needs to be constantly in sync with each other
and be able to recover when the downed instance is brought back up.
Well you would get that with both DRBD and EBS, EBS being /much/ more reliable and quite a bit faster given its design to be more durable than the local ephemeral drives.
One thing I've wondered is how you were able to use heartbeat in EC2.
I was under the impression that wouldn't work because the cloud
doesn't have static ips....or is that why you are using vtun?
That's what VTun is for. With VTun added to the mix, Heartbeat works like a dream :)
--
/M:D
M. David Peterson
Co-Founder & Chief Architect, 3rd&Urban, LLC
/M:D
M. David Peterson
Co-Founder & Chief Architect, 3rd&Urban, LLC
doughyjoey5
Sep 12, 2008, 12:27:03 PM9/12/08
to 3rd&Urban Development Discussion
> Why not just use an Elastic IP and remap that IP to the backup node if/when
> the primary node fails? I realize an Elastic IP can take a bit to remap to
> the backup instance, but the alternative would be to use a TTL of less than
> five minutes in your DNS settings which -- dependent on your use case --
> could bring down performance fairly dramatically if each client is forced to
> refresh the DNS cache every five minutes or less. Or is this less of a big
> deal in your particular case?
The fail over is only for the database not the whole system so I have
> the primary node fails? I realize an Elastic IP can take a bit to remap to
> the backup instance, but the alternative would be to use a TTL of less than
> five minutes in your DNS settings which -- dependent on your use case --
> could bring down performance fairly dramatically if each client is forced to
> refresh the DNS cache every five minutes or less. Or is this less of a big
> deal in your particular case?
my own bind server to update when they change. I suppose we may end up
changing it later and just map an elastic ip. That would be great.
> That's what VTun is for. With VTun added to the mix, Heartbeat works like a
> dream :)
Sweet. I'm having trouble understanding exactly how that works. Do you
think you could humor me and give me a brief overview of how that
works? Heartbbeat+VTun+DRBD sounds like a great solution!
On Sep 12, 1:29 am, "M. David Peterson" <xmlhac...@gmail.com> wrote:
> Mobile: (206) 999-0588http://3rdandUrban.com|http://amp.fm|http://www.oreillynet.com/pub/au/2354|http://news.oreilly.com/m-david-peterson/
Reply all
Reply to author
Forward
0 new messages