Go in Production

Showing 1-30 of 30 messages
Go in Production hcatlin 8/30/12 6:08 PM
I've posted before, but thought I should do a bit of a follow-up.

100% of our production system is now running Go as 95% of our
application (C being the other bits). Not side bits, but our critical
path.

We're running around 800 million requests a month
through our various public-facing sites run on more than
500 servers on AWS.

Sites we power that are in pure-Go..
...and a lot more.

We are running this on our production servers, but we
also have a local development env that our customers run
and that is compiled in all three major operating systems.
We have a binary executable available for Linux, Mac OSX,
and *gasp* Windows!

Since 1.0, Go has been ridiculously stable for us and
the decision to switch has been one of the better decisions
we've made as a business.

The power and speed of a compiled language, with the 
flexibility and agility that we've come to expect in a 
modern language.

-Hampton Catlin.
VP of Technology, Moovweb

PS: Also included a photo of our team with various Gopher
talisman's we keep around.

PPS: If you are in SF and want to chat with us, send me an email
and we'll chat! 
Re: [go-nuts] Go in Production Andrew Gerrand 8/30/12 6:41 PM
Great news! Thanks for sharing.
Re: Go in Production ISockGo 8/30/12 7:15 PM
Great work brother, Keep rocking Goians :). I have checked the site, looks really good and fast. What you have used as database.
Re: Go in Production Brad Clawsie 8/30/12 8:48 PM
congrats. i have also been using Go heavily in an industrial context with great results.

well thought-out typing, concurrency, and great unicode support out of the box. great libraries for building network apps and wrangling json. etc etc

i haven't had this much productive fun with a tool since i first picked up perl in the mid 90s.

glad to hear it has been a success for you too
Re: [go-nuts] Go in Production Uriel DeLarge 8/30/12 9:10 PM
On Fri, Aug 31, 2012 at 3:08 AM, hampton catlin <hca...@gmail.com> wrote:
> I've posted before, but thought I should do a bit of a follow-up.
>
> 100% of our production system is now running Go as 95% of our
> application (C being the other bits). Not side bits, but our critical
> path.
>
> We're running around 800 million requests a month
> through our various public-facing sites run on more than
> 500 servers on AWS.

Awesome!

And you just made it to the Hacker News front page:

http://news.ycombinator.com/item?id=4457854
Re: [go-nuts] Re: Go in Production notedit 8/30/12 10:15 PM


2012/8/31 <tjholo...@gmail.com>
hot! I love a lot of things about Go, I'm not sure I can overcome some of the
conventions, sounds silly since most of that is just aesthetics but uppercase-everything
seems really funky, realllllly not a fan of the python-style imports either that's a major
downside IMO


hope there is go web framework like express. 
 

On Thursday, 30 August 2012 18:08:43 UTC-7, hcatlin wrote:
I've posted before, but thought I should do a bit of a follow-up.

100% of our production system is now running Go as 95% of our
application (C being the other bits). Not side bits, but our critical
path.

We're running around 800 million requests a month
through our various public-facing sites run on more than
500 servers on AWS.

Sites we power that are in pure-Go..
...and a lot more.

We are running this on our production servers, but we
also have a local development env that our customers run
and that is compiled in all three major operating systems.
We have a binary executable available for Linux, Mac OSX,
and *gasp* Windows!

Since 1.0, Go has been ridiculously stable for us and
the decision to switch has been one of the better decisions
we've made as a business.

The power and speed of a compiled language, with the 
flexibility and agility that we've come to expect in a 
modern language.

-Hampton Catlin.
VP of Technology, Moovweb

PS: Also included a photo of our team with various Gopher
talisman's we keep around.

PPS: If you are in SF and want to chat with us, send me an email
and we'll chat! 

Re: [go-nuts] Re: Go in Production R. Rajesh Jeba Anbiah 8/30/12 10:39 PM
On Friday, 31 August 2012 10:45:32 UTC+5:30, notedit wrote:
hope there is go web framework like express. 


 
Re: Go in Production R. Rajesh Jeba Anbiah 8/30/12 10:41 PM
On Friday, 31 August 2012 06:38:43 UTC+5:30, hcatlin wrote:
   <snip> 
Sites we power that are in pure-Go..
...and a lot more.
  <snip>

    Can you share some insights about the technology/framework?  HTTP header signatures don't give any clue.
Re: [go-nuts] Re: Go in Production Dave Cheney 8/30/12 10:42 PM
http://gorilla-web.appspot.com/

It is not a framework, but a toolkit. That is a good thing.

If you are looking for an application server, Google App Engine offers
everything that opens and shuts.
Re: [go-nuts] Re: Go in Production Patrick Mylund Nielsen 8/30/12 11:05 PM
Go doesn't have those problems. In that way it is nothing like C. C
was about speed above all else; Go is about safety..and speed.
Statically typed/compiled languages are, or at least can be, much
safer than dynamic ones.

That's not to say there aren't security vulnerabilities -- buffer
overflows, non-nil-terminated strings, etc. just aren't what you need
to worry about.

I'm honestly a lot more worried about using monstrous Python/Ruby
frameworks than Go and its standard library.

On Fri, Aug 31, 2012 at 7:38 AM,  <adi...@gmail.com> wrote:
> Congrats - but how proven is the security aspect? Given how easy it has been
> in the past to cause overflow/underflow attacks on C style compile
> languages, are you not worried?
>
>
> On Thursday, August 30, 2012 9:08:43 PM UTC-4, hcatlin wrote:
>>
>> I've posted before, but thought I should do a bit of a follow-up.
>>
>> 100% of our production system is now running Go as 95% of our
>> application (C being the other bits). Not side bits, but our critical
>> path.
>>
>> We're running around 800 million requests a month
>> through our various public-facing sites run on more than
>> 500 servers on AWS.
>>
>> Sites we power that are in pure-Go..
>>   http://m.macys.com
>>   http://m.1800flowers.com
>> ...and a lot more.
>>
Re: [go-nuts] Re: Go in Production vision media [ Tj Holowaychuk ] 8/30/12 11:05 PM
gah yeah those are the kinds of conventions I dont think I can deal with.

io.WriteString(w, "hello, "+req.URL.Query().Get(":name")+"!\n")
could have been so much easier to read:

io.write(w, "hello, " + req.url.query().get(":name") + "!\n")
at worst, capitals all over creates a lot of noise, but I've heard how explicit
the Go community is about conventions so I haven't felt tempted to stray from that haha.. plus
the massive stdlib kinda forces you to use these same conventions regardless of
how you feel about them. import() is absolutely terrible though IMHO, I'm not sure if there
would have been technical limitations with something more explicit but the whole "import everything"
concept is really bad
Re: [go-nuts] Re: Go in Production Patrick Mylund Nielsen 8/30/12 11:07 PM
Not to be rude, but do you know what the capitalization means? It's a
clean way to distinguish what's exported from what isn't. I don't know
of anything's that's less noisy.
Re: [go-nuts] Re: Go in Production Patrick Mylund Nielsen 8/30/12 11:09 PM
And it's not "exported" or "unexported" in a Python sense where
appending an underscore conveys you don't want people to use a given
function. Types and functions that begin with a lower-case letter
cannot be used by other packages.
Re: [go-nuts] Re: Go in Production Patrick Mylund Nielsen 8/30/12 11:15 PM
> I'm not sure if there would have been technical limitations with something more explicit but the whole "import everything" concept is really bad

What do you mean? Imported packages are qualified in Go, e.g. import
foo; foo.Foo(); foo.Bar(). It doesn't matter if you can't specify that
you're only using a class or 2-3 functions from a package--only what's
actually needed to run your program will be included anyway.

In any case, these things are not conventions. They actually mean
something in Go.

On Fri, Aug 31, 2012 at 8:05 AM, vision media [ Tj Holowaychuk ]
<t...@vision-media.ca> wrote:
Re: [go-nuts] Re: Go in Production vision media [ Tj Holowaychuk ] 8/30/12 11:35 PM
I'm just looking at it from an outsiders perspective since I haven't used it for much. That makes 
more sense now, though it's still a little odd (to me at least), vs having some "export" keyword or
similar which conveys meaning and is explicit, binding meaning to the casing of an identifier is I guess
only weird coming from other languages where that's convention only.

Do imported packages just define whatever they like? Or must all values hang off
of one identifier? Compared to something explicit like "foo = require('bar')" it's not
immediately obvious, though it seems like Go is using the last path segment? :s
I'll just shut up and go read docs haha, sorry for the noise
Re: [go-nuts] Re: Go in Production Patrick Mylund Nielsen 8/30/12 11:43 PM
If you just import a package, it's qualified by its package name (from
the source files), e.g. "foo". You can import a package with a custom
qualifier like so:

import bar "github.com/baz/foo"

bar.Foo()
bar.Bar()
...

I found that the upper-case exporting and "backwards" variable
declaration seemed odd as well, but only until I had used the language
for a while (a couple of hours.) A lot of these "odd" choices will
"click"/make sense quickly. They're just odd because you're
conditioned to things that will actually seem more tedious in
retrospect, IMO.

On Fri, Aug 31, 2012 at 8:35 AM, vision media [ Tj Holowaychuk ]
Re: [go-nuts] Re: Go in Production brainman 8/30/12 11:45 PM
On Friday, 31 August 2012 16:35:08 UTC+10, vision media [ Tj Holowaychuk ] wrote:
... it's still a little odd (to me at least), vs having some "export" keyword or
similar which conveys meaning and is explicit, ...

Sure thing. But "export" keyword could be pages away (or even in a different source file) from where you are looking now. Capital letter is right there in your face.

Alex
Re: [go-nuts] Re: Go in Production Patrick Mylund Nielsen 8/30/12 11:48 PM
I write a lot of Haskell as well. I very rarely add a new type and
func and remember to add it to the exports list at the top before
trying to compile. I can't remember actually missing it in Go, because
if I'm calling a lower-cased function from another package, I know
I've missed something.
Re: [go-nuts] Re: Go in Production vision media [ Tj Holowaychuk ] 8/30/12 11:54 PM
yeah I can't say I'd like listing them at the top like erlang etc either, though
I really don't mind exports.foo = bar in node-land, that's good that you can
rename the import though I was worried it was more like Ruby. Doesn't hurt
to try new things! I'll have to dig into it some more
Re: [go-nuts] Re: Go in Production Rémy Oudompheng 8/31/12 12:09 AM
On 2012/8/31 vision media [ Tj Holowaychuk ] <t...@vision-media.ca> wrote:
> gah yeah those are the kinds of conventions I dont think I can deal with.
>
> io.WriteString(w, "hello, "+req.URL.Query().Get(":name")+"!\n")
>
> could have been so much easier to read:
>
> io.write(w, "hello, " + req.url.query().get(":name") + "!\n")

Both lines of code are totally unreadable to me.

Rémy.
Re: [go-nuts] Re: Go in Production vision media [ Tj Holowaychuk ] 8/31/12 12:16 AM
punctuation doesn't bother me at all I prefer that to something like coffeescript, but
capitalization will definitely take some getting used to
Re: [go-nuts] Re: Go in Production kortschak 8/31/12 12:35 AM
When I read that comment, I though he was making a joke...

Re: [go-nuts] Re: Go in Production David Leimbach 8/31/12 12:41 AM
Iam going to try to port Go to our BSD like stuff at work based on Devon's FreeBSD stuff one of these dang days and see what kind of stuff we can do here.  There's mostly C, C++ and python today, but there is some trickiness with Python and all the dependencies (mostly libraries and memory consumption).  Just a skunkworks idea for now but I think we've got some good opportunities here to leverage Go.  Glad to see it's going well for others!
Re: Go in Production hcatlin 8/31/12 9:49 AM
I'll just hop in to comment on the # of servers thing... I can't give specific breakdowns, but basically,
not all of those are request churning machines, many are management and test and routing machines,
and we massively over provision most of our customers to help ensure reliability and redundancy. 

We definitely are not attempting to maximize requests-to-machines at this point. However, request speed
and latency and redundancy are our primary focus.

And, the non-cached requests are relatively processor heavy...

And, I will make one correction for myself... We are running rails as our deploy and user management server, etc.
I don't count that in our "critical path" flow, aka customer uptime isn't measured by our user front-end, but by
the stability of their hosted application.

And the C is executed via CGO.

-hampton.

On Thursday, August 30, 2012 8:20:21 PM UTC-7, dcr...@gmail.com wrote:
I'd be curious to here more about the server breakdown and any kind of numbers you can give regarding normal load/request cycles.

For example, last time we did some math, we were doing about 250m requests to Python a day, and we had less than 200 machines (serving the entirety of our applications, most of which wasnt the actual Python web machines). That said, we now have about 300 machines and do a little over 5 billion "monthly requests" which I'd constitute as pageviews, though I'm not sure how many requests hit Python day to day off hand.
Re: [go-nuts] Re: Go in Production Kyle Lemons 8/31/12 10:37 AM
On Fri, Aug 31, 2012 at 12:09 AM, Rémy Oudompheng <remyoud...@gmail.com> wrote:
Were this to come before me in a code review, I'm pretty sure I'd suggest
fmt.Fprintf(w, "Hello, %s!", req.FormValue("name")) 

Re: [go-nuts] Re: Go in Production Paul 8/31/12 11:50 AM
Thank you very much for that very interesting success story about using Go on an industrial scale. +1

On Friday, August 31, 2012 7:37:47 PM UTC+2, Kyle Lemons wrote:
On Fri, Aug 31, 2012 at 12:09 AM, Rémy Oudompheng <remyoud...@gmail.com> wrote:
On 2012/8/31 vision media [ Tj Holowaychuk ] <t...@vision-media.ca> wrote:
> gah yeah those are the kinds of conventions I dont think I can deal with.
>
> io.WriteString(w, "hello, "+req.URL.Query().Get(":name")+"!\n")
>
> could have been so much easier to read:
>
> io.write(w, "hello, " + req.url.query().get(":name") + "!\n")

Both lines of code are totally unreadable to me.

Were this to come before me in a code review, I'm pretty sure I'd suggest
fmt.Fprintf(w, "Hello, %s!", req.FormValue("name")) 

Re: Go in Production Shuai Lin 8/31/12 6:29 PM

在 2012年8月31日星期五UTC+8上午9时08分43秒，hcatlin写道：
The power and speed of a compiled language, with the 
flexibility and agility that we've come to expect in a 
modern language.

Yeah, go build/go test is so fast.
Re: Go in Production mortdeus 9/4/12 6:32 AM
I am a gopher. 

On Thursday, August 30, 2012 9:15:37 PM UTC-5, ISockGo wrote:
Great work brother, Keep rocking Goians :). I have checked the site, looks really good and fast. What you have used as database.

On Thursday, August 30, 2012 9:08:43 PM UTC-4, hcatlin wrote:
I've posted before, but thought I should do a bit of a follow-up.

100% of our production system is now running Go as 95% of our
application (C being the other bits). Not side bits, but our critical
path.

We're running around 800 million requests a month
through our various public-facing sites run on more than
500 servers on AWS.

Sites we power that are in pure-Go..
...and a lot more.

We are running this on our production servers, but we
also have a local development env that our customers run
and that is compiled in all three major operating systems.
We have a binary executable available for Linux, Mac OSX,
and *gasp* Windows!

Since 1.0, Go has been ridiculously stable for us and
the decision to switch has been one of the better decisions
we've made as a business.

The power and speed of a compiled language, with the 
flexibility and agility that we've come to expect in a 
modern language.

-Hampton Catlin.
VP of Technology, Moovweb

PS: Also included a photo of our team with various Gopher
talisman's we keep around.

PPS: If you are in SF and want to chat with us, send me an email
and we'll chat! 
Re: Go in Production ISockGo 9/7/12 2:16 PM

Cool, Is there a way to show this to my boss that this is completely in GO.

I checked the home page source and have no idea why it's showing this JAVA thing..

<!--
Server Date: 09/06/2012 03:42:34 EDT
Content Version: 2.14.8 Revision: 227974
Content Build Date: Thu Aug 30 17:12:24 EDT 2012
Application Version: 12G.32.1 from 2.14.8 
Application Build Date: Thu Aug 30 17:10:31 EDT 2012 
SVN Revision: 227974 
Pagecode: xx-xx-xx-xx.index
Secure: false
Request URI: /index.jsp
Server Physical Name: fsgwws182Node_B
Server Clone Name: macys_B_c01_s182_m06
JSP Page Encoding: ISO-8859-1
-->

Re: [go-nuts] Re: Go in Production hcatlin 9/8/12 8:25 AM
We're a on-demand transformation proxy. That client must have a desktop
site that is powered by Java.

Check the headers for moovweb related headers... but we try to leave
our fingerprints off things.

-hampton.
More topics »