show and tell?

[Matthew Lyon]

Hi group, long-time lurker first-time poster.

As part of my work at AppFog, I've built out a graphing system that I think is pretty cool. It provides an aggregated view of the characteristics of certain types of events, and lets you drill down by facets of the events.

For example, this is from a report of our API, showing incoming HTTP requests and below that, the "dispatched" requests to the datacenters we provide hosting in around the world (I'm on an 11" MacBook, can't make larger screenshots):

http://dl.dropbox.com/u/330142/Screenshots/u1co3dr_vxgt.png

The top half of each graph shows three things: in blue, the "onset" or number of new events in a 15 second period, in grey, a histogram of the number "concurrent" events over the interval that are open but haven't completed, for any given set of unique facets (useful for finding resource leaks or unclosed connections), and in red the number of events that completed with an error of some kind.

The bottom half of the graph shows a histogram of the time taken to complete events over the interval given. These graphs use a 15-second-per-pixel interval.

Since I see that around 8:50 the dispatch was slow, I can drill down by "target" and see that the problem lies with the AWS Singapore region:

http://dl.dropbox.com/u/330142/Screenshots/0v5gk2lj82ro.png

I've thrown this type of measurement on everything I can think of: calls between different parts of our SOA, database / memcache, workers for various parts of our system, and have started using it for error reporting as well. It's proven invaluable in hunting down problems with our code, with our network operation and configuration, and has given some interesting insight into what happens during an AWS outage.

The data is gathered by a very simple nodejs app, stored in postgresql, and the graphs are rendered client-side using the canvas API from the results of a SQL query. Currently we're trimming the data to the past three hours, but are working to make it more archival.

I plan to open source this system soon, but would love to show it to an interested audience and get some feedback first. Is this something that would interest the group for a future meetup?

Thanks

Matthew Lyon

@mattly / 503-927-3357

Sr. Software Architect, AppFog

[Jesse Cooke]

I would love to hear more about this!

--------------------------------------------
Jesse Cooke :: N-tier Engineer
jc00ke.com / @jc00ke

--
You received this message because you are subscribed to the Google Groups "pdx-visualization" group.
To post to this group, send email to pdx-visu...@googlegroups.com.
To unsubscribe from this group, send email to pdx-visualizat...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/pdx-visualization?hl=en.

[Ryan Miller]

me too.  

I might also have something to show off, depending on the meeting date.

[Amber Case]

Awesome! If we can lock in one more speaker, it'd be great to have another meetup.

Best,

Amber 

---

Amber Case

caseo...@gmail.com 

http://caseorganic.com

(503) 729-8632

[Kim Rees]

Hi all,

 

Coincidentally, we are in the process of securing a meetup space. We’re hoping to get this on the docket for the end of the month.

 

Amber, were there other potential presenters who contacted you personally?

 

Kim

 

Periscopic :: Do good with data

www.periscopic.com

503-295-7946

@krees, @periscopic

[Amber Case]

Hi, Kim! None to my knowledge, although I just sent out a tweet. Feel free to organize the event/set the date, and I'll let you know if anyone else pops up!

Best,

Amber 

---

Amber Case

caseo...@gmail.com 

http://caseorganic.com

(503) 729-8632

[Ryan Miller]

Kim,  Will we see the latest from Periscopic?  It doesn't seem like a real PDX Data Viz meetup without some new work from you guys :-)

[Amber Case]

Agreed! If so, that's 3 presenters! 

Best, 

Amber 

--

Amber Case, Director

Esri R&D Center, Portland

920 SW 3rd Ave. #400

---

Em: ac...@esri.com

Ph: 1-971-275-6757

http://esri.com

--

[M. Edward (Ed) Borasky]

Did we get a data/venue yet? A couple of people asked me about this at
the #ONAPDX meeting tonight.

Twitter: http://twitter.com/znmeb; Computational Journalism Publishers
Workbench: http://znmeb.github.com/Computational-Journalism-Publishers-Workbench/

How the Hell can the lion sleep with all those people singing "A weem
oh way!" at the top of their lungs?

[Kim Rees]

We're are still settling on a venue. As for dates, what day the week of Feb 4th looks good for people?

Kim

[Mike Lonergan]

Monday or Tuesday would be great - Agile Open Northwest conference is Wed-Fri that week.

Mike

[Ryan Miller]

I concur on Monday - Tuesday.  PDX-UX is having Noah Pepper of Lucky Sort present on the 7th.  Lots of folks here should be interested in that.

[Amber Case]

Agreed.

Ryan, do you know if Thetus might be up for hosting? Geoloqi no longer has room as our team has grown to fill in a lot of the empty floor space, and we no longer have access to chairs.

Urban Airship might be another option. Barb Stark is the contact there.

Best,

Amber 

---

Amber Case

caseo...@gmail.com 

http://caseorganic.com

(503) 729-8632

[Christopher Rivard]

Jive is willing to host again - although the acoustics in the 5th floor room are not stellar and the training room we were in last time we hosted was a little small. Let me know.

What about Elemental?

[Amber Case]

Chris,

Last time I was at Jive it was 3 years ago. Is there still a main room that can hold about 70 people? Is there a way to get a PA system in there?

Has anyone been to an event at the new Elemental HQ? Sounds like a great place, and Bailey's is below, which is good for drinks after.

Best,

Amber 

---

Amber Case

caseo...@gmail.com 

http://caseorganic.com

(503) 729-8632

[Christopher Rivard]

I attended a Refresh Portland event (w/ Sheetal Dube) at Jive before I worked here (about 3 years ago). That was held on the 3rd floor - I think that's the space you recall. That space is now filled with desks. 

The large rooms we have available are a training room (we held the meeting in that room when we hosted) and the 5th floor cafeteria area. We did drinks/networking on 5 and then went down to the training room on 4. 

You're happy to stop by and take a look if you want. 

I'll ping Tom Quick at Elemental and ask if they are interested and have space.

[Jesse Cooke]

On Thu, Jan 17, 2013 at 10:40 AM, Amber Case <caseo...@gmail.com> wrote:

Chris,

Last time I was at Jive it was 3 years ago. Is there still a main room that can hold about 70 people? Is there a way to get a PA system in there?

Has anyone been to an event at the new Elemental HQ? Sounds like a great place, and Bailey's is below, which is good for drinks after.

I went to a Ruby meeting at Elemental last year. It was a bit small for the unexpectedly large number of attendees we had that night, but it worked out really well.

Plus I think they live-streamed it ;-)

[Ryan Miller]

The training room at Jive was pretty crowded and the cafeteria space didn't seem like it would be the right shape for a presentation.  Urban Airship has a great space (or did last time I was there about 5 months ago).

I'll be at Thetus tonight for PDX-UX and I'll check on that.  I don't work there anymore so I need to find someone who does willing to stick around for the whole meeting.

[Davy Stevenson]

Hey Everyone, Davy Stevenson from Elemental piping in. We would definitely be interested in hosting a meeting for you guys. Our largest conference room isn't the biggest around, but it can hold a decent amount of people (I think the count for the Ruby Brigade meeting was ~70 and that was doable) and we have a secondary monitor to help make sure everyone can see the presentation. Our space is also amplified, and we have a sweet recording system set up with capabilities to live-stream, or at least to record the presentations for archiving and later distribution.

[M. Edward (Ed) Borasky]

On Thu, Jan 17, 2013 at 11:08 AM, Ryan Miller <ninjas...@gmail.com> wrote:
> The training room at Jive was pretty crowded and the cafeteria space didn't
> seem like it would be the right shape for a presentation. Urban Airship has
> a great space (or did last time I was there about 5 months ago).

Yes - and the Data Science group meets there, so there are some
overlaps in people. I don't know who the visualization pros at Urban
Airship are, however.

[Amber Case]

Just asked Barb Stark at UA. Will let you know if we can get it! If not, Thetus would be a great backup if we can get it. 

Best,

Amber 

---

Amber Case

caseo...@gmail.com 

http://caseorganic.com

(503) 729-8632

[Kim Rees]

Let’s do it!  I like the idea of LiveStreaming. I know Amber did that for one of the events and it was very popular.

 

Does the evening of Feb 5^th work for you all at Elemental? Once we set a date, I’ll put out a Meetup notice.

 

Kim

To view this discussion on the web visit https://groups.google.com/d/msg/pdx-visualization/-/oBbzU2Obw8YJ.

[Amber Case]

Awesome! Sounds fantastic!

Best,

Amber 

---

Amber Case

caseo...@gmail.com 

http://caseorganic.com

(503) 729-8632

[Davy Stevenson]

Yes, Feb 5th will work for us. I'll unfortunately miss things due to being on vacation, but there are plenty of other people here who would be interested in being host for this event. One question for the group -- Elemental could provide some refreshments if that is a standard thing people would like. Beer would be easy to provide.

Davy

[Davy Stevenson]

Yes, Feb 5th definitely works for Elemental. Unfortunately I'll be gone on vacation on that date, but there will be no problem finding an interested host here. I could also plan to have Elemental provide refreshments (beer?) if that's something the group is interested in.

Davy

* Trying again with my other email -- hopefully I don't spam the group twice!

On Thursday, January 17, 2013 12:55:22 PM UTC-8, caseorganic wrote:

[Jesse Cooke]

Looks like Davy isn't able to join this group, or her posts are getting flagged or something.

Can someone with SUPER COW POWERS approve her if necessary?

[Matthew Lyon]

Ack, I can't do Tuesday nights. I'd be happy to do something on my event graphs another time, but Tuesday nights are out for me until March.

--
Matthew Lyon
503.927.3357

[Kim Rees]

How about the following week? Does Feb 13^th work?

 

Davy, what other dates work for you and Elemental?

[Davy Stevenson]

Feb 13th would work for us as well. And I'd be available that day too.

[Ryan Miller]

I would love some extra time to prep my stuff.  Matthew gave me a preview of his work last night at PDX-UX.  We don't want to miss it!  good stuff

[Davy Stevenson]

I'm so sorry guys, Feb 13th actually doesn't work for Elemental, we are hosting the PDX Cloud group that night. It's on Calagator but wasn't placed on our company calendar yet. We could handle Feb 11th or 12th (or pretty much any other night).

Davy

[Amber Case]

Urban Airship can host whenever. Let me know the date and Barb can put it on the calendar! 

---

Amber Case

Director, Esri R&D Center, Portland 920 SW 3rd Ave Suite 400 | Portland, OR 97204

---

e: ac...@esri.com    |   m: 1-503-729-8632  |   c: 1-971-275-6757

w: http://caseorganic.com/blog tw: @caseorganic

---

[M. Edward (Ed) Borasky]

Excellent!

[Matthew Lyon]

The 11th or the 13th work for me.

--
Matthew Lyon
503.927.3357