Groups
Groups
Sign in
Groups
Groups
peregrine-mapreduce
Conversations
About
Send feedback
Help
peregrine-mapreduce
1–27 of 27
Mark all as read
Report group
0 selected
lucie lina
2/20/18
What is the best source for learning about MapReduce?
MapReduce course curricula deal with the concepts of implementation for processing and generating
unread,
What is the best source for learning about MapReduce?
MapReduce course curricula deal with the concepts of implementation for processing and generating
2/20/18
Info Cim
9/6/16
Hadoop training in bay area
Hadoop has moved far beyond its beginnings in web indexing and is now used in many businesses for a
unread,
Hadoop training in bay area
Hadoop has moved far beyond its beginnings in web indexing and is now used in many businesses for a
9/6/16
burtonator
7/18/12
Wikipedia rank works with the production wikipedia graph.
OK. Looks like the version of Peregrine in burton-structreader-release is executing for larger graphs
unread,
Wikipedia rank works with the production wikipedia graph.
OK. Looks like the version of Peregrine in burton-structreader-release is executing for larger graphs
7/18/12
burtonator
6/10/12
Fixed another class of memory leaks (and a video of my new kitten)
I fixed another class of memory leaks in Peregrine so we can now run longer jobs. There are still 1-2
unread,
Fixed another class of memory leaks (and a video of my new kitten)
I fixed another class of memory leaks in Peregrine so we can now run longer jobs. There are still 1-2
6/10/12
burtonator
5/29/12
Wikipedia rank working and another memory issue.
I have most of the work done for wikipedia rank. Really happy how this is working out because this is
unread,
Wikipedia rank working and another memory issue.
I have most of the work done for wikipedia rank. Really happy how this is working out because this is
5/29/12
burtonator
5/28/12
Distributing a demo with wikipedia graph data.
It turns out that Wikipedia distributes a full dump of their page graph. It's about 8GB
unread,
Distributing a demo with wikipedia graph data.
It turns out that Wikipedia distributes a full dump of their page graph. It's about 8GB
5/28/12
burtonator
5/28/12
Fixed a significant memory leak in reduce due to incorrect rlimit.
Larger jobs exhibited a crash bug due to artificially constrained rlimit on locked memory. This
unread,
Fixed a significant memory leak in reduce due to incorrect rlimit.
Larger jobs exhibited a crash bug due to artificially constrained rlimit on locked memory. This
5/28/12
burtonator
,
Roland Gude
4
5/28/12
Fixed a significant OutOfMemoryError on default.
I just fixed one due to the fact that we weren't setting the right memlock limit... I'll post
unread,
Fixed a significant OutOfMemoryError on default.
I just fixed one due to the fact that we weren't setting the right memlock limit... I'll post
5/28/12
Roland Gude
,
burtonator
12
5/4/12
is something like a combiner available
It may be easier for you to just clone and push some where so that we can land actual branches. I
unread,
is something like a combiner available
It may be easier for you to just clone and push some where so that we can land actual branches. I
5/4/12
Roland Gude
,
burtonator
9
5/3/12
trouble with current tip
Ah... ok. I'm seeing similar things on my end. I think one of the issues is that the integration
unread,
trouble with current tip
Ah... ok. I'm seeing similar things on my end. I think one of the issues is that the integration
5/3/12
burtonator
,
Roland Gude
2
4/23/12
Landed large commit/branch into default
this sounds great. i will port the combiner to the new head asap and see if it helps me with my
unread,
Landed large commit/branch into default
this sounds great. i will port the combiner to the new head asap and see if it helps me with my
4/23/12
Roland Gude
,
burtonator
2
4/17/12
poke
Yup... sorry. I'm working on tracking down a bug with SM right now which has caused me to totally
unread,
poke
Yup... sorry. I'm working on tracking down a bug with SM right now which has caused me to totally
4/17/12
burtonator
3/23/12
Peregrine presentation video from my talk at the Wikimedia Analytics Day
Here it is... http://www.youtube.com/watch?feature=player_detailpage&v=-_iCo1fFSeQ#t=14s I'm
unread,
Peregrine presentation video from my talk at the Wikimedia Analytics Day
Here it is... http://www.youtube.com/watch?feature=player_detailpage&v=-_iCo1fFSeQ#t=14s I'm
3/23/12
OG
,
burtonator
2
3/6/12
Java systat
On Tuesday, March 6, 2012 5:13:30 AM UTC-8, OG wrote: Hey Kevin, long time no speak... Hope all is
unread,
Java systat
On Tuesday, March 6, 2012 5:13:30 AM UTC-8, OG wrote: Hey Kevin, long time no speak... Hope all is
3/6/12
burtonator
,
Roland Gude
13
3/5/12
Just merged Cassandra support into default...
OK... I mostly have this implemented now... So if the key IS a hash you can map over it... however,
unread,
Just merged Cassandra support into default...
OK... I mostly have this implemented now... So if the key IS a hash you can map over it... however,
3/5/12
burtonator
3/5/12
Peregrine Presentation at Wikimedia
I gave a presentation on Peregrine at the Wikimedia Foundation on Friday of last week. Here is the
unread,
Peregrine Presentation at Wikimedia
I gave a presentation on Peregrine at the Wikimedia Foundation on Friday of last week. Here is the
3/5/12
burtonator
2/17/12
Finished Cassandra job output and now merged to default.
I also finished up the Cassandra job output... so you can read AND write to Cassandra now. I'm
unread,
Finished Cassandra job output and now merged to default.
I also finished up the Cassandra job output... so you can read AND write to Cassandra now. I'm
2/17/12
burtonator
2/10/12
Peregrine continuous integration
I set it up here: http://peregrine.integration.spinn3r.com/ It's just a basic python script that
unread,
Peregrine continuous integration
I set it up here: http://peregrine.integration.spinn3r.com/ It's just a basic python script that
2/10/12
Roland Gude
,
burtonator
2
2/8/12
integrate.py beautification
Ah... I will see what I can integrate back in... I didn't realize you were contributing patches.
unread,
integrate.py beautification
Ah... I will see what I can integrate back in... I didn't realize you were contributing patches.
2/8/12
burtonator
2/6/12
Cassandra input support now working ...
I have Cassandra input support working ... I need to test it some more but the general functionality
unread,
Cassandra input support now working ...
I have Cassandra input support working ... I need to test it some more but the general functionality
2/6/12
Roland Gude
,
burtonator
9
2/6/12
Controller write timeout
Cool!
unread,
Controller write timeout
Cool!
2/6/12
burtonator
1/21/12
Release Peregrine 0.5.3
I just released peregrine 0.5.3... it fixes a bug on Linux that may have locked up production jobs.
unread,
Release Peregrine 0.5.3
I just released peregrine 0.5.3... it fixes a bug on Linux that may have locked up production jobs.
1/21/12
Roland Gude
,
Kevin Burton
11
1/17/12
reducing - bug or mistake?
hm… ok. Well I will work on getting variable width keys here shortly anyway. On Mon, Jan 16, 2012 at
unread,
reducing - bug or mistake?
hm… ok. Well I will work on getting variable width keys here shortly anyway. On Mon, Jan 16, 2012 at
1/17/12
burtonator
,
Roland Gude
3
1/13/12
High scalability interviews me about Peregrine.
Sweet… I'm actually working on an implementation here soon.. I want to finish up the page rank
unread,
High scalability interviews me about Peregrine.
Sweet… I'm actually working on an implementation here soon.. I want to finish up the page rank
1/13/12
burtonator
,
Brian O'Neill
3
1/11/12
Cassandra + Hadoop JobConf example?
Yup… exactly what I needed! Thanks! On Wed, Jan 11, 2012 at 2:18 PM, Brian O'Neill <bone@
unread,
Cassandra + Hadoop JobConf example?
Yup… exactly what I needed! Thanks! On Wed, Jan 11, 2012 at 2:18 PM, Brian O'Neill <bone@
1/11/12
Roland Gude
, …
Kevin Burton
24
1/10/12
loading writing to/from cassandra
ok, got it pipes would be grat i guess
unread,
loading writing to/from cassandra
ok, got it pipes would be grat i guess
1/10/12
burtonator
12/27/11
Peregrine: A new map reduce framework for iterative/pipelined jobs.
I'm pleased to announce Peregrine 0.5.0 - a new map reduce framework optimized for iterative and
unread,
Peregrine: A new map reduce framework for iterative/pipelined jobs.
I'm pleased to announce Peregrine 0.5.0 - a new map reduce framework optimized for iterative and
12/27/11