Groups

peregrine-mapreduce

1–27 of 27

0 selected

2/20/18

What is the best source for learning about MapReduce?

MapReduce course curricula deal with the concepts of implementation for processing and generating

unread,

What is the best source for learning about MapReduce?

MapReduce course curricula deal with the concepts of implementation for processing and generating

2/20/18

9/6/16

Hadoop training in bay area

Hadoop has moved far beyond its beginnings in web indexing and is now used in many businesses for a

unread,

Hadoop training in bay area

Hadoop has moved far beyond its beginnings in web indexing and is now used in many businesses for a

9/6/16

7/18/12

Wikipedia rank works with the production wikipedia graph.

OK. Looks like the version of Peregrine in burton-structreader-release is executing for larger graphs

unread,

Wikipedia rank works with the production wikipedia graph.

OK. Looks like the version of Peregrine in burton-structreader-release is executing for larger graphs

7/18/12

6/10/12

Fixed another class of memory leaks (and a video of my new kitten)

I fixed another class of memory leaks in Peregrine so we can now run longer jobs. There are still 1-2

unread,

Fixed another class of memory leaks (and a video of my new kitten)

I fixed another class of memory leaks in Peregrine so we can now run longer jobs. There are still 1-2

6/10/12

5/29/12

Wikipedia rank working and another memory issue.

I have most of the work done for wikipedia rank. Really happy how this is working out because this is

unread,

Wikipedia rank working and another memory issue.

I have most of the work done for wikipedia rank. Really happy how this is working out because this is

5/29/12

5/28/12

Distributing a demo with wikipedia graph data.

It turns out that Wikipedia distributes a full dump of their page graph. It's about 8GB

unread,

Distributing a demo with wikipedia graph data.

It turns out that Wikipedia distributes a full dump of their page graph. It's about 8GB

5/28/12

5/28/12

Fixed a significant memory leak in reduce due to incorrect rlimit.

Larger jobs exhibited a crash bug due to artificially constrained rlimit on locked memory. This

unread,

Fixed a significant memory leak in reduce due to incorrect rlimit.

Larger jobs exhibited a crash bug due to artificially constrained rlimit on locked memory. This

5/28/12

burtonator, Roland Gude4

5/28/12

Fixed a significant OutOfMemoryError on default.

I just fixed one due to the fact that we weren't setting the right memlock limit... I'll post

unread,

Fixed a significant OutOfMemoryError on default.

I just fixed one due to the fact that we weren't setting the right memlock limit... I'll post

5/28/12

Roland Gude, burtonator12

5/4/12

is something like a combiner available

It may be easier for you to just clone and push some where so that we can land actual branches. I

unread,

is something like a combiner available

It may be easier for you to just clone and push some where so that we can land actual branches. I

5/4/12

Roland Gude, burtonator9

5/3/12

trouble with current tip

Ah... ok. I'm seeing similar things on my end. I think one of the issues is that the integration

unread,

trouble with current tip

Ah... ok. I'm seeing similar things on my end. I think one of the issues is that the integration

5/3/12

burtonator, Roland Gude2

4/23/12

Landed large commit/branch into default

this sounds great. i will port the combiner to the new head asap and see if it helps me with my

unread,

Landed large commit/branch into default

this sounds great. i will port the combiner to the new head asap and see if it helps me with my

4/23/12

Roland Gude, burtonator2

4/17/12

Yup... sorry. I'm working on tracking down a bug with SM right now which has caused me to totally

unread,

Yup... sorry. I'm working on tracking down a bug with SM right now which has caused me to totally

4/17/12

3/23/12

Peregrine presentation video from my talk at the Wikimedia Analytics Day

Here it is... http://www.youtube.com/watch?feature=player_detailpage&v=-_iCo1fFSeQ#t=14s I'm

unread,

Peregrine presentation video from my talk at the Wikimedia Analytics Day

Here it is... http://www.youtube.com/watch?feature=player_detailpage&v=-_iCo1fFSeQ#t=14s I'm

3/23/12

OG, burtonator2

3/6/12

On Tuesday, March 6, 2012 5:13:30 AM UTC-8, OG wrote: Hey Kevin, long time no speak... Hope all is

unread,

On Tuesday, March 6, 2012 5:13:30 AM UTC-8, OG wrote: Hey Kevin, long time no speak... Hope all is

3/6/12

burtonator, Roland Gude13

3/5/12

Just merged Cassandra support into default...

OK... I mostly have this implemented now... So if the key IS a hash you can map over it... however,

unread,

Just merged Cassandra support into default...

OK... I mostly have this implemented now... So if the key IS a hash you can map over it... however,

3/5/12

3/5/12

Peregrine Presentation at Wikimedia

I gave a presentation on Peregrine at the Wikimedia Foundation on Friday of last week. Here is the

unread,

Peregrine Presentation at Wikimedia

I gave a presentation on Peregrine at the Wikimedia Foundation on Friday of last week. Here is the

3/5/12

2/17/12

Finished Cassandra job output and now merged to default.

I also finished up the Cassandra job output... so you can read AND write to Cassandra now. I'm

unread,

Finished Cassandra job output and now merged to default.

I also finished up the Cassandra job output... so you can read AND write to Cassandra now. I'm

2/17/12

2/10/12

Peregrine continuous integration

I set it up here: http://peregrine.integration.spinn3r.com/ It's just a basic python script that

unread,

Peregrine continuous integration

I set it up here: http://peregrine.integration.spinn3r.com/ It's just a basic python script that

2/10/12

Roland Gude, burtonator2

2/8/12

integrate.py beautification

Ah... I will see what I can integrate back in... I didn't realize you were contributing patches.

unread,

integrate.py beautification

Ah... I will see what I can integrate back in... I didn't realize you were contributing patches.

2/8/12

2/6/12

Cassandra input support now working ...

I have Cassandra input support working ... I need to test it some more but the general functionality

unread,

Cassandra input support now working ...

I have Cassandra input support working ... I need to test it some more but the general functionality

2/6/12

Roland Gude, burtonator9

2/6/12

Controller write timeout

unread,

Controller write timeout

2/6/12

1/21/12

Release Peregrine 0.5.3

I just released peregrine 0.5.3... it fixes a bug on Linux that may have locked up production jobs.

unread,

Release Peregrine 0.5.3

I just released peregrine 0.5.3... it fixes a bug on Linux that may have locked up production jobs.

1/21/12

Roland Gude, Kevin Burton11

1/17/12

reducing - bug or mistake?

hm… ok. Well I will work on getting variable width keys here shortly anyway. On Mon, Jan 16, 2012 at

unread,

reducing - bug or mistake?

hm… ok. Well I will work on getting variable width keys here shortly anyway. On Mon, Jan 16, 2012 at

1/17/12

burtonator, Roland Gude3

1/13/12

High scalability interviews me about Peregrine.

Sweet… I'm actually working on an implementation here soon.. I want to finish up the page rank

unread,

High scalability interviews me about Peregrine.

Sweet… I'm actually working on an implementation here soon.. I want to finish up the page rank

1/13/12

burtonator, Brian O'Neill3

1/11/12

Cassandra + Hadoop JobConf example?

Yup… exactly what I needed! Thanks! On Wed, Jan 11, 2012 at 2:18 PM, Brian O'Neill <bone@

unread,

Cassandra + Hadoop JobConf example?

Yup… exactly what I needed! Thanks! On Wed, Jan 11, 2012 at 2:18 PM, Brian O'Neill <bone@

1/11/12

Roland Gude, … Kevin Burton24

1/10/12

loading writing to/from cassandra

ok, got it pipes would be grat i guess

unread,

loading writing to/from cassandra

ok, got it pipes would be grat i guess

1/10/12

12/27/11

Peregrine: A new map reduce framework for iterative/pipelined jobs.

I'm pleased to announce Peregrine 0.5.0 - a new map reduce framework optimized for iterative and

unread,

Peregrine: A new map reduce framework for iterative/pipelined jobs.

I'm pleased to announce Peregrine 0.5.0 - a new map reduce framework optimized for iterative and

12/27/11

Search

Clear search

Close search

Google apps

Main menu