Project: to make fully daily updating clone/mirror of en.wikipedia.org

[BlueTape]

Greetings,

I am looking for a turn key solution (software wise) for fully daily updating clone/mirror of en.wikipedia.org, including all meta, back to day 0.

What to do with commons in this project is TBD.

I am willing to pay for this.

I will provide hardware and bandwidth.

:-)

[Federico Leva (Nemo)]

BlueTape, 27/01/2018 00:24:

> I am looking for a turn key solution (software wise) for fully daily
> updating clone/mirror of en.wikipedia.org, including all meta, back to
> day 0.

Do you mean, including deleted content? You can pull new revisions from
the API on a live basis (a bit like the speedy deletion wikia does), but
if you don't need that you can have a snapshot at an arbitrary time just
by using the ~monthly dumps.

Cf. <https://phabricator.wikimedia.org/T164654>.

Federico

[BlueTape]

Hi Federico,

Deletions (that are not in en.wiki) not needed.

But all revisions (meta) needed. 

As well as being updated and not behind more than 24 hours.

Essentially, lets pick any date, we will have a exact copy of en.wiki and it will be updated daily with all changes.

If the internet ended tomorrow, this copy would have everything en.wiki would have (current articles, talk pages, and all past revisions of both).

Full Commons to be decided upon later due to size.

[Emilio J. Rodríguez-Posada]

Do you know about incremental daily dumps?

https://dumps.wikimedia.org/other/incr/

--
You received this message because you are subscribed to the Google Groups "wikiteam-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wikiteam-discuss+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[BlueTape]

Helpful, but am looking for someone to 'make' this for me.

any takers of the project?

To unsubscribe from this group and stop receiving emails from it, send an email to wikiteam-discu...@googlegroups.com.