Does anyone know what is the maximum number of PURLs a single PURLZ
server could hold, being able to provide the reasonable response time?
I am testing a local PURLZV1.6.3 (hsql db) with 100,000 PURLs.
Creation of high valume of PURL data on server side takes quite a
while (with an average speed about 6000 PURLs per hour and getting
slower as the number of creation rises). Is this normal?
I wondering if anyone is experienced in this area, or similar
information has been already documented in somewhere? Many thanks.
It is normal for a PURL server's database to slow during ingest. It may take days to load hundreds of thousands of PURLs. However, we have production users with nearly one million PURLs, so we know that works.
Please note that hsqldb is an in-memory database. Lack of RAM is no way to test scalability. Hsqldb is provided for small installations: We recommend that you use MySQL and calculate sufficient RAM to keep indexes in memory.
For example, one production site is currently running 100,000 PURLs with 8 GB of RAM.
> Does anyone know what is the maximum number of PURLs a single PURLZ > server could hold, being able to provide the reasonable response time? > I am testing a local PURLZV1.6.3 (hsql db) with 100,000 PURLs. > Creation of high valume of PURL data on server side takes quite a > while (with an average speed about 6000 PURLs per hour and getting > slower as the number of creation rises). Is this normal?
> I wondering if anyone is experienced in this area, or similar > information has been already documented in somewhere? Many thanks.
> It is normal for a PURL server's database to slow during ingest. It may take days to load hundreds of thousands of PURLs. However, we have production users with nearly one million PURLs, so we know that works.
> Please note that hsqldb is an in-memory database. Lack of RAM is no way to test scalability. Hsqldb is provided for small installations: We recommend that you use MySQL and calculate sufficient RAM to keep indexes in memory.
> For example, one production site is currently running 100,000 PURLs with 8 GB of RAM.
> Does that help you?
> Regards,
> Dave
> On Sep 8, 2011, at 09:51, Landong Zuo wrote:
> > Hi,
> > Does anyone know what is the maximum number of PURLs a single PURLZ
> > server could hold, being able to provide the reasonable response time?
> > I am testing a local PURLZV1.6.3 (hsql db) with 100,000 PURLs.
> > Creation of high valume of PURL data on server side takes quite a
> > while (with an average speed about 6000 PURLs per hour and getting
> > slower as the number of creation rises). Is this normal?
> > I wondering if anyone is experienced in this area, or similar
> > information has been already documented in somewhere? Many thanks.
One comment from the peanut gallery: It might make sense to disable MySQL indexes during a bulk load and re-enable them afterwards. That can cut down on load times significantly, since the indexes don't need to be updated for every row insert.
On Thu, Sep 15, 2011 at 10:51 AM, David Wood <da...@hyland-wood.org> wrote: > Hi Landong,
> It is normal for a PURL server's database to slow during ingest. It may take days to load hundreds of thousands of PURLs. However, we have production users with nearly one million PURLs, so we know that works.
> Please note that hsqldb is an in-memory database. Lack of RAM is no way to test scalability. Hsqldb is provided for small installations: We recommend that you use MySQL and calculate sufficient RAM to keep indexes in memory.
> For example, one production site is currently running 100,000 PURLs with 8 GB of RAM.
> Does that help you?
> Regards, > Dave
> On Sep 8, 2011, at 09:51, Landong Zuo wrote:
>> Hi,
>> Does anyone know what is the maximum number of PURLs a single PURLZ >> server could hold, being able to provide the reasonable response time? >> I am testing a local PURLZV1.6.3 (hsql db) with 100,000 PURLs. >> Creation of high valume of PURL data on server side takes quite a >> while (with an average speed about 6000 PURLs per hour and getting >> slower as the number of creation rises). Is this normal?
>> I wondering if anyone is experienced in this area, or similar >> information has been already documented in somewhere? Many thanks.
