Idol Server IDX file

[inigo]

Dear all
WE are having problems with IDX file generated for the Index services.
It seems that messages have been process by the content by they are
still located in the share drive.
I mean it seems that have not been processed completed.
Any of you have see the same problem ?

Thanks !!!!

[Bagot, Janet]

You need to add the following parameter to your content.cfg file (and
then stop and start content)

DeleteAfterAdd=True

Then IDOL will delete your temp files after processing

Essential Support

The content of this email and any attachment is private and may be legally privileged. If you are not the intended recipient of this email, any use, disclosure, copying or forwarding of this email and/or its attachments is unauthorised. If you have received this communication in error please notify the sender and delete this email and any attachments immediately.

Essential Computing is a division of SevernSoft Ltd., a company registered in England and Wales
Company number 1870663, VAT registration number GB 433 7332 63
Registered office: 26 Bay Road, Clevedon, BS21 7BT, UK.
Tel: +44 (0)1275 343199, Email: in...@essential.co.uk. Web: www.essential.co.uk

[Herda Silverman]

Hi

 

We have a similar problem. We have been struggling for 8 months to build an idol index. We start building the index from scratch and everything runs at a reasonable pace initially and the idx files are processed. As the index grows the speed at which it processes the idx files slows down considerably, eventually it almost grinds to a halt.

 

Our vendor has tried various configurations for us over the last 9 months and we have still not succeeded in building a complete index. We have about 21 million docs to index and the best we get too is about 5 million docs indexed.

 

Quite honestly this product is not doing more for us other than reduce the size of our mailfiles. Even on the archiving side we  continually experience cases where users are unable to retrieve archived mails. I could spend time on webbex's with our vendor trying to sort each of these issues out, but there are so many and my perception is that the support from autonomy is so poor that I do not waste my time anymore, I just restore from tape.

 

Our vendor is most helpful and have spent many hours/days trying to help us. They send endless amounts of logs to autonomy and we still have no resolution on many of the problems we experience.

 

Sorry, this is doom and gloom, but maybe someone at autonomy will start to take note.

[inigo]

Hi all
We have around 70 million of messages.
A question when you say " 21 million docs to index and the best we get

too is about 5 million docs

indexed" How do you know the amount of messages already processed.?

Also DeleteAfterAdd=True is already add

On Jan 22, 5:42 am, Herda Silverman <herdasilver...@gmail.com> wrote:
> Hi
>
> We have a similar problem. We have been struggling for 8 months to build an
> idol index. We start building the index from scratch and everything runs at
> a reasonable pace initially and the idx files are processed. As the index
> grows the speed at which it processes the idx files slows down considerably,
> eventually it almost grinds to a halt.
>
> Our vendor has tried various configurations for us over the last 9 months
> and we have still not succeeded in building a complete index. We have about
> 21 million docs to index and the best we get too is about 5 million docs
> indexed.
>
> Quite honestly this product is not doing more for us other than reduce the
> size of our mailfiles. Even on the archiving side we  continually experience
> cases where users are unable to retrieve archived mails. I could spend time
> on webbex's with our vendor trying to sort each of these issues out,
> but there are so many and my perception is that the support from autonomy is
> so poor that I do not waste my time anymore, I just restore from tape.
>
> Our vendor is most helpful and have spent many hours/days trying to help us.
> They send endless amounts of logs to autonomy and we still have no
> resolution on many of the problems we experience.
>
> Sorry, this is doom and gloom, but maybe someone at autonomy will start to
> take note.
>
> On Wed, Jan 21, 2009 at 5:28 PM, Bagot, Janet

> <Janet.Ba...@essential.co.uk>wrote:

> > Tel: +44 (0)1275 343199, Email: i...@essential.co.uk. Web:
> >www.essential.co.uk- Hide quoted text -
>
> - Show quoted text -

[Henrik.Grevelund]

Hi Inigo

Will you reveal how you got that into IDOL ?

Are you using several servers, content DB's and DIH/DAH ?

How long did it take you to convert from Altavista ?

With regards,
Henrik

[Archives - R -US]

In several installations, have found that a number of things can be done to help resolve IDX processing.  This is in reference to IDOL for EAS.

1. Set the Content and DISH services to be owned by the EAS Admin account.

            1.a.  It is best that EAS IDOL Indexer DOES NOT reside on the IDOL Content server.

2. Set up an IDX directory location (preferably on the IDOL content Server) and set permissions to let EAS Admin have FULL access to the Share and Permissions, also give EAS IIS Modify for both aspects.

3. In the EAS Indexer panel, make sure you have a version that lets you set the IDX location and set it to the UNC path location of the shared IDX directory you set up just above.

4. Make sure you are using at least the 6.1HF1 version of posted IDOL components  (if you are not at least on 6.1HF1 of EAS and the related IDOL release, I am afraid you are not ever going to be successful).

5. Restart the Content and DISH services after this is set up.

 

Every time the above has been done in production installations, IDX files have gotten submitted and then processed out correctly.  This has also lead to IDOL being a bit more stable and not corrupting.  If you are trying to load multiple services on one system, you should make sure you have at least two systems.  You can match up EAS and EASIDOLINDEXER on one server, and can put EASIIS and IDOL Content on another server (but preferably put IDOL on its own system, and STRONGLY suggest 64-Bit servers with some real heft to the resources).

If you have existing IDX files sitting, it is possible they have been processed, you can use direct IDOL commands in a web browser to confirm what you may have.  There are lots in the IDOL manuals on Automater, one fast basic to use is

 

http://IPADDROFCONTENTSERVER:9002/a=getstatus&format=html

 

If you don't see documents counted up in this page, then you do have a problem.  If you see results, you might try submitting IDX files manually again via the browser (there is not an easy batch submission method).  Keep in mind, get the IDOL manuals on hand, they will help a lot (it is not light reading, but it is in depth).

http://IDOL_CONTENT_SERVER:9001/DREADD?DRIVE:\PATH\NAME\STRING\FILENAME.IDX&DREDBNAME=easmapi&DELETE

 

You can also do some manual queries right to the IDOL content, to see if results are even available

http://IDOL_MACHINE_IP:9002/action=query&text=<queried_text>

[rasmussendk]

Why is it, that since Autonomy aqcuired this product, more problems
have been introduced, the "much better" IDOL indexing seems a big
joke, documentation is poor, and Autonomy support sucks 100%?

Just a thought.

/Søren

> http://IPADDROFCONTENTSERVER:9002/a=getstatus&format=html<http://ipaddrofcontentserver:9002/a=getstatus&format=html>

>
> If you don't see documents counted up in this page, then you do have a
> problem.  If you see results, you might try submitting IDX files manually
> again via the browser (there is not an easy batch submission method).  Keep
> in mind, get the IDOL manuals on hand, they will help a lot (it is not light
> reading, but it is in depth).
>

> http://IDOL_CONTENT_SERVER:9001/DREADD?DRIVE:\PATH\NAME\STRING\FILENAME.IDX&DREDBNAME=easmapi&DELETE<http://idol_content_server:9001/DREADD?DRIVE:\PATH\NAME\STRING\FILENAME.IDX&DREDBNAME=easmapi&DELETE>

>
> You can also do some manual queries right to the IDOL content, to see if
> results are even available
>

> http://IDOL_MACHINE_IP:9002/action=query&text=<queried<http://idol_machine_ip:9002/action=query&text=%3Cqueried_text>
> _text <http://idol_machine_ip:9002/action=query&text=%3cqueried>>

>
>
>
> On Wed, Jan 21, 2009 at 7:47 AM, inigo <iniab...@hotmail.com> wrote:
>
> > Dear all
> > WE are having problems with IDX file generated for the Index services.
> > It seems that messages have been process by the content by they are
> > still located in the share drive.
> > I mean it seems that have not been processed completed.
> > Any of you have see the same problem ?
>

> > Thanks !!!!- Hide quoted text -

[dominique docquier]

You're right Soren. I don't have a huge experience with EAS but this version as well as his updates don't push me the way I am willing to continue working with it. However, version 5 was working pretty good.

Does someone know if some equivalent softwares ? Do they run in a smoother and performant way or do they walk the same path as EAS ?
Cheers,

Dominique

[nymiggoog]

I understand Autonomy is also acquiring Interwoven (we use this as our
document management system).

We are f***ed! (:

[nymiggoog]

Inigo,

You can check the status if the IDX file has been processed completely
by running "http://IDOLserver:IDOLserverport/action=indexergetstatus"

You should see something similar:

<percentage_processed>100</percentage_processed>
<status>-1</status>
<description>Finished</description>
<index_command>/DREADD?%5C%5CNygexidol2%5CIDX
%5C5252_5588_1.idx&dihmodseed=0&KillDuplicates=REFERENCE</
index_command>

This means that "5C5252_5588_1.idx" has been processed completely.
You can manually delete this file from the IDX share. Or, you can
write vbscript that queries IDOL and deletes the completed .idx files.
Run the script as scheduled task.