Trouble unning performance evaluations on RavenDB, build 701.
102 views
Skip to first unread message
spokey
Mar 6, 2012, 3:57:55 PM3/6/12
to rav...@googlegroups.com
Running performance evaluations on RavenDB, build 701.
Backed up a 1 million document database using Smuggler (created using build 616)
Restored 1 million document database using Smuggler.
Though the restore function seemed to process normally, the subsequent indexing is still running
after approximately 20 hours.
There are two indexes:
- The default "Raven/DocumentsByEntityName
- A custom index: "ProviderSearchIndex". It is a complex index, (as you can see below), but I don't
think it should still be calculating.
The debug file is exceedingly verbose and I don't seem to be able to find any instructions on how
to reduce the logging level.
There is no "NLog.config" file in the Server directory.
My machine stats:
Intel Xeon W3505 2.53 GHz
Memory 12.0 GB
64-bit OS
CPU averages between 80%-90%
Memory usage climbs to over 11mb
So far, the log has a combined length (over last 24 hours) of over 27GB.
Here is a short snippet from the server log file:
time,logger,level,message,exception
2012-03-06 00:00:00.0051,Raven.Database.Indexing.Index.Indexing,Debug,"Indexing on ProviderPtos/224079 result in index ProviderSearchIndex gave document: __document_id IS: providerptos/224079
NetworksPto_Network_Identifier I-: PN - Peterson-55
NetworksPto_Network_InternalId I-: 55
NetworksPto_EffectiveFrom I-: 20110101000000000
NetworksPto_EffectiveThrough I-: NULL_VALUE
NetworkSetsPto_NetworkSet_Identifier I-: PN Set-16
NetworkSetsPto_NetworkSet_InternalId I-: 16
NetworkSetsPto_EffectiveFrom I-: 19990101000000000
NetworkSetsPto_EffectiveThrough I-: NULL_VALUE
CategoriesPto_Category_InternalId I-: 4
CategoriesPto_Category_InternalId_Range I-: 4
CategoriesPto_Category_Identifier I-: Specialist
CategoriesPto_EffectiveFrom I-: 20110101000000000
CategoriesPto_EffectiveThrough I-: NULL_VALUE
SpecialtiesPto_Specialty_Identifier I-: MSR
SpecialtiesPto_Specialty_InternalId I-: 152
SpecialtiesPto_Specialty_InternalId_Range I-: 152
SpecialtiesPto_EffectiveFrom I-: 20110101000000000
SpecialtiesPto_EffectiveThrough I-: NULL_VALUE
ProviderIdentifiersPto_ConcatTypeAndIdentifier I-: 9|PI 241568
ProviderIdentifiersPto_EffectiveFrom I-: 20110101000000000
ProviderIdentifiersPto_EffectiveThrough I-: NULL_VALUE
Gender_Identifier I-: Male
Gender_InternalId I-: 1
Gender_InternalId_Range I-: 1
ProviderType_Identifier I-: Company
ProviderType_InternalId I-: 2
ProviderType_InternalId_Range I-: 2
NamesPto_FirstName I-: NULL_VALUE
NamesPto_LastOrOrganizationalName I-: Long - 229589
NamesPto_NullName I-: NULL_VALUE
LanguagesPto_Language_Identifier_IsArray IS: true
LanguagesPto_Language_InternalId_IsArray IS: true
AddressesPto_EffectiveFrom I-: 20110101000000000
AddressesPto_EffectiveThrough I-: NULL_VALUE
AddressesPto_Address_Address1 I-: 7652 Express Lane
AddressesPto_Address_Address2 I-: NULL_VALUE
AddressesPto_Address_City I-: Fort Worth
AddressesPto_Address_State I-: TX
AddressesPto_Address_Zip I-: 76102
AddressesPto_Address_AddressTypePto I-: ProviderAddress
PracticeOfficesPto_EffectiveFrom I-: 20110101000000000
PracticeOfficesPto_EffectiveThrough I-: NULL_VALUE
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_AddressEffectiveFrom I-: 20110101000000000
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_AddressEffectiveThrough I-: NULL_VALUE
IsPrimary I-: true
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_ContactEffectiveFrom I-: 20110101000000000
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_ContactEffectiveThrough I-: NULL_VALUE
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Address1 I-: 7652 Express Lane
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Address2 I-: NULL_VALUE
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_City I-: Fort Worth
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_State I-: TX
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Zip I-: 76102
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_AddressTypePto I-: PracticeOfficeAddress
PayTosPto_EffectiveFrom I-: 20110101000000000
PayTosPto_EffectiveThrough I-: NULL_VALUE
PayTosPto_PayTo_PrimaryContact_City I-: Los Angeles
PayTosPto_PayTo_PrimaryContact_State I-: CA
PayTosPto_PayTo_PrimaryContact_Zip I-: 90001
PayTosPto_PayTo_PrimaryContact_Address1 I-: 6096 Paved Road
PayTosPto_PayTo_PrimaryContact_Address2 I-: NULL_VALUE
PayTosPto_PayTo_PrimaryContact_AddressTypePto I-: PayToAddress
",
2012-03-06 00:00:00.0051,Raven.Database.Indexing.Index.Indexing,Debug,"Indexing on ProviderPtos/224079 result in index ProviderSearchIndex gave document: __document_id IS: providerptos/224079
NetworksPto_Network_Identifier I-: PN - Peterson-55
NetworksPto_Network_InternalId I-: 55
NetworksPto_EffectiveFrom I-: 20110101000000000
etc...
Backed up a 1 million document database using Smuggler (created using build 616)
Restored 1 million document database using Smuggler.
Though the restore function seemed to process normally, the subsequent indexing is still running
after approximately 20 hours.
There are two indexes:
- The default "Raven/DocumentsByEntityName
- A custom index: "ProviderSearchIndex". It is a complex index, (as you can see below), but I don't
think it should still be calculating.
The debug file is exceedingly verbose and I don't seem to be able to find any instructions on how
to reduce the logging level.
There is no "NLog.config" file in the Server directory.
My machine stats:
Intel Xeon W3505 2.53 GHz
Memory 12.0 GB
64-bit OS
CPU averages between 80%-90%
Memory usage climbs to over 11mb
So far, the log has a combined length (over last 24 hours) of over 27GB.
Here is a short snippet from the server log file:
time,logger,level,message,exception
2012-03-06 00:00:00.0051,Raven.Database.Indexing.Index.Indexing,Debug,"Indexing on ProviderPtos/224079 result in index ProviderSearchIndex gave document: __document_id IS: providerptos/224079
NetworksPto_Network_Identifier I-: PN - Peterson-55
NetworksPto_Network_InternalId I-: 55
NetworksPto_EffectiveFrom I-: 20110101000000000
NetworksPto_EffectiveThrough I-: NULL_VALUE
NetworkSetsPto_NetworkSet_Identifier I-: PN Set-16
NetworkSetsPto_NetworkSet_InternalId I-: 16
NetworkSetsPto_EffectiveFrom I-: 19990101000000000
NetworkSetsPto_EffectiveThrough I-: NULL_VALUE
CategoriesPto_Category_InternalId I-: 4
CategoriesPto_Category_InternalId_Range I-: 4
CategoriesPto_Category_Identifier I-: Specialist
CategoriesPto_EffectiveFrom I-: 20110101000000000
CategoriesPto_EffectiveThrough I-: NULL_VALUE
SpecialtiesPto_Specialty_Identifier I-: MSR
SpecialtiesPto_Specialty_InternalId I-: 152
SpecialtiesPto_Specialty_InternalId_Range I-: 152
SpecialtiesPto_EffectiveFrom I-: 20110101000000000
SpecialtiesPto_EffectiveThrough I-: NULL_VALUE
ProviderIdentifiersPto_ConcatTypeAndIdentifier I-: 9|PI 241568
ProviderIdentifiersPto_EffectiveFrom I-: 20110101000000000
ProviderIdentifiersPto_EffectiveThrough I-: NULL_VALUE
Gender_Identifier I-: Male
Gender_InternalId I-: 1
Gender_InternalId_Range I-: 1
ProviderType_Identifier I-: Company
ProviderType_InternalId I-: 2
ProviderType_InternalId_Range I-: 2
NamesPto_FirstName I-: NULL_VALUE
NamesPto_LastOrOrganizationalName I-: Long - 229589
NamesPto_NullName I-: NULL_VALUE
LanguagesPto_Language_Identifier_IsArray IS: true
LanguagesPto_Language_InternalId_IsArray IS: true
AddressesPto_EffectiveFrom I-: 20110101000000000
AddressesPto_EffectiveThrough I-: NULL_VALUE
AddressesPto_Address_Address1 I-: 7652 Express Lane
AddressesPto_Address_Address2 I-: NULL_VALUE
AddressesPto_Address_City I-: Fort Worth
AddressesPto_Address_State I-: TX
AddressesPto_Address_Zip I-: 76102
AddressesPto_Address_AddressTypePto I-: ProviderAddress
PracticeOfficesPto_EffectiveFrom I-: 20110101000000000
PracticeOfficesPto_EffectiveThrough I-: NULL_VALUE
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_AddressEffectiveFrom I-: 20110101000000000
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_AddressEffectiveThrough I-: NULL_VALUE
IsPrimary I-: true
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_ContactEffectiveFrom I-: 20110101000000000
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_ContactEffectiveThrough I-: NULL_VALUE
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Address1 I-: 7652 Express Lane
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Address2 I-: NULL_VALUE
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_City I-: Fort Worth
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_State I-: TX
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Zip I-: 76102
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_AddressTypePto I-: PracticeOfficeAddress
PayTosPto_EffectiveFrom I-: 20110101000000000
PayTosPto_EffectiveThrough I-: NULL_VALUE
PayTosPto_PayTo_PrimaryContact_City I-: Los Angeles
PayTosPto_PayTo_PrimaryContact_State I-: CA
PayTosPto_PayTo_PrimaryContact_Zip I-: 90001
PayTosPto_PayTo_PrimaryContact_Address1 I-: 6096 Paved Road
PayTosPto_PayTo_PrimaryContact_Address2 I-: NULL_VALUE
PayTosPto_PayTo_PrimaryContact_AddressTypePto I-: PayToAddress
",
2012-03-06 00:00:00.0051,Raven.Database.Indexing.Index.Indexing,Debug,"Indexing on ProviderPtos/224079 result in index ProviderSearchIndex gave document: __document_id IS: providerptos/224079
NetworksPto_Network_Identifier I-: PN - Peterson-55
NetworksPto_Network_InternalId I-: 55
NetworksPto_EffectiveFrom I-: 20110101000000000
etc...
Oren Eini (Ayende Rahien)
Mar 7, 2012, 2:34:57 AM3/7/12
to rav...@googlegroups.com
Spokey,
a) Any perf testing with the logging set to debug isn't really going to be valid.
You can set the log level using: <logger name="*" minlevel="Info" writeTo="logfile" />
b) Those numbers do NOT sound right.
b) Those numbers do NOT sound right.
On a system with more indexes and more data, indexing takes a lot less time for us.
Memory usage is 11 GB or 11 MB?
do you have any special config values?
Oren Eini (Ayende Rahien)
Mar 7, 2012, 2:35:08 AM3/7/12
to rav...@googlegroups.com
What does your index looks like?
spokey
Mar 7, 2012, 12:23:52 PM3/7/12
to rav...@googlegroups.com
Ayende:
I stopped the server, set the log minlevel = "Info" and restarted server. Does that mean it will now completely rebuild the index??
(By the way, the indexing was still running this morning (2nd morning), but I stopped the Reven server yesterday, so it may have restarted from scratch again.)
Memory usage was around 11.5 GB.
No, there is no special configuration in this case. Out of the box build 701.
Admittedly, the index is complex, but recently (don't remember the exact build) we were able to import 1 million of these documents in 4 hrs 13 minutes
and index them in 3 hrs 46 minutes.
-----------------------------------------------
Here's how the index is created:
public class ProviderSearchIndex : AbstractIndexCreationTask<ProviderPto, ProviderSearchIndex.Result>
{
public class Result
{
public string NamesPto_FirstName { get; set; }
public string NamesPto_LastOrOrganizationalName { get; set; }
public string AddressesPto_Address_Address1 { get; set; }
public string AddressesPto_Address_Address2 { get; set; }
public string AddressesPto_Address_City { get; set; }
public string PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Address1 { get; set; }
public string PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Address2 { get; set; }
public string PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_City { get; set; }
public string PayTosPto_PayTo_PrimaryContact_City { get; set; }
public string PayTosPto_PayTo_PrimaryContact_Address1 { get; set; }
public string PayTosPto_PayTo_PrimaryContact_Address2 { get; set; }
}
public ProviderSearchIndex()
{
Map = providerPtos =>
from provider in providerPtos
from category in (provider.CategoriesPto).DefaultIfEmpty()
from network in provider.NetworksPto.DefaultIfEmpty()
from networkSet in provider.NetworkSetsPto.DefaultIfEmpty()
from practiceOfficeAssignment in provider.PracticeOfficesPto.DefaultIfEmpty()
from practiceOfficeContact in practiceOfficeAssignment.PracticeOffice.PracticeOfficeContactsPto.DefaultIfEmpty()
from specialty in provider.SpecialtiesPto.DefaultIfEmpty()
from providerIdentifer in provider.ProviderIdentifiersPto.DefaultIfEmpty()
from payToAssign in provider.PayTosPto.DefaultIfEmpty()
from payToIdentifier in payToAssign.PayTo.PayToIdentifiersPto.DefaultIfEmpty()
from name in provider.NamesPto.DefaultIfEmpty()
from providerAddress in provider.AddressesPto.DefaultIfEmpty()
select new
{
// Networks
NetworksPto_Network_Identifier = network.Network.Identifier,
NetworksPto_Network_InternalId = network.Network.InternalId,
NetworksPto_EffectiveFrom = network.EffectiveFrom,
NetworksPto_EffectiveThrough = network.EffectiveThrough,
// NetworkSets
NetworkSetsPto_NetworkSet_Identifier = networkSet.NetworkSet.Identifier,
NetworkSetsPto_NetworkSet_InternalId = networkSet.NetworkSet.InternalId,
NetworkSetsPto_EffectiveFrom = networkSet.EffectiveFrom,
NetworkSetsPto_EffectiveThrough = networkSet.EffectiveThrough,
// Categories
CategoriesPto_Category_InternalId = category.Category.InternalId,
CategoriesPto_Category_Identifier = category.Category.Identifier,
CategoriesPto_EffectiveFrom = category.EffectiveFrom,
CategoriesPto_EffectiveThrough = category.EffectiveThrough,
// Specialities
SpecialtiesPto_Specialty_Identifier = specialty.Specialty.Identifier,
SpecialtiesPto_Specialty_InternalId = specialty.Specialty.InternalId,
SpecialtiesPto_EffectiveFrom = specialty.EffectiveFrom,
SpecialtiesPto_EffectiveThrough = specialty.EffectiveThrough,
// Provider Identifiers
ProviderIdentifiersPto_ConcatTypeAndIdentifier = providerIdentifer.ConcatTypeAndIdentifier,
ProviderIdentifiersPto_EffectiveFrom = providerIdentifer.EffectiveFrom,
ProviderIdentifiersPto_EffectiveThrough = providerIdentifer.EffectiveThrough,
// Gender & ProviderType
Gender_Identifier = provider.Gender.Identifier,
Gender_InternalId = provider.Gender.InternalId,
ProviderType_Identifier = provider.ProviderType.Identifier,
ProviderType_InternalId = provider.ProviderType.InternalId,
// Name
NamesPto_FirstName = name.FirstName,
NamesPto_LastOrOrganizationalName = name.LastOrOrganizationalName,
NamesPto_NullName = name.NullName,
// Language
LanguagesPto_Language_Identifier = provider.LanguagesPto.Select(l => l.Language.Identifier),
LanguagesPto_Language_InternalId = provider.LanguagesPto.Select(l => l.Language.InternalId),
// Provider Address
AddressesPto_EffectiveFrom = providerAddress.EffectiveFrom,
AddressesPto_EffectiveThrough = providerAddress.EffectiveThrough,
AddressesPto_Address_Address1 = providerAddress.Address.Address1,
AddressesPto_Address_Address2 = providerAddress.Address.Address2,
AddressesPto_Address_City = providerAddress.Address.City,
AddressesPto_Address_State = providerAddress.Address.State,
AddressesPto_Address_Zip = providerAddress.Address.Zip,
AddressesPto_Address_AddressTypePto = providerAddress.Address.AddressTypePto,
// PracticeOffice
PracticeOfficesPto_EffectiveFrom = practiceOfficeAssignment.EffectiveFrom,
PracticeOfficesPto_EffectiveThrough = practiceOfficeAssignment.EffectiveThrough,
// PracticeOffice Contact
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_AddressEffectiveFrom = practiceOfficeContact.AddressEffectiveFrom,
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_AddressEffectiveThrough = practiceOfficeContact.AddressEffectiveThrough,
// This is the only field which does not need the qualified naming convention to distinguish it from other like-named fields.
// In fact, Raven does not like the qualified name.
practiceOfficeContact.IsPrimary,
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_ContactEffectiveFrom = practiceOfficeContact.ContactEffectiveFrom,
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_ContactEffectiveThrough = practiceOfficeContact.ContactEffectiveThrough,
// PracticeOffice Contact Address
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Address1 = practiceOfficeContact.Address.Address1,
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Address2 = practiceOfficeContact.Address.Address2,
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_City = practiceOfficeContact.Address.City,
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_State = practiceOfficeContact.Address.State,
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Zip = practiceOfficeContact.Address.Zip,
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_AddressTypePto = practiceOfficeContact.Address.AddressTypePto,
// PayTo
PayTosPto_EffectiveFrom = payToAssign.EffectiveFrom,
PayTosPto_EffectiveThrough = payToAssign.EffectiveThrough,
// PayTo Identifiers
PayTosPto_PayTo_PayToIdentifiersPto_ConcatTypeAndIdentifier = payToIdentifier.ConcatTypeAndIdentifier,
PayTosPto_PayTo_PayToIdentifiersPto_EffectiveFrom = payToIdentifier.EffectiveFrom,
PayTosPto_PayTo_PayToIdentifiersPto_EffectiveThrough = payToIdentifier.EffectiveThrough,
// PayTo PrimaryContact Address
PayTosPto_PayTo_PrimaryContact_City = payToAssign.PayTo.PrimaryContact.City,
PayTosPto_PayTo_PrimaryContact_State = payToAssign.PayTo.PrimaryContact.State,
PayTosPto_PayTo_PrimaryContact_Zip = payToAssign.PayTo.PrimaryContact.Zip,
PayTosPto_PayTo_PrimaryContact_Address1 = payToAssign.PayTo.PrimaryContact.Address1,
PayTosPto_PayTo_PrimaryContact_Address2 = payToAssign.PayTo.PrimaryContact.Address2,
PayTosPto_PayTo_PrimaryContact_AddressTypePto = payToAssign.PayTo.PrimaryContact.AddressTypePto,
};
Index(x => x.NamesPto_FirstName, FieldIndexing.Analyzed);
Index(x => x.NamesPto_LastOrOrganizationalName, FieldIndexing.Analyzed);
Index(x => x.AddressesPto_Address_Address1, FieldIndexing.Analyzed);
Index(x => x.AddressesPto_Address_Address2, FieldIndexing.Analyzed);
Index(x => x.AddressesPto_Address_City, FieldIndexing.Analyzed);
Index(x => x.PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Address1, FieldIndexing.Analyzed);
Index(x => x.PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Address2, FieldIndexing.Analyzed);
Index(x => x.PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_City, FieldIndexing.Analyzed);
Index(x => x.PayTosPto_PayTo_PrimaryContact_City, FieldIndexing.Analyzed);
Index(x => x.PayTosPto_PayTo_PrimaryContact_Address1, FieldIndexing.Analyzed);
Index(x => x.PayTosPto_PayTo_PrimaryContact_Address2, FieldIndexing.Analyzed);
}
}
I stopped the server, set the log minlevel = "Info" and restarted server. Does that mean it will now completely rebuild the index??
(By the way, the indexing was still running this morning (2nd morning), but I stopped the Reven server yesterday, so it may have restarted from scratch again.)
Memory usage was around 11.5 GB.
No, there is no special configuration in this case. Out of the box build 701.
Admittedly, the index is complex, but recently (don't remember the exact build) we were able to import 1 million of these documents in 4 hrs 13 minutes
and index them in 3 hrs 46 minutes.
-----------------------------------------------
Here's how the index is created:
public class ProviderSearchIndex : AbstractIndexCreationTask<ProviderPto, ProviderSearchIndex.Result>
{
public class Result
{
public string NamesPto_FirstName { get; set; }
public string NamesPto_LastOrOrganizationalName { get; set; }
public string AddressesPto_Address_Address1 { get; set; }
public string AddressesPto_Address_Address2 { get; set; }
public string AddressesPto_Address_City { get; set; }
public string PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Address1 { get; set; }
public string PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Address2 { get; set; }
public string PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_City { get; set; }
public string PayTosPto_PayTo_PrimaryContact_City { get; set; }
public string PayTosPto_PayTo_PrimaryContact_Address1 { get; set; }
public string PayTosPto_PayTo_PrimaryContact_Address2 { get; set; }
}
public ProviderSearchIndex()
{
Map = providerPtos =>
from provider in providerPtos
from category in (provider.CategoriesPto).DefaultIfEmpty()
from network in provider.NetworksPto.DefaultIfEmpty()
from networkSet in provider.NetworkSetsPto.DefaultIfEmpty()
from practiceOfficeAssignment in provider.PracticeOfficesPto.DefaultIfEmpty()
from practiceOfficeContact in practiceOfficeAssignment.PracticeOffice.PracticeOfficeContactsPto.DefaultIfEmpty()
from specialty in provider.SpecialtiesPto.DefaultIfEmpty()
from providerIdentifer in provider.ProviderIdentifiersPto.DefaultIfEmpty()
from payToAssign in provider.PayTosPto.DefaultIfEmpty()
from payToIdentifier in payToAssign.PayTo.PayToIdentifiersPto.DefaultIfEmpty()
from name in provider.NamesPto.DefaultIfEmpty()
from providerAddress in provider.AddressesPto.DefaultIfEmpty()
select new
{
// Networks
NetworksPto_Network_Identifier = network.Network.Identifier,
NetworksPto_Network_InternalId = network.Network.InternalId,
NetworksPto_EffectiveFrom = network.EffectiveFrom,
NetworksPto_EffectiveThrough = network.EffectiveThrough,
// NetworkSets
NetworkSetsPto_NetworkSet_Identifier = networkSet.NetworkSet.Identifier,
NetworkSetsPto_NetworkSet_InternalId = networkSet.NetworkSet.InternalId,
NetworkSetsPto_EffectiveFrom = networkSet.EffectiveFrom,
NetworkSetsPto_EffectiveThrough = networkSet.EffectiveThrough,
// Categories
CategoriesPto_Category_InternalId = category.Category.InternalId,
CategoriesPto_Category_Identifier = category.Category.Identifier,
CategoriesPto_EffectiveFrom = category.EffectiveFrom,
CategoriesPto_EffectiveThrough = category.EffectiveThrough,
// Specialities
SpecialtiesPto_Specialty_Identifier = specialty.Specialty.Identifier,
SpecialtiesPto_Specialty_InternalId = specialty.Specialty.InternalId,
SpecialtiesPto_EffectiveFrom = specialty.EffectiveFrom,
SpecialtiesPto_EffectiveThrough = specialty.EffectiveThrough,
// Provider Identifiers
ProviderIdentifiersPto_ConcatTypeAndIdentifier = providerIdentifer.ConcatTypeAndIdentifier,
ProviderIdentifiersPto_EffectiveFrom = providerIdentifer.EffectiveFrom,
ProviderIdentifiersPto_EffectiveThrough = providerIdentifer.EffectiveThrough,
// Gender & ProviderType
Gender_Identifier = provider.Gender.Identifier,
Gender_InternalId = provider.Gender.InternalId,
ProviderType_Identifier = provider.ProviderType.Identifier,
ProviderType_InternalId = provider.ProviderType.InternalId,
// Name
NamesPto_FirstName = name.FirstName,
NamesPto_LastOrOrganizationalName = name.LastOrOrganizationalName,
NamesPto_NullName = name.NullName,
// Language
LanguagesPto_Language_Identifier = provider.LanguagesPto.Select(l => l.Language.Identifier),
LanguagesPto_Language_InternalId = provider.LanguagesPto.Select(l => l.Language.InternalId),
// Provider Address
AddressesPto_EffectiveFrom = providerAddress.EffectiveFrom,
AddressesPto_EffectiveThrough = providerAddress.EffectiveThrough,
AddressesPto_Address_Address1 = providerAddress.Address.Address1,
AddressesPto_Address_Address2 = providerAddress.Address.Address2,
AddressesPto_Address_City = providerAddress.Address.City,
AddressesPto_Address_State = providerAddress.Address.State,
AddressesPto_Address_Zip = providerAddress.Address.Zip,
AddressesPto_Address_AddressTypePto = providerAddress.Address.AddressTypePto,
// PracticeOffice
PracticeOfficesPto_EffectiveFrom = practiceOfficeAssignment.EffectiveFrom,
PracticeOfficesPto_EffectiveThrough = practiceOfficeAssignment.EffectiveThrough,
// PracticeOffice Contact
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_AddressEffectiveFrom = practiceOfficeContact.AddressEffectiveFrom,
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_AddressEffectiveThrough = practiceOfficeContact.AddressEffectiveThrough,
// This is the only field which does not need the qualified naming convention to distinguish it from other like-named fields.
// In fact, Raven does not like the qualified name.
practiceOfficeContact.IsPrimary,
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_ContactEffectiveFrom = practiceOfficeContact.ContactEffectiveFrom,
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_ContactEffectiveThrough = practiceOfficeContact.ContactEffectiveThrough,
// PracticeOffice Contact Address
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Address1 = practiceOfficeContact.Address.Address1,
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Address2 = practiceOfficeContact.Address.Address2,
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_City = practiceOfficeContact.Address.City,
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_State = practiceOfficeContact.Address.State,
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Zip = practiceOfficeContact.Address.Zip,
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_AddressTypePto = practiceOfficeContact.Address.AddressTypePto,
// PayTo
PayTosPto_EffectiveFrom = payToAssign.EffectiveFrom,
PayTosPto_EffectiveThrough = payToAssign.EffectiveThrough,
// PayTo Identifiers
PayTosPto_PayTo_PayToIdentifiersPto_ConcatTypeAndIdentifier = payToIdentifier.ConcatTypeAndIdentifier,
PayTosPto_PayTo_PayToIdentifiersPto_EffectiveFrom = payToIdentifier.EffectiveFrom,
PayTosPto_PayTo_PayToIdentifiersPto_EffectiveThrough = payToIdentifier.EffectiveThrough,
// PayTo PrimaryContact Address
PayTosPto_PayTo_PrimaryContact_City = payToAssign.PayTo.PrimaryContact.City,
PayTosPto_PayTo_PrimaryContact_State = payToAssign.PayTo.PrimaryContact.State,
PayTosPto_PayTo_PrimaryContact_Zip = payToAssign.PayTo.PrimaryContact.Zip,
PayTosPto_PayTo_PrimaryContact_Address1 = payToAssign.PayTo.PrimaryContact.Address1,
PayTosPto_PayTo_PrimaryContact_Address2 = payToAssign.PayTo.PrimaryContact.Address2,
PayTosPto_PayTo_PrimaryContact_AddressTypePto = payToAssign.PayTo.PrimaryContact.AddressTypePto,
};
Index(x => x.NamesPto_FirstName, FieldIndexing.Analyzed);
Index(x => x.NamesPto_LastOrOrganizationalName, FieldIndexing.Analyzed);
Index(x => x.AddressesPto_Address_Address1, FieldIndexing.Analyzed);
Index(x => x.AddressesPto_Address_Address2, FieldIndexing.Analyzed);
Index(x => x.AddressesPto_Address_City, FieldIndexing.Analyzed);
Index(x => x.PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Address1, FieldIndexing.Analyzed);
Index(x => x.PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Address2, FieldIndexing.Analyzed);
Index(x => x.PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_City, FieldIndexing.Analyzed);
Index(x => x.PayTosPto_PayTo_PrimaryContact_City, FieldIndexing.Analyzed);
Index(x => x.PayTosPto_PayTo_PrimaryContact_Address1, FieldIndexing.Analyzed);
Index(x => x.PayTosPto_PayTo_PrimaryContact_Address2, FieldIndexing.Analyzed);
}
}
Oren Eini (Ayende Rahien)
Mar 7, 2012, 1:21:06 PM3/7/12
to rav...@googlegroups.com
inline
On Wed, Mar 7, 2012 at 7:23 PM, spokey <jeffrey....@gmail.com> wrote:
Ayende:
I stopped the server, set the log minlevel = "Info" and restarted server. Does that mean it will now completely rebuild the index??
No, it will restart doing that.
(By the way, the indexing was still running this morning (2nd morning), but I stopped the Reven server yesterday, so it may have restarted from scratch again.)
Is this in a tenant database, by the way?
spokey
Mar 7, 2012, 10:47:12 PM3/7/12
to rav...@googlegroups.com
The app runs in a tenant database, but for the performance test I am running in the default database.
Though I've seen some strange behavior today related to the logging and the CPU utilization, I haven't had the time to spend to nail down the issue any better. It may take a few days to get any more info together.
Is there a log level I could use that would tell me succintly when the indexing starts and stops but wouldn't overwhelm the log with 30-40 GB of data the way it did yesterday?
Though I've seen some strange behavior today related to the logging and the CPU utilization, I haven't had the time to spend to nail down the issue any better. It may take a few days to get any more info together.
Is there a log level I could use that would tell me succintly when the indexing starts and stops but wouldn't overwhelm the log with 30-40 GB of data the way it did yesterday?
Oren Eini (Ayende Rahien)
Mar 8, 2012, 12:32:35 AM3/8/12
to rav...@googlegroups.com
No
Indexing is logged at a low priority by design
You can check the stats for that
Indexing is logged at a low priority by design
You can check the stats for that
spokey
Mar 8, 2012, 7:54:40 PM3/8/12
to rav...@googlegroups.com
I shut down the RavenServer this morning (it had been running all night), made some adjustments to the NLog config file and restarted it. 8 hours later it is still running; the server process is using 50 % CPU (fairly constant, on average) and over 9GB of memory.
The server has been running, on and off, for the past couple of days and never seems to complete indexing. The ProviderSearchIndex still shows that it is stale. No changes have been made to any of the documents since using Smuggler to import the data.
I need to find out how to gather relevant information to help resolve this issue.
The server has been running, on and off, for the past couple of days and never seems to complete indexing. The ProviderSearchIndex still shows that it is stale. No changes have been made to any of the documents since using Smuggler to import the data.
I need to find out how to gather relevant information to help resolve this issue.
Paul Hinett
Mar 8, 2012, 8:00:56 PM3/8/12
to rav...@googlegroups.com
I think I have been noticing a similar issue.
I index around 400,000 documents, takes about 20 minutes or so.
However my indexing never seems to complete, it stuck in stale state. I’m not sure if it actually is still indexing or stuck though, where would I look to find this out, can you see just how much indexing work is left to finish anywhere, would be pretty handy.
Paul
spokey
Mar 8, 2012, 8:25:54 PM3/8/12
to rav...@googlegroups.com
Shorty after my previous post, the indexing finished (> 8 hrs total) for 1 Million documents.
Not that it is relevant, but I noticed that there were several of the following exceptions in the log just before it finished:
2012-03-08 16:46:04.3217,Raven.Database.Server.HttpServer,Error,"Failed to properly handle error, further error handling is ignored","System.Net.HttpListenerException (0x80004005): An operation was attempted on a nonexistent network connection
at System.Net.HttpResponseStream.Write(Byte[] buffer, Int32 offset, Int32 size)
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
at System.IO.StreamWriter.Write(String value)
at Newtonsoft.Json.Utilities.JavaScriptUtils.WriteEscapedJavaScriptString(TextWriter writer, String s, Char delimiter, Boolean appendDelimiters) in d:\Development\Releases\Json\Working\Src\Newtonsoft.Json\Utilities\JavaScriptUtils.cs:line 120
at Newtonsoft.Json.JsonTextWriter.WritePropertyName(String name) in d:\Development\Releases\Json\Working\Src\Newtonsoft.Json\JsonTextWriter.cs:line 206
at Raven.Json.Linq.RavenJObject.WriteTo(JsonWriter writer, JsonConverter[] converters) in c:\Builds\RavenDB-Stable\Raven.Abstractions\Json\Linq\RavenJObject.cs:line 253
at Raven.Json.Linq.RavenJArray.WriteTo(JsonWriter writer, JsonConverter[] converters) in c:\Builds\RavenDB-Stable\Raven.Abstractions\Json\Linq\RavenJArray.cs:line 167
at Raven.Json.Linq.RavenJObject.WriteTo(JsonWriter writer, JsonConverter[] converters) in c:\Builds\RavenDB-Stable\Raven.Abstractions\Json\Linq\RavenJObject.cs:line 257
at Raven.Json.Linq.RavenJArray.WriteTo(JsonWriter writer, JsonConverter[] converters) in c:\Builds\RavenDB-Stable\Raven.Abstractions\Json\Linq\RavenJArray.cs:line 167
at Raven.Database.Extensions.HttpExtensions.WriteJson(IHttpContext context, RavenJToken obj) in c:\Builds\RavenDB-Stable\Raven.Database\Extensions\HttpExtensions.cs:line 107
Not that it is relevant, but I noticed that there were several of the following exceptions in the log just before it finished:
2012-03-08 16:46:04.3217,Raven.Database.Server.HttpServer,Error,"Failed to properly handle error, further error handling is ignored","System.Net.HttpListenerException (0x80004005): An operation was attempted on a nonexistent network connection
at System.Net.HttpResponseStream.Write(Byte[] buffer, Int32 offset, Int32 size)
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
at System.IO.StreamWriter.Write(String value)
at Newtonsoft.Json.Utilities.JavaScriptUtils.WriteEscapedJavaScriptString(TextWriter writer, String s, Char delimiter, Boolean appendDelimiters) in d:\Development\Releases\Json\Working\Src\Newtonsoft.Json\Utilities\JavaScriptUtils.cs:line 120
at Newtonsoft.Json.JsonTextWriter.WritePropertyName(String name) in d:\Development\Releases\Json\Working\Src\Newtonsoft.Json\JsonTextWriter.cs:line 206
at Raven.Json.Linq.RavenJObject.WriteTo(JsonWriter writer, JsonConverter[] converters) in c:\Builds\RavenDB-Stable\Raven.Abstractions\Json\Linq\RavenJObject.cs:line 253
at Raven.Json.Linq.RavenJArray.WriteTo(JsonWriter writer, JsonConverter[] converters) in c:\Builds\RavenDB-Stable\Raven.Abstractions\Json\Linq\RavenJArray.cs:line 167
at Raven.Json.Linq.RavenJObject.WriteTo(JsonWriter writer, JsonConverter[] converters) in c:\Builds\RavenDB-Stable\Raven.Abstractions\Json\Linq\RavenJObject.cs:line 257
at Raven.Json.Linq.RavenJArray.WriteTo(JsonWriter writer, JsonConverter[] converters) in c:\Builds\RavenDB-Stable\Raven.Abstractions\Json\Linq\RavenJArray.cs:line 167
at Raven.Database.Extensions.HttpExtensions.WriteJson(IHttpContext context, RavenJToken obj) in c:\Builds\RavenDB-Stable\Raven.Database\Extensions\HttpExtensions.cs:line 107
spokey
Mar 9, 2012, 6:04:55 PM3/9/12
to rav...@googlegroups.com
I'm not sure whether or not this problem is related to my indexing issues (I suspect not), but when I inserted 10 (randomly generated) documents into RavenDB,
my index remained permanently stale. The log file keeps repeating the following over and over again:
2012-03-09 12:08:57.4947,Raven.Database.Indexing.Index.Indexing,Debug,"Indexing on QJZQRTXQVRUQC result in index ProviderSearchIndex gave document: __document_id IS: qjzqrtxqvruqc
NetworksPto_Network_Identifier I-: RENHBEXDJCAWJVZKW
NetworksPto_Network_InternalId I-: FYMIXKMJZILTI
NetworksPto_EffectiveFrom I-: 20120313120103461
NetworksPto_EffectiveThrough I-: 20120309120103461
NetworkSetsPto_NetworkSet_Identifier I-: HPOYT
NetworkSetsPto_NetworkSet_InternalId I-: LQFLNVXKH
NetworkSetsPto_EffectiveFrom I-: 20120316120103462
NetworkSetsPto_EffectiveThrough I-: 20120311120103462
CategoriesPto_Category_InternalId I-: 91960
CategoriesPto_Category_InternalId_Range I-: 91960
CategoriesPto_Category_Identifier I-: PAFKE
CategoriesPto_EffectiveFrom I-: 20120311120103459
CategoriesPto_EffectiveThrough I-: 20120310120103459
SpecialtiesPto_Specialty_Identifier I-: BYOWXUVGVJYPXXCI
SpecialtiesPto_Specialty_InternalId I-: 81262
SpecialtiesPto_Specialty_InternalId_Range I-: 81262
SpecialtiesPto_EffectiveFrom I-: 20120313120103485
SpecialtiesPto_EffectiveThrough I-: 20120312120103485
ProviderIdentifiersPto_ConcatTypeAndIdentifier I-: 21431|UCRMETQHHKPZXBR
ProviderIdentifiersPto_EffectiveFrom I-: 20120312120103487
ProviderIdentifiersPto_EffectiveThrough I-: 20120318120103487
Gender_Identifier I-: ORGVOXIUUMVCQTRFC
Gender_InternalId I-: 72289
Gender_InternalId_Range I-: 72289
ProviderType_Identifier I-: OTA
ProviderType_InternalId I-: 98628
ProviderType_InternalId_Range I-: 98628
NamesPto_FirstName I-: CCCWUKCZY
NamesPto_LastOrOrganizationalName I-: ZCALLCECAXCPISJUOL
NamesPto_NullName I-: NULL_VALUE
LanguagesPto_Language_Identifier_IsArray IS: true
LanguagesPto_Language_Identifier I-: H
LanguagesPto_Language_Identifier I-: HI
LanguagesPto_Language_InternalId_IsArray IS: true
LanguagesPto_Language_InternalId I-: 47941
LanguagesPto_Language_InternalId_Range I-: 47941
LanguagesPto_Language_InternalId I-: 32675
LanguagesPto_Language_InternalId_Range I-: 32675
AddressesPto_EffectiveFrom I-: 20120309120103458
AddressesPto_EffectiveThrough I-: 20120310120103458
AddressesPto_Address_Address1 I-: ORHRHEPPTPKUUTMOOYV
AddressesPto_Address_Address2 I-: E
AddressesPto_Address_City I-: JGNAUTRSYEV
AddressesPto_Address_State I-: DHBVRZBTTGQOUNFL
AddressesPto_Address_Zip I-: BDNUZHBEGJVXAQGZZ
AddressesPto_Address_AddressTypePto I-: ProviderAddress
PracticeOfficesPto_EffectiveFrom I-: 20120312120103481
PracticeOfficesPto_EffectiveThrough I-: 20120315120103482
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_AddressEffectiveFrom I-: 20120315120103481
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_AddressEffectiveThrough I-: 20120311120103481
IsPrimary I-: false
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_ContactEffectiveFrom I-: 20120313120103481
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_ContactEffectiveThrough I-: 20120312120103481
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Address1 I-: GCACNJWDNCVWHNI
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Address2 I-: KWUOJZFWJP
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_City I-: FGFIJHRSP
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_State I-: MDYMUQDPOMAXM
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Zip I-: GULJHXDSYYRWHTL
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_AddressTypePto I-: ProviderAddress
PayTosPto_EffectiveFrom I-: 20120314120103493
PayTosPto_EffectiveThrough I-: 20120315120103493
PayTosPto_PayTo_PayToIdentifiersPto_ConcatTypeAndIdentifier I-: 18648|LOOKMUTBNAPXRTFEER
PayTosPto_PayTo_PayToIdentifiersPto_EffectiveFrom I-: 20120318120103492
PayTosPto_PayTo_PayToIdentifiersPto_EffectiveThrough I-: 20120317120103492
PayTosPto_PayTo_PrimaryContact_City I-: KIOIOMYC
PayTosPto_PayTo_PrimaryContact_State I-: CTCZRZ
PayTosPto_PayTo_PrimaryContact_Zip I-: C
PayTosPto_PayTo_PrimaryContact_Address1 I-: YSXODWAHWGUNHCTQNG
PayTosPto_PayTo_PrimaryContact_Address2 I-: K
PayTosPto_PayTo_PrimaryContact_AddressTypePto I-: ProviderAddress
",
2012-03-09 12:08:57.4947,Raven.Database.Indexing.Index.Indexing,Debug,"Indexing on QJZQRTXQVRUQC result in index ProviderSearchIndex gave document: __document_id IS: qjzqrtxqvruqc
NetworksPto_Network_Identifier I-: RENHBEXDJCAWJVZKW
NetworksPto_Network_InternalId I-: FYMIXKMJZILTI
NetworksPto_EffectiveFrom I-: 20120313120103461
etc...
The issue is repeatable; There is a backup of the database at:
https://docs.google.com/open?id=0BynL9_7fHqysaEJ2enJMNDBTM2U3M3BZR3dIMjg3Zw
If you restore this database (using build 702), the problem should manifest itself.
my index remained permanently stale. The log file keeps repeating the following over and over again:
2012-03-09 12:08:57.4947,Raven.Database.Indexing.Index.Indexing,Debug,"Indexing on QJZQRTXQVRUQC result in index ProviderSearchIndex gave document: __document_id IS: qjzqrtxqvruqc
NetworksPto_Network_Identifier I-: RENHBEXDJCAWJVZKW
NetworksPto_Network_InternalId I-: FYMIXKMJZILTI
NetworksPto_EffectiveFrom I-: 20120313120103461
NetworksPto_EffectiveThrough I-: 20120309120103461
NetworkSetsPto_NetworkSet_Identifier I-: HPOYT
NetworkSetsPto_NetworkSet_InternalId I-: LQFLNVXKH
NetworkSetsPto_EffectiveFrom I-: 20120316120103462
NetworkSetsPto_EffectiveThrough I-: 20120311120103462
CategoriesPto_Category_InternalId I-: 91960
CategoriesPto_Category_InternalId_Range I-: 91960
CategoriesPto_Category_Identifier I-: PAFKE
CategoriesPto_EffectiveFrom I-: 20120311120103459
CategoriesPto_EffectiveThrough I-: 20120310120103459
SpecialtiesPto_Specialty_Identifier I-: BYOWXUVGVJYPXXCI
SpecialtiesPto_Specialty_InternalId I-: 81262
SpecialtiesPto_Specialty_InternalId_Range I-: 81262
SpecialtiesPto_EffectiveFrom I-: 20120313120103485
SpecialtiesPto_EffectiveThrough I-: 20120312120103485
ProviderIdentifiersPto_ConcatTypeAndIdentifier I-: 21431|UCRMETQHHKPZXBR
ProviderIdentifiersPto_EffectiveFrom I-: 20120312120103487
ProviderIdentifiersPto_EffectiveThrough I-: 20120318120103487
Gender_Identifier I-: ORGVOXIUUMVCQTRFC
Gender_InternalId I-: 72289
Gender_InternalId_Range I-: 72289
ProviderType_Identifier I-: OTA
ProviderType_InternalId I-: 98628
ProviderType_InternalId_Range I-: 98628
NamesPto_FirstName I-: CCCWUKCZY
NamesPto_LastOrOrganizationalName I-: ZCALLCECAXCPISJUOL
NamesPto_NullName I-: NULL_VALUE
LanguagesPto_Language_Identifier_IsArray IS: true
LanguagesPto_Language_Identifier I-: HI
LanguagesPto_Language_InternalId_IsArray IS: true
LanguagesPto_Language_InternalId I-: 47941
LanguagesPto_Language_InternalId_Range I-: 47941
LanguagesPto_Language_InternalId I-: 32675
LanguagesPto_Language_InternalId_Range I-: 32675
AddressesPto_EffectiveFrom I-: 20120309120103458
AddressesPto_EffectiveThrough I-: 20120310120103458
AddressesPto_Address_Address1 I-: ORHRHEPPTPKUUTMOOYV
AddressesPto_Address_Address2 I-: E
AddressesPto_Address_City I-: JGNAUTRSYEV
AddressesPto_Address_State I-: DHBVRZBTTGQOUNFL
AddressesPto_Address_Zip I-: BDNUZHBEGJVXAQGZZ
AddressesPto_Address_AddressTypePto I-: ProviderAddress
PracticeOfficesPto_EffectiveFrom I-: 20120312120103481
PracticeOfficesPto_EffectiveThrough I-: 20120315120103482
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_AddressEffectiveFrom I-: 20120315120103481
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_AddressEffectiveThrough I-: 20120311120103481
IsPrimary I-: false
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_ContactEffectiveFrom I-: 20120313120103481
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_ContactEffectiveThrough I-: 20120312120103481
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Address1 I-: GCACNJWDNCVWHNI
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Address2 I-: KWUOJZFWJP
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_City I-: FGFIJHRSP
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_State I-: MDYMUQDPOMAXM
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_Zip I-: GULJHXDSYYRWHTL
PracticeOfficesPto_PracticeOffice_PracticeOfficeContactsPto_Address_AddressTypePto I-: ProviderAddress
PayTosPto_EffectiveFrom I-: 20120314120103493
PayTosPto_EffectiveThrough I-: 20120315120103493
PayTosPto_PayTo_PayToIdentifiersPto_ConcatTypeAndIdentifier I-: 18648|LOOKMUTBNAPXRTFEER
PayTosPto_PayTo_PayToIdentifiersPto_EffectiveFrom I-: 20120318120103492
PayTosPto_PayTo_PayToIdentifiersPto_EffectiveThrough I-: 20120317120103492
PayTosPto_PayTo_PrimaryContact_City I-: KIOIOMYC
PayTosPto_PayTo_PrimaryContact_State I-: CTCZRZ
PayTosPto_PayTo_PrimaryContact_Zip I-: C
PayTosPto_PayTo_PrimaryContact_Address1 I-: YSXODWAHWGUNHCTQNG
PayTosPto_PayTo_PrimaryContact_Address2 I-: K
PayTosPto_PayTo_PrimaryContact_AddressTypePto I-: ProviderAddress
",
2012-03-09 12:08:57.4947,Raven.Database.Indexing.Index.Indexing,Debug,"Indexing on QJZQRTXQVRUQC result in index ProviderSearchIndex gave document: __document_id IS: qjzqrtxqvruqc
NetworksPto_Network_Identifier I-: RENHBEXDJCAWJVZKW
NetworksPto_Network_InternalId I-: FYMIXKMJZILTI
NetworksPto_EffectiveFrom I-: 20120313120103461
etc...
The issue is repeatable; There is a backup of the database at:
https://docs.google.com/open?id=0BynL9_7fHqysaEJ2enJMNDBTM2U3M3BZR3dIMjg3Zw
If you restore this database (using build 702), the problem should manifest itself.
Oren Eini (Ayende Rahien)
Mar 11, 2012, 6:40:34 AM3/11/12
to rav...@googlegroups.com
This isn't related, it is just a network connection error from a client that was shut down mid response.
Oren Eini (Ayende Rahien)
Mar 11, 2012, 7:43:05 AM3/11/12
to rav...@googlegroups.com
Okay, this is really strange.
What does your index looks like in code, not the way it looks in RavenDB, but the actual code itself.
Note that the _index definition alone_ is 16KB in size, which is pretty huge.
It is hard to follow, but it looks like you are doing a Cartesean product between all the collection properties that you have there.
Just from tracking things, it seems like the 10 docs you have there generate several tens of thousands of index entries.
Reply all
Reply to author
Forward
0 new messages