Stability 4.0 Nightly (20-10-2017)

85 views
Skip to first unread message

Derek den Haas

unread,
Oct 21, 2017, 3:51:36 AM10/21/17
to RavenDB - 2nd generation document database
1. When patching, with only 1 index:

Processed: 31.744 / 114.907
It crashed

Last line:
 "2017-10-20T18:05:05.0842554Z, 63, Operations, A, Raven.Server.ServerWide.Maintenance.ClusterObserver, It appears that all nodes of the EasyFlor-ef-area51 database are not responding to the supervisor, the database is not reachable,
which is there a lot of times, no prev lines indicating it was going to crash, Kubernetes reports: Terminated: OOMKilled. Database is now crashing all the time (14 restarts).

2. Indexing is a lot stricter, let's say I had the document:
Purchase
- ArticleId
- Quantity
- Price

I added the name and description from the ArticleId in a later version:
Purchase
- Name (Dictionary to accommodate multiple languages)
- Description (Dictionary to accommodate multiple languages)
- ArticleId
- Quantity
- Price

Now I have 2 types of documents laying around, without and with Name and Description. Since it's old data, I'm fine with that, also the index next was working fine in 3.5:
Map = from a in Purchases
let language = LoadDocument("Config/Language") // Which is the default language for the company
select new {
    Name = a.Name[language.Code],
    Description = a.Description[language.Code]
}

language.Code = enum = int.

Now in the new version it is complaining about:
Failed to execute mapping function on purchases/126767. Exception: Microsoft.CSharp.RuntimeBinder.RuntimeBinderException: The best overloaded method match for 'Sparrow.Json.LazyStringValue.this[int]' has some invalid arguments

Will this be editted? Or is the new engine more strict in handling data?

Oren Eini (Ayende Rahien)

unread,
Oct 22, 2017, 1:39:53 AM10/22/17
to ravendb
1) Is this reproducible? If so, can you send it to us?

2) The error is an issue, yes.  Can you send us a way to reproduce this? Anything that was indexed in 3.5 should be indexed in 4.0

In general, a nightly is whatever we have on github when the build runs. There is no check made except that it compiles, basically. 



Hibernating Rhinos Ltd  

Oren Eini l CEO Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 


--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Derek den Haas

unread,
Oct 24, 2017, 10:19:37 AM10/24/17
to RavenDB - 2nd generation document database
1) No, not yet.

2) See attached, ListWithRatingNullAsync should fail.


Op zondag 22 oktober 2017 07:39:53 UTC+2 schreef Oren Eini:
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.
-.cs

Derek den Haas

unread,
Oct 24, 2017, 10:59:18 AM10/24/17
to RavenDB - 2nd generation document database
1) First off, it shouldn't be the local OOM killer (the one that's running serverside, not in the Docker node)
It got 3 levels of QOS:
Best Efford:  oom_level: 1000
Burstable:    oom_level: 1000 - (1000 * dockerMemoryLimit) / serverMemoryLimit
Guaranteed:   oom_level: -998

Only excluded some things:
- Normally it is set to Burstable, which due to rounding, when assigning 2.00gB, the score of a 3gB server will be 666, which is 1.998gB RAM in which it might be killed
- Set it to Guaranteed, if I place a rather large index I still get a OOMKilled

Since the above 2 might indicate that it's due to low resources on the Server, I excluded that by adding it as only member on a 6gB server, which will still kill RavenDB.

So first some questions:
- What is the minimum of RAM RavenDB can run on?
-- I can see that when I only load 1DB it's already on 1.5gByte of memory
-- When restricting it to 1.5GB of RAM, every index I try to create, will kill the server
-- When set to 1GB of RAM, the database will not load, server will crash
-- When set to 2GB of RAM, the database will eventually fail (restarted 24 times in the last week)

Database: 2.000.000 records, 5 test indexes (non-stale), so I don't know where to go, if you say, it should remain stable on 6gB or RAM, I'll try that (or 3gB).



Op dinsdag 24 oktober 2017 16:19:37 UTC+2 schreef Derek den Haas:

Oren Eini (Ayende Rahien)

unread,
Oct 24, 2017, 4:01:33 PM10/24/17
to ravendb
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

Oren Eini (Ayende Rahien)

unread,
Oct 24, 2017, 4:03:19 PM10/24/17
to ravendb
We imported & indexed the entire stack overflow dataset, over 52GB of data on a Pi, with 1GB or RAM.

What do you mean, large index?

That said, we just fixed an issue related to responding to low memory in indexes (we reacted only when the memory was 100% full by mistake.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

Derek den Haas

unread,
Oct 25, 2017, 5:02:07 AM10/25/17
to RavenDB - 2nd generation document database
Wasn't the Pi running in a different mode? One of your programmers adviced in another thread that (I thought the 32bit mode) would run better with low specs. I will try and see if I can get it killed on the new version in a reproducible manner. On 1GB of memory my test database won't even load without crashing, and it's not even close to your 52GB of data.

Op dinsdag 24 oktober 2017 22:03:19 UTC+2 schreef Oren Eini:

Oren Eini (Ayende Rahien)

unread,
Oct 25, 2017, 5:34:58 AM10/25/17
to ravendb
Yes, the 32 bits mode is more conservative for memory usage.
set Storage.ForceUsing32BitsPager to true and it should help
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

Derek den Haas

unread,
Oct 25, 2017, 10:29:06 AM10/25/17
to RavenDB - 2nd generation document database
Ok, what would you say is the minimum amount of RAM to get things running? I'm more into multiple small servers instead of one large, or do you prefer 2 large servers with multiple (50) databases in stead of 10 smaller with 10 databases on each server (each database is stored twice).

Moreover, if it crashes on 2GB of memory, is this because it's actually unsupported, or should it work fine with just 2gb or should 2gb and lower always use the 32bit mode.

Op woensdag 25 oktober 2017 11:34:58 UTC+2 schreef Oren Eini:

Oren Eini (Ayende Rahien)

unread,
Oct 25, 2017, 3:19:30 PM10/25/17
to ravendb
It should work just fine on 64 bits on 1GB mode.

Can you send us a way to reproduce the crash? 
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

Derek den Haas

unread,
Oct 26, 2017, 4:33:44 AM10/26/17
to RavenDB - 2nd generation document database
I'll first update my docker to the latest release, and will then again work on a reliable way to reproduce it.

P.s. Great work (you and your team) on the Linq to RQL tickets, saw that the last ticket was merged 19 hours ago!

Op woensdag 25 oktober 2017 21:19:30 UTC+2 schreef Oren Eini:
Reply all
Reply to author
Forward
0 new messages