peter.neuba...@neotechnology.com> wrote:
> Marko,
> since the BatchInserter is more of a unsafe utility to use in initial
> imports and has far less guarantees than the full API, we moved it to
> Does that mean that maybe after a few more releases it won't be supported anymore??
> Cheers,
> Pablo
> On Sat, Apr 28, 2012 at 3:54 PM, Peter Neubauer <peter.neuba...@neotechnology.com> wrote:
> Marko,
> since the BatchInserter is more of a unsafe utility to use in initial
> imports and has far less guarantees than the full API, we moved it to
> https://github.com/neo4j/community/tree/master/kernel/src/main/java/o... > where it better signals the risks you are taking ;)
Ok.
By the way, do you have any numbers of how much is actually the difference
(in terms of insertion time) between the standard API and the Batch
insertion mode?
michael.hun...@neotechnology.com> wrote:
> Right now that's not on the horizon. It was only deprecating the code in
> the old package and having it delegate to the place in the new package.
> Michael
> Am 28.04.2012 um 22:20 schrieb Pablo Pareja:
> Does that mean that maybe after a few more releases it won't be supported
> anymore??
> Cheers,
> Pablo
> On Sat, Apr 28, 2012 at 3:54 PM, Peter Neubauer <
> peter.neuba...@neotechnology.com> wrote:
>> Marko,
>> since the BatchInserter is more of a unsafe utility to use in initial
>> imports and has far less guarantees than the full API, we moved it to
- tx size
- do you add to indexes
- do you query indexes during insert
- available memory
- disk performance (for tx-flushes)
- transactions have to be kept in memory (thread local) the use much more concurrency aware datastructures (e.g. CHM) and they must be flushed to disk during commit
I had no issues inserting 3M nodes/s with the batch inserter on my Mac.
> Ok.
> By the way, do you have any numbers of how much is actually the difference (in terms of insertion time) between the standard API and the Batch insertion mode?
> Pablo
> On Sat, Apr 28, 2012 at 4:23 PM, Michael Hunger <michael.hun...@neotechnology.com> wrote:
> Right now that's not on the horizon. It was only deprecating the code in the old package and having it delegate to the place in the new package.
> Michael
> Am 28.04.2012 um 22:20 schrieb Pablo Pareja:
>> Does that mean that maybe after a few more releases it won't be supported anymore??
>> Cheers,
>> Pablo
>> On Sat, Apr 28, 2012 at 3:54 PM, Peter Neubauer <peter.neuba...@neotechnology.com> wrote:
>> Marko,
>> since the BatchInserter is more of a unsafe utility to use in initial
>> imports and has far less guarantees than the full API, we moved it to
>> https://github.com/neo4j/community/tree/master/kernel/src/main/java/o... >> where it better signals the risks you are taking ;)
- tx size --> (It would be very variable mainly because of the index
retrieving/flushing operations)
- do you add to indexes --> YES
- do you query indexes during insert --> YES
- available memory --> I could have up to 32 GB RAM
- disk performance (for tx-flushes) --> I don't know....
michael.hun...@neotechnology.com> wrote:
> depends on many factors:
> - tx size
> - do you add to indexes
> - do you query indexes during insert
> - available memory
> - disk performance (for tx-flushes)
> - transactions have to be kept in memory (thread local) the use much more
> concurrency aware datastructures (e.g. CHM) and they must be flushed to
> disk during commit
> I had no issues inserting 3M nodes/s with the batch inserter on my Mac.
> Am 28.04.2012 um 22:32 schrieb Pablo Pareja:
> Ok.
> By the way, do you have any numbers of how much is actually the difference
> (in terms of insertion time) between the standard API and the Batch
> insertion mode?
> Pablo
> On Sat, Apr 28, 2012 at 4:23 PM, Michael Hunger <
> michael.hun...@neotechnology.com> wrote:
>> Right now that's not on the horizon. It was only deprecating the code in
>> the old package and having it delegate to the place in the new package.
>> Michael
>> Am 28.04.2012 um 22:20 schrieb Pablo Pareja:
>> Does that mean that maybe after a few more releases it won't be supported
>> anymore??
>> Cheers,
>> Pablo
>> On Sat, Apr 28, 2012 at 3:54 PM, Peter Neubauer <
>> peter.neuba...@neotechnology.com> wrote:
>>> Marko,
>>> since the BatchInserter is more of a unsafe utility to use in initial
>>> imports and has far less guarantees than the full API, we moved it to
perhaps you can find the time to write a quick data generator (in memory no file reading) and test it with the batch-inserter and the core-API and publish it as a GH project (and blog post)?
> - tx size --> (It would be very variable mainly because of the index retrieving/flushing operations)
> - do you add to indexes --> YES
> - do you query indexes during insert --> YES
> - available memory --> I could have up to 32 GB RAM
> - disk performance (for tx-flushes) --> I don't know....
> Pablo
> On Sat, Apr 28, 2012 at 4:39 PM, Michael Hunger <michael.hun...@neotechnology.com> wrote:
> depends on many factors:
> - tx size
> - do you add to indexes
> - do you query indexes during insert
> - available memory
> - disk performance (for tx-flushes)
> - transactions have to be kept in memory (thread local) the use much more concurrency aware datastructures (e.g. CHM) and they must be flushed to disk during commit
> I had no issues inserting 3M nodes/s with the batch inserter on my Mac.
> Am 28.04.2012 um 22:32 schrieb Pablo Pareja:
>> Ok.
>> By the way, do you have any numbers of how much is actually the difference (in terms of insertion time) between the standard API and the Batch insertion mode?
>> Pablo
>> On Sat, Apr 28, 2012 at 4:23 PM, Michael Hunger <michael.hun...@neotechnology.com> wrote:
>> Right now that's not on the horizon. It was only deprecating the code in the old package and having it delegate to the place in the new package.
>> Michael
>> Am 28.04.2012 um 22:20 schrieb Pablo Pareja:
>>> Does that mean that maybe after a few more releases it won't be supported anymore??
>>> Cheers,
>>> Pablo
>>> On Sat, Apr 28, 2012 at 3:54 PM, Peter Neubauer <peter.neuba...@neotechnology.com> wrote:
>>> Marko,
>>> since the BatchInserter is more of a unsafe utility to use in initial
>>> imports and has far less guarantees than the full API, we moved it to
>>> https://github.com/neo4j/community/tree/master/kernel/src/main/java/o... >>> where it better signals the risks you are taking ;)
> perhaps you can find the time to write a quick data generator (in memory
> no file reading) and test it with the batch-inserter and the core-API and
> publish it as a GH project (and blog post)?
> That would be awesome.
> Cheers
> Michael
> Am 28.04.2012 um 22:48 schrieb Pablo Pareja:
> In my case:
> - tx size --> (It would be very variable mainly because of the index
> retrieving/flushing operations)
> - do you add to indexes --> YES
> - do you query indexes during insert --> YES
> - available memory --> I could have up to 32 GB RAM
> - disk performance (for tx-flushes) --> I don't know....
> Pablo
> On Sat, Apr 28, 2012 at 4:39 PM, Michael Hunger <
> michael.hun...@neotechnology.com> wrote:
>> depends on many factors:
>> - tx size
>> - do you add to indexes
>> - do you query indexes during insert
>> - available memory
>> - disk performance (for tx-flushes)
>> - transactions have to be kept in memory (thread local) the use much more
>> concurrency aware datastructures (e.g. CHM) and they must be flushed to
>> disk during commit
>> I had no issues inserting 3M nodes/s with the batch inserter on my Mac.
>> Am 28.04.2012 um 22:32 schrieb Pablo Pareja:
>> Ok.
>> By the way, do you have any numbers of how much is actually the
>> difference (in terms of insertion time) between the standard API and the
>> Batch insertion mode?
>> Pablo
>> On Sat, Apr 28, 2012 at 4:23 PM, Michael Hunger <
>> michael.hun...@neotechnology.com> wrote:
>>> Right now that's not on the horizon. It was only deprecating the code in
>>> the old package and having it delegate to the place in the new package.
>>> Michael
>>> Am 28.04.2012 um 22:20 schrieb Pablo Pareja:
>>> Does that mean that maybe after a few more releases it won't be
>>> supported anymore??
>>> Cheers,
>>> Pablo
>>> On Sat, Apr 28, 2012 at 3:54 PM, Peter Neubauer <
>>> peter.neuba...@neotechnology.com> wrote:
>>>> Marko,
>>>> since the BatchInserter is more of a unsafe utility to use in initial
>>>> imports and has far less guarantees than the full API, we moved it to
