Best way to convert existing Sqlite database to Realm?

[Abbas Mousavi]

What is the best way to convert an existing Sqlite database to Realm? is there any tool or script for this?

And my other question is: Does Realm compresses data? Is there any comparison regarding size of same data in Sqlite and Realm?

[JP Simard]

Hi Abbas, there's no tool to automatically convert a SQLite database to Realm, but we do plan on releasing such a tool in the near future.

In the meantime, you could easily adapt the RealmJSONImportExample to read from a SQLite database instead of a JSON file.

And Realm doesn't compress its data, which would require additional CPU time, power and disk space. However, Realm leverages bit-packing and smart column optimizations to be generally ~50% smaller than SQLite.

Let me know how how that goes for you and feel free to ask more questions here.

JP

On Friday, July 18, 2014, Abbas Mousavi <abbasm...@gmail.com> wrote:

What is the best way to convert an existing Sqlite database to Realm? is there any tool or script for this?

And my other question is: Does Realm compresses data? Is there any comparison regarding size of same data in Sqlite and Realm?

--
You received this message because you are subscribed to the Google Groups "Realm" group.
To unsubscribe from this group and stop receiving emails from it, send an email to realm-users...@googlegroups.com.
To post to this group, send email to realm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/realm-users/45efcf75-39b7-4dec-a5dc-9520ce67044d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Abbas Mousavi]

Thanks JP

This is my sample code to convert an Sqlite database

// Define your models

@interface Word : RLMObject

@property NSInteger _id;

@property NSString *word;

@property NSString *definition;

@end

@implementation Word

@end

RLM_ARRAY_TYPE(Word)

@implementation AppDelegate

- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions

{

    // Override point for customization after application launch.

    

  

    

    RLMRealm *realm = [RLMRealm defaultRealm];

    NSString * path = [[NSBundle mainBundle] pathForResource:@"1" ofType:@"db"];

    FMDatabase * db = [FMDatabase databaseWithPath:path];

    [db open];

    NSString * query = [NSString stringWithFormat:@"select * from Words"];

    

    FMResultSet *s = [db executeQuery:query];

    

   [realm beginWriteTransaction];

    while ([s next]) {

        Word * word = [[Word alloc] init];

        word._id = [s intForColumn:@"_id"];

        word.word = [s stringForColumn:@"word"];

        word.definition = [s stringForColumn:@"explanation"];

        [realm addObject:word];

    }

    [realm commitWriteTransaction];

    NSLog(@"done");

    RLMArray *words = [Word objectsWhere:@"_id = 203"];

    NSLog(@"%@", words);

    

    return YES;

}

but there is some problems. the source database has 50k rows, the destination realm has exactly same number of rows but 

the max id in _id column is 500 (instead of 50k). many rows are repetitions, and many fields are empty, also many strings are truncated from random positions.

What is wrong in the code?

On Saturday, July 19, 2014 1:43:50 AM UTC+4:30, JP Simard wrote:

Hi Abbas, there's no tool to automatically convert a SQLite database to Realm, but we do plan on releasing such a tool in the near future.

In the meantime, you could easily adapt the RealmJSONImportExample to read from a SQLite database instead of a JSON file.

And Realm doesn't compress its data, which would require additional CPU time, power and disk space. However, Realm leverages bit-packing and smart column optimizations to be generally ~50% smaller than SQLite.

Let me know how how that goes for you and feel free to ask more questions here.

JP

On Friday, July 18, 2014, Abbas Mousavi <abbasm...@gmail.com> wrote:

What is the best way to convert an existing Sqlite database to Realm? is there any tool or script for this?

And my other question is: Does Realm compresses data? Is there any comparison regarding size of same data in Sqlite and Realm?

--
You received this message because you are subscribed to the Google Groups "Realm" group.

To unsubscribe from this group and stop receiving emails from it, send an email to realm-users+unsubscribe@googlegroups.com.

[Abbas Mousavi]

I have changed main.m of RealmJSONImportExample to this:

#import <Foundation/Foundation.h>

#import <Realm/Realm.h>

#import "FMDatabase.h"

// Define your models

@interface Word : RLMObject

@property NSInteger _id;

@property NSString *word;

@property NSString *definition;

@end

@implementation Word

// No need for implementation

@end

RLM_ARRAY_TYPE(Word)

int main(int argc, const char * argv[])

{

    @autoreleasepool {

       

        NSMutableArray * array = [[NSMutableArray alloc] init];

        

        RLMRealm *realm = [RLMRealm defaultRealm];

        

        NSString * path = [[NSBundle mainBundle] pathForResource:@"1" ofType:@"db"];

        

        FMDatabase * db = [FMDatabase databaseWithPath:path];

        

        [db open];

        

        NSLog(@"%@", db.databasePath);

        

        

        NSString * query = [NSString stringWithFormat:@"select * from Words"];

        

        FMResultSet *s = [db executeQuery:query];

        

        

       

        

        while ([s next]) {

            

            

            

            Word * word = [[Word alloc] init];

            word._id = [s intForColumn:@"_id"];

            word.word = [s stringForColumn:@"word"];

            word.definition = [s stringForColumn:@"explanation"];

            

            

            //NSLog(@"id: %ld", (long)word._id);

            [array addObject:word];

            

            

            

        }

        

        

        NSLog(@"done1");

        

        [realm beginWriteTransaction];

        [realm addObjectsFromArray:array];

        [realm commitWriteTransaction];

        NSLog(@"done2");

        

        

        

        

        RLMArray *words = [Word objectsWhere:@"_id = 203"];

        

        

        

        NSLog(@"%@", words);

       

        

    }

    return 0;

}

now it works and converts the database correctly. but it is very very slow. It takes more than 15 minutes on my MacBook Pro to convert a 35 MB sqlite database with 50k row.

[Ari Lazier]

Is that 15 minutes to run the whole process or is that just the time between NSLog(@"done1") and NSLog(@"done2")? It would be cool to see the times for each log message/ie the splits for each step.

It definitely shouldn't take this long to insert 50k rows - make sure you are running as release as the debug version should be significantly slower.

Ari

--
You received this message because you are subscribed to the Google Groups "Realm" group.

To unsubscribe from this group and stop receiving emails from it, send an email to realm-users...@googlegroups.com.

To post to this group, send email to realm...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/realm-users/ca39d344-7ab2-4ace-b482-9ab34ef00fa2%40googlegroups.com.

[Abbas Mousavi]

Thanks Ari

Using release mode the time drops under 2 minutes for 50k inserts.

regarding size, my data in CSV is about 35 MB, in sqlite it is about 37 MB and in Realm it is about 39 MB, the data is mostly text.

[Brian Munkholm]

Hi Abbas,
We do pack numbers, so you would see the biggest "compression" on databases with integer attributes. We also have a similar feature for strings which hasn't been enabled in ObjC yet, but will be relatively shortly. You should then see the size reduced.

//Brian

[Abbas Mousavi]

Hi Brian

 Just asking if that "text compression" feature has been enabled for objective C in realm as of version 0.88?

[Brian Munkholm]

Hi,

Not yet.
The file will always contain some free unused space. But we will make a "compact" feature available that can help you remove that unused space.  That's mostly useful if your database at some point is not written to anymore. Once you do start to add stuff the file will grow again. That has currently been added to Android.

Cheers

Brian

--
You received this message because you are subscribed to the Google Groups "Realm" group.

To unsubscribe from this group and stop receiving emails from it, send an email to realm-cocoa...@googlegroups.com.

To post to this group, send email to realm...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/realm-cocoa/2b88ae25-7998-41e1-93e0-0d5f743e0132%40googlegroups.com.

[Brian Munkholm]

The size of the database can be reduced with 0-50% roughly. The size of the database is doubled when free space runs out. So just adding one object can be enough to double the database size. In that situation you can almost reduce the file with 50%.

Actually I remembered incorrectly - is has already been added in the latest release, it's just called writeCopyToPath:error.

Cheers

Brian

On Tue, Dec 16, 2014 at 1:51 PM, Abbas Mousavi <abbasm...@gmail.com> wrote:

I want to use realm to store a huge dictionary of Persian language, so there is no write after some point, but I want database to be small and very fast. Can you estimate the time for this "compact" feature to be ready? and can you estimate the impact of this "compacting" on the size of database?

[JP Simard]

Hi Abbas, we still haven't exposed string compression in Realm, but we now have a method to make a compacted copy of the realm, which usually makes it significantly smaller. See its documentation at -[RLMRealm writeCopyToPath:].

--
JP Simard

--
You received this message because you are subscribed to the Google Groups "Realm" group.
To unsubscribe from this group and stop receiving emails from it, send an email to realm-cocoa...@googlegroups.com.
To post to this group, send email to realm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/realm-cocoa/2b88ae25-7998-41e1-93e0-0d5f743e0132%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

{#HS:60770433-180#}