Direction for C# Sqlite.
Miguel de Icaza
Noah, I am wondering where you would like to take the project from
here. It is a fantastic thing to have Sqlite ported to C#. The
current downside is that it is very much a one-to-one mapping to C and
it does not take advantage of C# features available to it.
I would personally want to C#-ify the code, and I am wondering
how do you feel about that?
Some things that could be done:
* Replace manual memory management (strlen30, free, sqlite_sprintf and
others with the C# equivalents where possible).
* Avoid using the emulated C library functions and instead use
the .NET ones (for example there are many of those on src/shell.cs at
the bottom).
* Avoid using printf, but use Console.WriteLine instead.
* Use the "using" construct for automatically disposing objects.
* Avoid depending on #ifdef WIN32 for example to produce libraries
that are OS specific, and instead make those runtime detectable
features.
Noah
still 9 issues to be resolved. These really need to be looked at
first to confirm there is not some hidden flaw that will create major
problems later.
That being said, I'd like to start slowly.
This last weekend, I removed all calls to malloc and free. This
increased the speed in many cases. See http://code.google.com/p/csharp-sqlite/wiki/Benchmarks
for details
> * Replace manual memory management (strlen30, free, sqlite_sprintf and
> others with the C# equivalents where possible).
They caused real problems for me. So printf, sqlite_sprintf, etc are
good candidates for rewrite, esp since they are so easy to test.
> * Avoid using the emulated C library functions and instead use
> the .NET ones (for example there are many of those on src/shell.cs at
> the bottom).
trying to determine what the C# way was. Never got back to it, once
it was working. In fact, the whole shell.cs can be rewritten, as long
as we maintain equivalent functionality.
> * Avoid using printf, but use Console.WriteLine instead.
of the printf calls still exist and can be converted
> * Use the "using" construct for automatically disposing objects.
> * Avoid depending on #ifdef WIN32 for example to produce libraries
> that are OS specific, and instead make those runtime detectable
> features.
However, me need to distinguish between Vista, XP, Mono, Silverlight,
and that can be runtime detectable I'm sure
Noah
Phil Haack
I'd be happy to help out with the CSharpitization of the source as well as pitch in elsewhere. Of course I can't do this on work time at all, unlike Miguel there. :)
Is there an ADO.NET provider as part of this library? If not, I could ping the guy who does the ADO.NET Provider for Sqlite to see if he'd have interest in providing one for this. His provider works with the unmanaged SQLite.
Phil
Miguel de Icaza
C#ing the code makes sense. However, one concern is that there are
still 9 issues to be resolved. These really need to be looked at
first to confirm there is not some hidden flaw that will create major
problems later.
That being said, I'd like to start slowly.
This makes sense;
I think it would be very useful if we could at least fix the line-ending situation to either be always Unix-style or always be DOS-style; The current mix of DOS/Unix confuses Mercurial, Emacs and every other tool I use, and makes producing useful patches very hard.
This last weekend, I removed all calls to malloc and free. This
increased the speed in many cases. See http://code.google.com/p/csharp-sqlite/wiki/Benchmarks
for details
It seems like the old benchmarks and the new benchmarks use different measurements.
Yup ... shell.cs was one of my first rewrites into C#, and I still was
trying to determine what the C# way was. Never got back to it, once
it was working. In fact, the whole shell.cs can be rewritten, as long
as we maintain equivalent functionality.
Lovely to hear.
> * Use the "using" construct for automatically disposing objects.Not sure what you mean here...
When dealing with resources like file IO, I found parts in shell.cs that did things like:
try {
a = open file;
if (a != null) Dosomething
} catch {
} finally {
close a;
}
The using statement lets you automatically cope with disposing objects properly, like this:
using (a = fileopen ()){
DoSmoethign ();
}
The a is automatically closed when the using statement ends (there is an implicit finally added by the compiler plus a null check).
Good suggestion. We can rip out much of the unneeded detection code.
However, me need to distinguish between Vista, XP, Mono, Silverlight,
and that can be runtime detectable I'm sure
All of those we can detect.
Miguel
Miguel de Icaza
I'd be happy to help out with the CSharpitization of the source as well as pitch in elsewhere. Of course I can't do this on work time at all, unlike Miguel there. :)
Hah! I am actually not doing this on work time either. I just love the project.
Miguel
Phil Haack
I’ve wanted to use SQLite for Subtext for a long time. J
From: Miguel de Icaza
[mailto:miguel....@gmail.com]
Sent: Monday, August 10, 2009 10:41 AM
To: Phil Haack
Cc: Noah; C#-SQLite
Subject: Re: Direction for C# Sqlite.
Hello,
Jay Beavers
- Keep a "port" branch that is basically just the "C#-ized" C source. This is great to have both as a "how to port C to C#" reference as well as a useful tool for taking updated patches from the SQLite source tree.
- Create a "polish" branch that uses "C# best practices" including things like working on StyleCop and FxCop cleanliness. This will diverge significantly from the "port" branch quickly so keeping these two in sync will be tough.
- Both "port" and "polish" branch should stay compatible with SQLite-C by keeping the file format and the calling API the same.
- Move the System.Data.SQLite functionality (ADO.Net & Linq calling APIs) on top of the "polished" branch.
- Get all SQLite tests working.
- Get the test harness into mstest (or nunit) format to make it easier to verify correct functionality.
- Find and fix performance bugs to get the C# code within say 20% of the performance of the C code.
- Get the performance harness cleaned up a bit so we have a better "apples to apples" comparison of C# to C performance.
Phil Haack
Regarding the test harness. Do you mean the TCL tests? It seems to me that trying to port all the tests to a unit testing framework would make it hard to receive patches to the new tests. If that’s the route you want to go, I can help there. I know Miguel’s not a fan of unit testing. :P Though I’m partial to XUnit.NET rather than MS Test or NUnit, but it’s your choice. J
I tried to run the tests but I really don’t understand the TCL environment. Can anyone provide me a simple walkthrough, ideally on the Wiki?
Phil
Jay Beavers
Noah Hart
know TCL and needed to figure out what it was doing.
You can think of TCL as a powerful scripting language, which allow you
to define verbs and procedures, as well as call external programs
sqlite.org does all their unit testing via the tcl test harness, which
runs some canned scripts for various tests
Noah
Jay Beavers wrote:
> I'm trying to figure it out right now.
>
> On Mon, Aug 10, 2009 at 11:05 PM, Phil Haack <phi...@microsoft.com> wrote:
>
>> Regarding the test harness. Do you mean the TCL tests? It seems to me
>> that trying to port all the tests to a unit testing framework would make it
>> hard to receive patches to the new tests. If that’s the route you want to
>> go, I can help there. I know Miguel’s not a fan of unit testing. :P Though
>> I’m partial to XUnit.NET rather than MS Test or NUnit, but it’s your choice.
>> J
>>
>>
>>
>> I tried to run the tests but I really don’t understand the TCL environment.
>> Can anyone provide me a simple walkthrough, ideally on the Wiki?
>>
>>
>>
>> Phil
>>
>>
>>
>>
>>
>>
>>
>> *From:* csharp...@googlegroups.com [mailto:
>> csharp...@googlegroups.com] *On Behalf Of *Jay Beavers
>> *Sent:* Monday, August 10, 2009 9:37 PM
>> *To:* csharp...@googlegroups.com
>> *Subject:* RE: Direction for C# Sqlite.
>>
>>
>>
>> My opinion (take it for what it's worth):
>>
>>
>>
>> - Keep a "port" branch that is basically just the "C#-ized" C source.
>> This is great to have both as a "how to port C to C#" reference as well as
>> a useful tool for taking updated patches from the SQLite source tree.
>> - Create a "polish" branch that uses "C# best practices" including
>> things like working on StyleCop and FxCop cleanliness. This will diverge
>> significantly from the "port" branch quickly so keeping these two in sync
>> will be tough.
>> - Both "port" and "polish" branch should stay compatible with SQLite-C
>> by keeping the file format and the calling API the same.
>> - Move the System.Data.SQLite functionality (ADO.Net & Linq calling
>> APIs) on top of the "polished" branch.
>>
>> Before we consider the "polish" work, I think there are some fundamentals
>> that need to be ironed out first:
>>
>> - Get all SQLite tests working.
>> - Get the test harness into mstest (or nunit) format to make it easier
>> to verify correct functionality.
>> - Find and fix performance bugs to get the C# code within say 20% of
>> the performance of the C code.
>> - Get the performance harness cleaned up a bit so we have a better
>> "apples to apples" comparison of C# to C performance.
>>
>> We should also consider what it means to test for Silverlight, Moonlight,
>> and Mono compatibility breaks in addition to the desktop Windows CLR before
>> accepting checkins. I'm not certain what best practices we need to have in
>> place here.
>>
>>
>>
>> On Mon, Aug 10, 2009 at 12:11 PM, Phil Haack <phi...@microsoft.com> wrote:
>>
>> I’ve wanted to use SQLite for Subtext for a long time. J
>>
>>
>>
>> *From:* Miguel de Icaza [mailto:miguel....@gmail.com]
>> *Sent:* Monday, August 10, 2009 10:41 AM
>> *To:* Phil Haack
>> *Cc:* Noah; C#-SQLite
>>
>>
>> *Subject:* Re: Direction for C# Sqlite.
Jay Beavers
Phil Haack
Phil
Phil Haack
Ah! I see. I thought I could run the tests from the interactive environment. But you simply need to pass the file path to the exe. Why don’t we make that the default so that when you run the exe, it’ll pass in that path?
Tim Anderson (itwriting)
Sent: 11 August 2009 5:37 AM
> * Move the System.Data.SQLite functionality (ADO.Net & Linq calling APIs)
> on top of the "polished" branch.
I am not sure about the practicality of maintaining two ports.
I do think that ADO.Net and Linq wrappers should be in separate DLLs, or easily excluded, since "small" is a feature.
Tim
Max Kosenko
In case API will be same as SQLite it would be very simple to create
support of managed SQLite in System.Data.SQLite.
In fact - many project wouldn't use ADO and that's -50% of dll size.
Max
On Aug 11, 12:22 pm, "Tim Anderson (itwriting)" <t...@itwriting.com>
wrote:
Miguel de Icaza
> * Move the System.Data.SQLite functionality (ADO.Net & Linq calling APIs)
> on top of the "polished" branch.I am not sure about the practicality of maintaining two ports.
I do think that ADO.Net and Linq wrappers should be in separate DLLs, or easily excluded, since "small" is a feature
Phil Haack
I agree. As for the ADO.NET API and LINQ API’s, I think that ought to be part of this VS Solution, but as different class lib projects. It’d be nice to have a top level solution file that includes everything and makes it easy to figure out the starting point.
BTW, if we’ve come to any decisions on these things, let’s add issues to the issue tracker and I’d be happy to take some one.
Phil
From:
csharp...@googlegroups.com [mailto:csharp...@googlegroups.com] On
Behalf Of Miguel de Icaza
Sent: Tuesday, August 11, 2009 6:39 AM
To: Tim Anderson (itwriting)
Cc: csharp...@googlegroups.com
Subject: Re: Direction for C# Sqlite.
> * Move the System.Data.SQLite functionality (ADO.Net & Linq calling APIs)