Finding Provider Types

[Jonathan Pryor]

While working on the database-independent tests, I ran across ReflectonObjectFactory.Parse(), and wondered if there was a way to improve this method.  It currently contains:

var assemblyTypes = assembly.GetTypes();
foreach (Type type in assemblyTypes)
{
    if (type.IsAbstract)
        continue;
    foreach (Type i in type.GetInterfaces())
    {
        if (i.Assembly.GetCustomAttributes(typeof(DbLinqAttribute), false).Length > 0)
        {
            IList<Type> types;
            if (!interfaceImplementations.TryGetValue(i, out types))
                interfaceImplementations[i] = types = new List<Type>();
            types.Add(type);
        }
    }
}

This is less than ideal for two reasons:

1. It uses Assembly.GetTypes(), which thus "pulls in" private types which we probably don't care about.
2. It increases memory requirements, as we're loading more Type instances than we actually need.
3. It increases disk usage (as more Reflection information needs to be read from-disk to populate the Type instances).
4. It's slower than it could be, due to (1), (2) and (3).

(1) is "easily" fixed by using Assembly.GetExportedTypes(), which only returns public types, but the biggest performance and memory issues are due to (2) and (3).

A better solution (in terms of memory and execution time) is to use an assembly-level attribute:

// within DbLinq.dll:
[AttributeUsage(AttributeTargets.Assembly)]
public class DbLinqProviderAttribute {
    public DbLinqProviderAttribute(Type providerType);
    public Type ProviderType {get;}
}

// within e.g. DbLinq.Sqlite.dll
[assembly:DbLinqProvider(typeof(SqliteVendor))]

This turns the O(n*m) (n=types, m=interfaces for each type) algorithm used in ReflectonObjectFactory.Parse() into an O(1) algorithm, improving execution time and lowering memory requirements (as we don't need to allocate a potentially large array from Assembly.GetTypes(), etc.).

The downside is that it slightly complicates the provider assembly, as they need to use the DbLinqProvider attribute, but I think this may be worthwhile.

However, my biggest problem with the algorithm isn't within Parse() but instead within ParseAppDomain():

4. It processes most assemblies within the current AppDomain (and only those assemblies).

This limitation turned up when I was creating the DbLinq.SqlServer_test_ndb project: it required that I have a strong reference to the DbLinq.SqlServer assembly from my test assembly.

The problem, specifically, is the use of the NullConnection type, e.g. from MsSqlDataContextTest.cs:

return new DataContext(new NullConnection(), new AttributeMappingSource());

In this case, we're not providing a connection string (deliberately, as we don't actually want to connect to a database), and thus there is no explicit reference to the provider that we want to test.  This is undesirable; it's only acceptable for now because the default provider is the SqlServer provider, which is what I was testing, but this should not remain the situation for long.

Thus, a question: when using the DataContext(IDbConnection,MappingSource) constructor, how should we specify which provider assembly should actually be used?  The current code within DataContext.cs doesn't support using an IDbConnection with a non-SqlServer provider:

private void Init(IDatabaseContext databaseContext, MappingSource mappingSource, IVendor vendor)
{
    if (databaseContext == null)
        throw new ArgumentNullException("databaseContext");

    _VendorProvider = ObjectFactory.Get<IVendorProvider>();
    if (vendor == null)
        Vendor = _VendorProvider.FindVendorByProviderType(typeof(SqlClient.Sql2005Provider));
    else
        Vendor = vendor;

Non-SqlServer providers aren't supported because vendor will be null, thus prompting Init() to try to find the SqlClient.Sql2005Provider provider.  Oops.

Possible solutions would be to check to see if the IDbConnection instance implements the IVendor interface, and use that if appropriate (though this doesn't seem quite right, given the IVendor.CreateDbConnection() method), or have the IDbConnection type implement IVendorProvider (and ignore the providerType parameter in the IVendorProvider.FindVendorByProviderType() method).

I'm not really sure what the ideal solution here is, and would welcome any additional input.

Thanks,
- Jon

[Andrus]

> I'm not really sure what the ideal solution here is, and would welcome any
> additional input.

As I posted some time ago in this list, Linq to Sql author, Matt Warren blog
contains sample about replacing default MS Linq-To-Sql provider with
other linq provider implementing provider interface.

Initially MS desided to make this interface public but later desicion was to
change it to internal.

Andrus Moor.

[Pascal Craponne]

Hi Jon,

For ObjectFactory, you may proceed (I wrote it, and it was supposed to be temporary).

Regarding the fact of loading a given vendor, without referencing it, the following threads were talking about it:

http://groups.google.com/group/dblinq/browse_thread/thread/9e3b0ffa5f2d7e1f/6716fb3496c10b5f

http://groups.google.com/group/dblinq/browse_thread/thread/ce397e5f58b36a12/d7378bd1c401957a 

Pascal.

jabber/gtalk: pas...@jabber.fr
msn: pas...@craponne.org

[Jonathan Pryor]

On Mon, 2009-03-02 at 23:44 +0200, Andrus wrote:
> > I'm not really sure what the ideal solution here is, and would welcome any
> > additional input.
>
> As I posted some time ago in this list, Linq to Sql author, Matt Warren blog
> contains sample about replacing default MS Linq-To-Sql provider with
> other linq provider implementing provider interface.

I assume you mean this entry:

http://blogs.msdn.com/mattwar/archive/2008/05/04/mocks-nix-an-extensible-linq-to-sql-datacontext.aspx

I think it should be fairly obvious that we don't want to reimplement
Microsoft's internal interfaces, and that we should come up with a
public, well supported mechanism here. Using TransparentProxy's and
Reflection to muck about with private code is NOT the way to go.

- Jon

[Pascal Craponne]

And also, working this way requires to rewrite a lof of things for each database, since the provider handles almost everything in the LINQ to SQL request.

DbLinq is based on sharing as much logic as possible between vendors, so the philosophy is quite at the opposite.

Pascal.

jabber/gtalk: pas...@jabber.fr
msn: pas...@craponne.org

[Jonathan Pryor]

On Tue, 2009-03-03 at 13:13 +0100, Pascal Craponne wrote:

Regarding the fact of loading a given vendor, without referencing it, the following threads were talking about it:

http://groups.google.com/group/dblinq/browse_thread/thread/9e3b0ffa5f2d7e1f/6716fb3496c10b5f

This seems to discuss the DataContext(string) constructor, which seems to have been solved with the DbLinqProvider "parameter" within the connection string (a useful solution).

Alas, this seems to have no relevance to the DataContext(IDbConnection) constructor.

http://groups.google.com/group/dblinq/browse_thread/thread/ce397e5f58b36a12/d7378bd1c401957a 

This also seems to be about the DbLinqProvider connection string parameter.

However, it seems that I was blind, or at least not reading all the sources; an appropriate DataContext constructor already exists within src/DbLinq/Data/Linq/DataContext.Extended.cs:

        public DataContext(IDbConnection dbConnection, MappingSource mappingSource, IVendor vendor)

So we've already added an appropriate mechanism.  My apologies for not noticing this earlier.

- Jon

[Pascal Craponne]

I had in mind that the IDbConnection.ConnectionString was a string, so it could be handled the same way... :)

[Jonathan Pryor]

On Tue, 2009-03-03 at 20:12 +0100, Pascal Craponne wrote:

I had in mind that the IDbConnection.ConnectionString was a string, so it could be handled the same way... :)

It took me awhile to understand what you were getting at, but I like that idea.

How's this patch look?

- Jon

[Pascal Craponne]

Probably fine, if there is no regression (I didn't check) ;)

Pascal.

jabber/gtalk: pas...@jabber.fr
msn: pas...@craponne.org