Performance Tests

[aptq257]

Curious if anyone has any performance information when using Unity or
any other DI framework in a large application with many container
controlled objects?

I have created a small test to instantiate one object 4 different
ways.
My test object does not have any DI injected types.
When creating 100000 objects here are the results in milliseconds:

Create by Activator.CreateInstance<>
206 milliseconds

Create by using a IL generated Dynamically Created Method and Delegate
and storing the delegate in cache for subsequent instantiations:
49 milliseconds

Create by standard new MyObeject()
10 milliseconds

Create by Unity Resolve (Note: container was created, configured and
stored in a local var one time, before starting the Stopwatched)
475 milliseconds

Create by Ninject Resolve(Note: container was created, configured and
stored in a local var one time, before starting the Stopwatched)

1850 milliseconds

I tested two DI frameworks, Unity and Ninject. Unity is able to
resolve my type much faster then Ninject, but is still 2x slower then
Activator.CreateInstance.
Does anyone have any thoughts on the overall performance impact
application wide when using such a framework.

I really believe that a substantial performance hit is warranted
giving the architecture gains DI provides, but I am realistically
trying evaluate the performance impact.

Thanks.
Joe

[Nate Kohari]

Simple answer to this: the reason you tested using 100,000 calls is that
it takes that many calls to notice a difference in performance. In
practice, you would never ask Ninject to create 100,000 objects. (Or if
you were, I would strongly suggest you rethink your design! :)

There is a performance hit for using any DI framework, including
Ninject. However, you will find it to be absolutely insignificant if you
compare it to the latency involved in communicating across a network,
performing I/O on a disk, or accessing a database -- all of which most
typical applications do on a constant basis.


-Nate

[aptq257]

Hi Nate,
If i run the test on 1000 objects is 15 times slower then Activator.
So even on small amounts of objects, it seems the difference is
significant.
Although I agree that even this sort of significant amount of extra
time during instantiations will likely have no noticable impact on the
final application.
But again, that assumption is what i am trying to investigate.

Thanks!

> > Joe- Hide quoted text -
>
> - Show quoted text -

[Jason Dentler]

The second or two per year you will save your users is insignificant compared to the days you will spend wiring everything by hand. 

[aptq257]

I have read that Ninject uses Dynamically Created Methods and
Delegates to newUp the objects.
In my little test i found that method to be reasonably close in speed
to doing new().
So I was expecting Ninject to instantiate much faster then Activator,
but that does not seem to be the case.

> > > - Show quoted text -- Hide quoted text -

[Nate Kohari]

Not to be rude, but the time you spent writing these email chains is probably 1,000,000 times the cumulative difference in speed between Ninject and manually newing objects over the lifetime of your application. ;)

Is it slower? Yes. Maybe even 15 times slower. A 15-fold difference sounds like a huge deal, until you remember you're talking about nano- or milliseconds. Inconceivably small amounts of time.

Believe me, it's something I worried about a lot when I first started working with DI. It's one of those things that you can show a major difference in a sterile benchmark, but once you use it in a real-world app, you realize worrying about DI performance is micro-optimization and matters very little, if at all.

Think about it this way: it's similar to worrying about how long it would take NHibernate to hydrate an object from the raw data returned from your RDBMS. In the grand scheme of things, it just doesn't matter.


-Nate

[aptq257]

Nate,
The reason I am spending the time writing these posts is to get
feedback such as your last post and to try and inform myself the best
I can that i am on the right path.
I have used Ninject and Unity on small apps with great results and I
want to be sure I will not see diminishing returns once i employ DI in
a large scale (company owned) app with a lots of injected types.
JOe

[Nate Kohari]

Sure, I definitely understand, and like I said, I don't mean to sound rude or dismissive. I shared your concern at one point as well, and it was only experience that convinced me otherwise. I've had great success using Ninject in everything from large-scale applications to handheld devices.

As long as you don't activate hundreds of thousands of instances (which again, you shouldn't do anyway), you'll be fine!


-Nate

[J.R. Garcia]

We use it here on several large apps. You can't tell a difference between it, or StructureMap, or Unity, etc. The reason I chose Ninject over xyz is because it is lightweight and very easy to use. I love the syntax. It's better than any other IoC container out there in my opinion. The point is, those nanoseconds are not something you should ever worry about until you have to. If you are just concerned about using it on large, enterprise apps, I can tell you that it works very well.

J.R. Garcia
mrjohn...@gmail.com

[x97mdr]

Hi Nate,

I'm not sure I agree with your statement that you would never ask
Ninject for 100,000 objects. While I might agree that if you are
using Ninject for mainly web applications then your statement would
hold true but what about the people who have adapted the IoC philosphy
to other domains? I've been using Ninject in my application for about
8 months now. The application is an edit and imputation console
program written in C#. Typically, edit and imputation involves taking
a household that has failed a set of rules and finding similar
households to impute data and make the original household pass. In
our domain, the original household is known as the fail and the
households we search for are known as donors. Donors have a state and
they have business logic associated with them. For instance, a donor
needs to be able to calculate the distance between itself and the fail
and it might also need to retain that distance as part of its state.
When trying to find a donor for a fail we may search tens of thousands
of households as donors. As you can hopefully see Donors are
transient objects and in our case we have to make millions (sometimes
tens of millions) of them. I use Ninject to reduce the complexity of
my program (it's tough enough understanding the algorithm without
bloating the code with a bazillion factories and new operators) and it
works extremely well (excellent job by the way!!) but I need to find a
way to reconcile the need for a large number of transient objects with
Ninject?

These objects are typically created in one line in the code and in one
small run 4,000,000+ objects of a particular type were created. When
I use Ninject there is a noticeable (one user reported 600 times
slower) performance hit. The performance hit happened on code that
had not been optimized at all and I had read about a performance bug
in v1 (which I am using) that may reduce the performance hit
dramatically but not entirely. However, I am gambling that the
difference will still be significant!

What are your thoughts on using a new operator in very specific places
to enhance performance? Are there other tricks one can take with
Ninject to enhance performance? I hope to switch over to v2 soon and
re-evalaute performance with that version of Ninject, but in the
meantime is there any interest in adding syntax similar to Autofac
that would allow someone to use lambda expressions to employ the new
operator to speed things along when a user does need a large number of
transient operators?

Thanks in advance

On Oct 27, 3:17 pm, Nate Kohari <nkoh...@gmail.com> wrote:

[Nate Kohari]

Jeffrey:

I would never advocate using a general-purpose dependency injection framework like Ninject to activate millions of instances of anything. What you're describing is injecting services into your entities, which is something I would strongly recommend against. There's always a better solution.

Remember, using Ninject is not an all-or-nothing situation. I've run into quite a few scenarios where I've preferred to use reflection or factories instead of leaning on the framework.


-Nate

[Sean Chambers]

Someone may correct me but this seems to be an odd usage of IOC. I
personally never put my model creation onto my container. At the most,
I employ factories to give me those instances as there is often a fair
amount of state required to construct those instances and doesn't
really belong in a container directly.

You did mention that factories in your domain would be too complex,
however the alternative that you are using doesn't seem all that ideal
IMO. At the very least it feels like this isn't best solved by putting
it onto your container.

Sean Chambers

[x97mdr]

Thanks to both Nate and Sean for the replies.

It seems like the overriding theme here is that I should avoid IoC
containers for resolving a lot of transient instances and I shouldn't
rely solely on the IoC container ... point taken ... use all of the
tools in my toolbox! :)

From both of your responses I appear to be doing something odd and
that distresses me a little since I thought I've been trying very hard
to follow best practices! My donors (from the earlier example) are
entities, not services ... the problem is that there are just so many
of them (like an unending zombie horde). I'm not a big believer in
the Anemic Domain Model pattern so my donors, being entities, have
meaningful methods on them. For instance, they know how to perform a
complicated task and may use other helper objects (the services that
are injected into the donor) to deal with it. Is this not what I
should be doing?

Nate: when you're talking entities and services do you mean the formal
definition of entities and services from DDD or were you using those
terms more generally?

Sean: It's not so much that factories would be complex themselves, the
sheer number of them would bring complexity. When I introduced
Ninject, I trimmed at least a dozen different factory objects. Some
of the factories even use different means to resolve things (sometimes
an enumeration, sometimes a type, etc.). This is the complexity I
meant. Also: if you don't put model creation on the container ...
what do you put on the container?

Sorry I'm asking a lot of questions, but I really want to see how
other people use the tool and try to improve my own usage. It feels
like I'm the only person at my workplace who wants to try to improve
themselves sometimes so an outside perspective is appreciated.

[Ben Ellis]

Ioc is best used for injecting dependencies into long-lived
singleton services into other long-lived singleton services.

Since services are generally long-lived and singleton they only need
to be injected once. If you want to inject a dependency into many
short-lived transient entities it would be better to create a
Factory/Provider service (not a ninject provider) that has the same
depenencies as the required dependencies for the entities, then inject
manually on creation of the entity (plus any constructor parameters needed
for creation). This obviously removes the need to use a more complex IoC
container to resolve the dependencies each time.

For the above reasons, it's a bit of a smell if an entity has a
dependency on a service. There are various ways of getting around this, I
prefer to use events and adhering to CQS. I've also seen suggestions of
injecting dependencies on the calls to the entity instead of on
construction, this allows the caller to choose what service the entity
should use (e.g. IErrorNotification service, depending on the context in
which your entity is used, you may want a ConsoleNotification,
EmailNotification or SmsNotification) on a per call level instead of for the
lifetime of the entity.

Ben Ellis

[x97mdr]

Very interesting, thank you Ben.

I did a lot of searching around on the net before about using IoC when
I first came across Ninject but don't recall ever coming across a blog
or article detailing situations where *not* to use it. It would have
been very useful. I'm going to take the factory suggestion and
refactor my code to take advantage of it.

Thanks again for all the patient responses!