Events could be a problem, but not alone mine. You could try enabling
the experimental cache, but it may even decrease performance, due to
sb/ltn issues. See the calendar properties dialog.
In general, your performance might go down if you have many recurring
events, especially infinitely recurring events (i.e birthdays). This
is of course not something that you should live with in the long run,
we are working on improving performance, but its something thats not
really easy to fix!
Aside from the perf problems we are having with lightning/sunbird,
there is also room for improvement within the provider. I haven't
opened a bug on that yet, since its rather low priority and I'm not
sure it will actually bring performance (since it crosses the
javascript<->C++ bridge), but its worth a try when there is nothing
else to fix :-)
Philipp