Building a custom formatter to profile code within before and after blocks

[Adam Magan]

Hey RSpec Team

I hope you are doing well!

Recently I've been trying to build a custom formatter that allows us to monitor code in our before and after hooks. To give a bit more context FactoryGirl subscribes to ActiveSupports instrumentation api and it allows you to monitor the length of time factories take to setup and I've been doing this inside a before and after block before a test suite is run in RSpec.

This seems like something that could be extracted to a gem and would potentially be useful to a lot of projects, so I tried to build a custom formatter that would do this for me. But I am just not sure on what to do in order to hook into the before and after blocks in RSpec. 

So my two questions are;

Is this something I should even be doing with a custom formatter?

If so is there some sort of stable public API in rspec-core that I am supposed to hook into, in order to gain access to the before and after blocks that are run for the test suite?

Thanks 

[Jon Rowe]

Hi Adam

This is something that can be done with a formatter, but you'd have to use example_started and example_finished to replicate before / after but the notification should include the example run time.

If up needed more detail you'd probably have to ask us to add more hooks as we currently don't do any notifications between start and stop iirc but we do in include all hook time in our run time.

Make sense?

Cheers

Jon

--
You received this message because you are subscribed to the Google Groups "rspec" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rspec+un...@googlegroups.com.
To post to this group, send email to rs...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rspec/70b0b368-588f-4c8a-a76b-20e27fb23881%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ad...@mrageh.com]

Thanks for the swift reply Jon, your reply kind of makes sense. 

So just to confirm example_started would allow me to monitor the before hook for a an example?

On Wednesday, November 25, 2015 at 2:20:07 AM UTC, Jon Rowe wrote:

Hi Adam

This is something that can be done with a formatter, but you'd have to use example_started and example_finished to replicate before / after but the notification should include the example run time.

If up needed more detail you'd probably have to ask us to add more hooks as we currently don't do any notifications between start and stop iirc but we do in include all hook time in our run time.

Make sense?

Cheers

Jon

On Wednesday, 25 November 2015, Adam Magan <adam....@gmail.com> wrote:

Hey RSpec Team

I hope you are doing well!

Recently I've been trying to build a custom formatter that allows us to monitor code in our before and after hooks. To give a bit more context FactoryGirl subscribes to ActiveSupports instrumentation api and it allows you to monitor the length of time factories take to setup and I've been doing this inside a before and after block before a test suite is run in RSpec.

This seems like something that could be extracted to a gem and would potentially be useful to a lot of projects, so I tried to build a custom formatter that would do this for me. But I am just not sure on what to do in order to hook into the before and after blocks in RSpec. 

So my two questions are;

Is this something I should even be doing with a custom formatter?

If so is there some sort of stable public API in rspec-core that I am supposed to hook into, in order to gain access to the before and after blocks that are run for the test suite?

Thanks 

--
You received this message because you are subscribed to the Google Groups "rspec" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rspec+unsubscribe@googlegroups.com.

[ad...@mrageh.com]

Jon Rowe 

I think it's just as I expected but there's no public API exposed to custom formatters that allows us to track what happens within 

```
RSpec.configure do |config|

   config.before(:suite) do 

     #some setup happening before a test is run that I need to track

   end

   config.after(:suite) do 

      #display stuff that I tracked in before block above

   end

end 

```

Is does something we can expose to custom formatters from the reporter?

On Wednesday, November 25, 2015 at 2:20:07 AM UTC, Jon Rowe wrote:

Hi Adam

This is something that can be done with a formatter, but you'd have to use example_started and example_finished to replicate before / after but the notification should include the example run time.

If up needed more detail you'd probably have to ask us to add more hooks as we currently don't do any notifications between start and stop iirc but we do in include all hook time in our run time.

Make sense?

Cheers

Jon

On Wednesday, 25 November 2015, Adam Magan <adam....@gmail.com> wrote:

Hey RSpec Team

I hope you are doing well!

Recently I've been trying to build a custom formatter that allows us to monitor code in our before and after hooks. To give a bit more context FactoryGirl subscribes to ActiveSupports instrumentation api and it allows you to monitor the length of time factories take to setup and I've been doing this inside a before and after block before a test suite is run in RSpec.

This seems like something that could be extracted to a gem and would potentially be useful to a lot of projects, so I tried to build a custom formatter that would do this for me. But I am just not sure on what to do in order to hook into the before and after blocks in RSpec. 

So my two questions are;

Is this something I should even be doing with a custom formatter?

If so is there some sort of stable public API in rspec-core that I am supposed to hook into, in order to gain access to the before and after blocks that are run for the test suite?

Thanks 

--
You received this message because you are subscribed to the Google Groups "rspec" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rspec+unsubscribe@googlegroups.com.

[Xavier Shay]

not exactly what you're asking, but some prior art for measuring test time (which is similar to profiling):

 

https://github.com/tpope/fivemat/blob/master/lib/fivemat/rspec3.rb

To unsubscribe from this group and stop receiving emails from it, send an email to rspec+un...@googlegroups.com.

To post to this group, send email to rs...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/rspec/6770daa7-e80b-4bbf-8395-85300dbc675d%40googlegroups.com.

[ad...@mrageh.com]

Thanks Xavier

I took a very similar approach when writing my formatter. I think the problem with that approach is that the notification I am trying to subscribe to has already ran before the 

RSpec examples start running. Should we expose a way to hook into:

```

RSpec.configure do |config|

   config.before(:suite) do 

    end

end

```

[ad...@mrageh.com]

I had a slight bug in the code I was writing so the hooks that are exposed to custom formatters worked fine for me.