I haven't forgotten the project
Stefano Maestri
sorry for this long silence, but I've been very busy both in my daytime
job (that very often get also some of my spare time) and Wise.
Anyway I'm thinking and refining the idea behind testedby.
Let me summarize here my recent thoughts hoping we can open the
discussion about this new ideas.
Well testedby aims to get 2 main results:
1. Define test on interface and inject implementation in test
2. Run test only against modified classes
I think I have good discussions with you all about these two points, and
yo all perfectly understand what I have in mind with these 2 points
without deeper details.
I try to rethink these two points keeping in mind all comments and
concerns I got from you and community. I tryied t focus mainly on
concerns about coupling and cluttering of code.
For point 1 I still convinced annotations are the best way to take.
For point 2, even if I still convince annotation could get some great
plus in BDD or DbC approach, isn't the only way. I think we can go for a
sw that continue to launch tests starting from class under test, but we
have at least 2 way to get wich tests we have to run:
* Well, annotations as said
* Using coverage infos. Open source coverage tools don't take care
of single test coverage, but for example Clover (commercial
coverage tool from atlassian does) as well described here
http://blogs.atlassian.com/developer/2008/11/stop_testing_so_much.html
Point 2 is the "magic button" Randall would like to see in testedby
isn't it?
So I'm saying we have to extend (or write from scratch) a coverage tool,
open source taking care of single test coverage integrated with and used
by testedby to run our test.
So testedby will be able to use both annotation (and they would have
precedence) for who define them, or "magically" use coverage data to run
right tests. Annotation will remain for test injection for "Test the
interfaces" approach.
Moreover we would get the first open source coverage tool with a
per-test report. Of course we still need ant, eclipse and moreover
HUDSON plugin.
What about?
Shih-gian Lee
I am new to this group. I like the idea of testedby. I have read most
of the comments on TSS and blog. I am still a little unclear about the
concerns of coupling. If I understand the argument correctly, we
already have logical coupling when doing JUnit testing. The logical
coupling comes from the test methods and test class. For best
practices, the test methods and test class have the names of the
actual class and methods we would like to test. I find this kind of
logical coupling is insufficient. Many developers tend to forget to
update the corresponding test method when updating a function. I find
it very frustrating when there is a lack of "stronger" linkage between
the test class and the class under test. So, I don't see it this kind
of coupling is necessary a bad thing.
As for code cluttering, can you list some of the use cases for
brainstorming?
I have a day job as well and I will try to contribute to this
discussion as much as possible. I really like the idea proposed by
testedby and would like to see something good comes out of it.
Thanks,
Lee
On Nov 9, 4:45 pm, Stefano Maestri <stefano.maes...@javalinux.it>
wrote:
Stefano Maestri
first of all welcome on board.
Shih-gian Lee wrote on 13/11/08 22:38:
> Hello,
>
> I am new to this group. I like the idea of testedby. I have read most
> of the comments on TSS and blog. I am still a little unclear about the
> concerns of coupling. If I understand the argument correctly, we
> already have logical coupling when doing JUnit testing. The logical
>
Yep, you are right about logical coupling. Concerns about a more
strictly coupling is because with testedby you would have to maintain
not only test classes but also clas under test when you refactoring a
test class. IMHO it' a pure theoretical concerns if you would have good
IDE tools. Anyway, my last post on this group aims to provide an
alternative for who don't like this coupling and cluttering (about
cluttering see my comment to your question)
> coupling comes from the test methods and test class. For best
> practices, the test methods and test class have the names of the
> actual class and methods we would like to test. I find this kind of
> logical coupling is insufficient. Many developers tend to forget to
> update the corresponding test method when updating a function. I find
> it very frustrating when there is a lack of "stronger" linkage between
> the test class and the class under test. So, I don't see it this kind
> of coupling is necessary a bad thing.
>
+1. And with BDD approach names convention could be absolutely bad.
> As for code cluttering, can you list some of the use cases for
> brainstorming?
>
Well very simple. If you have a lot of test stressing a single method
you would have a lot of annotation on the method under test. And it
drive t cluttering code. But it give also some advantage giving better
understandig of the code at a first look defining in fact contract for
methods, and exposing it in javadoc too.
John have also some ideas about how to get annotations more compacts and
easy to use? John, would you explain them to all?
But my last post aims to don't imposing use of annotations. You could
continue to use testedby defining contract with annotation on interfaces
(where is much more important) and still use testedby test executor
limiting test run only to ones stressing modified code. Being too much
impositive could drive community away from testedby, and I'm convinced
who will try testedby could get its plus and step by step will start to
use its features.
BTW removing the requirement of annotation make possible to start using
testedby on existing projects without modify existing code. And it's a
great plus to introduce testedby as common used tool. Isn't it?
> I have a day job as well and I will try to contribute to this
> discussion as much as possible. I really like the idea proposed by
> testedby and would like to see something good comes out of it.
>
And I hope you could lend an hand with the code too at some stage ;)
Thanks for the interest.
bye
S.
Shih-gian Lee
On Nov 14, 4:21 pm, Stefano Maestri <stefano.maes...@javalinux.it>
wrote:
> first of all welcome on board.
> But my last post aims to don't imposing use of annotations. You could
> continue to use testedby defining contract with annotation on interfaces
> (where is much more important) and still use testedby test executor
> limiting test run only to ones stressing modified code.
testedby by convention. I am not so sure what do we mean by using test
executor to stress modified code. Maybe I have not looked at the code,
yet. I will try to download the code and take a look at it.
> Being too much
> impositive could drive community away from testedby, and I'm convinced
> who will try testedby could get its plus and step by step will start to
> use its features.
> BTW removing the requirement of annotation make possible to start using
> testedby on existing projects without modify existing code. And it's a
> great plus to introduce testedby as common used tool. Isn't it?
testedby by convention when user annotates the interface. As use is
more comfortable with testedby, he/she can refine their testing.
> And I hope you could lend an hand with the code too at some stage ;)
much as possible. I think it is easier to communicate through code
than message-posting :-)
Thank you!
Lee
Stefano Maestri
Shih-gian Lee wrote on 17/11/08 22:02:
> I agree. If we only annotate the interfaces, everything will run on
> testedby by convention. I am not so sure what do we mean by using test
> executor to stress modified code. Maybe I have not looked at the code,
> yet. I will try to download the code and take a look at it.
>
>
example if you have class A, B,C and test classes tesA, testB, testC and
you modify class A and B but not C testedby will run only testA and testB.
> Yep. Going back to my point above, we can have everything run on
> testedby by convention when user annotates the interface. As use is
> more comfortable with testedby, he/she can refine their testing.
>
>
test and class under test relationships is exactly to avoid conventions.
Conventions in tests results on poor test class and test method name
(I'm a BDD fan where naming is very very important).
>> And I hope you could lend an hand with the code too at some stage ;)
>>
>
> I will download the code to play with and will help with the coding as
> much as possible. I think it is easier to communicate through code
> than message-posting :-)
>
>
to commit something more very very soon.
> Thank you!
> Lee
>
Thanks you too. BTW your timezone?
bye
S.
Shih-gian Lee
> example if you have class A, B,C and test classes tesA, testB, testC and
> you modify class A and B but not C testedby will run only testA and testB.
and which one is not? Also, running test during compile time is also
very interesting and useful. Care to share your idea on how to
implement that?
> Well, my point on use a coverage (or coverage like) metadata to track
> test and class under test relationships is exactly to avoid conventions.
> Conventions in tests results on poor test class and test method name
> (I'm a BDD fan where naming is very very important).
the purpose of BDD.
> Well you will not find a lot of code yet, just a ridiculous PoC. I hope
> to commit something more very very soon.> Thank you!
coding in anyway I can. I plan to start looking at your code to get an
idea and then evolve ideas from there.
> Thanks you too. BTW your timezone?
Cheers,
Lee
Stefano Maestri
Shih-gian Lee wrote on 17/11/08 22:43:
> I see. This will be cool. But, how do we know which one is modified
> and which one is not? Also, running test during compile time is also
> very interesting and useful. Care to share your idea on how to
> implement that?
>
>
plugin would know which are class modified, the same for javac in ant
(and maven? Some mavenized man help me here! Alessio, John, Randall, no
one?). But we could have different scenario using svn updated files (i.e
in Hudson plugin)
bye
S.
Randall Hauch
Don't some of these systems work by comparing the timestamps of the .class files with the timestamps of the source? If a source file is newer than the .class file, it has been modified. This is how I'd expect Maven or Ant works (or at least the compiling plugins). There are other approaches, though. An SVN-aware system might look at the .svn directory for the original file information, comparing timestamps and/or doing differences. (Remember that SVN keeps a local copy of every file that is checked out. This way, when you revert or do a difference, the SVN client doesn't have to consult the repository.) And an IDE like Eclipse probably is more sophisticated and may explicitly track the files that were modified. And JSR-203 will offer additional opportunities, like being able to receive file system events upon file modifications.
If we're talking about tools that make it easy to run the unit tests, then those tools would likely rely upon the platform in which they're running. Please correct me if I'm wrong, but and Eclipse "builder" extension is called when the Eclipse build system is executed and is supplied the set of changes to the files that were modified since the previous build. See http://help.eclipse.org/stable/topic/org.eclipse.platform.doc.isv/reference/extension-points/org_eclipse_core_resources_builders.html for more information.
Randall
Randall Hauch
Shih-gian Lee wrote on 17/11/08 22:02:
> I agree. If we only annotate the interfaces, everything will run onI mean run only test that point to modified class under test. For
> testedby by convention. I am not so sure what do we mean by using test
> executor to stress modified code. Maybe I have not looked at the code,
> yet. I will try to download the code and take a look at it.
>
>
example if you have class A, B,C and test classes tesA, testB, testC and
you modify class A and B but not C testedby will run only testA and testB.
Plus, one benefit I see is that after I've finished editing a source file, I can hit a button to run the unit tests for the class I just modified. Or maybe it runs my tests automatically. It allows me to stay focused on my class, yet the corresponding unit test is run automagically. What I get is not having to find my corresponding unit test, select it, and then run it. This would be similar to "autotest" in the Ruby/Rails world.
It sounds like this may be fundamentally different than a framework designed around specifying behaviors for interfaces and APIs. They both perhaps need the same information (what is the relationship between my code and my tests), but then use that information for different ends. Or maybe I'm seeing something that isn't there.
Randall
Stefano Maestri
Answer inline
Randall Hauch wrote on 18/11/08 02:20:
>
> and then run it. This would be similar to "autotest" in the
> Ruby/Rails world.
Yep, that is what "inspired" me
>
> It sounds like this may be fundamentally different than a framework
> designed around specifying behaviors for interfaces and APIs. They
> both perhaps need the same information (what is the relationship
> between my code and my tests), but then use that information for
> different ends. Or maybe I'm seeing something that isn't there.
Yep you are right they need same informations, so why don't provide both
features?
More general it's the focus changing from test class to class/interface
under tests. Goals are different, but focus is the same. Moreover
developer may force "autotest" link between test classes and classes
under test with annotation (automatic discovering may be disabled)
providing an explicit link which have other problems (cluttering
mainly), but that could have sense in some case. What I'm saying is: we
should leave to developer choice if and when use auto discovery and when
explicit define annotation.
Both features could be provided by testedby since they aren't logically
related, but as you said use same meta data. Am I totally bad?
bye
S.
JPAV
approaches a bit with Randall. It seems to me that the real magic
behind what your suggesting could be provided almost entirely by the
tooling (i.e., plug-ins/extensions for Eclipse, Ant, etc.). I
definitely see 2 very distinct concerns with what's been discussed:
1. Providing a contract for an interface such that all
implementations of that interface are guaranteed to adhere to its
contract.
2. Maintaining the relationship between classes (or perhaps just
certain methods) and their associated tests.
The first item seems to be a completely new provision that calls for a
new annotation. However, I'd agree with earlier concerns that adding
such annotations to production code will not be received well. So
what about going the other direction? Borrowing the example from your
original post (http://www.javalinux.it/wordpress/?p=116):
public interface APIInterface {
public int add(int a, int b);
}
@ContractFor( interface = "APIInterface")
public class APIContract {
private APIInterface instance;
public APIContract(APIInterface instance) {
this.instance = instance;
}
@Test // This is just the standard JUnit annotation
public void shouldAddTwoAndThree() {
assertThat(instance.add(3,2), is(5));
}
}
The expectation would be that developers would provide test classes
that extend or use the contract:
public class APIImplOne implements APIInterface {
public int add(int a, int b) {
return a + b;
}
}
public class APIImplOneTest extends APIContract {
public APIImplOneTest() {
super(new APIImplOne());
}
}
or the tooling could simply auto-generate and run contract tests for
simple scenarios.
Regarding the second item, is there any reason why generic tooling
couldn't be adapted to each tooling environment (e.g., Eclipse, Ant,
etc.) via its available plug-in/extension mechanisms that manages all
the metadata related to a workspace such that it would be entirely
responsible for maintaining the relationships between classes/methods
and their associated tests? In other words, why can't the tooling
alone allow me to right-click on a class/method and run all associated
tests that reference that class/method - without any annotations other
than those already provided by JUnit/TestNG/whatever?
Randall Hauch
...Borrowing the example from your
original post (http://www.javalinux.it/wordpress/?p=116):
public interface APIInterface {
public int add(int a, int b);
}
@ContractFor( interface = "APIInterface")
public class APIContract {
private APIInterface instance;
public APIContract(APIInterface instance) {
this.instance = instance;
}
...
Stefano Maestri
JPAV wrote on 18/11/08 14:39:
> I've been thinking about this for a while and discussed possible
> approaches a bit with Randall. It seems to me that the real magic
> behind what your suggesting could be provided almost entirely by the
> tooling (i.e., plug-ins/extensions for Eclipse, Ant, etc.).
even if IMHO add something (see later in this email.)
> I
> definitely see 2 very distinct concerns with what's been discussed:
>
>
>
> The first item seems to be a completely new provision that calls for a
> new annotation.
> However, I'd agree with earlier concerns that adding
> such annotations to production code will not be received well.
case (see later)
Depends on the value you are giving to this Behaviour/Contract. If you
put to them real value of contract I think it's better to have it in
your production code.
IMHO if you would design by contract it's a great plus to have contract
in your production code, since Java interface lack in behavioural
contract definition. Java interface are pure signature definition
without any behavioural enforcement. But when you are designing an
interface wouldn't you think also to its implementation boundaries? IOW
to its contract? And don't you provide a detailed description of them in
Javadoc?
But even if javadocs have its own great purpose they aren't for sure a
formal definition of contract. And for sure the can't verify them.
Moreover I have seen a lot of code where very complex description (and
some times not so clear) of behaviour stay in javadoc, and they are
cluttering very much code. But no ones claims to remove javadoc to
avoid code cluttering.
BTW a nice plus of using annotation on interface/class under test is to
have them into javadoc (as it happens for jcip annotations)
> So
> what about going the other direction? Borrowing the example from your
>
explained above, but I can see somevery good points here. If you would
define a more test oriented design by interface than a design by
contract it's good. (I hope I've been clear here...I've rewritten this
sentence 4 times, and I can't get a better one :) )
BUT MUCH MORE IMPORTANT: a well designed core for test extecution could
support both approach. As said in one of my last post: we should let the
user choice if he want avoid cluttering or would prefer to have clear
annotation defining contracts. I can see use case for both approach.
Isn't it?
> The expectation would be that developers would provide test classes
> that extend or use the contract:
>
with annotations on class under test. of course we need to define
priorities for this different approach. For example (but priority could
be changed, ideally with a config):
1. If there is annotations in classes under test (or interfaces
implemented by them) go for it
2. If there is annotations on test classes go for it
3. MAgic discovery based on metadata of eclipse (or collected by
coverage like tool)
>
> or the tooling could simply auto-generate and run contract tests for
> simple scenarios.
>
yep it would be useful for both approaches.
> Regarding the second item, is there any reason why generic tooling
> couldn't be adapted to each tooling environment (e.g., Eclipse, Ant,
> etc.) via its available plug-in/extension mechanisms that manages all
> the metadata related to a workspace such that it would be entirely
> responsible for maintaining the relationships between classes/methods
> and their associated tests? In other words, why can't the tooling
> alone allow me to right-click on a class/method and run all associated
> tests that reference that class/method - without any annotations other
> than those already provided by JUnit/TestNG/whatever?
>
We are in the same page here. Tool could do all itself.
IMHO we should let user specify annotations if they want for
"documentation" and to change the "standard" behaviour of the tool. For
example tool could get more tests than specified explicitly by user, but
user could exactly want to consider only test he have linked. In this
case tool should give a warning to user, but just run tests specified.
IOW I like your ideas because it's doable together mine, and I'm
convinced here that flexibility could give a serious spin to our TestedBy.
Please let me know if I'm totally wrong. Comment from all are very welcome.
Randall Hauch
I agree with Stefano. If the annotations specify the behavior contract, then it is very valuable (i) for implementers, and (ii) for users of the API.
Perhaps John was referring to the "tested by" annotations, which are important only for that project. I'd love to see the "tested-by" functionality (at least a good portion of it) work without any effort by the developer - and that includes not maintaining "tested-by" annotations.
However, remember that with each annotation definition you can specify how the information is retained (in the source only, in the classes, or at runtime). So if any tested-by annotations are needed, those annotations could be made to be retained in source (or class file if needed).
Stefano Maestri
Answer inline
Randall Hauch wrote on 18/11/08 22:02:
> On Tue, Nov 18, 2008 at 2:40 PM, Stefano Maestri
> <stefano...@javalinux.it <mailto:stefano...@javalinux.it>>
Not sure I'm following you. I'm referring to testedby annotation which
are defining contract/behaviour using unit test.
As said I'd like too to support great portion of tested by (the auto
test part IMHO) without any effort.
Just use annotation for 2 reasons:
1. Define contract
2. change default magic discover forcing your own link between test
class under tes.
>
> However, remember that with each annotation definition you can specify
> how the information is retained (in the source only, in the classes,
> or at runtime). So if any tested-by annotations are needed, those
> annotations could be made to be retained in source (or class file if
> needed).
Not sure to follow you also here. What do you mean exatly? For sure this
kind of annotation may stay in class or source (even if using pure
string as parameter the aren't a problem in runtime too).
bye
S.
Randall Hauch
Randall Hauch wrote on 18/11/08 22:02:
Not sure I'm following you. I'm referring to testedby annotation which> I agree with Stefano. If the annotations specify the behavior
> contract, then it is very valuable (i) for implementers, and (ii) for
> users of the API.
>
> Perhaps John was referring to the "tested by" annotations, which are
> important only for that project. I'd love to see the "tested-by"
> functionality (at least a good portion of it) work without any effort
> by the developer - and that includes not maintaining "tested-by"
> annotations.
are defining contract/behaviour using unit test.
As said I'd like too to support great portion of tested by (the auto
test part IMHO) without any effort.
Just use annotation for 2 reasons:
1. Define contract
2. change default magic discover forcing your own link between test
class under tes.
My bad. I should have siad "autotest" rather than "tested by". Yes, "tested by" annotations would be useful in production code, so they should be defined to have a retention of at least "CLASS" and probably "RUNTIME".
I was trying to say that any annotation that exists purely for autotest functionality (aka, "autotest annotations") may be considered by some (including John :-) as clutter. Granted, they may only be needed to override the default "magic" behavior. But the testing tooling would have access to the source, right? If so, then maybe these autotest annotations need to be defined as retained only in the "SOURCE".
> However, remember that with each annotation definition you can specifyNot sure to follow you also here. What do you mean exatly? For sure this
> how the information is retained (in the source only, in the classes,
> or at runtime). So if any tested-by annotations are needed, those
> annotations could be made to be retained in source (or class file if
> needed).
kind of annotation may stay in class or source (even if using pure
string as parameter the aren't a problem in runtime too).
See above. Hopefully my correction makes sense.
Stefano Maestri
Stefano Maestri wrote on 18/11/08 21:40:
>
> I don't totally agree here. Even if I like your purpose for some use
> case (see later)
>
be misunderstood for my english
bye
S.
Shih-gian Lee
But, one question. Does developer have to write the contract code or
it can be generated by tested-by?
JPAV
On Nov 19, 8:19 am, Shih-gian Lee <shihg...@gmail.com> wrote:
> But, one question. Does developer have to write the contract code or
> it can be generated by tested-by?
basically represents the behavioral requirements of the interface that
would otherwise only be specified via documentation. I think by
definition this has to be provided by the interface developer and
would have no source for generation.
JPAV
On Nov 18, 3:02 pm, "Randall Hauch" <rha...@gmail.com> wrote:
> I agree with Stefano. If the annotations specify the behavior contract,
> then it is very valuable (i) for implementers, and (ii) for users of the
> API.
there are 2 issues we're discussing, a new annotation seems
appropriate when talking about a contract. The only difference I'm
proposing is to move the annotation to the contract code (i.e., the
test code) rather than specifying it in the interface. Same net
effect, but environments that need source but for whatever reason
don't have access to test source wouldn't be confounded by not being
able to process annotations that refer to missing entities. Not
really sure how much of a real issue this is, but assuming it is, that
would be the advantage of moving the annotation. Otherwise, having
the annotation on the interface would be perfectly fine.
I'd also add that, IMO, something like @ContractFor(interface = XXX)
(or, if specified on the interface, maybe @Contract(class = XXX))
reads a bit better than @TestedBy and highlights that this is really
about contracts and not just any type of testing.
> Perhaps John was referring to the "tested by" annotations, which are
> important only for that project. I'd love to see the "tested-by"
> functionality (at least a good portion of it) work without any effort by the
> developer - and that includes not maintaining "tested-by" annotations.
> However, remember that with each annotation definition you can specify how
> the information is retained (in the source only, in the classes, or at
> runtime). So if any tested-by annotations are needed, those annotations
> could be made to be retained in source (or class file if needed).
interesting concern. I was more concerned about not having to
specify, or more importantly maintain, annotations unless absolutely
necessary, which seems only to establish a relationship between an
interface and its contract such that all implementations are forced to
adhere to both contract and interface. Not sure if I understand any
need to "retain" the annotations anywhere other than source.
Shih-gian Lee
> there are 2 issues we're discussing, a new annotation seems
> appropriate when talking about a contract. The only difference I'm
> proposing is to move the annotation to the contract code (i.e., the
> test code) rather than specifying it in the interface. Same net
> effect, but environments that need source but for whatever reason
> don't have access to test source wouldn't be confounded by not being
> able to process annotations that refer to missing entities. Not
> really sure how much of a real issue this is, but assuming it is, that
> would be the advantage of moving the annotation. Otherwise, having
> the annotation on the interface would be perfectly fine.
>
> I'd also add that, IMO, something like @ContractFor(interface = XXX)
> (or, if specified on the interface, maybe @Contract(class = XXX))
> reads a bit better than @TestedBy and highlights that this is really
> about contracts and not just any type of testing.
approach for those who have concerns with production code coupled with
test cases or those who may not have access to test cases.
I will be reluctant to write the additional contract code to enforce
the behavior. I like Stefano's design by interface approach where we
put annotation on the interface under test. As I mentioned in my
previous posts, I have no concern with this kind of coupling. Test
cases to me are production code. Not many people will agree with me,
but that is OK. Since test cases are production code, there is no
situation where I will have access to source code but not test code.
Stefano Maestri
JPAV wrote on 19/11/08 17:09:
> On Nov 18, 3:02 pm, "Randall Hauch" <rha...@gmail.com> wrote:
>
>> I agree with Stefano. If the annotations specify the behavior contract,
>> then it is very valuable (i) for implementers, and (ii) for users of the
>> API.
>>
> I'm not disagreeing with this either. Going back to the idea that
> there are 2 issues we're discussing, a new annotation seems
> appropriate when talking about a contract. The only difference I'm
> proposing is to move the annotation to the contract code (i.e., the
> test code) rather than specifying it in the interface.
>
But, I think Randall was agreeing with me on put annotation on interface
to give implementers and users evidence of contract.
> Same net
> effect, but environments that need source but for whatever reason
> don't have access to test source wouldn't be confounded by not being
> able to process annotations that refer to missing entities. Not
> really sure how much of a real issue this is, but assuming it is, that
> would be the advantage of moving the annotation. Otherwise, having
> the annotation on the interface would be perfectly fine.
>
Well, not sure to follow you here: IMHO it's the contrary. If you put
annotation on interface and that annotation use pure strings as
parameters you don't need at all test code to compile. You see the
contract defined (at least the name of contracts) and then you can
compile and use it. When you want to verify contracts you need test
code, but at least you are advised there is a contract during your
implementation or even reading javadocs (which contain our annotation
since they are defined as @Documented). The contract verifacation is
matter of test, let me call it a TCK.
With you approach you haven't any advice of contract and you need test
code to get it.
let me remark that using pure string as parameter of annotation give us
the opportunity to compile and use interface even without test code.
Annotation will be used only by testedby test runner. It's exactly the
reason for which I've used string instead of class reference, to avoid a
real coupling. I think a lot of people have misunderstood me on this point.
Annotation as defined in my original post (with string parameter) don't
introduce compile time coupling between test class and class under test
> I'd also add that, IMO, something like @ContractFor(interface = XXX)
> (or, if specified on the interface, maybe @Contract(class = XXX))
> reads a bit better than @TestedBy and highlights that this is really
> about contracts and not just any type of testing.
>
Absolutely agree here. I said a lot of time that annotations name need
to be reviewed. @TestedBy have been good just in my first post to keep
it easy to read and understand without a lot of names and changes.
>> Perhaps John was referring to the "tested by" annotations, which are
>> important only for that project. I'd love to see the "tested-by"
>> functionality (at least a good portion of it) work without any effort by the
>> developer - and that includes not maintaining "tested-by" annotations.
>>
> Yes, this is part of what I was getting at.
>
And I almost agree here, as said in last posts. I'd just remark to leave
users the choice, with configurable priorities as said in my last post.
>> However, remember that with each annotation definition you can specify how
>> the information is retained (in the source only, in the classes, or at
>> runtime). So if any tested-by annotations are needed, those annotations
>> could be made to be retained in source (or class file if needed).
>>
>
> I wasn't addressing this aspect of annotations, but it's an
> interesting concern. I was more concerned about not having to
> specify, or more importantly maintain, annotations unless absolutely
> necessary, which seems only to establish a relationship between an
> interface and its contract such that all implementations are forced to
> adhere to both contract and interface. Not sure if I understand any
> need to "retain" the annotations anywhere other than source
May I insist that is better to have to maintain annotation defining
contracts in production code. IMHO they have too much value (if users
decide to use it, and we wouldn't force anyone) to don't maintain them.
Retain annotation at runtime may be needed to put them on javadoc, but
perhaps my memory could fail here..I've to check. For sure it make test
executor able to run against .class without code, but maybe it's not
strictly needed (even if is how JUnit and TestNG works actually)
bye
S.
JPAV
On Nov 19, 10:37 am, Shih-gian Lee <shihg...@gmail.com> wrote:
> I will be reluctant to write the additional contract code to enforce
> the behavior. I like Stefano's design by interface approach where we
> put annotation on the interface under test.
things. I've been focusing entirely on the original TestedBy
annotation from Stefano's first post, and not annotations mentioned by
responses to that post that would cover the "low-hanging fruit" of
contracts, such as whether arguments can be null or if a method can
only be called when an object has a particular state. I'm now
thinking these are what you had in mind when talking about not having
to write additional contract code? If so, I'd have to completely agree
that it makes sense for these to be part of the interface itself, and
therefore an annotation referring to a test that further defines the
contract (e.g., @Contract(class = XXX)) would go there also. I'd
still say, though, that no generation should be necessary. The
tooling should be able to test contract adherence of implementations
just by using these annotations when the test is run.
Regarding annotation retain policies, I also was too focused on open-
source. It would definitely make sense that the policy for these
annotations would be at least CLASS (for IDEs that process bytecode
directly), if not RUNTIME (for IDEs that use reflection), to allow
user code to adhere to the contracts of binary distributions.
Jochen Mader
coupling between sourcecode.
I fear the problem here is that this decision strongly depends on coding style and project requirements.
Just two examples:
1: Several people are developing entities for our software based of a set of interfaces we defined.
Here we want to have a strong relation between interface and behaviour that should be visible to the developers.
The developers should be able to see the behaviour defining class. We also want to be able to verify behaviour at runtime
when we add a service from an external provider (basically testing a service before we allow it to enter the system).
2: We are currently also developing some serverside software. The software is meant to be run in a memory restricted environment.
We don't want to ship the behaviour classes because we are not doing runtime evaluation of the contracts and because we need to
save ram (sorry for that monster of a sentence). We also don't need to ship the contracts as there are only our own developers
working on this one.
In my eyes the best approach would be to support both scenarios:
strong coupling and no coupling
Cheers
Jochen
--
Jochen Mader
Development Lead
Dipl. Inf. (FH)
+49 941 604889701
Pramari
Bruderwöhrdstr. 15b
93055 Regensburg
Stefano Maestri
Jochen Mader wrote on 19/11/08 20:50:
> In my eyes the best approach would be to support both scenarios:
> strong coupling and no coupling
>
both approaches and let users decide.
I think a good implementation of the core could easily support both.
bye
S.
Jochen Mader
(http://www.contract4j.org/contract4j/example) The approach they take is
a little heavyweight in my eyes (contracts get evaluated during runtime
using aspects).
But it might give some ideas for testedby, especially the annotations
they use.
Cheers
jochen