Next Version of web4j

[John O'Hanley]

Hello,

I'm starting work on the next version of web4j.

I will look at any suggestions you may have for changes and additions to it.

I especially like to hear about ideas that will remove some sort of pain that you have experienced.

Regards,
John

[Thomas Hallgren]

Hi John,

The largest pain-point for me is not being able to collaborate and have discussions around the source code. The current
approach where you are the omnipotent owner of the code and doesn't let anyone have an option on the actual code that is
in the making until you release a source package is increasingly uncommon in the open source domain. And for good
reasons too.

I would very much like to see the source for web4j on GitHub so that I can submit pull-requests and have discussions
around concrete changes, submit issues to an issue tracker, and be able to view individual commits and branches. That
would also make it somewhat easier to keep my own clone in sync. Not to mention that it would make your own changes
visible (and hence, subject to discussion) long before they were released.

Today I keep my own private copy of web4j where I've made some changes:

1.
The ability to add dynamic sql statements. It's needed to support safe ways of building queries based on user input
since that in turn controls the number of parameters and sometimes also sort order of the SQL selects in many of the
use-cases that I've had to deal with. Being limited to static statements created a huge number of permutations to a
point where it was no longer feasible.

2.
I had to rewrite parts of the database initialization code since our app connects to a large number of data sources and
it's highly likely that some of them are offline at times. Hence, resilience to connection failure was of essence and we
wanted the app to boot OK regardless (as long as it didn't affect the database used for authentication) and then instead
fail when attempts were made to access the failing data sources from a feature.

3.
I've removed the web4j datetime implementation completely since we didn't consider the motivation for it strong enough.
The team I've been working with are all quite happy with the support that java.util provides and want to write reusable
code.

4.
All tests were removed from the actual source code and instead moved into a separate source tree.

Some suggestions:
It would be nice if each feature had it's own namespace for the SQL statements so that use of the same name on
statements didn't cause confusion in the overall app. Possible with the addition of a global namespace to enable the
current behavior.

web4j performs a lot of "injection" of custom classes when it boots up. It would be nice of all of that was controlled
using a proper injection framework such as Google Guice.

Why not use annotations rather than relying on global public fields of specific classes? Seems to me, that would be both
safer and offer more flexibility.

A proper build system should be set up (perhaps using Travic-CI or Maven)?

Regards,
- thomas

> --
> You received this message because you are subscribed to the Google Groups "web4j-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
> web4j-users...@googlegroups.com.
> To post to this group, send email to web4j...@googlegroups.com.
> Visit this group at http://groups.google.com/group/web4j-users.
> For more options, visit https://groups.google.com/groups/opt_out.

[John O'Hanley]

My responses are below,
>>like this...

The largest pain-point for me is not being able to collaborate and have discussions around the source code.
The current approach where you are the omnipotent owner of the code and doesn't let anyone have an option
on the actual code that is in the making until you release a source package is increasingly uncommon
in the open source domain. And for good reasons too.

>>This forum allows for discussion at any time. I send out the email before I start a new release just to prompt suggestions. I think that's a good idea, and I'll continue to do so.
The web4j code base isn't excessively complex, so informal methods seem to work fine in this case.
Yes, I'm the sole committer and omnipotent god-emperor of web4j ;) It's seems to have worked fairly well so far.

I would very much like to see the source for web4j on GitHub so that I can submit pull-requests and
have discussions around concrete changes, submit issues to an issue tracker, and be able to view
individual commits and branches. That would also make it somewhat easier to keep my own clone in sync.
Not to mention that it would make your own changes visible (and hence, subject to discussion)
long before they were released.

Today I keep my own private copy of web4j where I've made some changes:

1.
The ability to add dynamic sql statements. It's needed to support safe ways of building
queries based on user input since that in turn controls the number of parameters and sometimes
also sort order of the SQL selects in many of the use-cases that I've had to deal with.
Being limited to static statements created a huge number of permutations to a point where
it was no longer feasible.

>>Are you not using the DynamicCriteria class for this?:
http://www.web4j.com/UserGuide.jsp#SearchAndSortOperations
If not, why?

2.
I had to rewrite parts of the database initialization code since our app connects
to a large number of data sources and it's highly likely that some of them are offline at times.
Hence, resilience to connection failure was of essence and we wanted the app to boot OK regardless
(as long as it didn't affect the database used for authentication) and then instead fail when
attempts were made to access the failing data sources from a feature.

>>Yes, we talked about this before. I will take a look at it again. I will ask to take a look at
your code, but not today.

3.
I've removed the web4j datetime implementation completely since we didn't consider the motivation
for it strong enough. The team I've been working with are all quite happy with the support that
java.util provides and want to write reusable code.

4.
All tests were removed from the actual source code and instead moved into a separate source tree.

Some suggestions:
It would be nice if each feature had it's own namespace for the SQL statements so that use of the
same name on statements didn't cause confusion in the overall app. Possible with the addition of a
global namespace to enable the current behavior.

>>Yes, I have had this idea as well in the back of my mind for a long time too. I should be able
to do something for this for the next release.

web4j performs a lot of "injection" of custom classes when it boots up. It would be nice of all
of that was controlled using a proper injection framework such as Google Guice.

>>Where's the pain of the current system? I'm not in pain. If there's no pain, I won't be working on it.
I want to work on actual pain points. The current impl is rather simple, and rather elegant.
You just write code, with little or no explicit config at all.

Why not use annotations rather than relying on global public fields of specific classes?
Seems to me, that would be both safer and offer more flexibility.

>>Where's the pain? Show me the pain of the current system.

A proper build system should be set up (perhaps using Travic-CI or Maven)?

>>Yes, the source distribution is way too bare bones.
I will add an Ant build.xml and a README, and perhaps all of the jars I'm compiling against.
But that's all. Maven gives me feelings of existential nausea. Sorry, not going there.

- John

[Thomas Hallgren]

On 2013-09-24 14:07, John O'Hanley wrote:
>
> The largest pain-point for me is not being able to collaborate and have discussions around the source code.
> The current approach where you are the omnipotent owner of the code and doesn't let anyone have an option
> on the actual code that is in the making until you release a source package is increasingly uncommon
> in the open source domain. And for good reasons too.
>
> >>This forum allows for discussion at any time. I send out the email before I start a new release just to prompt
> suggestions. I think that's a good idea, and I'll continue to do so.
> The web4j code base isn't excessively complex, so informal methods seem to work fine in this case.
> Yes, I'm the sole committer and omnipotent god-emperor of web4j ;) It's seems to have worked fairly well so far.

It works well for you. I have sincere doubt it works well if the objective is to build a thriving community around the
code base. The amount of activity on this group can serve as good evidence that my suspicion is well founded.

>
> 1.
> The ability to add dynamic sql statements. It's needed to support safe ways of building
> queries based on user input since that in turn controls the number of parameters and sometimes
> also sort order of the SQL selects in many of the use-cases that I've had to deal with.
> Being limited to static statements created a huge number of permutations to a point where
> it was no longer feasible.
>
> >>Are you not using the DynamicCriteria class for this?:
> http://www.web4j.com/UserGuide.jsp#SearchAndSortOperations
> If not, why?
>

Because the DynamicCriteria was far too limited for what we were doing. In many cases we don't have a "static" and
"dynamic" part. For instance, we're building unions of queries. Each select included in the union needs the same
predicates. The predicates vary depending on user input, both in number and in what they filter.

Another example is when you have varying number of predicates in predicates that in themselves are sub selects.

We also have selects that will select different column for a specific column alias depending on user input (the
application is used for accessing old legacy databases and a few of them have really bad design).

Our solution for this is to have SQL that is assembled by named fragments. The actual assembly happens in runtime since
there's no way we can predict all combinations on beforehand.

> >>Where's the pain of the current system? I'm not in pain. If there's no pain, I won't be working on it.
> I want to work on actual pain points. The current impl is rather simple, and rather elegant.
> You just write code, with little or no explicit config at all.
>

The largest pain point is the mix of languages. I'm writing an application in Java. Why should I need to declare classes
in an XML file? I want my app to be tight and fit together with near 100% direct impact analysis from the IDE if I
misspell something.

> Why not use annotations rather than relying on global public fields of specific classes?
> Seems to me, that would be both safer and offer more flexibility.
>
> >>Where's the pain? Show me the pain of the current system.
>

An annotation makes a statement about the thing that it annotates. You don't intuitively see that the fields are used by
web4j. Also, turning it around, it's a bit cumbersome to see what things that I can add to an action and how it will be
recognized by web4j. A well documented set of annotations would make that much clearer. Now it's just "magic" that
happens under the cover.

> A proper build system should be set up (perhaps using Travic-CI or Maven)?
>
> >>Yes, the source distribution is way too bare bones.
> I will add an Ant build.xml and a README, and perhaps all of the jars I'm compiling against.
> But that's all. Maven gives me feelings of existential nausea. Sorry, not going there.
>

We all have our preferences. The advantages with Maven is probably more in distribution and ease of consumption than in
the actual build (which, in this case would be incredible simple). You can do everything with ant of course, it's just
ten times more lines of XML.

- thomas

[John O'Hanley]

My objective isn't to have a lot of traffic on this mailing list: my objective is to have
zero traffic on this mailing list. Why? Because every posting to a mailing list
represents a failure of some sort, a source of real pain to someone: a bug, or something is too hard to
understand, or poor documentation, or a feature that's not present but really should be.

web4j is my first born child. Forgive me if I trust no one to the point of giving them commit
access to it. Sorry. And I don't wish its feature list to grow uncontrollably either. That would be against its basic nature, and against the reason why it exists in the first place - as a way of keeping
things as simple as possible, for the most common use cases.

Re:complex dynamic sql.
Yes, web4j doesn't help when the sql gets really complex.
But should it? Could it do that without running into excessive complexity? That's the question.
Do you have a suggestion? If so, please post as a separate topic - this one is too big already...

- John

[Thomas Hallgren]

On 2013-09-24 14:07, John O'Hanley wrote:

My responses are below,
>>like this...

The largest pain-point for me is not being able to collaborate and have discussions around the source code.
The current approach where you are the omnipotent owner of the code and doesn't let anyone have an option
on the actual code that is in the making until you release a source package is increasingly uncommon
in the open source domain. And for good reasons too.

>>This forum allows for discussion at any time.

Perhaps I should clarify that when I say "discussions around the source code" then I mean discussions about specific commits, specific lines of code, etc. All the things that GitHub allows you to do very nicely. Example: https://github.com/blog/785-pull-request-diff-comments

- thomas

[Thomas Hallgren]

On 2013-09-24 15:17, John O'Hanley wrote:
>
> My objective isn't to have a lot of traffic on this mailing list: my objective is to have
> zero traffic on this mailing list. Why? Because every posting to a mailing list
> represents a failure of some sort, a source of real pain to someone: a bug, or something is too hard to
> understand, or poor documentation, or a feature that's not present but really should be.
>

To expect that all discussions around a piece of software are signs of failure, that's really weird in my view. What
about new improvements, elaborations, new use-cases, new ideas, or keeping up with ever changing surroundings?

> web4j is my first born child. Forgive me if I trust no one to the point of giving them commit
> access to it.

You don't need to. You need to be able to view pull requests, have discussions around them, and then perhaps eventually
accept them when they are to your liking.

> Sorry. And I don't wish its feature list to grow uncontrollably either. That would be against its basic nature, and
> against the reason why it exists in the first place - as a way of keeping things as simple as possible, for the most
> common use cases.
>

I doubt you would see it grow uncontrollably. But you would show some good faith and that would perhaps attract new
users. All of them interested in improving the quality. Perhaps people would start creating extensions in repositories
of their own, etc. etc.

- thomas