[Mifos-developer] Mifos i18n issues, may be 2000+ out of 7000+ keys are not being used

[Udai Gupta]

Hi Folks,

Here is some data which I wanted to share.

Property keys and their occurrence in java(but not Test.java), jsp,
ftl and rptdesign(BIRT) files from the code base
http://pastebin.com/raw.php?i=zHsugL9j (csv format)

Properties seems problematic to me. excluded these from search
http://pastebin.com/raw.php?i=23cLRtwz

Code
http://pastebin.com/raw.php?i=pKQLAzRw

I am not going remove anything yet. I have some more things to check
before giving any conclusion.

I am just opening a discussion for more inputs.

Cheers
Udai

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2dcopy1
Mifos-developer mailing list
mifos-d...@lists.sourceforge.net
Unsubscribe or change settings at:
https://lists.sourceforge.net/lists/listinfo/mifos-developer

[Keith Woodlock]

Udai,

>From the subject heading I am gathering that your main point is that
2000+ out of 7000~ properies are not being used in application. I am
not that surprised by this. Clearly its not ideal and I can see that
this would be a pain for the likes of niklas as they will end up
translating properties not even used in some cases.

The ones we know for certain being used are the ones used by .ftl files (newer).

Of the older property files, some of these maybe completely obsolete
due to UI refactoring but its hard to say. Does your analysis indicate
which property files are no longer in use in the application and what
properties in each file are not in use also?

Note: There is some work that could be done that would allow for a
number of legacy .jsp ==> property files to be removed. At present the
only 'product' related screen left for 'move to .ftl' is the loan
product screen (create, read,update)

see http://mifosforge.jira.com/browse/MIFOS-4465

doing this makes it easy to do MIFOS-4466

There also a number of bugs and suggested improvements around the loan
product screeen

query using the following: "project = MIFOS AND component = "Product
Definition" AND resolution = Unresolved ORDER BY key DESC" in jira

Keith.

[Udai Gupta]

> >From the subject heading I am gathering that your main point is that
> 2000+ out of 7000~ properies are not being used in application. I am
> not that surprised by this. Clearly its not ideal and I can see that
> this would be a pain for the likes of niklas as they will end up
> translating properties not even used in some cases.

Yes, and we want to do this cleanup.

> Of the older property files, some of these maybe completely obsolete
> due to UI refactoring but its hard to say. Does your analysis indicate
> which property files are no longer in use in the application and what
> properties in each file are not in use also?

The program does a simple string search among files in code base to
find the occurrence of properties which are well in namespace
"admin.role.create.instruction", it ignores one word properties(key)
like "click", "submit". There are about 100+ properties which are one
word and makes it hard to say whether those are being used or not just
by a simple search.

But, leaving those 100+ properties there are around 7000+ properties
out of which 2000+ had 0 occurrence.

The link I have mentioned in my prev mail has occurrence data. eg.

AdminResourceUI, admin.role.create.instruction, 10

which means "admin.role.create.instruction" property from
AdminResourceUI.properties is found to be used at 10 places.

I am still trying to make sense out of that, we store some labels in
the database.

I will spend some more time and find out what other ways those 2000+
properties might be used which are not captured by the simple search.

I started looking into this when I saw lot's of questions on TWN about
properties which were not being used.

> Note: There is some work that could be done that would allow for a
> number of legacy .jsp ==> property files to be removed. At present the
> only 'product' related screen left for 'move to .ftl' is the loan
> product screen (create, read,update)
>
>    see http://mifosforge.jira.com/browse/MIFOS-4465
>
> doing this makes it easy to do MIFOS-4466
>
> There also a number of bugs and suggested improvements around the loan
> product screeen

I agree, I think that dead code(jsp) from spring/ftl migration can be
one problem.

I also think that there should be some conventions about the property
key namespace
- key should be well defined e.g. "admin.role.create.instruction" (not
just simple one word like create, submit)
- key should be > 10 but < 100 chars
- key should not contains special characters like &, (), ', or , (link
in the previous mail to such properties)
- property files name should be camelCase instead of under_score to
keep things consistent.

[Keith Woodlock]

Udaid

> I also think that there should be some conventions about the property
> key namespace
> - key should be well defined e.g. "admin.role.create.instruction" (not
> just simple one word like create, submit)
> - key should be > 10 but < 100 chars
> - key should not contains special characters like &, (), ', or , (link
> in the previous mail to such properties)
> - property files name should be camelCase instead of under_score to
> keep things consistent.

Seem like a sensible list of guidelines around properties.

> Yes, and we want to do this cleanup.

Cleaning this up would be a good thing for sure but is it more
important than other things outstanding on the project. I am not
trying to dissuade you from this but just trying to understand where
the urgency has come from to do this?

Keith.

[Jasmine sandhu]

Good morning Keith and Udai,

In the past I was answering questions on the TWN site and submitting patches for unused messages. I was grepping for questionable messages, so this is definitely more efficient, I'll have to look and learn.

One of the things we ran into was that some of the *.loanProduct.* messages like (NotNull.loanProduct*, typeMismatch.loanProduct.* and so forth) are not going to show up anywhere except for the properties file. This is because they are constructed by the LoanProductFormBeanValidator when user input is invalid. This was because the loanProduct pages use JSR validation and if there are others where this has been implemented, then the messages will only show up in the properties file.

Here is the email question about it:
http://groups.google.com/group/mifosdeveloper/browse_thread/thread/a97fda08069b74cb/72202f4936dfc3ad?lnk=gst&q=typeMismatch#72202f4936dfc3ad

There was another, separate discussion about parameterizing these messages. Here are the bugs that were filed and some attempts at fixing:
http://mifosforge.jira.com/browse/MIFOS-3859
http://mifosforge.jira.com/browse/MIFOS-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#issue-tabs

The above might be a digression to the original question of wanting to clean up the messages.properties file, which would be great; but we should be careful not to remove these messages constructed on the fly by the validator that show up only in the properties file.

Jasmine

--
- Climbed Rainier to Raise funds for Asha - please sponsor my climb -
-------------------------------------------------------------------------------------

[Udai Gupta]

Hi Jasmine,

Thanks for pointing out JSR 303 validation properties. I will check
those. It's good to have more eyes, although I am not going to remove
anything without actually making sure that it is not being used at
all.

Cheers,

[Udai Gupta]

> Cleaning this up would be a good thing for sure but is it more
> important than other things outstanding on the project. I am not
> trying to dissuade you from this but just trying to understand where
> the urgency has come from to do this?

I get your point. I agree those issue have to be fixed at some point
which is more important.

[Kay Chau]

I say any work is good work though as long as progress is made :)  If this calls to your heart I say go for it. 

I will admit I have a bit of a bias as I've always wanted to see i18n and l10n cleaned up in mifos :)

Cheers
Kay

[Adam Monsen]

On 10/05/2011 04:33 PM, Kay Chau wrote:
> I say any work is good work though as long as progress is made :)
> If this calls to your heart I say go for it.
>
> I will admit I have a bit of a bias as I've always wanted to see
> i18n and l10n cleaned up in mifos :)

Heh, I was thinking the same thing.

I'm crossing my fingers for this one too!

Dead code slows down development. Translations are part of development.

[Udai Gupta]

Okay, so the latest updates are that after checking/excluding the
database(like permission label from lookup tables) and JSR303
properties there are still 2000+ properties which seems to be not
being used.

http://pastebin.com/4t2n1hyv

Although I would have to make sure that I am completely right, I would
appreciate more eyes on the results.

This code can detect and delete(disabled) the properties which are not
being used.
http://pastebin.com/KDh4JXJk

I am hoping that this way we would be able to automate deletion of
properties while throwing out the jsp/struts code.

There are around 500 properties which are excluded from this search
because of their nature which makes it difficult to include in search
(include JSR303 properties mess)
http://pastebin.com/Pk4jHGev

Apart from that I have very few strings which qualify for the
translation properties but doesn't makes any sense.
e.g. line # 1 of LoanUIResources.properties

Cheers,