[Mifos-developer] Update on Mifos Offline Client

[Vishwas]

Hi All,

The basic functionality of an Offline client is ready and the code base
is located at https://github.com/vishwasbabu/Mifos-Offline-Client.
The Client is built using XUL and Javascript, data is stored in a native
XML database (BaseX) with the aim of reducing coding complexity

For testing the same, restful services returning JSON/XML data for
authentication,getting loan officers,getting collection sheets based on
date and loan officers etc
were required.Since I fell a bit short of time I exposed these services
from mifos DB using native sql queries and a JAX-RS implementation
(RESTEasy), the code base
is located at http://mifoslive.git.sourceforge.net/git/gitweb-index.cgi

Next step would be to fork mifos on GitHub and expose RESTful services
with the same/similar contracts using spring MVC

Client Functionality (Only loan accounts for clients):

1) Data entry operator/Branch manager registers with the offline
client by entering his mifos username,password and url of their mifos
instance
2) Upon successful login, a hash of the users password is stored in
his home directory,future logins do not require an internet connection
3) After logging in,user is taken to "Today" tab where he can fetch
collection sheets for any loan officers in his branch (when internet is
available)
4) While offline, user can enter repayments, fees etc and save them
5) All saved and unsynced data from previous days is visible under
"Unsynced" tab. User can additionally modify any of the unsynced records
and sync them when an
internet connection is available
6)Once synced, records are archived and visible under "Synced" tab.
User can then optionally delete these archived records
7)The "Future" tab can be used to fetch collection sheet data for
future dates. Records displayed in this tab are non editable.
For purposes of ensuring data integrity,only collection sheets for the
next "N-1" days should be downloaded, where "N" is the loan repayment
frequency
in days

--
Thanks& Regards,
Vishwas
9986077286

------------------------------------------------------------------------------
Create and publish websites with WebMatrix
Use the most popular FREE web apps or write code yourself;
WebMatrix provides all the features you need to develop and
publish your website. http://p.sf.net/sfu/ms-webmatrix-sf

[Homo Sapience]

Very nice! Can we see XML structure?  Also some screenshots or demo would be good to see.

[Jakub Sławiński]

Thanks for bringing this up.

I found some screen shots at:
https://docs.google.com/viewer?a=v&pid=explorer&chrome=true&srcid=0B9dBOJINVE7XNGQwN2Y2YjgtYjZkMi00YzhmLTg2MGMtZmQ4MDBmN2Q5ZmE3&hl=en

Since 2.2.x will be exposing the REST JSON API, it should be possible to
easily integrate this tool with Mifos.

Regards,
Jakub.

On 28.10.2011 18:25, Homo Sapience wrote:
> Very nice! Can we see XML structure? Also some screenshots or demo would
> be good to see.
>
>
>
>

> ------------------------------------------------------------------------------
> The demand for IT networking professionals continues to grow, and the
> demand for specialized networking skills is growing even more rapidly.
> Take a complimentary Learning@Cisco Self-Assessment and learn
> about Cisco certifications, training, and career opportunities.
> http://p.sf.net/sfu/cisco-dev2dev
>
>
>
> Mifos-developer mailing list
> mifos-d...@lists.sourceforge.net
> Unsubscribe or change settings at:
> https://lists.sourceforge.net/lists/listinfo/mifos-developer

------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn
about Cisco certifications, training, and career opportunities.
http://p.sf.net/sfu/cisco-dev2dev
Mifos-developer mailing list
mifos-d...@lists.sourceforge.net
Unsubscribe or change settings at:
https://lists.sourceforge.net/lists/listinfo/mifos-developer

[Ed Cable]

Udai,

Do you want to send a brief update to the list on the offline client work that you are carrying on with Vishwas?

https://github.com/vishwasbabu/Mifos-Offline-Client

Perhaps we can consolidate under a new project page here:

http://mifosforge.jira.com/wiki/display/projects/Active+Projects

Ed

2011/10/28 Jakub Sławiński <jslaw...@soldevelo.com>

[Udai Gupta]

Hi,

Vishwas and Nayan created a prototype for offline collectionsheet
around May 2011, few weeks ago I started to think what should be the
client. I have decided to go forward the prototype of Mifos offline
client.

There are few things ahead to make it fully functional.
- Walk through of code. (for me)
- Document and understand client side storage structure (XML)
- REST API in Mifos which were created externally to make the client work.
- Create probably a free style build on ci.
- Make it available for users for general review/testing.
- Get back the feedback, polish the client and release it.

I will create the page under active projects section.

Cheers,
Udai

------------------------------------------------------------------------------
Get your Android app more play: Bring it to the BlackBerry PlayBook
in minutes. BlackBerry App World&#153; now supports Android&#153; Apps
for the BlackBerry&reg; PlayBook&#153;. Discover just how easy and simple
it is! http://p.sf.net/sfu/android-dev2dev

[Vishwas Babu A J]

Hi Udai,

I'll document the following tomorrow

1) Current XSD used for Offline storage
2) Expected rest contracts for the Client and check if similar contracts are available/planned to be released.

I had a look at existing contracts listed in http://mifosforge.jira.com/wiki/display/MIFOS/How+To+-+REST+API#HowTo-RESTAPI-Loans.
1)The documentation talks about JSON responses, do we also support XML also? (We could convert JSON to XML in the client if it is not available)
2)Are the  REST Contracts for the next release finalized? If so, where can I find them?

Thanks and Regards,
Vishwas

[Udai Gupta]

> I had a look at existing contracts listed in
> http://mifosforge.jira.com/wiki/display/MIFOS/How+To+-+REST+API#HowTo-RESTAPI-Loans.

> 1)The documentation talks about JSON responses, do we also support XML also?
> (We could convert JSON to XML in the client if it is not available)

No, but we can have XML output, I would rather keep it json only as we
already have json2xml and xml2json plugins in flamingo. But if you
think this will give us performance(or any other kind) benefit then I
can enable XML output also.

> 2)Are the  REST Contracts for the next release finalized? If so, where can I
> find them?

Those are probably not documented, but you can see issues created
(open) in last 20 days to see if new contract development are in
progress. I think let's just create our requirement without looking on
existing ones (as there are not many) and then we can figure out the
overlap and duplication.

[bankadmin]

Is that planned for release 2.2.2 ?
(even partly?)

[bankadmin]

XSD will be very good start and very useful doc!  Thank you very much,

[Udai Gupta]

> Is that planned for release 2.2.2  ?
> (even partly?)

Yes, we are trying to get REST API related work in 2.2.2

[Vishwas Babu A J]

Udai,

The REST services required for an Offline Client are described in
https://docs.google.com/a/confluxtechnologies.com/document/d/1dZWg7Dw6kGJbpowjHJRCgdX3r9EI3p2K2yCGcC0s6XM/edit?hl=en_US
(XSD's need to be updated with saving accounts)

While json services would be sufficient for retrieving Loan Officers
etc, It would probably be helpful if we could also have services
spitting out XML for getting/updating Collection sheets. This would be
helpful as we could store an entire collection(s) sheet fetched from the
server as is into the XML database and push data from the XML DB back to
the server (Would probably improve performance on a slow client machine
by removing overhead of xml->json conversion for large collection
sheets...though it could be argued that more network bandwidth would be
required)

Ideally we could have a single URL and pick the most appropriate
provider based on the Accept header sent from the client (JAX-RS
implementations like RESTEasy do this...though I am not sure about
Spring MVC..)

Thanks and Regards,
Vishwas

[Udai Gupta]

> The REST services required for an Offline Client are described in
> https://docs.google.com/a/confluxtechnologies.com/document/d/1dZWg7Dw6kGJbpowjHJRCgdX3r9EI3p2K2yCGcC0s6XM/edit?hl=en_US
>  (XSD's need to be updated with saving accounts)

Thanks

> While json services would be sufficient for retrieving Loan Officers etc, It
> would probably be helpful if we could also have services spitting out XML
> for getting/updating Collection sheets. This would be helpful as we could
> store an entire collection(s) sheet fetched from the server as is into the
> XML database and push data from the XML DB back to the server (Would
> probably improve performance on a slow client machine by removing overhead
> of xml->json conversion for large collection sheets...though it could be
> argued that more network bandwidth would be required)

Let's keep this in the TODO list, we can deal with JSON for now, and
later will do the performance v/s network overhead discussion. it's a
non issue for now as we have json2xml in client and we can enable xml
on server side very easily.

> Ideally we could have a single URL and pick the most appropriate provider
> based on the Accept header sent from the client (JAX-RS implementations like
> RESTEasy do this...though I am not sure about Spring MVC..)

This is the patch that is needed configuration with spring.oxm and
xstream library in pom.xml to enable XML output.
http://pastebin.com/raw.php?i=ZfTQL24h

The URL remains same, it also work based on Accept header too, but I
like using prefix to the url like .json or .xml by which you can avoid
Accept header.
http://static.springsource.org/spring/docs/current/spring-framework-reference/html/mvc.html#mvc-multiple-representations

[Binny Gopinath]

All,

I am afraid we are getting into a lot of technical details before analysing fully the requirements for the offline module.

There are some tricky parts to this module: for example:
- What happens if the meeting is on 28th and on 29th we are trying to "upload" a transaction which happened on 25th. Mifos by default will throw out this transaction because it is before the last meeting date.
- If we post a deposit to savings account that happened on 20th. The upload is done on 2nd of the next month. On 30th, Mifos would have already computed the interest for this savings account. How will the interest calculation be adjusted/corrected?
- If we have a loan with interest type as "decliniing balance with interest recalculation". What happens if the installment is due on 25th, the transaction actually happened on 29th and the "upload" of this transaction is happening on 3rd of the next month?

I am sure there will be more such scenarios which we will need to brainstorm about.

I am attaching a 2-pager with what I think is the requirement for this module. I will update this document in the next few days based on the discussions I am having with some of my prospective clients.

I have also attached this document at:
http://mifosforge.jira.com/wiki/display/projects/Offline+Data+Entry

Thanks
Binny

[Emily Tucker]

I have been very excited about the traction around an offline module—and I also share Binny’s concern.  It’s important to figure out how to handle all the nasty edge cases—so data doesn’t get corrupted.  The other edge case is how to handle when products/fees are updated in the Head Office, but haven’t been synched up with the offline offices.

Given that this is just the first release of the offline module, the best approach might be to specify a limited set of features that won’t be supported—things like savings, declining balance, etc.   Although I’m sure you are thinking about this already…..

 

Cheers,

Emily

[Binny Gopinath]

All,

I have published a document (OfflineDataCollection_RequestResponseFields_Draft) at:

https://docs.google.com/document/d/1XLW1fR8JX0QG7LC7t8n23pschk4wsxSFjjcZRhv7vzM/edit

This gives my draft view of what the requests and responses should contain for the various APIs that is needed (server side) to support an offline module.

I have classified my priorities as 1, 2 and 3 of the various APIs that is needed. I have only detailed the requests and responses for the Priority 1 APIs.

What I would suggest as next steps would be to have a sub-group call on this (Udai, Vishwas, Ed, Jakub, Kakhaber and anyone else who would like to join) to have a walk through of this. I can be available on Friday or Monday (4th Nov or 7th Nov for a call). Let me know if this makes sense.

Any feedback on this is appreciated.

Thanks
Binny

[George Iberia]

Just to remind : most important is  A) Server side API and Structure of the File (XML. XLS, CSV or whatever, I prefer XML). Also it is very important to get approval from all key members and to include it in next  release . This type of enhancement  can not be done by one member or even group.  CORE always must be

Centralized and agreed by all ( or significant part)  members.  

Questions:

1.      1.  Do you agree to use XML as import format ?

2. 2.      If yes is the schema suggested by Binny accepted? 

3. 3.    How much LOE is requested to have  1^st version of the API for import ?  Can it be done in 2.2.2? At least as a 1^st iteration?

To support community for those (I think useful )  changes I agree to use new (latest) release with import API in one of the implementation project and to test it in pre - production mode.  Also: if we will succeed  we have a small gift to programmer who will do that,  no big expectations please , I am also from poor part of the world, but we like to share success and profit with our friends  :-) .   

 

[Udai Gupta]

Given that you have logged issue.
http://mifosforge.jira.com/browse/MIFOS-5152

I wonder if you are looking for something m-pesa plugin with back
dated transaction enabled?

Does anyone know if you can get advance collectionsheet (next 2-3
meetings) in Mifos, I don't think it is possible currently in Mifos
(online).

Next collectionsheet depends on what is submitted in current collectionsheet.

Udai

------------------------------------------------------------------------------
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1

[Ed Cable]

Binny,

I would agree that a meeting would be best to discuss this.  Kakhaber from FINA will be sending out a meeting invite shortly for Friday.

Here is a link to his offline requirements that I've added to the project page as well:

http://projects.fina2.net/mfi/odc

Ed

------------------------------------------------------------------------------
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1

[Ed Cable]

Hello all,

A small group of us (Binny, Udai, Kakha, Lukasz, Ed) met today to discuss this further. Notes from the call:

http://willyou.typewith.me/p/mifos-offline-11-4-2011

[George Iberia]

Thanks everyone who participated in meeting. Please if someone has a data format and description for import publish it here whatever version and format you are using / suggesting. 

[Udai Gupta]

Hi Binny,

The rest API is documented here.
http://mifosforge.jira.com/wiki/display/MIFOS/REST+API+Reference#RESTAPIReference-Collectionsheet

STEP: Get the information in one format, convert it into save
collectionsheet format and submit.

I have used this acceptance test db dump.
https://github.com/mifos/head/blob/master/db/src/test/resources/sql/acceptance_test_dump.sql

check data on Mifos using "Enter collectionsheet"
MyOfficeDHMFT - > loan officer -> Default Center
this will set the date as 22-4-2011, this collectionsheet does not
include all cases, I will create better test case later.

You can use Google Chrome "Simple REST Client" extension for REST
manual testing.
https://chrome.google.com/webstore/detail/fhjcajmcbmldlhcimfajhfbgofnpcjmb

JSON to XML convertor.
http://jsontoxml.utilities-online.info/

I also think that once the client will be ready it will make more
sense as same data model is used for online collectionsheet.

Udai

------------------------------------------------------------------------------
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1

[George Iberia]

Found allot of   EXCEL to JSON converters.  But would like to coordinate it with the team. Which one you are using or recommending ?

[Binny Gopinath]

Hi Udai,

I need some help.

I installed the WAR file from Hudson. Installed a REST client.
I created a simple center with just 1 group and within that 1 client and 1 loan for the client.
I am trying to disburse this loan to this client using the saveCollectionSheet API.

And I am able to get a valid response to the getCollectionSheet.

But the saveCollectionSheet gives me errors.

Please see attached the getCollectionSheet response and the corresponding saveCollectionSheet request and the error response for that.

I am using the URL: http://localhost:8030/mifos/collectionsheet/save.json
Method: POST
(I am sending requests from Simple REST Client from Google Chrome as you suggested)

Please let me know if you need any further details to help me.

Thanks
Binny

[Udai Gupta]

Hi Binny,

There is nothing wrong with your JSON, it's the REST call unable to
understand this JSON, It's something I have to fix on the server side
in REST API.

Thanks,

[Udai Gupta]

Binny,

You would need the latest war, I have made the change so that save
collectionsheet works properly.

I had to remove one comma from the your save collection
"attendanceId":1}] which was earlier "attendanceId":1},]

[Binny Gopinath]

Hi Udai,

I downloaded the latest war file from the link below:

http://ci.mifos.org/hudson/job/head-g-release-archive/lastSuccessfulBuild/org.mifos$mifos-war/artifact/org.mifos/mifos-war/1.9-SNAPSHOT/mifos.war

Installed this. And then tried the following using the Simple REST Client from Google Chrome.

URL: http://localhost:8030/mifos/collectionsheet/save.json
Method: POST

Headers: <Blank>
Data: The contents of the request file attached.

I am getting the error: Required String parameter 'json' is not present

Please see attached the full response.

Any idea what I am doing wrong?

Thanks
Binny

[Udai Gupta]

Try this as url

http://localhost:8030/mifos/collectionsheet/save.json?json={"userId":1,"transactionDate":[2011,11,09],"receiptDate":null,"saveCollectionSheetCustomers":[{"customerId":10,"parentCustomerId":9,"saveCollectionSheetCustomerSavings":null,"saveCollectionSheetCustomerIndividualSavings":null,"saveCollectionSheetCustomerLoans":[{"accountId":17,"currencyId":84,"totalDisbursement":1000000,"totalLoanPayment":0}],"saveCollectionSheetCustomerAccount":{"accountId":16,"currencyId":84,"totalCustomerAccountCollectionFee":0},"attendanceId":1}],"paymentType":1,"receiptId":""}

the json string is send now as parameter json instead of data which
still can be passed as post for parameter.

Uda