[GSOC 2012] Improve eCommerce and Payment modules

377 views
Skip to first unread message

Ryan Dao

unread,
Mar 22, 2012, 9:48:47 AM3/22/12
to silverst...@googlegroups.com
Hi all,

I was playing around with the Silverstripe eCommerce module to gain more insights on how the Payment module works. I realized that eCommerce is a good module but still has room for some improvements. So I decided to propose a project for GSOC 2012 to improve both the Payment and eCommerce module. Here are the project objectives:

- Improve admin interface: The admin interface lacks the ability to change settings like confirmation email template, money currency, allow purchasing, etc. I don't think everybody is comfortable with modifying the config.php file every time they need to make some setting changes. Browsing, creating, and editing products are also not very user-friendly. If a store has hundreds or thousands of products, it would be very difficult for the admin to perform those tasks. So I think some improvements for the admin interface are much needed.

- Simplify installation: It took me quite a while to get the eCommerce module up and running properly. The documentation and the config.php file do not agree with each other. There are also some bugs here and there during the installation process. So I suggest we should have something like Drupal Commerce's Kickstart installation package http://drupal.org/project/commerce_kickstart. The idea is to put everything including the Payment module in one package and run an installation script that takes care of configuration, database schema as well as testing. It'd make it much easier for site owners to create an ecommerce website using Silverstripe.

- Add more features to Product: More features can be added to Product like multiple images, sales and promotion, stock availablity, etc.

- Improve Payment module (taken from the GSOC project idea document): There should be an admin interface to manage payment and configure payment gateways. I haven't studied the code much, but from what I see, the APIs can be more standardized to make it easier for developers to extend the module.

- Inventory tracking & analysis: An inventory tracking system can keep track of what is in stock, what is on order, and what has been sold. An analysis & reporting system can help site owners see which items are best-sellers, how much they've earned, and how they can improve their sales. I think they are very nice features to have.

I'm new to Silverstripe, so I'm not quite sure how feasible this proposal is. I hope to receive some inputs and suggestions from everyone.

Best regards,

Ryan Dao

Frank Mullenger

unread,
Mar 22, 2012, 4:11:30 PM3/22/12
to silverst...@googlegroups.com
Hey Ryan,

There are a few e-commerce modules for SilverStripe out there now. There are at least a couple of forks of the original SilverStripe ecommerce module afaik as well as SilverCart and a module I recently released called SwipeStripe.

The SilverStripe e-commerce forks and SwipeStripe both make use of the Payment module, it was after talking with Jeremy Shipman who maintains a fork of SilverStripe e-commerce that the Payment module project for GSOC came about. There is a group of developers who would really like to see some work on the Payment module and there's been some discussion on it for some time. So it's great that you're taking an interest in this area, I think you'll have lots of support and input.

Comments inline:

On Fri, Mar 23, 2012 at 2:48 AM, Ryan Dao <daoduyd...@gmail.com> wrote:
Hi all,

I was playing around with the Silverstripe eCommerce module to gain more insights on how the Payment module works. I realized that eCommerce is a good module but still has room for some improvements. So I decided to propose a project for GSOC 2012 to improve both the Payment and eCommerce module. Here are the project objectives:

- Improve admin interface: The admin interface lacks the ability to change settings like confirmation email template, money currency, allow purchasing, etc. I don't think everybody is comfortable with modifying the config.php file every time they need to make some setting changes. Browsing, creating, and editing products are also not very user-friendly. If a store has hundreds or thousands of products, it would be very difficult for the admin to perform those tasks. So I think some improvements for the admin interface are much needed.

There are definitely some improvements that can be made for admin interfaces. It might be an idea to look at new features in SS3 and how they fit in or improve existing interfaces. I think the module is intended to use the _config.php - as more of a tool for developers to create custom e-commerce sites rather than providing all the options for users to configure via a gui. 
 

- Simplify installation: It took me quite a while to get the eCommerce module up and running properly. The documentation and the config.php file do not agree with each other. There are also some bugs here and there during the installation process. So I suggest we should have something like Drupal Commerce's Kickstart installation package http://drupal.org/project/commerce_kickstart. The idea is to put everything including the Payment module in one package and run an installation script that takes care of configuration, database schema as well as testing. It'd make it much easier for site owners to create an ecommerce website using Silverstripe.

I think there is another GSOC project that might intersect here which concerns module management, dependencies, installing and upgrading. 
 

- Add more features to Product: More features can be added to Product like multiple images, sales and promotion, stock availablity, etc.

- Improve Payment module (taken from the GSOC project idea document): There should be an admin interface to manage payment and configure payment gateways. I haven't studied the code much, but from what I see, the APIs can be more standardized to make it easier for developers to extend the module.

There are lots of ideas for improving the Payment module, Jeremy Shipman has put together some here:

The payment module is really useful, I've used it in at least a couple of different projects selling bookings and custom house plans as well as the e-commerce module I've been working on.

There are some tricky use cases like recurring payments and processing refunds. It would be great to tie this sort of functionality into an admin interface in SilverStripe. 

There is quite a lot of work and complexity in recurring payments alone and no adequate system at the moment - but it would open up the door to make developing SaaS applications in SilverStripe a lot easier and I think that has huge potential especially with SS3.

I think there could be some interesting work to improve the API for the payment module and perhaps borrow some ideas from other projects like Zend, Pear etc. Also, there are a bunch of payment gateways that are not currently supported - some cool new ones like Stripe for instance, but also some big ones like Google checkout that are really needed (we need more alternatives to PayPal!). 

It's not too tricky to add new gateways in, but there are always some differences between each gateway and the module API needs to be flexible enough to support those differences. 

Most of the existing gateways in the module do not have any unit tests and this is a bit of a problem. Recently I rewrote the Paystation gateway integration, I'm not sure if the API has changed on Paystation's end since the gateway was originally added but it wasn't working for me and there were no Unit tests for me to quickly find out. I think the unit testing system is an important component to get right and could be improved quite a bit.
 

- Inventory tracking & analysis: An inventory tracking system can keep track of what is in stock, what is on order, and what has been sold. An analysis & reporting system can help site owners see which items are best-sellers, how much they've earned, and how they can improve their sales. I think they are very nice features to have.

I won't touch on this too much but personally have been looking at integration with VendHQ API to help clients handle stock etc.

Cheers,
Frank
 

I'm new to Silverstripe, so I'm not quite sure how feasible this proposal is. I hope to receive some inputs and suggestions from everyone.

Best regards,

Ryan Dao

--
You received this message because you are subscribed to the Google Groups "SilverStripe Core Development" group.
To view this discussion on the web visit https://groups.google.com/d/msg/silverstripe-dev/-/_iKrEw2NunUJ.
To post to this group, send email to silverst...@googlegroups.com.
To unsubscribe from this group, send email to silverstripe-d...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/silverstripe-dev?hl=en.

Jeremy Shipman

unread,
Mar 22, 2012, 5:36:19 PM3/22/12
to silverst...@googlegroups.com
Hello all,

Ryan, can you confirm which version of eCommerce you are talking about? As Frank has mentioned, there are a few different versions out there. Here all all I currently know of:

I understand all are still being actively developed, and I'm going to assume that it'll be up to you to choose which module you will work with, as long as SilverStripe LTD approves.

As the maintainer of the Shop module, which is a semi-recent fork of eCommerce, I'm happy to answer any questions you have regarding the module.

One major addition to your list of things to improve, would be mere need to upgrade the module to SilverStripe 3. I have started an attempt at migrating on a branch, but I have not yet pushed changes to github.

Regarding simplifying installation, I have created a fork of the SilverStripe installer to aid in setting up a shop: https://github.com/burnbright/silverstripe-installer/tree/shop . There is plenty of room for improvement however.
Regarding configuration, my approach has been to leave configuration out of the CMS, unless necessary. Most configuration is done when the site is set up, and then never touched again. Any changes would simply require contact with the site's developer, which I believe is a healthy thing to do to keep store-owner/developer relationship going.

Here is a document I wrote, with some more high-level thoughts on the module:
https://docs.google.com/document/d/1dQY6cimJXypTsAdfP_q-8UdjpB0Vl8PiVMEyrp7XmYQ/edit

I should also make you aware that there is some separate ongoing discussion for both eCommerce and shop in this google mailing list: https://groups.google.com/forum/?fromgroups#!forum/silverstripe-ecommerce.

regards,
Jeremy

Ryan Dao

unread,
Mar 23, 2012, 11:38:41 AM3/23/12
to silverst...@googlegroups.com
Hi,

Thanks Frank and Jeremy for your inputs. Just to confirm, I was talking about the eCommerce module. I wasn't aware of other versions like Shop, SilverCart...

Jeremy, can you tell me the core differences between Shop and eCommerce? I installed your Shop module and didn't notice much difference. Also I'm not sure whether it's just me but I found it quite buggy. I couldn't add image for a product, and some of the buttons didn't work properly. I'm gonna dig into the code and find out what's wrong. 

Thanks Frank on your detailed information on the Payment module. Now I understand why there's a whole GSOC on improving the Payment module. You mentioned that there's no adequate system for recurring payment at the moment. Can you let me know a bit more what are the difficulties the developers of the Payment module are facing regarding recurring payment?

Thank you.

Regards,

Ryan

Frank Mullenger

unread,
Mar 23, 2012, 8:07:45 PM3/23/12
to silverst...@googlegroups.com
Hi Ryan,

Regarding recurring payments, one of the trickiest areas I think is Dunning - which is basically sending emails to customers if a charge for a subscription does not go through.

There would need to be a way to customise content of the emails and set the schedule of the reminders. 
How many reminders before the subscription is put on hold, what frequency of reminders after payment is declined. 
How many different reminder emails are there - maybe 3 emails of differing content with increasingly insistent wording :-)
How many times is the card retried and over what period before sending the first reminder or pausing a subscription.
How to gather new credit card info from the customer if credit card is declined.

There is also the API of recurring payments - how can the schedule of the subscription be deducted from the product being sold. This might need to also facilitate upgrade of the subscription, downgrade, setup fee, trial period at a different cost, pausing or discontinuing a subscription and sending out invoices appropriately.

These are just a few ideas, not a complete feature set. I think you would need to first look at the current API for payments, see where that needs improvement keeping in mind some of this functionality for recurring payments and refunds that can be built on top.

Cheers,
Frank

Jeremy Shipman

unread,
Mar 23, 2012, 9:10:50 PM3/23/12
to silverst...@googlegroups.com
Hi Ryan,

With shop, I've taken a step backwards to put a strong focus on making the core easy to use for developers, before moving forward with various features. So yes, you will not notice much difference at this stage.

With images that you need to add the image (uploads image), then click save & publish (links image) to actually get it to work fully. If there's still an issue, please feel free to post a new issue on the github issue tracker. I'm keen to hear what buttons aren't working.

regards
Jeremy

Nicolaas Thiemen Francken - Sunny Side Up

unread,
Mar 25, 2012, 4:48:49 PM3/25/12
to silverst...@googlegroups.com
Hi Ryan

You will be able to find a new version of e-commerce on
www.silverstrripe-ecommerce.com in the next day or two. I have done
some really exciting new work that I am testing at the moment.

We are working really hard to get a 1.0 version released. Are you a
member of silverstrip...@googlegroups.com? Make sure you are,
because all updates will be announced on there.

Cheers

Nicolaas

Ryan Dao

unread,
Mar 26, 2012, 8:41:59 AM3/26/12
to silverst...@googlegroups.com
Hi Jeremy, Frank, and Nicolass,

Sorry for being quite quiet recently. I have been busy studying for a test coming up this Wednesday. I am still considering whether to propose a project on improving eCommerce as a whole or just the Payment module. Personally I think the Payment module has higher priority since it's an essential part of Silverstripe eCommerce but hasn't been taken care of for quite some time (It was last updated 8 months ago according to GitHub). However, I also find some interesting improvements that I can do for the eCommerce module(s).

Can you give me an advice on which project I should choose? 

Regards,

Ryan

Frank Mullenger

unread,
Mar 26, 2012, 5:24:40 PM3/26/12
to silverst...@googlegroups.com
Hi Ryan,

E-commerce is quite fragmented with many different projects and more coming out of the woodwork constantly, for example I read yesterday that Nivanka has an e-commerce system (closed source) that his company is developing and sounds like Tomas Bilek is working on one also.

There are many sites and applications using the Payment module, at least 3 of the e-commerce modules use it but many more apps use it stand alone to process payments for selling all sorts of things like bookings, single products, tickets etc.

Working on the payment module has quite a wide impact, improving the Payment module is only going to enable more of these kinds of projects and I think it could contribute a lot to the SilverStripe ecosystem - as you say it's been a while since much work has been done on the Payment module and it's in need of some attention.

Just the effort of improving the Payment module API and increasing the number of (unit tested) Payment gateways would have a bit impact, is very achievable and would demonstrate experience with payment systems and unit testing to future employers. 

Beyond that effort there are plenty of features that could be added to the Payment system, some kind of stand alone admin area would be useful, UI to manage refunds and/or recurring payments - the recurring payments system is a fair chunk of work also.

I think by working on the Payment module you can contribute to many of the e-commerce systems out there and help developers to create more, the payment module is a very important part of all e-commerce applications.

Cheers,
Frank

Jeremy Shipman

unread,
Mar 26, 2012, 5:35:13 PM3/26/12
to silverst...@googlegroups.com
I second Frank's recommendation to first focus on Payment.

I'm not exactly sure of the GSOC rules, but you could always get
involved with a commerce module once Payment is improved.

Jeremy

Sigurd Magnusson

unread,
Mar 26, 2012, 7:38:46 PM3/26/12
to silverst...@googlegroups.com
Jeremy is your question about whether you can change tack on a project or whether you can work on a commercial software package?

Jeremy Shipman

unread,
Mar 26, 2012, 7:42:38 PM3/26/12
to silverst...@googlegroups.com
My question is - does GSOC allow working on multiple projects within
SilverStripe for a student?, which I think I can answer 'yes' myself.
I'm sure it was done during the last one.

On Tue Mar 27 12:38:46 2012, Sigurd Magnusson wrote:
> Jeremy is your question about whether you can change tack on a project
> or whether you can work on a commercial software package?
>
>
> On 27 March 2012 10:35, Jeremy Shipman <jer...@burnbright.net

> <mailto:jer...@burnbright.net>> wrote:
>
> I second Frank's recommendation to first focus on Payment.
>
> I'm not exactly sure of the GSOC rules, but you could always get
> involved with a commerce module once Payment is improved.
>
> Jeremy
>

Nicolaas Thiemen Francken - Sunny Side Up

unread,
Mar 26, 2012, 8:07:08 PM3/26/12
to silverst...@googlegroups.com
Payment is a big enough project in its own right. It is also an
interesting one, because it serves several e-commerce modules and it
is a fair challenge to make it work in a wide variety of ways. You
will have to put together a solid concept and for me that is what GSoC
is all about.

Ryan Dao

unread,
Mar 28, 2012, 10:03:14 AM3/28/12
to silverst...@googlegroups.com
Hi all,

I've decided to work on the Payment module (in case I can get accepted to GSOC). Here are the list of the things I want to do:

(1) Improve payment gateway API: Make the API flexible enough to accomodate the differences between each gateway. 

(2) Add at least 2 more payment gateways including Google Checkout and Amazon Payments and some other gateways like WorldPay, Dwolla if time allows.

(3) Include unit testing: Create a standard unit testing API for all payment gateways. Each gateway will then have a set of test cases that can quickly verify whether the gateway works. 

(4) Improve recurring payment: Improve the API to handle the differences between payment service providers; Handle errors and send reminders when a payment does not go through. 

(5) Create admin interface to manage payments: Create a PaymentAdmin class that generates an admin UI to manage the two models: Payment and RecurringPayment. For Payment, the interface allows settings for currency and enabling/disabling payment gateways. For recurring payment, the settings include scheduling reminders, customize reminder emails, view/edit/delete existing recurring payments.   

(2), (3) and (5) are quite straightforward and I think I won't have any problem implementing them. For (1), I still haven't been able to come up with a proper new API given the large range of differences between payment gateways. I'll have to look at all payment gateways' APIs and see what they have in common. For (4), as far as I know, each gateway handles recurring payment a bit differently, some don't even support recurring payment. I've only studied Paypal API, no I'm not sure whether it can be done for the others.

 @Frank: Would it be ok for you to send me some working code that integrates the Payment module to a website? I need some reference to fully understand how online payment works. Also, how much detailed should I put into my proposal regarding the technical aspects? Do I have to layout the whole API I'm planning to build as well as test cases? And how can I demonstrate my competency and familiarity with Silverstripe.

Best regards,

Ryan

xeraa

unread,
Mar 28, 2012, 10:35:17 AM3/28/12
to silverst...@googlegroups.com
And how can I demonstrate my competency and familiarity with Silverstripe.


Cheers,
Philipp 

Frank Mullenger

unread,
Mar 28, 2012, 7:10:04 PM3/28/12
to silverst...@googlegroups.com
Hi Ryan,

Excellent, I think working on the Payment module is a good choice. 

Regarding getting set up with processing payments, a good start is getting test accounts for some of the payment gateways. 

DPS / PaymentExpress:
Probably want a pxpay test account - apply here or email in...@paymentexpress.com

PayPal:
Setting up a PayPal test account can be a bit fiddly, here are some instructions - let me know if you have any problems with these.

PayStation:
You just need to email in...@paystation.co.nz and ask for a developer account - some details for developers on PayStation.

After that you can get started by installing the payment module and either the payment test module or an e-commerce module.
My fork of payment module, much the same but with PayStation gateway updated


After installing both of those its just a matter of configuring the payment gateways with your dev account details, to get started really quickly you can just enable cheque payments only, this will allow you to start processing test orders immediately and get a feel for how the system works, in _config.php:
Payment::set_supported_methods(array(
    'ChequePayment' => 'Cheque Or Pay On Site'
  ));

Here is an example of some _config file settings for configuring other payment gateways:

I can help you with any of these settings if you are not sure what to put, the DPS settings are pretty straight forward as well:
//DPS payment gateway
DPSAdapter::set_receipt_from('site...@example.com'); 
DPSAdapter::set_pxpay_account('account', 'api code');

I'm not sure how much technical detail is required in the proposal, the API you implement is likely going to change from whatever you propose and the unit tests are a bit difficult to anticipate also. To demonstrate competency the thread by Philipp explains that the best, heres a quick link to the coding test:
http://www.silverstripe.org/php-coding-test/

Regarding (1) in your email, the existing API is already quite flexible but I think there are areas where it could improve and be tidied up. It's going to be hard to anticipate what the entire API is going to look like from the outset, I would start by tidying up what is there, add some payment gateways to see what little differences they have, then look to improve the API to accommodate those gateways.

I forgot to mention multiple currency support, currently you can set a site currency for the payment module:
Payment::set_site_currency('NZD');
But if the gateway allows your account to settle in more than one currency how can the payment module support a set of accepted currencies.

Cheers,
Frank

Nicolaas Thiemen Francken - Sunny Side Up

unread,
Mar 28, 2012, 7:20:16 PM3/28/12
to silverst...@googlegroups.com
Hi Ryan

That sounds great.  With the new payment module, it would also be important that it still has backward compatibility. 

I have worked on a lot of payment gateways, some of which are not published OS.  I would be good to at least list them as "existing" so that people can request a coy of needed.  I will also see if I can publish them OS.

Cheers

Nicolaas

Jeremy Shipman

unread,
Mar 28, 2012, 8:42:55 PM3/28/12
to silverst...@googlegroups.com
> That sounds great. With the new payment module, it would also be
> important that it still has backward compatibility.

I think if its necessary to break backwards compatibility in order to
achieve a more ideal solution, then that should be ok. SS3 has broken
backwards compatibility, which I assume this module improvement will
fall into line with. Otherwise, yes, maintaining backwards
compatibility is good.

To help with understanding what types of payment solution there are, I
created some sequence diagrams:
https://github.com/downloads/burnbright/silverstripe-payment/PaymentsSequenceDiagram.pdf
Perhaps it would be good to document all possible features, and we can
discuss which should go into the core API.

Question: what would payment gateway unit tests look like? Won't it be
difficult to run unit tests quickly if they need to interface with
remote servers? Are there other unit tests that similarly test web apis?

One more concern I have is that the providers included with the
existing payment module became out of date, and you had to create your
own payment classes like "ImprovedPaystationPayment". My thinking is
that if we don't include any payment types in the core module, this
won't happen. On the other hand, GIT is now making it easy to manage
provider updates. Thoughts?

Dan Rye

unread,
Mar 28, 2012, 9:19:28 PM3/28/12
to silverst...@googlegroups.com
Another payment gateway I highly recommend working with (Especially for reoccurring payments) is Authorize.net.

You can signup for a test account here: https://developer.authorize.net/testaccount/

--
You received this message because you are subscribed to the Google Groups "SilverStripe Core Development" group.
To post to this group, send email to silverstripe-dev@googlegroups.com.
To unsubscribe from this group, send email to silverstripe-dev+unsubscribe@googlegroups.com.

Frank Mullenger

unread,
Mar 28, 2012, 9:24:16 PM3/28/12
to silverst...@googlegroups.com
If we need to break backwards compatibility then perhaps we can just create a branch (or a tag?) in git to keep the old payment module available.

Discussing features sounds like a good idea and perhaps documenting the features of the current payment module would be a good place to start on that.

Unit testing could be difficult in some payment solutions where the customer is redirected to the payment gateway. I'm sure there are still things that can be tested though - for instance with Paystation when the return URL is called the payment is marked as paid etc.

Unit testing gateways where the cc details are sent and a response is sent back from the gateway is doable, there are DPS unit tests currently which I think will provide a good starting point. I'm writing some unit tests to for Xero API at the moment for a project, it can take a while for some tests to run and you generally need to provide some API keys for testing but it is doable.

Maybe it would be good to have the providers in seperate modules? It would reduce the number of unnecessary classes if you could pick just the payment gateways you are using. Depending on how many gateways are included this might not be a problem though. With the new module management system it should be clear when there is a dependency on the payment module so that shouldn't be a problem. Updating the code is pretty straight forward with git, especially if there are unit tests that can be run to check. We can always pull the payment gateways into seperate modules a bit later I guess. 

--
You received this message because you are subscribed to the Google Groups "SilverStripe Core Development" group.
To post to this group, send email to silverstripe-dev@googlegroups.com.
To unsubscribe from this group, send email to silverstripe-dev+unsubscribe@googlegroups.com.

Sigurd Magnusson

unread,
Mar 28, 2012, 9:33:48 PM3/28/12
to silverst...@googlegroups.com
Jeremy - 

I think the answer to your question is yes, but to clarify...

A student must have just one project.
That "project" might entail a single student working on several modules with one open source software package. If multiple modules are being worked on, there should be some common reason, e.g. "Upgrade these 3 popular modules from SS 2.4 to SS 3.0 compatibility". Or "Improve these several ecommerce/payment modules in order to <holistic reason>".


A mentor is permitted to work with more than one student.


(Preaching to the converted, no doubt, but being a student or a mentor is a strong commitment of time so anyone agreeing to take up such a role needs to ensure they can deliver in terms of having time to devote…)


Sig

Jeremy Shipman

unread,
Mar 28, 2012, 9:50:59 PM3/28/12
to silverst...@googlegroups.com
>
> Unit testing gateways where the cc details are sent and a response is
> sent back from the gateway is doable, there are DPS unit tests
> currently which I think will provide a good starting point. I'm
> writing some unit tests to for Xero API at the moment for a project,
> it can take a while for some tests to run and you generally need to
> provide some API keys for testing but it is doable.
Yeah, I guess it's not that difficult if you're mimicking the gateway
responses / requests, rather than actually connecting to them.

Ryan Dao

unread,
Mar 29, 2012, 6:59:02 AM3/29/12
to silverst...@googlegroups.com

Yeah, I guess it's not that difficult if you're mimicking the gateway responses / requests, rather than actually connecting to them.
Hi Jeremy, that's what I'm thinking also. I think the current best way to test a remote API fast is to mock both the request and response behavior rather than directly sending request to the remote server. I'm not sure whether this is a good idea, but I'm planning to gather enough HTTP request/response dialogs to set up the testing environment.

@Frank: Regarding putting the providers in separate modules, I think it's a good idea. It can further decouple the Payment module. And if the new module management system can handle dependencies well like in Drupal, I think developers will not have any problems installing some extra modules.

Frank Mullenger

unread,
Mar 29, 2012, 5:46:21 PM3/29/12
to silverst...@googlegroups.com
Hi Ryan,

Sounds like separating providers into different modules is a good plan then. Mock requests / responses for testing sounds like a good start, it might be nice to have the actual requests sent if test API keys exist - depending on how hard it is to implement with the different gateways. You can speed up the unit tests quite a bit by using the sqlite3 module btw.

Cheers,
Frank

To post to this group, send email to silverst...@googlegroups.com.
To unsubscribe from this group, send email to silverstripe-d...@googlegroups.com.

Jeremy Shipman

unread,
Mar 29, 2012, 5:53:57 PM3/29/12
to silverst...@googlegroups.com

> Sounds like separating providers into different modules is a good plan
> then.
For the record, here are the providers I've already created repositories
for:
https://github.com/burnbright/silverstripe-payment-paystation
https://github.com/burnbright/silverstripe-payment-paymentexpress
https://github.com/burnbright/silverstripe-payment-paypal
...anyone opposed to using that naming scheme? silverstripe-module-submodule

> Mock requests / responses for testing sounds like a good start,
> it might be nice to have the actual requests sent if test API keys
> exist - depending on how hard it is to implement with the different
> gateways.

Perhaps create a static variable like PaymentTest::$make_external_calls,
which determines whether to use external APIs or not. False by default.
I could imagine it would be frustrating if you're developing/running tests
without an internet connection.

Nicolaas Thiemen Francken - Sunny Side Up

unread,
Mar 29, 2012, 6:16:02 PM3/29/12
to silverst...@googlegroups.com
Here are some more I have worked on:



UNPUBLISHED:
payment_scnet                   


WITHOUT THIRD-PARTY PROVIDER


AND ALSO SEE:
payment_epaydk                       


Hope that helps

Nicolaas

Frank Mullenger

unread,
Mar 29, 2012, 6:18:03 PM3/29/12
to silverst...@googlegroups.com
+1 for that naming scheme: silverstripe-module-submodule. I think you should be able to set test API details for a gateway, if they exist send an actual request if not mock the request.

Nicolaas Thiemen Francken - Sunny Side Up

unread,
Mar 29, 2012, 6:33:34 PM3/29/12
to silverst...@googlegroups.com
most gateways provide a test gateway - I think an important part of any payment module should be clear instructions on how to test the gateway.

I also tend to think that the gateway should automatically switch to test if the site in in "dev" mode - and to real credentials when it is in "live" mode.

Here are some other notes about the payment module that will need attention:

1. the field stuff and how this is info is "passed" / "retrieved" from payment classes
    - required fields
    - validation (luhn test for credit cards?) - see http://rosettacode.org/wiki/Luhn_test_of_credit_card_numbers

2. payment specific form fields (e.g. credit card field) could be added to the core module

Hope that helps

Nicolaas

Sam Minnée

unread,
Mar 29, 2012, 6:49:18 PM3/29/12
to silverst...@googlegroups.com

>> Mock requests / responses for testing sounds like a good start,
>> it might be nice to have the actual requests sent if test API keys
>> exist - depending on how hard it is to implement with the different
>> gateways.
> Perhaps create a static variable like PaymentTest::$make_external_calls,
> which determines whether to use external APIs or not. False by default.
> I could imagine it would be frustrating if you're developing/running tests
> without an internet connection.

The best way of doing this is to create a class that is a really, really thin layer around the payment gateway. It might simply be the RestfulService class. Perhaps it's assigned to $this->gateway inside the Payment class. Then you have a setter, constructor argument, or Dependency Injection framework to specify what object should be provided to this gateway. In different parts of your application you can then pass one of 3 implementations:

1. The production service
2. The test service (but still connecting to the
3. A mock object that doesn't call anything

You don't necessarily need to have 3 separate classes - the first two might be the same class but with different properties.
If you like, you can use Phockito (or PHPUnit built-in mocking system, which I hate because of its verbosity) to create the 3rd one. Or you can hand-roll a mock object that does what your tests need it to.

#1 would be used for production deployment
#2 would be used for manual test environments and possibly automated integration testing
#3 would be used for unit tests

You might have a 4th environment, that provides a database-backed mock gateway, where that database back-end is exposed in a modeladmin. This would let you create a self-contained manual testing environment, but it means that you're testing more code that's not actually part of the production system, and it's more work. It's arguable as to whether or not this is worthwhile.

The point is that by using this dependency injection (even without a DI framework this is still dependency injection) you don't need to bake any knowledge of this stuff into your Payment class. It makes your code more testable, more maintainable, and more flexible.

Frank Mullenger

unread,
Mar 29, 2012, 7:42:09 PM3/29/12
to silverst...@googlegroups.com
Thanks, sounds good. I guess I'll need to put aside half a day to read this thread on DI a bit more thoroughly:

So, just to clarify:
  1. The production service connects to gateway and processes payment as it would in a live environment
  2. The test service connects to gateway and processes payment for say a dev environment with test API account settings for the payment gateway - or manual testing
  3. Mock object represents the gateway and responds like the gateway would in order to run unit tests, it would not connect to the gateway it just mimics it.
I don't really understand the database backed mock gateway. When I think of manual testing I think of manually processing a payment for an order, in which case the test service (#2) would be used. Is it for creating payments to test managing them without requiring a test service or can we use fixtures/unit tests instead to test the same kind of things?


Jeremy Shipman

unread,
Mar 29, 2012, 10:00:53 PM3/29/12
to silverst...@googlegroups.com
Yes, thanks Sam for pointing out how the pattern can be used. I like that approach.

I understand this would mean a single payment submodule may contain the following classes:

* MyPayment - DataObject storing payment, containing a DI reference
 * MyPaymentGateway - used for live production, and likely user testing.
 * MyPaymentTestGateway - optional user testing class, when the gateway user testing requires some custom code, and would probably just extend MyPaymentGateway.
 * MyPaymentUnitTestGateway - mock gateway to be used in MyPaymentTest unit tests


What about controllers? This also needs some thought.

You won't really need a controller at all if you are processing payments via a web API. The payment can be processed, and status recorded.

You will however need something if your gateway redirects back to your site on completion/cancel/instant payment notification...which is more common in my experience. Do we have a common Payment_Controller that handles responses from gateway? Do we pass a $controller reference somewhere for the gateway to call urls on?

How would we then allow the developer to specify where to redirect to? My own payment implementations have not cleanly solved this, where I just reference $payment->PaidObject() ...which requires storing PaidObjectClass and PaidObjectID on the Payment class. Not something I've seen encouraged by SS.



Some other areas that I think may need discussion at some point:
  • Providing forms / collecting the right information for a payment. Should this be left to the developer? How does the payment type validate the passed data.
  • Processing a payment - does it need attention? The $payment->processPayment($data,$form) function is doing a lot of the work at the moment. I think some responsibility could be given to a few other functions. Not sure how yet. Perhaps a $payment->validate() function will be useful for doing checks first. Maybe the data can be passed in through other functions also, and then when payment is ready to be processed, you just call $payment->process(), which then either submits web API call, or redirects user to gateway site.
Jeremy


On Fri Mar 30 12:42:09 2012, Frank Mullenger wrote:

Thanks, sounds good. I guess I'll need to put aside half a day to read
this thread on DI a bit more thoroughly:
http://groups.google.com/group/silverstripe-dev/browse_thread/thread/22de1481e6cc570b?fwc=2

So, just to clarify:

1. The production service connects to gateway and processes payment

as it would in a live environment
2. The test service connects to gateway and processes payment for say

a dev environment with test API account settings for the payment
gateway - or manual testing
3. Mock object represents the gateway and responds like the gateway

To unsubscribe from this group, send email to
silverstripe-d...@googlegroups.com

Jeremy Shipman

unread,
Mar 29, 2012, 10:06:41 PM3/29/12
to silverst...@googlegroups.com
Correction: MyPayment would extend Payment class, not DataObject

Jeremy Shipman

unread,
Mar 29, 2012, 10:17:37 PM3/29/12
to silverst...@googlegroups.com
I would also like to better understand how useful the Payment_Result
classes will be moving forward.

Briefly how they work is that they package up a result as either:

* Payment_Success - successful and not processing
* Payment_Processing - not successful and still processing - wait
* Payment_Failure - not successful and not processing

In addition these classes can contain a value, which is often just a
message. From what I've seen, these classes are only ever used in the
processPayment function.

Nicolaas Thiemen Francken - Sunny Side Up

unread,
Mar 29, 2012, 10:25:50 PM3/29/12
to silverst...@googlegroups.com
On 30 March 2012 15:17, Jeremy Shipman <jer...@burnbright.net> wrote:
I would also like to better understand how useful the Payment_Result classes will be moving forward.

Briefly how they work is that they package up a result as either:

* Payment_Success - successful and not processing
* Payment_Processing - not successful and still processing - wait
* Payment_Failure - not successful and not processing


I have just written a class for the scnet non-hosted payment gateway.  We do the soap call and depending on what is returned from the gateway we either return a success of a failure.
 

Frank Mullenger

unread,
Mar 29, 2012, 11:19:16 PM3/29/12
to silverst...@googlegroups.com
@Jeremy Correct me if I'm wrong, I don't think we need to extend Payment in the payment submodules, like it is now, just create the payment gateway classes that can be injected into the Payment class. Then the dependency is used when processPayment() or similar is called in the Payment class. You could have RecurringPayment extend Payment perhaps if there is some shared API/functionality.

Controllers can be quite useful for handling the return URLs like you mention, maybe we could use DI for that as well? Then a developer could inject a controller that will handle redirect back to the paid object or you could have a generic controller for that particular gateway that always redirects back to /x/y/z like Paystation.

I also use PaidObject(), I don't have a problem with storing this info in Payment personally. Unless you want to add a has_one PaidObject to Payment and developers need to extend PaidObject for their Order class or something?

Re collecting the right info and validating it - is there much difference in the info needed by different gateways? Perhaps we could have a set of form fields which validate themselves and get the fields from the gateway service class that is injected? A developer could pull these form fields into their checkout form or similar, different gateways might have a different set of required data like you say.

processPayment() and validate() would use the injected class for that particular gateway or something like that?

One other question, do we need to decorate Member with address fields and have sending receipts as part of this payment module? 
The payment class should be able to reference the object that was paid for and the member who paid for it, but often a shipping address and billing address are specified - maybe it's cleaner to decorate Member with has_many Address? And just have a hook for developers to use in order to send receipts.

Frank Mullenger

unread,
Apr 2, 2012, 6:30:09 AM4/2/12
to silverst...@googlegroups.com, Ryan Dao
Hey Ryan,

Hows it going with all this - do you have any questions and did you see Sam's post about dependency injection?

Cheers,
Frank 

Ryan Dao

unread,
Apr 2, 2012, 7:58:14 AM4/2/12
to frankmu...@gmail.com, silverst...@googlegroups.com
Hi all,

Sorry for being quiet for some time. Regarding the recent discussion about dependency injection and testing, here are my thoughts:

- I think it's a great idea to have 3 different implementations for production, API testing and unit testing. Dependency Injection is the proper pattern to achieve this (I should thank my Software Engineering professor for teaching me this :P)
- I agree with Frank that a subclasses Payment will no longer be needed in this case. Applying the pattern, a payment submodule will contain 3 payment gateway classes: ProductionTestGateway, APITestGateway, and UnitTestGateway, each of which can be injected to the Payment object when a payment is about to be processed. The Payment class will handle the payment info directly since the info is the same for all gateways. 
- Following the pattern, we can have a PaymentController class inside each submodule to handle redirection. It will also be injected to the Payment object. I think a default controller should also be included in the Payment module in case the submodule developer doesn't want to customize redirection. 
- Should there be a class for processing recurring payment in the submodule as well? If I'm not wrong, each gateway processes recurring payment differently. Using DI we can inject the submodule's RecurringPaymentProcessor to the RecurringPayment object. Let me know what you think about this.

I am aiming to finish my proposal by today. I saw from the Wiki that Frank and Jeremy will be the mentors for this project. Am I allowed to send you my proposal for reviewing before I submit it to GSOC?

Thanks so much for your wonderful supports so far!

Best regards,

Ryan   

Jeremy Shipman

unread,
Apr 2, 2012, 6:14:56 PM4/2/12
to silverst...@googlegroups.com
"One other question, do we need to decorate Member with address fields and have sending receipts as part of this payment module?"

I don't think a member should be required to make a payment, but it is useful having the existing PaidBy Member relationship on Payment. Re member shipping address and receipt sending, I think those should be left to your code/module that is making use of the Payment module. I think Payment module should be kept minimal, basically for making / recording payments. We can provide examples, or further code, or links to modules that implement those additional features. On the other hand, maybe a few features might be useful in the core module...I just wouldn't want them tripping up developers or creating unnecessary bloat. There are some things in Payment module as it stands that I don't understand or never touch.

Subclassing payment:
What if there are specific things to record for a payment type (eg TransactionID)? It would be great to have all the right fields already provided on Payment, but I can't imagine we'd catch all the edge cases. I think it should be possible at least to extend payment, if needed.
Would this mean we also need to store the payment name/type in a new field on Payment? Sub-classing did make identifying the payment type easy.
Subclassing can become an issue if you uninstall a payment type. Not a common thing to do, but retaining historical payment info is important. That said, I don't think SS just destroys the data, I think it might remove the ClassName field from records.

Controller / redirection:
Currently we do use a controller for each payment type. Eg PaypalExpressCheckoutaPayment_Handler extends Controller. Common things this type of controller will do:
  • Retrieve the payment, based on some passed ID or token that was stored before payment was processed
  • Check payment is successful - either by checking data passed in request, or connecting back to API
  • Set the $payment->Message to something
  • Direct the user somewhere meaningful, such as $payment->PaidObject()->Link();
  • Instant payment notification - some gateways can notify your site exactly when a payment is completed.

Regarding the $payment->Message field. It is not clear if this is for the user to see payment information, or for admin to see technical info such as error code. Perhaps we need also need a PaymentMessage class to store multiple messages? Not sure.

The PaidObject link seems a bit inflexible to me. Can we somehow provide a controller / action / id for a redirect link to be derived from? Or can we jump to some action on the same controller that the gateway was originally redirected from, then the developer can decide what do do from there, depending on success/fail etc.

Recurring payments:
Recurring payments is feature of some gateways, but is it common?. I think it would be good to cater for these, perhaps using DI as suggested. I'm not sure how complex this would be though, and if there's be enough similarity between systems for a generic API to work. I'm keen to see what the API would look like.
I wonder if we could build a SS-based system that allows making recurring payments using any payment type that the user desires. There appears to be some code already for this.
https://github.com/silverstripe-labs/silverstripe-payment/blob/master/code/RecurringPayment.php
Could that be combined with the gateway provided recurring payment features?

I like to make fake examples to get my head around API changes. Here's some draft ideas for this work:
https://gist.github.com/2287180
More draft code will be useful if you guys want to write/share any.

PS, if you haven't already...please read https://github.com/burnbright/silverstripe-payment/wiki , where I've listed some other thoughts and ideas.

Jeremy
<mailto:ma...@sunnysideup.co.nz>> wrote:



On 30 March 2012 15:17, Jeremy Shipman
<jer...@burnbright.net <mailto:jer...@burnbright.net>> wrote:

I would also like to better understand how useful the
Payment_Result classes will be moving forward.

Briefly how they work is that they package up a result
as either:

* Payment_Success - successful and not processing
* Payment_Processing - not successful and still
processing - wait
* Payment_Failure - not successful and not processing


I have just written a class for the
scnet non-hosted payment gateway. We do the soap call and
depending on what is returned from the gateway we either
return a success of a failure.

--
You received this message because you are subscribed to
the Google Groups "SilverStripe Core Development" group.
To post to this group, send email to
silverst...@googlegroups.com

To unsubscribe from this group, send email to
silverstripe-d...@googlegroups.com

Frank Mullenger

unread,
Apr 2, 2012, 10:18:35 PM4/2/12
to silverst...@googlegroups.com
I agree: Member is not required for Payment - that's probably a good feature to support. I'm all for keeping it simple/minimal as well - address details may be a requirement for some gateways that might be something that needs looking into. If they are just optional perhaps just leave them out but support a way for developers to include address info when sending data to the gateway for example: sending address data to PayPal and it is prepopulated in the payment form that the customer is redirected to.

If there are specific data that should be recorded for a payment for a given gateway perhaps decorate Payment? It saves you referencing and remembering the subclass but maybe there is a performance tradeoff with decorators? I think it would be good to have just the common db fields in Payment and custom fields from other submodules are added from the submodule. Also keep in mind custom form fields with validation for some of these custom fields that are required for a particular gateway.

$payment->Message I use for the error messages etc. from the payment gateway, but like Jeremy said I agree this needs to be looked at more closely. The cheque payment type puts HTML into this field but should it just be plain text and should it support multiple messages from certain gateways etc.?

I think pass/inject a controller to Payment class to handle redirection, for Paystation this would be a generic controller which would have a standardised redirect link for that Paystation account, in other instances it might be the controller of the Order/PaidObject which could provide a link and redirect to view the Order/PaidObject. Having a standard PaymentController API/Interface that developer could subclass/implement would probably be handy.
 
For recurring payments I think we just want to find gateways that support recurring payments: PayPal, DPS etc. and then support using these gateways for recurring payments with perhaps some more features like an admin and dunning emails etc. if appropriate. Supporting recurring payments for any gateway might be difficult because of storing cc details etc.?

Good idea about the gist, will try to write something up.

Jeremy Shipman

unread,
Apr 2, 2012, 11:22:28 PM4/2/12
to silverst...@googlegroups.com
On 3/04/12 2:18 PM, Frank Mullenger wrote:
I agree: Member is not required for Payment - that's probably a good feature to support. I'm all for keeping it simple/minimal as well - address details may be a requirement for some gateways that might be something that needs looking into. If they are just optional perhaps just leave them out but support a way for developers to include address info when sending data to the gateway for example: sending address data to PayPal and it is prepopulated in the payment form that the customer is redirected to.
Yeah, thats a good point about gateways requiring address details. They may or may not be optional. Lets generalize this issue to 'uncommon fields required by gateways'. I'm sure we'll find some gateways requiring fields which no others have. Developers will need to specifically supply these.
I remember thinking about this a while back, and that you could try mapping to common fields, eg developer supplies either Surname or LastName, and these always map to 'surname', the payment gateway implementation then always checks/grabs the 'surname' field, and doesn't need to worry about other spellings.
It still doesn't solve the uncommon required fields issue.

If there are specific data that should be recorded for a payment for a given gateway perhaps decorate Payment? It saves you referencing and remembering the subclass but maybe there is a performance tradeoff with decorators?I think it would be good to have just the common db fields in Payment and custom fields from other submodules are added from the submodule. Also keep in mind custom form fields with validation for some of these custom fields that are required for a particular gateway.
I definitely think common fields should go in Payment. I'm still not sold on Payment being the only class in question though. Why do you need to reference/remember the subclass, can you give an example? The way I see it, the ORM makes life a bit easier...you should only need to worry about specifically referencing subclasses when dealing specifically with that payment type. Am I missing something?



$payment->Message I use for the error messages etc. from the payment gateway, but like Jeremy said I agree this needs to be looked at more closely. The cheque payment type puts HTML into this field but should it just be plain text and should it support multiple messages from certain gateways etc.?
Yes, definitely should be plain text, especially considering the Message field is a Text DBField. Perhaps PaymentMessage could have an extra argument that specifies if the message is for admin or customer?


I think pass/inject a controller to Payment class to handle redirection, for Paystation this would be a generic controller which would have a standardised redirect link for that Paystation account, in other instances it might be the controller of the Order/PaidObject which could provide a link and redirect to view the Order/PaidObject. Having a standard PaymentController API/Interface that developer could subclass/implement would probably be handy.
For the sake of communication, I just want to make clear that I'm talking about two controllers.
The first is the one for handling redirects from gateways, and probably extends from the PaymentController. This controller might not be needed at all if your payment type is 'bank wire', 'pay on delivery', or 'cheque'...which I call "offline" payments, and then there's the payment APIs, called via SOAP/REST/etc, where you have collected the credit card details from your own form, sometimes known as "merchant hosted".
The second controller is that which you then direct the user to after making payment. This may also be the same as the controller which triggers the redirect to the gateway. I've updated my gist to further demonstrate how this might be recorded. https://gist.github.com/2287180
$controller = $this->controller;
$action = 'vieworder';
$id = $buyable->ID;
$payment->returnTo($controller,$action,$id,$data);
Somehow $payment->returnTo would store the passed args, and later PaymentController would use these to create a link and redirect wherever the developer wants.

Supporting recurring payments for any gateway might be difficult because of storing cc details etc.?
I assumed that recurring payments were semi-automatic, but yeah I wouldn't want to store cc details for a fully automatic system. In a semi-automatic system, the customer would be notified that they need to pay to renew. Perhaps this feature is better implemented in a 'subscription' module?

Jeremy Shipman

unread,
Apr 19, 2012, 5:41:06 PM4/19/12
to silverst...@googlegroups.com
Where is the best place to communicate with payment module development?
I might suggest we use the ecommerce mailing list, as the main dev list
will be rather loud, plus payment relates closely to ecommerce, and
stakeholders are on that list. What does SS want to happen?

I also wonder where the main fork of payment should reside. I assume
we're not going to keep it in silverstripe-labs, as we should have a
nice (non labs) module by the end of it.
I was thinking we could possibly transfer ownership to whatever student
wins this project, perhaps once they have proved themselves? Then again,
the student may not have an interest, or the time to support it once
GSOC is over.
Perhaps a bigger question to ask is a political one of who will be the
maintainer, and how will future changes be made? Since the payment
module doesn't need to change much (I hope), once finished, then perhaps
the consensus-based democracy approach will work. We'd need to create
some guidelines on how changes get accepted.
http://producingoss.com/en/consensus-democracy.html

Jeremy

Frank Mullenger

unread,
Apr 19, 2012, 6:26:25 PM4/19/12
to silverst...@googlegroups.com
If the payment module project is accepted and goes ahead (fingers crossed) I'm happy for communications to be in either channel but I think the dev list is more appropriate - there may be interested parties in the dev list that do not know about the ecommerce list (I only recently joined this list myself) and we have had good input for this project so far from the dev list - such as Sams post about DI.

Would it be possible for some extra collaborators to be added to https://github.com/silverstripe-labs/silverstripe-payment? That repository has 18 watchers and I think would be the best place to create a branch. For GSOC all we would need is someone that can merge pull requests at some point really.



Jeremy

--
You received this message because you are subscribed to the Google Groups "SilverStripe Core Development" group.
To post to this group, send email to silverstripe-dev@googlegroups.com.
To unsubscribe from this group, send email to silverstripe-dev+unsubscribe@googlegroups.com.

Jeremy Shipman

unread,
Apr 19, 2012, 6:35:05 PM4/19/12
to silverst...@googlegroups.com
Yes, I think you're right. Payment is actually quite an important
module, and there are probably a lot more people interested than I
realise.

Would it be a good idea for the module to move to silverstripe/payment
before adding collaborators and commencing work?

Ingo Schommer

unread,
Apr 19, 2012, 6:35:34 PM4/19/12
to silverst...@googlegroups.com, Normann Lou
Hey Frank and Jeremy,

I'd prefer posts on ecommerce list, to keep the core mailinglist low frequency.
For questions on payment affecting core (or requriing core changes), the core mailinglist is appropriate of course.

I'd keep the module location on labs for now, and add collabs as Frank suggested.
Just you two for now? I've CCed Normann, the current maintainer.
@Normann: How do you see yourself involved in development of the module
over the next ~six months, in context of GSOC, reviewing pull requests etc?

I don't think moving it to any student's repo makes sense until well after GSOC,
when he's proved some long-term interest in the project.

Ingo

To post to this group, send email to silverst...@googlegroups.com.
To unsubscribe from this group, send email to silverstripe-d...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages