Marketplace functionality - Split Orders

[Jonathan Meran]

Hello,

We have been working on customizing Oscar to support Marketplace functionality where we have several Shop owners under our umbrella site.

I am now working on the Checkout/Shipping/Order processing portion and that is going fairly smooth thanks to Oscar's ability to allow customization of almost anything which has been an awesome experience!

Has anyone done something similar where the Order object from the Basket is split into many Orders (1 per Shop owner)? Currently it appears that I need to change all methods that deal with producing an Order by passing in the Shop_Id as well to associate the Shop with the split order. My thought is that I can do this without affecting method signatures by passing the Shop_Id in the kwargs.

Also, I was thinking of still keeping a "top-level" Order that includes all of the Basket items purchased for record keeping purposes. However, I am afraid that this may cause issues since it will never become a completed and processed order so I am now taking the approach of splitting the Orders and not keeping any top-level record. Thoughts?

Thank you,

Jon

[David Winterbottom]

​This is one we've thought about before but never got round to working on properly. I suspect that Oscar's checkout and order placement code does assume there's only one order, and that customising this would require overriding quite a few methods (as you say). Not impossible but a little clunky.

I'd like to change this in core Oscar but it requires some careful thought. The link between payment and orders will require altering to accommodate this change. Right now, the payment.Source links to the order but when multiple orders can be paid for with one payment source (Eg one bankcard transaction), a different arrangement will be required. We'll need a "payment group" model that each source has a foreign key to, and each order can link to the payment group. This will require a data migration for all existing Oscar installations and so isn't something to take lightly.  ​

​I've written this up as a ticket: https://github.com/tangentlabs/django-oscar/issues/1519

If I were you, I would consider placing a single order at checkout time but then having a cron/celery job that splits the order up into smaller ones for each partner. Then you don't have to hack around in the checkout process too much.​

 

Thank you,

Jon

--
https://github.com/tangentlabs/django-oscar
http://django-oscar.readthedocs.org/en/latest/
https://twitter.com/django_oscar
---
You received this message because you are subscribed to the Google Groups "django-oscar" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-oscar...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-oscar.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-oscar/c9b3036b-a345-4898-b0c3-7bc8a1bbb393%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

David Winterbottom
Technical Director

Tangent Snowball

Threeways House

40-44 Clipstone Street

England, UK

[Maik Hoepfel]

Hi Jon,

how do you intend to let a customer pay for the orders? Will it be
possible to pay in one go?

Just for the record, I've taken something like this to the proof of
concept stage for my own site, but I didn't have to worry about payment.
But I basically split the orders if necessary, put the individual order
IDs into the session and then made sure to render them out in the
success template to be able to display them individually.

Until better support for this lands in Oscar, I suppose for payment you
could keep the top-level order around and mark that as paid. You'll have
to somehow mark your "child orders" and make sure they're handled
correctly. It'll also mess up Oscar's statistics. But generally, I feel
like it should be achievable.

Cheers,

Maik

On 13/10/14 09:37, David Winterbottom wrote:
>
>
> On 10 October 2014 13:26, Jonathan Meran <jonm...@gmail.com

> <mailto:django-oscar...@googlegroups.com>.

> Visit this group at http://groups.google.com/group/django-oscar.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-oscar/c9b3036b-a345-4898-b0c3-7bc8a1bbb393%40googlegroups.com

> <https://groups.google.com/d/msgid/django-oscar/c9b3036b-a345-4898-b0c3-7bc8a1bbb393%40googlegroups.com?utm_medium=email&utm_source=footer>.

> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
> --

> *David Winterbottom*

> Technical Director
>
> Tangent Snowball
> Threeways House
> 40-44 Clipstone Street
> England, UK
>

> --
> https://github.com/tangentlabs/django-oscar
> http://django-oscar.readthedocs.org/en/latest/
> https://twitter.com/django_oscar
> ---
> You received this message because you are subscribed to the Google
> Groups "django-oscar" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-oscar...@googlegroups.com

> <mailto:django-oscar...@googlegroups.com>.

> Visit this group at http://groups.google.com/group/django-oscar.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/django-oscar/CAA0jhkrX9nJDKXbY1Ywz1rG4ToH9W2Fdu8%3Df%3DCVuFVzFhyvN4g%40mail.gmail.com
> <https://groups.google.com/d/msgid/django-oscar/CAA0jhkrX9nJDKXbY1Ywz1rG4ToH9W2Fdu8%3Df%3DCVuFVzFhyvN4g%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[Jonathan Meran]

Hey guys,

I have this working so far by overriding the Order and Checkout applications. It actually wasn't all that bad once you get some familiarity of the checkout flow and what classes need to be subclassed. Essentially what I did was first create a top-level order but I make sure that no stock record updates are applied if it is the top level order (i.e. there is no Shop ID attached to that order). I use that top-level order to handle things like displaying the order confirmation page/email that the user sees at the end of the transaction. I then take that top-level order and split that into orders by Shop ID. The Shop level orders do the normal Order processing such as stock record updates etc.

For payment, customers are allowed to pay in one go. We are actually very happy with this so far as major Marketplaces such as Etsy force separate checkouts for each Shop. I use the top-level order since our Marketplace will take the initial funds and then break that up into transfers to our Shop partners when they prove that they have fulfilled their portion of the order.

I currently do not "tie" the top-level order to the child orders but I do have some correlation by some customization I did on the order number generation. Essentially the top-level order creates an order number in the traditional Oscar manner, and the child orders take this number and prefix the ShopID and a hyphen to that.

All in all, this was semi "hacky" as it is not built into Oscar's core modeling but I feel that one of the best features of Oscar is how much you can customize it for flows like this. If Oscar were to support Marketplace functionality out of the box at some point I can see making these customizations part of the core product.

I am more than happy to share the overridden apps if anyone wants an idea of what it takes to pull this off.

Thanks,

Jon

>     <mailto:django-oscar+unsub...@googlegroups.com>.

>     Visit this group at http://groups.google.com/group/django-oscar.
>     To view this discussion on the web visit
>     https://groups.google.com/d/msgid/django-oscar/c9b3036b-a345-4898-b0c3-7bc8a1bbb393%40googlegroups.com
>     <https://groups.google.com/d/msgid/django-oscar/c9b3036b-a345-4898-b0c3-7bc8a1bbb393%40googlegroups.com?utm_medium=email&utm_source=footer>.
>     For more options, visit https://groups.google.com/d/optout.
>
>
>
>
> --
> *David Winterbottom*
> Technical Director
>
> Tangent Snowball
> Threeways House
> 40-44 Clipstone Street
> England, UK
>
> --
> https://github.com/tangentlabs/django-oscar
> http://django-oscar.readthedocs.org/en/latest/
> https://twitter.com/django_oscar
> ---
> You received this message because you are subscribed to the Google
> Groups "django-oscar" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-oscar...@googlegroups.com

> <mailto:django-oscar+unsub...@googlegroups.com>.

[Jeff Bowman]

As a follow up to what Jon has done (I am working Jon on this), I am now attempting to modify the base_queryset for the orders dashboard so that the partners will only see the split order and NOT the top level order.

It was easy to modify the queryset_orders_for_user(user) method to do this.. But I am having trouble making this change propagate in the view (right now I am working on the OrderListView).. Since the dispatch method of the base class sets the base_queryset, I have not yet been able to find a way to not overwrite the base_queryset that I have generated. Do you guys know how I would extend the dispatch method of the OrderListView to not use super? (As this will execute the dispatch method of the base class and subsequently override any modified base_queryset that I generate.)

Regards,

-Jeff

[Maik Hoepfel]

Hi,

I'm glad it went reasonably well! David and me would be interested in
having a look at your customisations. It's helpful to have an actual
implementation in hand to learn where the pain points are when we'll get
to address https://github.com/tangentlabs/django-oscar/issues/1519.

I think David will want to do the modelling aspect of adding better
support for that. But anything beyond that, you're more than welcome to
chip in and improve functionality and break assumptions. It sounds like
the kind of thing that's actually better solved in Oscar than in your
custom version.

Jeff: Your best bet for now is to copy the dispatch method, adjust the
base_queryset and call the supersuper (for lack of a better word)
method, skipping Oscar's views dispatch method.
But those are the kind of pain points we will always be happy to remove.
If you could send a pull request that factors out building the base
queryset (probably into a one line method), I'd happily merge it to let
you get rid of the workaround.

Cheers,

Maik

> --
> https://github.com/tangentlabs/django-oscar
> http://django-oscar.readthedocs.org/en/latest/
> https://twitter.com/django_oscar
> ---
> You received this message because you are subscribed to the Google
> Groups "django-oscar" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-oscar...@googlegroups.com

> <mailto:django-oscar...@googlegroups.com>.

> Visit this group at http://groups.google.com/group/django-oscar.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/django-oscar/5607ff1b-7ff9-4082-9396-5c5bb506a5c7%40googlegroups.com
> <https://groups.google.com/d/msgid/django-oscar/5607ff1b-7ff9-4082-9396-5c5bb506a5c7%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Jeff Bowman]

Hi Maik,

Thank you for the advice. You are right, it would be quite easy to break out this out into a single line method as below.. When I get the time, I will create a fork of oscar so that I may make a pull request for this.

   

    def dispatch(self, request, *args, **kwargs):
        # base_queryset is equal to all orders the user is allowed to access
        self.set_base_queryset(request)
        return super(OrderListView, self).dispatch(self, request, *args, **kwargs)


    def set_base_queryset(self, request):
        self.base_queryset = queryset_orders_for_user(
            request.user).order_by('-date_placed')

 

With this one simply needs to override the set_base_queryset method to override the default base_queryset returned by oscar.

In the mean time I made this work by calling the View base classes dispatch method directly.

I am not a super expert with python.. and I was unable to find out how to get the supersuper. So instead I simply called the base django View class directly.

    def dispatch(self, request, *args, **kwargs):
        self.base_queryset = queryset_orders_for_user(
            request.user).order_by('-date_placed')
        return View.dispatch(self, request, *args, **kwargs)

Thanks for the advice!

Regards,

-Jeff

[Jonathan Meran]

Hey all,

Here are some code snippets of the meat of the changes I made to support the splitting of Orders by shop (partner). I'm still cleaning it up but this should show the idea. I've highlighted the important parts in yellow:

In my overridden checkout app, ignore the additions I made to support Stripe payments:

class PaymentDetailsView(CorePaymentDetailsView):
	template_name = "payment_details.html"
	def get_context_data(self, **kwargs):
	# ctx = super(PaymentDetailsView, self).get_context_data(**kwargs)
	#
	# if not hasattr(self, 'stripe_token'):
	# return ctx
	# This context generation only runs when in preview mode
	# ctx.update({
	# 'STRIPE_PUBLIC_KEY': settings.STRIPE_PUBLISHABLE_KEY
	# })
	return {'STRIPE_PUBLIC_KEY': settings.STRIPE_PUBLISHABLE_KEY}
	def post(self, request, *args, **kwargs):
	error_msg = (
	"A problem occurred communicating with PayPal "
	"- please try again later"
	)
	try:
	self.token = request.POST['stripe_token']
	except KeyError:
	# Probably suspicious manipulation if we get here
	messages.error(self.request, error_msg)
	return HttpResponseRedirect(reverse('home'))
	submission = self.build_submission()
	return self.submit(**submission)
	def build_submission(self, **kwargs):
	submission = super(
	PaymentDetailsView, self).build_submission(**kwargs)
	# Pass the user email so it can be stored with the order
	# submission['order_kwargs']['guest_email'] = self.txn.value('EMAIL')
	# Pass Stripe params
	submission['payment_kwargs']['stripe_token'] = self.token
	return submission
	def handle_payment(self, order_number, total, **kwargs):
	stripe_ref = Facade().charge(
	order_number,
	total,
	card=self.request.POST['stripe_token'],
	description=self.payment_description(order_number, total, **kwargs),
	metadata=self.payment_metadata(order_number, total, **kwargs))
	source_type, __ = SourceType.objects.get_or_create(name=PAYMENT_METHOD_STRIPE)
	source = Source(
	source_type=source_type,
	currency=settings.STRIPE_CURRENCY,
	amount_allocated=total.incl_tax,
	amount_debited=total.incl_tax,
	reference=stripe_ref)
	self.add_payment_source(source)
	self.add_payment_event(PAYMENT_EVENT_PURCHASE, total.incl_tax)
	def payment_description(self, order_number, total, **kwargs):
	# Jon M TODO - Add case for anonymous user with email
	return self.request.user.email
	# return self.request.POST[STRIPE_EMAIL]
	def payment_metadata(self, order_number, total, **kwargs):
	return {'order_number': order_number}
	def get_shipping_method(self, basket, shipping_address=None, **kwargs):
	"""
	Return the shipping method used
	"""
	if not basket.is_shipping_required():
	return NoShippingRequired()
	method = Free
	return method
	def submit(self, user, basket, shipping_address, shipping_method, # noqa (too complex (10))
	order_total, payment_kwargs=None, order_kwargs=None):
	"""
	Submit a basket for order placement.
	The process runs as follows:
	* Generate an order number
	* Freeze the basket so it cannot be modified any more (important when
	redirecting the user to another site for payment as it prevents the
	basket being manipulated during the payment process).
	* Attempt to take payment for the order
	- If payment is successful, place the order
	- If a redirect is required (eg PayPal, 3DSecure), redirect
	- If payment is unsuccessful, show an appropriate error message
	:basket: The basket to submit.
	:payment_kwargs: Additional kwargs to pass to the handle_payment method
	:order_kwargs: Additional kwargs to pass to the place_order method
	"""
	if payment_kwargs is None:
	payment_kwargs = {}
	if order_kwargs is None:
	order_kwargs = {}
	# Taxes must be known at this point
	assert basket.is_tax_known, (
	"Basket tax must be set before a user can place an order")
	assert shipping_method.is_tax_known, (
	"Shipping method tax must be set before a user can place an order")
	# Freeze the basket so it cannot be manipulated while the customer is
	# completing payment on a 3rd party site. Also, store a reference to
	# the basket in the session so that we know which basket to thaw if we
	# get an unsuccessful payment response when redirecting to a 3rd party
	# site.
	self.freeze_basket(basket)
	self.checkout_session.set_submitted_basket(basket)
	# We define a general error message for when an unanticipated payment
	# error occurs.
	error_msg = _("A problem occurred while processing payment for this "
	"order - no payment has been taken. Please "
	"contact customer services if this problem persists")
	signals.pre_payment.send_robust(sender=self, view=self)
	# Generate an order number for the top level that includes orders from all the shops
	top_level_order_number = self.generate_order_number(basket)
	self.checkout_session.set_order_number(top_level_order_number)
	logger.info("Order #%s: beginning submission process for basket #%d",
	top_level_order_number, basket.id)
	items_by_shop = {}
	top_level_order = None
	try:
	top_level_order = self.handle_order_placement(
	top_level_order_number, user, basket, shipping_address, shipping_method,
	prices.Price(currency=basket.currency, excl_tax=basket.total_excl_tax, incl_tax=basket.total_excl_tax), **order_kwargs)
	except UnableToPlaceOrder as e:
	# It's possible that something will go wrong while trying to
	# actually place an order. Not a good situation to be in as a
	# payment transaction may already have taken place, but needs
	# to be handled gracefully.
	msg = six.text_type(e)
	logger.error("Order #%s: unable to place order - %s",
	top_level_order_number, msg, exc_info=True)
	self.restore_frozen_basket()
	return self.render_preview(
	self.request, error=msg, **payment_kwargs)
	# Collect information to split into an order for each shop
	for line in basket.all_lines():
	if(line.product.shop not in items_by_shop):
	items_by_shop[line.product.shop] = {"products": [], "order_total": 0}
	items_by_shop[line.product.shop]["products"].append(line.product),
	items_by_shop[line.product.shop]["order_total"] += line.line_price_excl_tax
	for shop in items_by_shop.keys():
	# We generate the order number first as this will be used
	# in payment requests (ie before the order model has been
	# created). We also save it in the session for multi-stage
	# checkouts (eg where we redirect to a 3rd party site and place
	# the order on a different request).
	order_number = self.generate_order_number(basket, shop.id)
	order = None
	try:
	order_kwargs['shop'] = shop
	order = self.handle_order_placement(
	order_number, user, basket, shipping_address, shipping_method,
	prices.Price(currency=basket.currency, excl_tax=items_by_shop[shop]["order_total"],
	incl_tax=items_by_shop[shop]["order_total"]),
	**order_kwargs)
	except UnableToPlaceOrder as e:
	# It's possible that something will go wrong while trying to
	# actually place an order. Not a good situation to be in as a
	# payment transaction may already have taken place, but needs
	# to be handled gracefully.
	msg = six.text_type(e)
	logger.error("Order #%s: unable to place order - %s",
	order_number, msg, exc_info=True)
	self.restore_frozen_basket()
	return self.render_preview(
	self.request, error=msg, **payment_kwargs)
	try:
	self.handle_payment(top_level_order_number, order_total, **payment_kwargs)
	except RedirectRequired as e:
	# Redirect required (eg PayPal, 3DS)
	logger.info("Order #%s: redirecting to %s", top_level_order_number, e.url)
	return HttpResponseRedirect(e.url)
	except UnableToTakePayment as e:
	# Something went wrong with payment but in an anticipated way. Eg
	# their bankcard has expired, wrong card number - that kind of
	# thing. This type of exception is supposed to set a friendly error
	# message that makes sense to the customer.
	msg = six.text_type(e)
	logger.warning(
	"Order #%s: unable to take payment (%s) - restoring basket",
	top_level_order_number, msg)
	self.restore_frozen_basket()
	# We assume that the details submitted on the payment details view
	# were invalid (eg expired bankcard).
	return self.render_payment_details(
	self.request, error=msg, **payment_kwargs)
	except PaymentError as e:
	# A general payment error - Something went wrong which wasn't
	# anticipated. Eg, the payment gateway is down (it happens), your
	# credentials are wrong - that king of thing.
	# It makes sense to configure the checkout logger to
	# mail admins on an error as this issue warrants some further
	# investigation.
	msg = six.text_type(e)
	logger.error("Order #%s: payment error (%s)", top_level_order_number, msg,
	exc_info=True)
	self.restore_frozen_basket()
	return self.render_preview(
	self.request, error=error_msg, **payment_kwargs)
	except Exception as e:
	# Unhandled exception - hopefully, you will only ever see this in
	# development...
	logger.error(
	"Order #%s: unhandled exception while taking payment (%s)",
	top_level_order_number, e, exc_info=True)
	self.restore_frozen_basket()
	return self.render_preview(
	self.request, error=error_msg, **payment_kwargs)
	signals.post_payment.send_robust(sender=self, view=self)
	# If all is ok with payment, try and place order
	logger.info("Order #%s: payment successful, placing order",
	top_level_order_number)
	basket.submit()
	return self.handle_successful_order(top_level_order)
	def generate_order_number(self, basket, shop_id=None):
	order_num = 100000 + basket.id
	if shop_id is not None:
	order_num = str(shop_id) + '-' + str(order_num)
	return order_num
	def handle_order_placement(self, order_number, user, basket,
	shipping_address, shipping_method,
	total, **kwargs):
	"""
	Write out the order models and return the appropriate HTTP response
	We deliberately pass the basket in here as the one tied to the request
	isn't necessarily the correct one to use in placing the order. This
	can happen when a basket gets frozen.
	"""
	order = self.place_order(
	order_number, user, basket, shipping_address, shipping_method,
	total, **kwargs)
	return order

In my overridden Order app:

from django.conf import settings
	# from oscar.apps.checkout.mixins import OrderNumberGenerator Jon M WARNING!! For some reason importing this makes this module not be loadable!
	from oscar.apps.order.utils import OrderCreator as CoreOrderCreator
	from oscar.apps.shipping.methods import Free
	from oscar.core.loading import get_model, get_class
	from django.utils.translation import ugettext_lazy as _
	from decimal import Decimal as D
	Order = get_model('order', 'Order')
	Line = get_model('order', 'Line')
	order_placed = get_class('order.signals', 'order_placed')
	class OrderCreator(CoreOrderCreator):
	def place_order(self, basket, total,
	user=None, shipping_method=None, shipping_address=None,
	billing_address=None, order_number=None, status=None,
	**kwargs):
	"""
	Placing an order involves creating all the relevant models based on the
	basket and session data.
	"""
	if basket.is_empty:
	raise ValueError(_("Empty baskets cannot be submitted"))
	if not shipping_method:
	shipping_method = Free()
	if not status and hasattr(settings, 'OSCAR_INITIAL_ORDER_STATUS'):
	status = getattr(settings, 'OSCAR_INITIAL_ORDER_STATUS')
	try:
	Order._default_manager.get(number=order_number)
	except Order.DoesNotExist:
	pass
	else:
	raise ValueError(_("There is already an order with number %s")
	% order_number)
	shop = kwargs.pop('shop', None)
	# Ok - everything seems to be in order, let's place the order
	order = self.create_order_model(
	user, basket, shipping_address, shipping_method, billing_address,
	total, order_number, status, **kwargs)
	for line in basket.all_lines():
	if shop is None or line.product.shop.id == shop.id:
	# Top level order and Shop orders track product lines
	self.create_line_models(order, line)
	if shop and line.product.shop.id == shop.id:
	# Stock is updated only for the Shop orders, not the top level order
	self.update_stock_records(line)
	for application in basket.offer_applications:
	# Trigger any deferred benefits from offers and capture the
	# resulting message
	application['message'] \
	= application['offer'].apply_deferred_benefit(basket)
	# Record offer application results
	if application['result'].affects_shipping:
	# Skip zero shipping discounts
	if shipping_method.discount <= D('0.00'):
	continue
	# If a shipping offer, we need to grab the actual discount off
	# the shipping method instance, which should be wrapped in an
	# OfferDiscount instance.
	application['discount'] = shipping_method.discount
	self.create_discount_model(order, application)
	self.record_discount(application)
	for voucher in basket.vouchers.all():
	self.record_voucher_usage(order, voucher, user)
	# Send signal for analytics to pick up
	order_placed.send(sender=self, order=order, user=user)
	return order