Fedex Module - RFC

[Chris Moffitt]

I recently had the need to go in and make some custom changes to Fedex. In the process, I learned that Satchmo's current XML implementation is a bit old and needs to be updated. The current version works with the legacy Fedex servers but won't migrate cleanly to the newer and more up to date ones.

After fiddling around with it for a while, I think we have two options going forward.

1. Modify the existing XML version to use the newer syntax and keep it mostly the same as the current implementation.
2. Migrate to a newer API version using SOAP. It looks like there have been a few folks that have done this so I think the implementation effort is fairly limited but it would introduce new dependencies for the Fedex module, namely suds and python-fedex - http://code.google.com/p/python-fedex/

Under the hood there aren't that many differences between SOAP and the XML api but after trying to read the Fedex docs for a while, they are really pushing towards the SOAP approach. Having some work already in place for the Python-fedex module, I think it's good to leverage instead of recreating our own.

As you can tell, I'm leaning towards jumping to option #2 but wanted some feedback from the group. Also, some folks have already started working on this integration so I'd appreciate any updates or code contribution to make this easier.

Interested in the group's input.

-Chris

[Dave Brown]

Greg Newman wrote this soap module for satchmo here: https://bitbucket.org/gregnewman/fedex_soap/overview

I've been messing with it a bit and it seems to work pretty well. 

Dave

-Chris

--
You received this message because you are subscribed to the Google Groups "Satchmo users" group.
To post to this group, send email to satchm...@googlegroups.com.
To unsubscribe from this group, send email to satchmo-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/satchmo-users?hl=en.

--
Dave Brown
CEO/Founder
Rampframe.com - Action Sports Network

[Chris Moffitt]

Yes. I've see that one as well as this one - https://bitbucket.org/arlolra/fedex_web_services/overview

They are both very similar so I was waiting to see if either of the authors jumped in with other ideas before I started using their code.

Thanks for letting me know that it's working for you.

-Chris

[hynekcer]

I fixed international characters for legacy method in collaboration
with user "Ionic Drive" in e-mail communication following the thread
groups.google.com/group/satchmo-users/browse_thread/thread/
86ea6bfeeae3955b/ . I did not published the solution becase he found
other bug caused by outdating Fedex code (no "time_in_transit") and he
did not want to continue.
I think the new interface would be better if you have time for it now.

I would like if no new dependencies would be imported if Fedex is not
used. New packages would be of course included in automatic Quick
Start Installation but it would be possible a manual installation of
unmodified Satchmo without them. It is not as serious as deep
dependencies on livesettings etc. but nobody knows ahead.
Please send a link some day before you commit it to trunk.

On 26 lis, 23:56, Chris Moffitt <ch...@moffitts.net> wrote:
> I recently had the need to go in and make some custom changes to Fedex. In
> the process, I learned that Satchmo's current XML implementation is a bit
> old and needs to be updated. The current version works with the legacy
> Fedex servers but won't migrate cleanly to the newer and more up to date
> ones.
>
> After fiddling around with it for a while, I think we have two options
> going forward.
>
> 1. Modify the existing XML version to use the newer syntax and keep it
> mostly the same as the current implementation.
> 2. Migrate to a newer API version using SOAP. It looks like there have been
> a few folks that have done this so I think the implementation effort is
> fairly limited but it would introduce new dependencies for the Fedex

> module, namely suds and python-fedex -http://code.google.com/p/python-fedex/

[Chris Moffitt]

Yes any new dependencies would only be required if you use Fedex. I will try to setup a fork for integrating and make sure to let people look at it before integrating it with trunk.

-Chris

[ionic drive]

@Hynekcer
Its not that I did not want to continue. But there is no nanosecond time
left for me right now.
The only thing I remember, when I was sniffing in the Fedex response one
day. I think I remember that Fedex did not respond with a "time in
transit" or similar parameter.

Maybe "time in transit" is only for ground delivery not for aircraft
shipments.
But its hard to say as there is no Fedex API and parameter description
available. I think its a good idea to proceed to the newer Fedex API if
they provide better and more detailed information.

ionic

[Chris Moffitt]

Based on the work that others have done, I was able to get the module working for me. I'd appreciate it if others would take a look at it and let me know if there's anything I'm missing.

You can view the fork here -
https://bitbucket.org/chris1610/satchmo-fedex/

I still need to update the docs but it seems to be working well for me.

-Chris

[Dave Brown]

Chris, I installed it and it seems to work fine.

A comment in general about the shipping methods:  It seems strange to have the calculate() method depend on a contact object being passed in.  For example if a user wants to pop in a zip code and get back the shipping rates for the current items in the cart before proceeding to checkout, it can't be done without hacking the modules.  As far as i know (atleast for fedex and USPS), you only need the zipcode and the country... so why require there be a contact object? 


Back to fedex:
I actually haven't been able to implement fedex on my site because I haven't been able to get any Fedex module I've tried to get a descent response time.
The one that ships with satchmo currently gives me a ~6 second response time using only 2 shipping methods (express & 2 day) and one item in the cart.
The soap method that Greg Newman built takes ~5.5 seconds
and the new method of yours I just tried out takes 8.5 seconds. 

I cannot figure out what in the world is going on.. Using the USPS i can get 1.5 second response times without a problem. 
The response from fedex is fine (~500ms) so I know its not an external issue, any ideas??

Dave

[Chris Moffitt]

Dave,

Thanks for installing and checking and verifying it works fine.

As far as the performance issues, I have a couple of questions:
- How long does it take if you just have Fedex Ground enabled?
- Look in your satchmo.log file and see if there appear to be any big time lags in the time stamps

Regarding the contact object, I think the original thought is that you might want more info than just a zip code to calculate costs. Since I wasn't sure about all the possible API's, I used a contact object. If someone wanted to run a query with just a zip code, they could create a dummy contact and call it. I'm open to other ideas though.

-Chris

[Dave Brown]

Chris, With only fedex ground, it takes ~4 seconds.

If i bump it up one more (add 2 day), ~7 seconds

Add one more (express), ~10.29 seconds.

I'm assuming since the standard satchmo checkout has the payment and shipping tied together, you are using the payment/shipping form to get the dynamic rates?  There isn't any other 'simple' form that your using that may make it quicker?

(i created a simple form since i want to get rates anywhere on the site with the current cart items, not just at checkout, but that still didn't make a difference).

The biggest thing I've noticed between fedex and usps is that even if i have verbose logs unchecked, the log file is filled with thousands of lines of fedex request.  That can't be good for it speed.

Does this sound normal?  

In regard to using the contact information, I'm about 99% sure that fedex/usps only uses zip code & country to get rates, so having more than that available wont be of any benefit.  I'm not sure about UPS though.  

thanks!

Dave

[hynekcer]

Chris, I confirm that new Fedex modules does not disturb other parts
of Satchmo.

I see that all new satchmo changes are applied to satchmo-fedex. So I
think it should be anyway all pushed to Satchmo, but due to David's
notes without large publicity yet. The speed can be tried fixed
continuously on the fly. It is not blocking for other development.
Thank you for giving the time to think.

[Dave Brown]

I figured out the speed issue, it was something stupid i was doing.

I now get 3 methods back in about 4.5 seconds, before it was taking ~10 seconds.  

Is 1.5seconds per method about right you think?

Dave

[Chris Moffitt]

Dave,

That sounds about right for what I'm seeing. I haven't done excessive testing but that's not outside the realm of what I'm seeing.

-Chris

[Josh]

Hey Dave any chance you could let me/us know what was causing the
speed issue. I have noticed that FeDex is generally slow and I wonder
if I am doing something similar.

Thanks,
Josh

> ...
>
> read more »

[Dave Brown]

Josh,
I had the log level set to DEBUG  :(

How slow are you getting?  I was getting 10 seconds for 3 shipping methods... now im getting ~4-5 seconds for 3

Dave

[Josh]

Sorry it's taken me a week to get back. I've been seeing 10+ seconds.
Sometimes it also just errors out lately though, so I may have other
issues going on. Thanks for the info.

Regards,
Josh

> ...
>
> read more »

[Josh]

Actually, is anybody else getting this:
AttributeError at /checkout/credit/

'NoneType' object has no attribute 'HighestSeverity'

I'm getting it when I try to submit the contact info form during
checkout.

It comes from this line:
if rate_request.response.HighestSeverity in ['SUCCESS', 'WARNING',
'NOTE']:

Regards,
Josh

On Dec 14, 9:32 am, Josh <gro...@j.oshua.net> wrote:
> Sorry it's taken me a week to get back. I've been seeing 10+ seconds.
> Sometimes it also just errors out lately though, so I may have other
> issues going on.  Thanks for the info.
>
> Regards,
> Josh
>
> On Dec 7, 1:27 pm, Dave Brown <wakeb0a...@gmail.com> wrote:
>
>
>
>
>
>
>
> > Josh,
> > I had the log level set to DEBUG  :(
>
> > How slow are you getting?  I was getting 10 seconds for 3 shipping
> > methods... now im getting ~4-5 seconds for 3
>
> > Dave
>
> > On Wed, Dec 7, 2011 at 3:52 PM, Josh <gro...@j.oshua.net> wrote:
> > > Hey Dave any chance you could let me/us know what was causing the

> > > speed issue.  I have noticed thatFeDexis generally slow and I wonder

> > > if I am doing something similar.
>
> > > Thanks,
> > > Josh
>
> > > On Dec 3, 1:28 pm, Dave Brown <wakeb0a...@gmail.com> wrote:
> > > > I figured out the speed issue, it was something stupid i was doing.
>
> > > > I now get 3 methods back in about 4.5 seconds, before it was taking ~10
> > > > seconds.
>
> > > > Is 1.5seconds per method about right you think?
>
> > > > Dave
>
> > > > On Sat, Dec 3, 2011 at 4:10 PM, hynekcer <hy...@sdb.cz> wrote:

> > > > > Chris, I confirm that newFedexmodules does not disturb other parts

> > > > > of Satchmo.
>
> > > > > I see that all new satchmo changes are applied to satchmo-fedex. So I
> > > > > think it should be anyway all pushed to Satchmo, but due to David's
> > > > > notes without large publicity yet. The speed can be tried fixed
> > > > > continuously on the fly. It is not blocking for other development.
> > > > > Thank you for giving the time to think.
>
> > > > > On 30 lis, 02:56, Dave Brown <wakeb0a...@gmail.com> wrote:

> > > > > > Chris, With onlyfedexground, it takes ~4 seconds.

> > > > > > If i bump it up one more (add 2 day), ~7 seconds
> > > > > > Add one more (express), ~10.29 seconds.
>
> > > > > > I'm assuming since the standard satchmo checkout has the payment and
> > > > > > shipping tied together, you are using the payment/shipping form to
> > > get
> > > > > the
> > > > > > dynamic rates?  There isn't any other 'simple' form that your using
> > > that
> > > > > > may make it quicker?
>
> > > > > > (i created a simple form since i want to get rates anywhere on the
> > > site
> > > > > > with the current cart items, not just at checkout, but that still
> > > didn't
> > > > > > make a difference).
>

> > > > > > The biggest thing I've noticed betweenfedexand usps is that even

> > > if i
> > > > > > have verbose logs unchecked, the log file is filled with thousands of
> > > > > lines

> > > > > > offedexrequest.  That can't be good for it speed.

> > > > > > Does this sound normal?
>
> > > > > > In regard to using the contact information, I'm about 99% sure that
> > > > > >fedex/usps only uses zip code & country to get rates, so having more
> > > than
> > > > > > that available wont be of any benefit.  I'm not sure about UPS
> > > though.
>
> > > > > > thanks!
> > > > > > Dave
>
> > > > > > On Tue, Nov 29, 2011 at 8:30 PM, Chris Moffitt <ch...@moffitts.net>
> > > > > wrote:
> > > > > > > Dave,
>
> > > > > > > Thanks for installing and checking and verifying it works fine.
>
> > > > > > > As far as the performance issues, I have a couple of questions:

> > > > > > > - How long does it take if you just haveFedexGround enabled?

> > > > > > >>fedexand USPS), you only need the zipcode and the country... so

> > > why
> > > > > > >> require there be a contact object?
>
> > > > > > >> Back tofedex:

> > > > > > >> I actually haven't been able to implementfedexon my site
> > > because I
> > > > > > >> haven't been able to get anyFedexmodule I've tried to get a

> > > descent
> > > > > > >> response time.
> > > > > > >> The one that ships with satchmo currently gives me a ~6 second
> > > > > response
> > > > > > >> time using only 2 shipping methods (express & 2 day) and one item
> > > in
> > > > > the
> > > > > > >> cart.
> > > > > > >> The soap method that Greg Newman built takes ~5.5 seconds
> > > > > > >> and the new method of yours I just tried out takes 8.5 seconds.
>
> > > > > > >> I cannot figure out what in the world is going on.. Using the
> > > USPS i
> > > > > can
> > > > > > >> get 1.5 second response times without a problem.

> > > > > > >> The response fromfedexis fine (~500ms) so I know its not an

> > > > > > >>>> day. I think I remember thatFedexdid not respond with a "time

> > > in
> > > > > > >>>> transit" or similar parameter.
>
> > > > > > >>>> Maybe "time in transit" is only for ground delivery not for
> > > aircraft
> > > > > > >>>> shipments.

> > > > > > >>>> But its hard to say as there is noFedexAPI and parameter

> > > > > description
> > > > > > >>>> available. I think its a good idea to proceed to the newerFedex
> > > > > API if
> > > > > > >>>> they provide better and more detailed information.
>
> > > > > > >>>> ionic
>
> > > > > > >>>> On Sat, 2011-11-26 at 16:28 -0800, hynekcer wrote:
> > > > > > >>>> > I fixed international characters for legacy method in
> > > > > collaboration
> > > > > > >>>> > with user "Ionic Drive" in e-mail communication following the
> > > > > thread
> > > > > > >>>> > groups.google.com/group/satchmo-users/browse_thread/thread/
> > > > > > >>>> > 86ea6bfeeae3955b/ . I did not published the solution becase he
> > > > > found

> > > > > > >>>> > other bug caused by outdatingFedexcode (no

> > > > > > >>>> > >Fedexservers but won't migrate cleanly to the newer and

> > > > > > >>>> > > api but after trying to read theFedexdocs for a while,

> > > they
> > > > > are
> > > > > > >>>> really
> > > > > > >>>> > > pushing towards the SOAP approach. Having some work already
> > > in
> > > > > > >>>> place for

> > > > > > >>>> > > the Python-fedexmodule, I think it's good to leverage

> > > instead
> > > > > of
> > > > > > >>>> > > recreating our own.
>
> > > > > > >>>> > > As you can tell, I'm leaning towards jumping to option #2

> > > but...
>
> read more »

[Chris Moffitt]

I am not seeing this error. Do you see any other information in the log files that might explain the issue?

-Chris

>
> read more »

--
You received this message because you are subscribed to the Google Groups "Satchmo users" group.
To post to this group, send email to satchm...@googlegroups.com.

[Josh]

The logs weren't showing anything conclusive. I flushed the cache and
it started working again. I had been switching back and forth between
the new fedex module and the version I had created some time ago
(which arlorla's fedex_web_services is a fork of) so corrupted data
probably ended up in the cache.

I had noticed this on a site with my old module when I wasn't
switching between modules so there could be some sort of unknown
caching issues, in case anyone runs into this particular issue again.
Thanks for the help!

Josh

> ...
>
> read more »

[hynekcer]

I think that I fixed logging for Fedex.
https://bitbucket.org/chris1610/satchmo/issue/1379/
(wait several hours before Chris probably submits to trunk)
Now it should be easy to find where is a problem ;-/)

I restricted also default logging of SQL queries. People mostly solve
other problems then number of queries. Both logging restrictions
affects only new installations by modified local_settings.py in the
skeleton.

Today I got several tracebacks caused by experiments of some hackers.
They are trying urls "/?page=../../../../some_nonsense"
If it is anying also for you now, apply two lines patch
https://bitbucket.org/hynekcer/satchmo-outgoing/changeset/d5e50a285c8c

-- Hynek

> ...
>
> číst dál »

[Chris Moffitt]

All those patches have been applied to tip.

Thanks,
Chris

[Josh Cartmell]

I just figured out issue I was having and it turns out it had nothing
to do with caching. The problem was that some of the items in my
database have a weight unit of lb instead of LB, which is why the
error appeared to be random. I'm not sure why my db is like that, but
it was causing the shipping module to choke because it wants LB and
will not accept lb. It was throwing a suds schema validation error,
unfortunately thats as far as the error went. I thought I would post
here in case anyone else runs into something similar.

I also wonder if anyone knows a way to get suds to give better
errors? For example is it possible for it to explicitly tell you what
the schema validation error is rather than just that there is one?

Thanks,
Josh

> ...
>
> read more »

[hynekcer]

Josh,
please write an issue to be possible to fix it later. Try to write it
the best you can.
(Imagine that you will once later select an issue which you will want
to fix from many issues written by other and you would prefer issues
clearly written, good explained etc.)

Could you write the wrong lowercase "lb" by admin? Or did you it
programmatically?

I am here frequently, but usually I debug complicated things which I
do no want to interrupt or to forget by reconfiguring patched Satchmo,
Django, etc.
Though I do not use weight field and I have no Fedex account, I can
understand something.
e.g. I suspected that caching problem is a popular alibi for something
what could not be reproduced in your case.

If you place some example of parsed code to pastebin and some bad
error message, maybe more people (including me) answer something or
try it for him- or herself.

Is it easier to find anything in the log after my fix?

-- Hynek

> ...
>
> číst dál »

[Chris Moffitt]

The Fedex module is picky about the weight units and needs the values in all caps. It should be simple to do an uppercase on the weight measurement in order to minimize this problem.
-Chris

[Josh Cartmell]

Thanks for the help guys and it definitely has been easy to fix. For
now I am just forcing the fedex to use the default weight value rather
than checking if the product has it's own because this site is all in
pounds so that is a safe assumption for me.

Hynek, your fix definitely makes the logs more readable and usable and
it is much appreciated, thanks! I'm not sure how lowercase got into
the db in this case. I inherited this site awhile ago so I don't know
all the history behind it.

The bad error messages I am referring to are not part of the satchmo
module but error messages from python-fedex and or suds. Inside
python-fedex there is a SchemaValidationError which always provides
the following error message:
"suds encountered an error validating your data against this service's
WSDL schema. Please double-check for missing or invalid values,
filling all required fields."

I was thinking it would be nice if it told you what schema validation
error you had, but I don't know too much about suds and though maybe
someone knew of a quick fix. It's not important now since I figured
out the problem I was having, but it could be nice for future
debugging.

Thanks,
Josh

> ...
>
> read more »

[Chris Moffitt]

There is an issue with suds and the python-fedex module (not satchmo's) where these types of exceptions get buried and not returned in the best format.

I think the only work around is hacking fedex and/or suds. I'm sure the creator of the python-fedex module would be willing to pull in updates if someone is willing to propose them.

-Chris

[hynekcer]

Josh,
this was not a raw message from suds but a "nice" message from fedex
(package pypi fedex). I wrote a fix for that fedex. It combines the
best of both messages:
http://dpaste.com/676494/ (can be downloaded within 7 days)
Please test it. I can not do it without an account.

Chris,
I tried to test it with a weight for the product, but got an
exception:
TypeError: unsupported operand type(s) for +=: 'float' and 'Decimal'
.../shipping/modules/fedex_web_services/shipper.py: 141
box_weight += item_weight
Variables
box_weight = 0.0
item_weight = Decimal("1.00")

A strong note about only allowed weight units LB and KG should be
written to Fedex module documentation or a conversion to the default
unit should be implemented. Other unit "g" (gram) not only "KG" is
frequently used in Europe for many products. It can not be easy
changed in the database if it is displayed also for users. If products
are imported from other site, different units can be in one site and
the actual model allows it, but KG and G would be summarized
numerically together now.

-- Hynek

> ...
>
> číst dál »