Verbose tracing of HTTP Requests - possible?

87 views
Skip to first unread message

Joachim Tuchel

unread,
Mar 8, 2021, 11:02:16 AM3/8/21
to VAST Community Forum
Hi there!

I am a bit lost about some quite simple REST / HTTPs request I am sending out via VAST.
I constantly get a 400 Bad Request from the server. Sending the same request via curl from the command line does work, however.

I have taken a look at all kinds of stuff during the sending of the request and it really is a simple request, but still some subtle difference between the Smalltalk and the curl version must exist.

I was wondering if there is something similar to curl's --verbose option in VAST/SSt?
Do you guys have any tips on what to look at and how to trace what goes out and if SSL handshaking etc. work?


Joachim



Joachim Tuchel

unread,
Mar 8, 2021, 11:19:47 AM3/8/21
to VAST Community Forum
Ok, some more info:

When I do

response := client send: request to: theURL.

response contains an errorObject with an Error Code of 153, which is an SstReceiveError (definde in SstErrorConstants). But the response also contains HTML code and an HTTP Response Code of 400 Bad Request, so it seems the request reached the remote server. I get the same response to very simple GET requests as well as POSTs.

I'd like to see what came back to the VAST image befor it is wrapped in an SstByteMessage. Maybe I find something useful in the raw response?

Joachim

Joachim Tuchel

unread,
Mar 8, 2021, 12:47:50 PM3/8/21
to VAST Community Forum
Okay,


my problem gets more interesting. It seems to be unrelated to the structure of my request.
I tried this:


Which, as expected, answers with 401 Not Authorized. What I get in an inspector, however, is  an  SstReceiveError with Error Code 153 which holds onto the response in its errorObject instVar. That response looks good, it contains JSON with an error message and lots of headers and stuff.

So far it seems I can tell:
  • The endpoint is reached and an answer is coming back
  • it seems to not be an SSL configuration problem (I had suspected that in the meantime, but I have requests to other servers that work)
So, what else can I try?
  •  'https://google.com' sstAsUrl fetch also returns an SstReceiveError with an HTTP Response of 301 moved Permanently inside.
  • . 'https://www.google.com' sstAsUrl fetch returns a nice HTML page - no redirects or anything
So am I having a problem with redirects or such? Is there a way around them in VAST?
No, I don't think so: entering https://api.sendinblue.com/v3/contacts into the address bar of Firefox and watching the network tracing, it shows no redirects or such. The request is immediately answered with a 401 unauthorized.

One thing that might play a role here is that teh Browser's network logger says the request was HTTP/2. I don't think VAST 9.2.2 supports HTTP/2.

So I tried another curl with --http1.1 and the curl request gets a proper answer....


What else can I try now?

Mariano Martinez Peck

unread,
Mar 8, 2021, 1:02:14 PM3/8/21
to VA Smalltalk
I think you should contact the provider. Everything seems fine from your end. I just tried with curl too


and I get a 401 with {"message":"Key not found","code":"unauthorized"}

So..I think so far is not a problem in the http client but in how you should be calling that service. 




--
You received this message because you are subscribed to the Google Groups "VAST Community Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to va-smalltalk...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/va-smalltalk/9cc2fe64-7807-42f8-a5ba-f9932fa0003cn%40googlegroups.com.


--
Mariano Martinez Peck
Software Engineer, Instantiations Inc.

Louis LaBrunda

unread,
Mar 8, 2021, 2:51:26 PM3/8/21
to VAST Community Forum
Hi Guys,

This is going to get a little confusing, so bear with me.

I converting some of my V9.2.2 programs to V10.0.0.  One gets email from a server using the Totally Objects email stuff in Socket Set.  I'm converting this to the new VAST SstImap... stuff.

When I try to use the SstImap4ClientExample I get an error  'Function not in library: SSLv23_method', that leads me to think there is a DLL problem.  The abt.ini file says SSL_LIB=libssl but libssl.dll doesn't exist with V10 (I do find a copy with an old Ecap).  When I try that file, things still don't work.

The V9.2.2 abt.ini says SSL_LIB=ssleay32.  Now ssleay32.dll doesn't exist with V9.2.2 but I have a V8.6 version that does.

Now to  Joachim's problem.   When I try:

'https://www.google.com' sstAsUrl fetch inspect.

with V10.0.0, it fails.  When I try it in V10.0.0 with the old ssleay32.dll, it still fails.  But when I try:

'http://www.google.com' sstAsUrl fetch inspect.

it works.  Note, the 'http:..' and not 'https:...' gets it to work.  My conclusion is that V10.0.0 and maybe V9.2.2 don't have the latest libssl.dll.

Am I nuts or on to something?

Lou

Joachim Tuchel

unread,
Mar 8, 2021, 3:06:05 PM3/8/21
to VAST Community Forum
Lou,

I also have problems getting this stuff to work in V10.0.0. There I get an SstSendError which advises me to check the OpenSSL version.
As far as I understand, you need to use libcrypto.dll instead of libeay32 for VAST 10. But I can't get this to work either.

I tried copying libcrypto and libssl from an 1.1.0.h version of openSSL into the VAST10.0.0 runtime directory and still get the SstSenError.

But the problems I mentioned in the beginning are in VAST 9.2.2, where I also have a working REST client in the same image that connects to another server. This other server is also using SSL / HTTPS, so I don't think it is an openSSL configuration problem. Also the fact that https://www.google.de works...


Joachim

Seth Berman

unread,
Mar 8, 2021, 3:16:52 PM3/8/21
to VAST Community Forum
Hello All,

The first resource to look at before my explanation about OpenSSL is the migration guide.
Please check these after a release.
https://www.instantiations.com/vast-support/documentation/1000/#page/mi/migr9222.html#

The second resource is the FAQ
https://www.instantiations.com/vast-support/documentation/FAQ/#page/FAQ/va09002.html

OpenSSL Important Information
1. We do not ship OpenSSL libraries with the product.  There are export restrictions with crypto so we (Instantiations) have never shipped them.  We did ship the old 0.9.x series that IBM packaged with the product.  Supposable, they had some sort of legal waiver to do so but its not really important to this conversation.  We don't package them with the product, and therefore the user must get installers to download such as https://slproweb.com/products/Win32OpenSSL.html.

2. Once the dlls or shared libraries are installed, the ini files must be updated to point to them.
ssleay32 and libeay32 are the old names on windows from the 1.0.x and below series of OpenSSL.  ALL of those versions are now OBSOLETE.
The new names are libssl.dll and libcrypto.dll.  BUT depending on who you get the libraries from...it may be named something like libcrypto-1_1-x64.dll or libssl-1_1-x64.dll.
This is something you have to know...our product can't know that.
But, we changed the default names in the ini files from ssleay32 and libeay32 to libssl and libcrypto to reflect the prefixed names of supported versions of OpenSSL.

A quick setup guide for success

1. Download Windows Installer for OpenSSL 

v1.1.1j Light 32-bit
https://slproweb.com/download/Win32OpenSSL_Light-1_1_1j.msi

v1.1.1j Light 64-bit
https://slproweb.com/download/Win64OpenSSL_Light-1_1_1j.msi

2. Execute Installers

- Win32OpenSSL_Light-1_1_1i.msi to install. (this will install to C:\OpenSSL-Win32)
- Win64OpenSSL_Light-1_1_1i.msi to install. (this will install to C:\OpenSSL-Win64)   

3. Change the following platform name mappings in abt.ini (STAGE 1: ABSOLUTE PATHS)
32-bit
CRYPTO_LIB=C:\OpenSSL-Win32\libcrypto-1_1.dll
SSL_LIB=C:\OpenSSL-Win32\libssl-1_1.dll

64-bit
CRYPTO_LIB=C:\OpenSSL-Win64\libcrypto-1_1.dll
SSL_LIB=C:\OpenSSL-Win64\libssl-1_1.dll

4. Restart VAST and try

5. Change the following platform name mappings in abt.ini (STAGE 2: RELATIVE PATHS)

32-bit
CRYPTO_LIB=libcrypto-1_1.dll
SSL_LIB=libssl-1_1.dll

64-bit
CRYPTO_LIB=libcrypto-1_1.dll
SSL_LIB=libssl-1_1.dll

6. Restart VAST and try


Hope this helps

Joachim Tuchel

unread,
Mar 8, 2021, 3:21:38 PM3/8/21
to VAST Community Forum
Mariano,

The funny thing is that I get an 401 in VAST, but it is wrapped in a SstReceiveError. Look at this screenshot from my Debugger when I evaluate


SStReceiveError.png

So it seems everything is fine, except VAST throws an error resp. returns an SstReceiveError instead of an SstByteMessage with the 401 header.

In my "full" case with an api-key and query JSON in the request, the result is also a SstReceiveError with the "real" response in its errorObject variable, which is a 401 Bad Request.

So actually I have two problems:

  1. I get an SstReceiveError instead of a response object, but the response is inside of the error object...?
  2. My "real" query doesn't work from VAST while an equivalent curl does, and I need a way to spot the difference

Any more ideas would be greatly appreciated.

Joachim

Joachim Tuchel

unread,
Mar 8, 2021, 4:03:03 PM3/8/21
to VAST Community Forum
HI Seth,


thanks for clarifying. I had found both resources earlier today.  But I seem to have followed the instructions there completely wrong.

I had downloaded OpenSSL 1.1.0.h from the second  site mentioned on the OpenSSL site (indy.fulgan.com) and copied the two DLLs libcrypto and libssl from that ZIP to the VAST 10 installation directory (the path where abt.exe is). I renamed them to libcrypto.dll and libssl.dll to make them match the symbilic names in the abt.ini of my image. Didn't work, threw an SstSendError.

I tried your description given here and with the full path in the abt.ini for the DLLs I now get the very same behavior as in VAST 9.2.2

So I can confirm downloading from slproweb, installing and setting the full path name of the DLLs works. I somehow couldn't get it to work without the path. I used V10.0.0x64 on a Windows 10 VM.

One more question: If I understand the FAQ correctly, VAST 9.2 works with openSSL 1.0 only, right?


So the 10.0. problem is solved, but now I am still chewing on that SstReceiveError problem.


This is what I see in VAST 10.0.0x64 when I inspect


So communication works in VAST 10 now.

SStReceiveError_V10.png

Thanks again, Seth.


My other problem, however, is still open. A request that I think looks exactly the same as its curl counterpart entered via command line does fail with a 401 Bad request, both in VAST 9.2.2 and 10.0.0.
I know nobody here on this list can tell me why that is, but maybe somebody has an idea how I can find the difference between the two.


Joachim

Seth Berman

unread,
Mar 8, 2021, 4:24:29 PM3/8/21
to VAST Community Forum
Hello Joachim,

That can be confusing, so here is the relevant part from the FAQ

From FAQ:
VA Smalltalk 8.6.2 and above supports OpenSSL version 1.0.x. Anything below this version level is not just unsupported; it is known to be incompatible.
VA Smalltalk 8.6.3 and above supports OpenSSL version 1.1.0.
VA Smalltalk 9.1 and above supports OpenSSL version 1.1.1.

This means the bindings in 9.2 supports 1.0.x and 1.1.x.
However, should you be using 1.0.x at this point?
Answer: No, not unless you are a company paying for extended support from OpenSSL (https://www.openssl.org/support/contracts.html)

If folks want more clarity on End-Of-Life from OpenSSL versions, see the following:

- Seth

Mariano Martinez Peck

unread,
Mar 8, 2021, 4:45:35 PM3/8/21
to VA Smalltalk


  1. I get an SstReceiveError instead of a response object, but the response is inside of the error object...?

Not sure if this is a problem. I guess it was designed to work this way and it's debatable. 
 
  1. My "real" query doesn't work from VAST while an equivalent curl does, and I need a way to spot the difference


Can you share the curl command, the output and your smalltalk code?


Best, 

Joachim Tuchel

unread,
Mar 9, 2021, 3:24:43 AM3/9/21
to VAST Community Forum
Hi Mariano,

Well, I agree: it seems that the returning of an SstRceiveError is intended, but also very strange. Especially given that #isCommunicationsError returns true for errorCode 153. Getting a 401 Bad Request from the remote server is not a problem receiving a message via a socket, nor any other kind of communications error. It is a logical error at best. But  hey, someone at IBM made this strange decision two decades ago, we'll have to live with it... So let's not concentrate on this one, it's strange, but it can be handled.

Here is my code in VAST that gets a 401 Bad Request from the server:

|client request response |
client := SstHttpClient forTransportScheme: 'httpsl'.
request := client templateHttpGETMessage copy.
request header
    at: 'api-key' put: 'my private API key';
    at: 'content-type' put: WAMimeType applicationJson greaseString;
    at: 'accept' put:  WAMimeType applicationJson greaseString.
request contents: ''. "Doing this or not doesn't make a difference"

client startUp.
[response := client send: request to: 'https://api.sendinblue.com/v3/contacts' sstAsUrl]     ensure: [client shutDown].
response inspect.



And this is the curl that works like a charm (with or without the added --http1.1 parameter, just to be sure):

curl  --url https://api.sendinblue.com/v3/contacts   --header 'api-key:my private API key'   --header 'Content-Type: application/json'   --header 'accept: application/json' --http1.1  

I really can't find the reason why the curl call works and the VAST call returns an 401 Bad Request.... Even debugging into Sst, I see no reason why the server shouldn't like the request.
So what tools can I use to get more insight? Do you see any difference?

Joachim

Joachim Tuchel

unread,
Mar 9, 2021, 3:27:26 AM3/9/21
to VAST Community Forum
... also it doesnt change anything if I change content-type to Content-Type in the Smalltalk code, I just double-checked....

Mariano Martinez Peck

unread,
Mar 9, 2021, 10:53:46 AM3/9/21
to VA Smalltalk
Joachim, 

The problem was that such a server required a client side CA (Certificate Authority) validation [1]. When you run the curl command:

curl -v  --url https://api.sendinblue.com/v3/contacts   --header 'api-key:MY_KEY'   --header 'Content-Type: application/json'   --header 'accept: application/json'

It works because it finds the "ca bundle". See the following lines in the output:

* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem


That pem file is usually known as the CA bundle and it's just a list of all known CAs. The problem is who keeps that list up to date. Linux seems to do it and depending on the different distros it can be found in different places [2]. Git is usually another tool that does that. I took my Linux one and copied it to Windows as c:\Users\mpeck\ca-bundle.pem. The format is valid and you can save it with .pem or .crt, up to you. 

Your Smalltalk side wasn't very clear to me so I wrote it from scratch in the way I believe the SstHttpClient should be used:

| url client headers |

url := 'https://api.sendinblue.com/v3/contacts'.

client := SstHttpClient forTransportScheme: 'httpsl'.  

headers := Dictionary new

at: 'api-key' put: 'MY_KEY';

at: SstHttpConstants::HttpAcceptKey put: WAMimeType applicationJson greaseString;

yourself.

[

client configuration securityConfiguration caFile: 'c:\Users\mpeck\ca-bundle.pem'.

client startUp.

responseMsg := client 

get: url 

typed: WAMimeType applicationJson greaseString 

using: nil 

withHeaders: headers

] ensure: [client shutDown].

responseMsg basicContents asString inspect.

responseMsg inspect.




The key line here is the "client configuration securityConfiguration caFile: 'c:\Users\mpeck\ca-bundle.pem'."


Let me know if that works. 

Best, 

Mariano




--
You received this message because you are subscribed to the Google Groups "VAST Community Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to va-smalltalk...@googlegroups.com.

Joachim Tuchel

unread,
Mar 9, 2021, 2:00:09 PM3/9/21
to VAST Community Forum
Mariano,

I can't get it to work. Strange: I tried in VAST 10.0.0 on Linux (Kubunt 18.04). I have the full paths to libcrypto and libssl in the abt.ini of my image.
I tried with


client configuration securityConfiguration caFile: '/etc/ssl/certs/ca-certificates.crt'.
And I still get 401 Unauthorized


After all you say I guess this is an OpenSSL config problem....

VAST claims to be on OpenSSL Version 1.0.2n while 'openssl version' in a command line answers "OpenSSL 1.1.1 Sep 2018".

How does VASt find the SSL libs if not (only) via the name mappings in abt.ini?


Joachim

Mariano Martinez Peck

unread,
Mar 9, 2021, 3:17:20 PM3/9/21
to VA Smalltalk
I just tried it on Linux too and it works perfect for me.

Mariano Martinez Peck

unread,
Mar 9, 2021, 3:18:16 PM3/9/21
to VA Smalltalk
For Linux the only difference was:

client configuration securityConfiguration caFile: '/etc/ssl/certs/ca-certificates.crt'.

Joachim Tuchel

unread,
Mar 10, 2021, 3:05:27 AM3/10/21
to VAST Community Forum
Mariano, Seth, all,


Once again: Thank you very much for your patient and kind support. This whole story turned out to be my own ignorance towards simple rules of HTTP.
I took you on a long journey around a very simple problem: The request I had created in my initial code was missing two very essential parts: the path in the GET header and the complete HOST header.

While Mariano's code travelled through a few methods deep down in SstHttpClient, my approach of nailing the Request together was incomplete. It seems some servers out there don't care about these errors, and this one does. The road to the riddle's solution started with Mariano finding a stupid error in my code snippet. Other than teh one posted here in the list, I had repeated the word api-key in the api-key header's value: at: 'api-key' put: 'api-key:xyz'. Once I had corrected this, I could see that Marianos code worked and mine didn't and I could spot the difference between the HTTP request created by Mariano's snippet and the one I had created...

So this didn't have much to do with SSL at all, it seems. When I create a valid HTTP Request, the snippet works as expected, even without setting the caFile:

I've learned a lot in this little episode, and I am happy with Mariano's and Seth's support. It's good to have you as co-pilots!

Joachim

Mariano Martinez Peck

unread,
Mar 10, 2021, 8:19:11 AM3/10/21
to VA Smalltalk
Greetings Joachim,

We are glad to read that indeed the problem is fixed now and the code is working. As always, it is our pleasure to help. As you are a customer/reseller, we were able to allocate some engineering time to analyze and resolve it

Best regards, 

Mariano

Seth Berman

unread,
Mar 10, 2021, 8:22:57 AM3/10/21
to VAST Community Forum
Hi Joachim,

Just echoing what Mariano said...great to hear you are past the problem!

Kind Regards,

- Seth

Reply all
Reply to author
Forward
0 new messages