How to use the Google SOAP API
Ssentale
I've had a problem presented here for weeks and I am still stuck.
I want to use Tcl to submit a query to google. For example I may want
to access some website (i.e www.greenland.ac.uk) but I do not want to
use my web browser. I just want Tcl to automate this process.
I've tried to practice with lots of sample code provided on
comp.lang.tcl but with no success.
Then someone advises that the only way to do it is to use the Google
SOAP API. I downloaded the package and the key. But the package seems
to have java class files and C# and I think .NET etc..
The question is what do I do with all that stuff. How do I call this
API from within a TcL application?
Thanks for your time.
Harm Olthof
Note however that queries using the SOAP interface of Google run on special
servers, containing a
subset of Google. This gives sometimes unexpected results when you would
want to have the number of hits.
Harm
"Ssentale" <skiy...@hotmail.com> wrote in message
news:1163342160....@f16g2000cwb.googlegroups.com...
Cameron Laird
Ssentale's report concerns me. Ssentale, what you describe
is DEFINITELY possible with Tcl, and does NOT require Java,
.NET, or so forth. Mr. Olthof's suggestion is a good start-
ing point; if you don't quickly get satisfying results,
*please* describe the specifics of your situation, and I'm
confident we'll be able to help you achieve success.
Arnulf Wiedemann
Try to use Gerald Lesters Webservices package, it has an example for
accessing google with soap.
Arnulf
Bryan Oakley
I'm working on an article on that very subject for tclscripting.com; I
should have it posted laster this evening.
It's remarkably simple using Gerald Lester's Web Services for Tcl. Give
yourself a head start and install that package and any dependencies you
don't already have. Start here:
http://members.cox.net/~gerald.lester/WebServicesForTcl.html
The code to make an API call to google looks something like this:
::WS::Client::GetAndParseWsdl \
http://api.google.com/GoogleSearch.wsdl"
set args {}
lappend args key "<your license key here>"
lappend args q "site:tclscripting.com bindtags"
lappend args start "0 "
lappend args maxResults "10"
lappend args filter "true"
lappend args restrict ""
lappend args safeSearch "false"
lappend args lr ""
lappend args ie "latin1"
lappend args oe "latin1"
set result [::WS::Client::DoCall \
GoogleSearchService doGoogleSearch $args]
(that's not a canonically correct example as it uses a list rather than
a dict for the arguments, but it works just as well...)
Donal K. Fellows
> set args {}
> lappend args key "<your license key here>"
> lappend args q "site:tclscripting.com bindtags"
I'd do that like this;
set args {
key "<your license key here>"
q "site:tclscripting.com bindtags"
etc "and so on..."
}
It always seemed cleaner to me to put everything in one place without
littering it all up with lots of [lappend args] noise. OK, I admit I
like [array set] a lot too. :-)
> (that's not a canonically correct example as it uses a list rather than
> a dict for the arguments, but it works just as well...)
For this sort of thing, the difference does not matter in the
slightest. The cost of communicating over the network will swamp any
performance differences. :-)
Donal.
Bruce
> Bryan Oakley wrote:
>> set args {}
>> lappend args key "<your license key here>"
>> lappend args q "site:tclscripting.com bindtags"
> [...]
>
> I'd do that like this;
> set args {
> key "<your license key here>"
> q "site:tclscripting.com bindtags"
> etc "and so on..."
> }
for statically defined things I do this too, but
as soon as some of the values (or possibly the keys)
are stored in variable, then the lappend is actually
cleaner since you can avoid quoting mess
bruce
Ssentale
The question is what do I do with the zipped file that is downloadable
from http://members.cox.net/~gerald.lester/WebServicesForTcl.html
Do I extract it into the directore where my Wish84 is?
Silver
Bryan Oakley
> Thank you very much for your help.
> The question is what do I do with the zipped file that is downloadable
> from http://members.cox.net/~gerald.lester/WebServicesForTcl.html
>
> Do I extract it into the directore where my Wish84 is?
Start up a tcl shell. Run the following command:
puts $::auto_path
That will show a list of directories. Copy the folder with the web
services code into any one of those directories.
Bryan Oakley
> Ssentale wrote:
>
>> Hi there,
>> I've had a problem presented here for weeks and I am still stuck.
>> I want to use Tcl to submit a query to google. For example I may want
>> to access some website (i.e www.greenland.ac.uk) but I do not want to
>> use my web browser. I just want Tcl to automate this process.
>>
>> I've tried to practice with lots of sample code provided on
>> comp.lang.tcl but with no success.
>> Then someone advises that the only way to do it is to use the Google
>> SOAP API. I downloaded the package and the key. But the package seems
>> to have java class files and C# and I think .NET etc..
>> The question is what do I do with all that stuff. How do I call this
>> API from within a TcL application?
>
>
> I'm working on an article on that very subject for tclscripting.com; I
> should have it posted laster this evening.
I have finally posted that article.
Donal K. Fellows
> for statically defined things I do this too, but
> as soon as some of the values (or possibly the keys)
> are stored in variable, then the lappend is actually
> cleaner since you can avoid quoting mess
Actually, the most common case is a mix of mostly static stuff and some
dynamic bits. In that case, put the static parts first as illustrated
above and then add the dynamic bits. Also, if you use [dict set] instead
of [lappend], you get a natural way of managing defaults too since that
will override existing mappings if there is a duplicate.
set args {
key ... q ... etc ...
}
dict set args foo $bar
if {some($condition)} {
dict set args q "optional replacement"
}
Donal.
Bruce
> Bruce wrote:
>> for statically defined things I do this too, but
>> as soon as some of the values (or possibly the keys)
>> are stored in variable, then the lappend is actually
>> cleaner since you can avoid quoting mess
>
> Actually, the most common case is a mix of mostly static stuff and some
> dynamic bits. In that case, put the static parts first as illustrated
> above and then add the dynamic bits. Also, if you use [dict set] instead
> of [lappend], you get a natural way of managing defaults too since that
> will override existing mappings if there is a duplicate.
>
except I use arrays rather than dict or lappend, so after
the static stuff I use individual set on an element to get
expanded stuff.
array set Info {
a "lkdfjbh"
b " klbnsd"
...
}
set Info(x) [query_val]
set Info(z) $input_param
....
Bruce
Donal K. Fellows
> and you just described the top of most of my programs ;
> except I use arrays rather than dict or lappend, so after
> the static stuff I use individual set on an element to get
> expanded stuff.
Perhaps we should go ahead and claim that it is a Tcl Software Pattern.
Donal.
chawki...@gmail.com
Hello Bryan.
I've read your article. I've followed everything and was able to run
all but the very last script. I get the following error:
missing value to go with key
while executing
"dict get $item title"
(procedure "main" line 19)
invoked from within
"main"
(file "C:\MACROS\google_soap_api.tcl" line 30)
Process tclsh exited with code 1
Any thoughts on what the problem might be?
Thanks very much,
CJ.
walto...@gmail.com
a protocol, and it is the protocol Google uses for a special API to
access their services. However, you don't need to use Google's
complicated API in order to access their site (or any site) though Tcl.
All that you need is HTTP.
With HTTP, you'll be communicating directly with the site's web server,
just as your browser does. Try the following code:
# load the http extension.
package require http
# download the web page.
set token [http::geturl http://www.google.com]
set data [http::data $token]
http::cleanup $token
# save it to a file.
set token [open google.html w]
puts $token $data
close $token
Bryan Oakley
> Hello Bryan.
>
> I've read your article. I've followed everything and was able to run
> all but the very last script. I get the following error:
>
> missing value to go with key
> while executing
> "dict get $item title"
> (procedure "main" line 19)
> invoked from within
> "main"
> (file "C:\MACROS\google_soap_api.tcl" line 30)
> Process tclsh exited with code 1
>
> Any thoughts on what the problem might be?
Hmmm..... I may have to rethink my solution. You'll get that error if
you $item is not a well-formed dict (read: does not have an even number
of words). I *thought* $item would be a valid dict but maybe it is not
always the case. Are you using the search parameters from my article or
something different?
Can you email me your script?
Bryan Oakley
Oddly, I just tried my example today and it doesn't work any more. I
think maybe Google has changed something about the data it returns --
I'm getting a DOM parse error.
Ssentale
provided by Walton and many other solutions provided by those who have
responded to my query. However, all the time I get "Couldn't open
socket: wrong urguments" error. I am using Windows XP and I am
connected to a work network.
Is there anyone who can figure out what the problem is?
Mark Janssen
Ssentale wrote:
> I have copied and pasted into TcL's concole the tcl code provided
> provided by Walton and many other solutions provided by those who have
> responded to my query. However, all the time I get "Couldn't open
> socket: wrong urguments" error. I am using Windows XP and I am
> connected to a work network.
>
> Is there anyone who can figure out what the problem is?
>
Coorporate firewall? If so and it is a normal Socks proxy, have a look
at autoproxy in Tcllib.
Mark
Larry W. Virden
Ssentale wrote:
> I get "Couldn't open
> socket: wrong urguments" error. I am using Windows XP and I am
> connected to a work network.
>
> Is there anyone who can figure out what the problem is?
Can you identify the exact line that is raising the error, and provide
to us the values of any variables in that line?
Ssentale
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
(bin) 1 % package require http
2.5.2
(bin) 2 % package require dict
8.5.3
(bin) 3 % package require WS::Client
1.0.2
(bin) 4 % set token [http::geturl
http://api.google.com/GoogleSearch.wsdl]
couldn't open socket: invalid argument
(bin) 5 % puts errorInfo
errorInfo
(bin) 6 % puts $::errorInfo
couldn't open socket: invalid argument
while executing
"http::geturl http://api.google.com/GoogleSearch.wsdl"
(bin) 7 %
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Mark Janssen
Ssentale schreef:
> This is the error as requested:
> :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
> (bin) 1 % package require http
> 2.5.2
> (bin) 2 % package require dict
> 8.5.3
> (bin) 3 % package require WS::Client
> 1.0.2
> (bin) 4 % set token [http::geturl
> http://api.google.com/GoogleSearch.wsdl]
> couldn't open socket: invalid argument
This matches the symptoms when you are trying to use the http package
behind a proxy. Are you using/Do you have to use a proxy server? If so,
to get http to use the proxy for retrieving web pages you can use the
autoproxy package from tcllib. The easiest thing to try is to insert
before the http::geturl
package require autoproxy
autoproxy::init
If that doesn't work (because auotproxy cannot determine your proxy
settings by itself) use:
package require autoproxy
autoproxy::configure -proxy_host {proxyserver} -proxy_port {port}
-basic -username {proxy_username or empty} -password {password or
empty}
Mark
Gerald W. Lester
> This is the error as requested:
> :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
> (bin) 1 % package require http
> 2.5.2
> (bin) 2 % package require dict
> 8.5.3
> (bin) 3 % package require WS::Client
> 1.0.2
1.0.4 got posted this week, you may want to down load it, even though you
seem not to be using it.
> (bin) 4 % set token [http::geturl
> http://api.google.com/GoogleSearch.wsdl]
> couldn't open socket: invalid argument
> (bin) 5 % puts errorInfo
> errorInfo
> (bin) 6 % puts $::errorInfo
> couldn't open socket: invalid argument
> while executing
> "http::geturl http://api.google.com/GoogleSearch.wsdl"
The geturl and the ::WS::Client::GetAndParseWsdl both work for me.
It almost looks like your socket command is broken.
Are you using ActiveTcl from ActiveState?
What does:
info patchlevel
return?
> (bin) 7 %
> ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
> Larry W. Virden wrote:
>
>> Ssentale wrote:
>>> I get "Couldn't open
>>> socket: wrong urguments" error. I am using Windows XP and I am
>>> connected to a work network.
>>>
>>> Is there anyone who can figure out what the problem is?
>> Can you identify the exact line that is raising the error, and provide
>> to us the values of any variables in that line?
>
--
+--------------------------------+---------------------------------------+
| Gerald W. Lester |
|"The man who fights for his ideals is the man who is alive." - Cervantes|
+------------------------------------------------------------------------+
Mark Janssen
> This matches the symptoms when you are trying to use the http package
> behind a proxy. Are you using/Do you have to use a proxy server? If so,
> to get http to use the proxy for retrieving web pages you can use the
> autoproxy package from tcllib. The easiest thing to try is to insert
> before the http::geturl
> empty}
>
> Mark
By the way the reason you get the not so informative error message is
because on windows:
% socket doesnotexist 80
couldn't open socket: invalid argument
% set errorInfo
couldn't open socket: invalid argument
while executing
"socket doesnotexist 80"
Look at
http://sourceforge.net/tracker/index.php?func=detail&aid=219237&group_id=10894&atid=110894
for details.
Mark