Creating a 'Click' event using tIP ??

[Mel Smith]

Hi:

Is there a way to create a 'Click' event thru the tIP modules, or do I
have to use the CreateObject("InternetExploration.Application") to create an
oIE instance ?

I've used the oIE way before but now I've come to like the tIP way
better.

(of course, I can fall back to the Curl executable --- curl.exe if I
have to. And the hbcurl library way is still inoperable)


Thanks,

-Mel

[eisoftserv]

Hello,

Please describe your:
- at server-side I want to achieve...
- at client-side when the user does...I want to...

Ella

[Mel Smith]

Ella said:

> Please describe your:
> - at server-side I want to achieve...
> - at client-side when the user does...I want to...

Server-Side: a public organization whose pages I have downloaded, parsed,
and placed needed info into two databases at about 1 page every 1.5 seconds.

Client-Side: Me, on my Win Xp Pro (sp3) machine runnnig anxHarbour app with
the tIP modules.

Below I show the tIP-related getpage() function which 'gets' the needed
page. (The page retrieved is approx 64K thru 68K)

Of course, there is a bunch of work before the function is called and a
large amoun of 'parsing' code that I use to pick out needed pieces of info
which I place in my databases. I have now finished the downloading work
(over the last week) and have 18,580 records in one database, and approx
110,000 records in the second.

Now, my further problem:

The 'page' downloaded has a button that needs to be clicked
programmatically, so that it submits a request to the server for more info.
I believe this is an AJAX request because the page address doesn't change,
the viewable section doesn't change, except that another table of data opens
up on the screen where there is *more* information that I need and want to
get at.

So, I want to click that button (can it be done thru tIP ???) , and
start investigating what happens.

Thanks for your response.

-Mel

-------------------------------------------------------------------------

Here is the operating portion that 'gets' the needed page

***********************************************
FUNCTION GETPAGE(nKKKKCODE,cHTXFILE)
STATIC cUrlTpl :=
"http://www.xxxxxxxxxxx.org/yyyyyy/yyyyy/Profile/yyyyyyProfile.aspx?ID=#fzzzid#"
LOCAL lOK := .T.
LOCAL oCon, oUrl
LOCAL cURL
LOCAL cLOG := "",lDOLOG := .F.
LOCAL cDATA,cDATAFILE := "\aaa\htx\C_#fzzzid#.htx"
LOCAL cKKKKCODE

cLOG := CRLF
cURL := cURLTPL // GET THE TEMPLATE
cKKKKCODE := NTOC(nKKKKCODE)
cUrl := STRTRAN(cUrl,"#fzzzid#",cKKKKCODE)
oUrl := tURL():New(cUrl)
oCon := TipClientHttp():New( oUrl )
oCon:nConnTimeout := 20000

IF oCon:Open(cUrl)
cDATA := oCon:ReadAll()
cHTXFILE := cDATAFILE
cHTXFILE := STRTRAN(cHTXFILE,"#fzzzid#",cKKKKCODE)
MEMOWRIT(cHTXFILE,cDATA)
ELSE
lDOLOG := .T. // MAKE THE LOG
cLOG += CRLF + "Can't connect with "+ oUrl:cServer
IF oCon:SocketCon == NIL
cLOG += CRLF + "Connection not initiated"
ELSEIF InetErrorCode( oCon:SocketCon ) == 0
cLOG += CRLF + "Server said: " + oCon:cReply
ELSE
cLOG += CRLF + "Error in connection: " + InetErrorDesc(
oCon:SocketCon )
ENDIF
ENDIF

oCon:Close()
IF .NOT. lDOLOG
RETURN lOK
ENDIF

cLOG += CRLF + "Done" + CRLF
MEMOWRIT("\aaa\LOGS\HTTPLOG.TXT",cLOG)

RETURN lOK
*************************************************

[eisoftserv]

> the tIP modules.

> STATIC cUrlTpl :=
>
> "http://www.xxxxxxxxxxx.org/yyyyyy/yyyyy/Profile/yyyyyyProfile.aspx?ID=#fzzzid#"

I'm using ASP.NET, please let me summarize what I've understood so far:

- you are a scraper proggy written in xHarbour
- above is a sample URL similar to the real one
- initially you are doing a http GET request to the server, and you are receiving a HTML stream (file), which can be crunched

Please open your browser (IE or whatever)
- navigate to the above URL or the .aspx which includes the desired button (the button you want to push)
- now open up the source code of the web page in your browser (with F12 etc.) and identify the HTML element ( an INPUT element, or Image...) which is representing your desired button
- hunt down the exact URL used when it's clicked ( it could be a ONCLICK event handler with inline JavaScript, or a JavaScript function associated to the event handler, or it can be a jQuery code sequence in the html header, which is using the button's ID attribute value

In ASP.NET when you click a button on a form, ALWAYS a http POST happens, and that might be targeting the same .ASPX file, a WebMethod of the same .ASPX file, a web service, or a web handler file...and in your browser it might produce a complete or partial page refresh, regardless of what you see after the POST operation.

So you need to know the URL targeted by the button :-)

Ella

[Mel Smith]

Hi Ella:

I understand completely what you mean (because I spend half my day
building Javascript code and HTML for my new project :)) )

I took a quick look for the 'button' code on one of the 'pages' a week
or so ago, and this code was very effectively hidden in a see of
downloadable js stuff. So I dropped my search until now. But, with your
advice, I'll go looking again.

Thanks for detailed instructions !

-Mel Smith

[eisoftserv]

Hello,

Short answer: the task is not doable in conventional manner, and it's not doable at all without loading the page normally in a web browser window.

Long answer: ASP.NET works like this:

- in most cases a control on the page corresponds to a PEER of classes: one of them sitting at server-side, the other sitting at client-side (managed by JavaScript stuff) and communicating between them

- the communication between server-side and client-side "parts" is packaged inside at least a stream (VIEWSTATE), which are passed back and forth all the time, carried by server responses and client PostBacks

- VIEWSTATE is basically carrying a collection of sequentialized objects and it's encrypted

Example to how it works the classic ASP.NET PostBack:

- the user is clicking a button in the browser (or does other action which produces a PostBack)

- a PostBack is made to the server (right before POSTing the VIEWSTATE is being updated and encrypted "automatically")

- IF the button's server-side peer has an OnClick method defined, that method is fired on the server (in your case an additional table is created and displayed)

- the new code of the HTML page and the VIEWSTATE are prepared and sent back to the client

Your INPUT element contains this code:

onclick="javascript:__doPostBack( ObjectName, SomeValue)"

This is a classic ASP.NET PostBack, which is working as described above, and is producing a complete page refresh (not ajax).



Ella

[Mel Smith]

Hi Ella:

Thank you for this very clear explanation !

Before reading the above I replied to you privately by email. Please
ignore my emailed response :((

The above sequence makes it clear that I have a lot more to learn here.

Thanks again,

-Mel