I am trying to use Firebug to help me create in VB.net 2005 the correct Httpwebresponse to a asp.net/IIS web portal

[Penny Willoughby]

I am trying to modifiy a vb.net 2005 in production Clinician software suite.  Prior to a recent update by the state ran insurance site this program successfully used a screen scrape to log into the site using the users login information and would upload, download etc to the site using HTTPWebRequest.  Most of the work was done using the HTTPWebRequest and the HTTPWebResponse.  The download had needed to use SOAP, but all of this was working prior to my employment for several years.

It was recommended to download Firebug to 'see' what is happening, and it was totally cool, but I had only ever used Firebug in school to modify HTML.  I am hoping someone could help me understand more about how Firebug can show me what the page is expecting as it's post and how to possible set up my httpwebrequest accordingly. thank you

Last week the state site changed significantly and the state agency is not really working with me so I am on my own. This is in the body of the page when I look at source. The new site is ASP.Net/IIS with DotNetNuke and lots of JavaScript and redirects.

    <form method="post" action="/hcp/Default.aspx?alias=www.ohcaprovider.com/hcp/provider"  onsubmit="javascript:return WebForm_OnSubmit();" id="Form" enctype="multipart/form-data" autocomplete="off">

The first difference I noticed is that the first page is doing a PostBack to itself we used to post the parameters on the end of the next pages Url.  

    function __doPostBack(eventTarget, eventArgument) {
    if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
        theForm.__EVENTTARGET.value = eventTarget;
        theForm.__EVENTARGUMENT.value = eventArgument;
        theForm.submit();
    }

Also the first page only wants a login name and no matter what you put in manually will proceed to the next page (as this is a post back I am assuming it has a redirect going on too).  However if I set up the HTTPWebRequest it always gives me a Status 200 and the response is the default page (which is also the first page) of the web portal.

I have really researched and hunted to find answers.  I am a new person to posting to forums, and I really welcome and need some help. 

I have installed firebug and noticed that when I post manually it shows me in the post what appears to be a design for a multipart/form data.  I have attempted to copy and put on HTTPWebRequest, but It is not giving me anything but a status 200 and the response is the default page again.  Below I will try to piece the code together as it is in different OOP pieces. 

Here is my attempt to modify what worked in the past for the new web site:
Basically I set up Httpwebrequest, add headers, get page, scrape for __ViewState, set up the multipart/form, set up post httpwebrequest, post and then I don't get what I expect.  I'm not sure what to expect or if one(or more) pieces of this code is not working right. Thank you again for any help.

        Dim lsViewState As String = "__VIEWSTATE"" value="""
        Try
            'Section of code to get the upload form GET
            chwrequest = WebRequest.Create("https://www.ohcaprovider.com/hcp/Default.aspx?alias=www.ohcaprovider.com/hcp/provider")
            chwrRequest.Method = "GET"
            chwrRequest.KeepAlive = True
            chwrRequest.CookieContainer = cckcCookieContainer
            ' Configure the web request to work with a proxy, like ACT
            If pobjProxy Is Nothing Then
               pobjProxy = System.Net.WebRequest.DefaultWebProxy
               pobjProxy.Credentials = System.Net.CredentialCache.DefaultCredentials
            End If
            chwrRequest.Proxy = pobjProxy
           
            'ADD Headers
            chwrRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0" 
            chwrRequest.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
            chwrRequest.Headers.Add("Accept-Language", "en")
            chwrRequest.Headers.Add("Accept-Charset", "windows-1252, utf-8, utf-16, iso-8859-1;q=0.6, *;q=0.1")
            chwrRequest.KeepAlive = True
           
            'Get Page
            chrsResponse = chwrRequest.GetResponse()
            cstmStream = chrsResponse.GetResponseStream()
            lsResp = CSubmitterUtils.GetStreamContent(cstmStream)

            cstmStream.Close()
            chrsResponse.Close()
            CSubmitterUtils.WriteFileContent(psSaveAs, lsResp)  **writes to file for debug purposes

            'Store cookie Date
            fsCookieData = cckcCookieContainer.GetCookieHeader(New Uri(OHCA_WEB_NEW))
           
            'Section of code to do fill form and upload file SCRAPE for viewSTATE
            Dim lnViewStateURLIndex As Integer = csResp.IndexOf(lsViewState)

            If lnViewStateURLIndex < 0 Then
                WriteLog("ViewState not found")
                lbReturn = False
            End If

            Dim lnStartIndex As Integer = lnViewStateURLIndex + lsViewState.Length
            Dim lnEqualIndex As Integer = csResp.IndexOf("=", lnStartIndex)
            Dim lsViewStateContents As String = csResp.Substring(lnStartIndex, lnEqualIndex - lnStartIndex)

            'Setup to POST
            chwrequest = WebRequest.Create(psUrl)
            chwrRequest.Method = "POST"
            chwrRequest.KeepAlive = True
            chwrRequest.CookieContainer = cckcCookieContainer
            ' Configure the web request to work with a proxy, like ACT
            If pobjProxy Is Nothing Then
               pobjProxy = System.Net.WebRequest.DefaultWebProxy
               pobjProxy.Credentials = System.Net.CredentialCache.DefaultCredentials
            End If
            chwrRequest.Proxy = pobjProxy
 
            'ADD Headers
            chwrRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0" 
            chwrRequest.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
            chwrRequest.Headers.Add("Accept-Language", "en")
            chwrRequest.Headers.Add("Accept-Charset", "windows-1252, utf-8, utf-16, iso-8859-1;q=0.6, *;q=0.1")
            chwrRequest.KeepAlive = True
            chwrRequest.AllowAutoRedirect = False

            'Setup multipart/form
            SetupLogonFileSubmit(lsViewStateContents)
            Dim lmpBuffer As MultiPartBuffer
            Dim lsContentType As String = "Content-Disposition: form-data; name="
            dim csBoundary = "------------------------------" & DateTime.Now.Ticks.ToString("x")

            lmpBuffer.ContentTypeHeader = "multipart/form-data; boundary=" & csBoundary.Substring(2)
            lmpBuffer.WriteLine(csBoundary)
            lmpBuffer.WriteLine(lsContentType + "__EVENTTARGET") 
            lmpBuffer.WriteLine()
            lmpBuffer.WriteLine("")

            lmpBuffer.WriteLine(csBoundary)
            lmpBuffer.WriteLine(lsContentType + "__EVENTARGUMENT") 
            lmpBuffer.WriteLine()
            lmpBuffer.WriteLine("")

            lmpBuffer.WriteLine(csBoundary)
            lmpBuffer.WriteLine(lsContentType +  "__LASTFOCUS")
            lmpBuffer.WriteLine()
            lmpBuffer.WriteLine("")

            lmpBuffer.WriteLine(csBoundary)
            lmpBuffer.WriteLine(lsContentType + "__VIEWSTATE")
            lmpBuffer.WriteLine()
            lmpBuffer.WriteLine(lsViewStateContents + "=")

            lmpBuffer.WriteLine(csBoundary)
            lmpBuffer.WriteLine(lsContentType + "__VIEWSTATEENCRYPTED")
            lmpBuffer.WriteLine()
            lmpBuffer.WriteLine("")

            lmpBuffer.WriteLine(csBoundary)
            lmpBuffer.WriteLine(lsContentType + "dnn$ctr1842$Login$UserIdCmnTextBox$Control")
            lmpBuffer.WriteLine()
            lmpBuffer.WriteLine(psLogName)

            lmpBuffer.WriteLine(csBoundary)
            lmpBuffer.WriteLine(lsContentType + "dnn$ctr1842$Login$LoginCmnButton")
            lmpBuffer.WriteLine()
            lmpBuffer.WriteLine("Log In")

            lmpBuffer.WriteLine(csBoundary)
            lmpBuffer.WriteLine(lsContentType + "ScrollTop")
            lmpBuffer.WriteLine()
            lmpBuffer.WriteLine("")

            lmpBuffer.WriteLine(csBoundary)
            lmpBuffer.WriteLine(lsContentType + "__dnnVariable")
            lmpBuffer.WriteLine()
            lmpBuffer.WriteLine("{""__scdoff"":""1""}")

            lmpBuffer.CloseBuffer()

            Dim lsMpContent As String = lmpBuffer.ToString()
            chwrRequest.ContentLength = lsMpContent.Length
            chwrRequest.ContentType = lmpBuffer.HttpContentTypeHeader

            Dim lbyBytesBuff As Byte()
            lbyBytesBuff = Encoding.UTF8.GetBytes(lsMpContent)

            cstmStream = chwrRequest.GetRequestStream()
            cstmStream.Write(lbyBytesBuff, 0, lbyBytesBuff.Length)
            cstmStream.Close()

           'Get the Response
            chrsResponse = chwrRequest.GetResponse()
           'Put it in a stream
            cstmStream = chrsResponse.GetResponseStream()
           
            If chrsResponse.StatusCode = HttpStatusCode.OK Or chrsResponse.StatusCode = HttpStatusCode.Found Then
                lsResp = CSubmitterUtils.GetStreamContent(cstmStream)
                cstmStream.Close()
            Else
                lsResp = ""
            End If

            chrsResponse.Close()
            CSubmitterUtils.WriteFileContent(psSaveAs, lsResp) **Previously this was then used to move on to the next page for scraping/posting

[Sebastian Zartner]

It's not clear to me what exactly you want to do. As far as I understand you want to log into https://www.ohcaprovider.com/hcp/Default.aspx?alias=www.ohcaprovider.com/hcp/provider to get information from there and use it on your site.
The easier way would of course be to use an API to get that information. I.e. you should get in contact with the site owner and ask for it.
If that's not possible for whatever reason, you have to continue the way you started.
It seems you already saw that you can use Firebug's Net panel to see the headers for each request. So what you need to do is to rebuild each page request you see in Firebug when manually following the steps. AFAICS you are first doing a GET request to the page. This doesn't seem to be necessary. You can directly post the login data.

Please note that your actual question seems to be unrelated to Firebug but rather requires some VB.net knowledge. So for further help you may better ask in a VB.net forum.

Sebastian

[Penny Willoughby]

Thank you for your suggestions.  you are correct, I am trying to use the firebug results in VB 2005.  Also I can now navigate  through most of the pages on the new site.  I was able to use firebug's post response to build the correct multipart form and find the parameters for session id's after the three page login process.

One last question if this has helped clarify what I need firebug for. 

The last piece is to download a specific file.  I need to set my GET request right.  Any suggestions on which pieces/places to look in firebug to correctly set up the HTTP response would be very helpful.  The IIS server/ASP.Net app gives a link, but I cannot seem to use it programmatically.  So if there are any extra suggestions for what I am looking for and where I might find it using firebug would be very help.

Again thank you for your suggestions.

[Sebastian Zartner]

For the file download you should also see a request inside the Net panel like this:

The GET request should normally look the same as the other requests. So just try it with the headers you had in the GET request of the code you posted before.

Sebastian