Trying to Login to a Forum via C# Winforms application using HttpWebRequest

[minigweek]

Hi,

I am working on building a C# Winforms application which will allow me to login to a forum or site and perform certain tasks. I am stuck at the very 1st step. I cannot login to the forum. Here is what I have tried so far,
the Login button has the following code :-

try
            {
                Uri uri = new Uri("http://www.thinkdigit.com/forum/login.php");
                string data = "vb_login_username=userid&vb_login_password=password";
                HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(uri);
                request.Method = WebRequestMethods.Http.Post;
                request.ContentLength = data.Length;
                request.ContentType = "application/x-www-form-urlencoded";
                StreamWriter writer = new StreamWriter(request.GetRequestStream());
                writer.Write(data);
                writer.Close();
                HttpWebResponse response = (HttpWebResponse)request.GetResponse();
                StreamReader reader = new StreamReader(response.GetResponseStream());
                string tmp = reader.ReadToEnd();
                response.Close();
                richTextBox1.AppendText(tmp);               
            }
            catch (Exception ex)
            {
                textBox3.Text = ex.ToString();
            }

Thats what I scrounged up after google busting and visiting so many forum posts. Unfortunately this does not seem to log me in the forum. I am checking the HTML generated content in t my richtextbox control and there's the login fields there still. Which means it didn't login.

I checked the login page.

and these are the contents there

<form name="loginfrm" id="loginfrm" action="login.php?do=login" method="post" onsubmit="md5hash_2(vb_login_password, vb_login_md5password, vb_login_md5password_utf, 0);">

<input type="text" class="bginput" style="font-size: 11px" name="vb_login_username" id="navbar_username" size="10" accesskey="u" tabindex="101" value="User Name" onblur="myUnTip();" onmouseout="myUnTip();" onfocus="if (this.value == 'User Name') this.value = '';myTip();"   />

<input type="password" class="bginput" style="font-size: 11px" name="vb_login_password" id="navbar_password" size="10" tabindex="102" />

<input type="submit" class="button" value="Log in" tabindex="104" title="Enter your username and password in the boxes provided to login, or click the 'register' button to create a profile for yourself." accesskey="s" />

those are the relevant parts IMHO

so I am filling up the username field, filling up the password, posting to the login.php page with the data ...what am i mising out ?

Also I have another query. Is there a way to load the response , as in HttpGetResponse in a webbroswer control ? If I am login in to the forum, then it would be nice to have the logged in page loaded in my webbrowser control!!

Any Help Would be appreciated.

thanks a bunch folks for your time!!

[sknake]

Out of curiosity what task are you trying to perform when you log in?

[minigweek]

Hi sknake ( i almost typed snake!! )

thanks for dropping by. I have various ideas floating around which I want to do. Like there's a news /technology and gadget section in the forum, I would like to have first 10 topic snippets in my application , and reading it can reply in 1/2 sentences using my app without loggin into the forum via browser.

Then there's this forum where latest upcoming novels description are posted, I would like to get snippets of novel description from there. and so on!! :)

I also need to understand how this works for another more serious reason too.
@ Work we use a tool for keeping track of various JVMs in servers , its web based tool which fetches data from the JVMs. I plan to develop a desktop client which can query that webserver and get few important data into my tool where I can do more analysis and set up alerts as per my requirement. And so many more stuffs. Basically I am trying to self improvise and come up with new ideas. Any help ? :)

[minigweek]

Hello anyhelp folks ?

[sknake]

Oh man I forgot about this thread :( I'm about to run out for the day but i'll post some code this evening

[sknake]

Actually it looks like they use client side encryption for the login values so this will be a little more of an undertaking. I don't have time right now to work on it :(

You can find examples of browser automation on google. Take a look at what people have done with mshtml -- you can get the same functionality working with webrequests.

[minigweek]

Yes I see they are hashing the password using md5 hashing. I have opened their vbulletin_md5.js file and am attempting to call their md5hash function in an html file and check if their md5hashed result is same that as of one of the md5 hashed generated value from the C# programs I found online. If its the same I will try posting the hidden input field which stores their md5 hash value, instead of the password field. I think the password is sent to the md5hash function,which returns the md5 hash and then this md5 hash is being posted as a password to the login form. Looks like it will take some time to understand the md5hash function as it has 4 arguments and I am yet to make head or tails of it.

Sigh. thanks for dropping by. I will keep updated on my progress. Learning lots stuffs.

[sknake]

Alright I got it .. but you have to sniff your login and get the md5 hash. I tested it and it doesn't look like it changes based on session or anything, it is just a hash of your password. I used Wireshark to sniff the post and grab the MD5 hash.

private void button1_Click(object sender, EventArgs e)
    {
      const string login = @"sknake";
      const string password = @"";
      const string md5 = @"";

      CookieContainer cookieMonster = new CookieContainer();
      string postArgs = string.Format(
        @"raw_login_password={0}&do=login&url=%2Fforum%2Fusercp.php&vb_login_md5password={2}&vb_login_md5password_utf={2}&s=&securitytoken=guest&vb_login_username={1}&vb_login_password=",
        password,
        login,
        md5
         );

      HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create("http://www.thinkdigit.com/forum/login.php?do=login");
      req.Method = "POST";
      req.CookieContainer = cookieMonster;
      req.Referer = @"http://www.thinkdigit.com/forum/usercp.php";
      req.ContentType = @"application/x-www-form-urlencoded";
      byte[] buffer = System.Text.ASCIIEncoding.UTF8.GetBytes(postArgs);
      req.ContentLength = buffer.Length;
      req.AllowAutoRedirect = true;
      using (Stream reqStream = req.GetRequestStream())
      {
        reqStream.Write(buffer, 0, buffer.Length);
        reqStream.Close();
      }
      HttpWebResponse response = (HttpWebResponse)req.GetResponse();
      using (Stream respStream = response.GetResponseStream())
      {
        using (StreamReader sr = new StreamReader(respStream))
        {
          string s = sr.ReadToEnd();
          System.Diagnostics.Debugger.Break();
        }
      }
    }

This is what I got back:

<form action="http://www.thinkdigit.com/forum/usercp.php?" method="post" name="postvarform">
<table class="tborder" cellpadding="6" cellspacing="1" border="0" width="70%" align="center">
<tr>
	<td class="tcat">Redirecting...</td>
</tr>
<tr>
	<td class="panelsurround" align="center">
	<div class="panel">

		<blockquote>
			<p>&nbsp;</p>
			<p><strong>Thank you for logging in, sknake.</strong></p>
			
				<p class="smallfont"><a href="http://www.thinkdigit.com/forum/usercp.php?">Click here if your browser does not automatically redirect you.</a></p>
				<div>&nbsp;</div>
			
		</blockquote>

	</div>
	</td>
</tr>
</table>
</form>

[serkan sendur]

>Hi sknake ( i almost typed snake!! )

that happens to me a lot :)

[minigweek]

Thanks Scott for the code. I am reading it now, will be trying to understand what happened here, and then come back and ask if I have any doubts! :)

[minigweek]

Update : Just installed wireshark, and was going thru its helppages to learn the filters. Its amazing tool ! oh wow!

Now I can see whts the url encoded text the login form submitted! So much intersting info!! :))

http://www.adamek.biz/md5-generator.php

allowed me to confirm that its a just plain md5 function, nothing fancy :) which is good.

I will keep exploring your code and learn stuffs. thnks again!! :) will keep you posted!

[sknake]

Have you made any progress?

[minigweek]

Hi Sknake , thanks for your time. Yes I am able to login to the forum , as using wireshark ( as you showed me!! ) I am able to find out the hidden fields they are posting to their forms. I can now see which inputs I need to get from the user ( me , the userid and pass ) .

I understand the CookieContainer is important to store the session cookie. Thanks for showing me its use.

I have some more queries and doubts which I have been trying to find out about.

1. I am logging into the forum my user id and password. Its writing the HTML code in a rich text box and later loading it into a webbrowser control shows me the "Thank you for logging in Thor" and "Click here if your browser does not automatically redirect you." . Because of the redirection script snippet in the html, the webbrowser is redirected to the next page. But as the Webbrowser doesn't show me logged in and I see the login user id and password field.

I understand this is happening because the Cookie is stored in the CookieContainer object of HttpWebRequest object. So i would need to find a way to use the same cookie in the webbrowser if I plan to use the form posting via httpwebrequest and webbrowser control simultaneously. I have been trying but to no success.

Ok why I am trying ? The page is being redirected, and as far as I read, automatic redirection of the HttpWebRequest is enabled by default, so I should be able to get the redirected page using WebResponse too and able to write the Html Code as usual. But its not happening. sigh.

So I thought maybe using WebBrowser cntrol i can get to the redirected page. Again not happening.

Your help here would be so helpful sigh.

2. I have tried the login and it works like magic @ other php forums too as it should. And I have tried various actions like click on "thanks" , "create a new thread" , "preview a post" , "submit a post" actions and have closely monitored the data being sent by HTTP GET or HTTP POST using Wireshark.

I noticed this securitytoken appearing time and again. You used its value as guest in the login code just as the browser does ( established by wireshark too ). But once logged into the forum , its value changes with every action and I don't think trying guest there would work at all.

So to do any action on the forum using the Winforms application I would probably need this securitytoken taken care off. And the rest of the fields are easy , like subject , message title, postbody and stuff and can be filled with ease.

It would be tremendous help if you can explain or guide me in this regards.

3. @Work there's a site we use which is SSL enabled , we use it for various kind of tracking of application related issue. I used wireshark there too , but unfortunately , i am unable to sniff out any form of recognizable POST/GET data. Guess this can still wait, but I am intensely interested in being able to work with this site of ours too!! :D

[sknake]

1. There are many ways to "redirect" a page. The HttpWebRequest will follow redirection if the server issues a HTTP-301 Permanent Redirect or a HTTP-302 Temporary Redirect but it will not redirect when javascript inside of the page contains the logic for the redirect, as is the case here. The server assumes you have javascript enabled because you obviously had to run javascript to get the hash value to login, right? ;) Remember the HttpWebRequest is dumb. It follows HTTP codes and thats about it.

2. I stopped once I had it logging in. I will have to take a look later on today.

3. You can decode SSL. I usually use ettercap for this instead of Wireshark and I swap out the security certificates for one where i have the public and private key so I can decode the secure data. It throws those "Invalid security certificate!" messages in the browser but I just click through them to learn how the website behaves when i'm sniffing. I will have to take a look at this later on too.

[minigweek]

1. There are many ways to "redirect" a page. The HttpWebRequest will follow redirection if the server issues a HTTP-301 Permanent Redirect or a HTTP-302 Temporary Redirect but it will not redirect when javascript inside of the page contains the logic for the redirect, as is the case here. The server assumes you have javascript enabled because you obviously had to run javascript to get the hash value to login, right? ;) Remember the HttpWebRequest is dumb. It follows HTTP codes and thats about it.

Right! :D .... I will be using the dumbness of httpwebrequest and get back to you later on how I am faring . Again! Much thanks for breaking it down.

[hohy]

Hi Sknake , thanks for your time. Yes I am able to login to the forum , as using wireshark ( as you showed me!! ) I am able to find out the hidden fields they are posting to their forms. I can now see which inputs I need to get from the user ( me , the userid and pass ) .

I understand the CookieContainer is important to store the session cookie. Thanks for showing me its use.

I have some more queries and doubts which I have been trying to find out about.

1. I am logging into the forum my user id and password. Its writing the HTML code in a rich text box and later loading it into a webbrowser control shows me the "Thank you for logging in Thor" and "Click here if your browser does not automatically redirect you." . Because of the redirection script snippet in the html, the webbrowser is redirected to the next page. But as the Webbrowser doesn't show me logged in and I see the login user id and password field.

I understand this is happening because the Cookie is stored in the CookieContainer object of HttpWebRequest object. So i would need to find a way to use the same cookie in the webbrowser if I plan to use the form posting via httpwebrequest and webbrowser control simultaneously. I have been trying but to no success.

Ok why I am trying ? The page is being redirected, and as far as I read, automatic redirection of the HttpWebRequest is enabled by default, so I should be able to get the redirected page using WebResponse too and able to write the Html Code as usual. But its not happening. sigh.

So I thought maybe using WebBrowser cntrol i can get to the redirected page. Again not happening.

Your help here would be so helpful sigh.

2. I have tried the login and it works like magic @ other php forums too as it should. And I have tried various actions like click on "thanks" , "create a new thread" , "preview a post" , "submit a post" actions and have closely monitored the data being sent by HTTP GET or HTTP POST using Wireshark.

I noticed this securitytoken appearing time and again. You used its value as guest in the login code just as the browser does ( established by wireshark too ). But once logged into the forum , its value changes with every action and I don't think trying guest there would work at all.

So to do any action on the forum using the Winforms application I would probably need this securitytoken taken care off. And the rest of the fields are easy , like subject , message title, postbody and stuff and can be filled with ease.

It would be tremendous help if you can explain or guide me in this regards.

3. @Work there's a site we use which is SSL enabled , we use it for various kind of tracking of application related issue. I used wireshark there too , but unfortunately , i am unable to sniff out any form of recognizable POST/GET data. Guess this can still wait, but I am intensely interested in being able to work with this site of ours too!! :D

can u help me in post data from c# form to php page and navigate the page with data in web browser control