How to send request for responded page to server without Rendering on Page?

[Hakoo]

I am developing app where I need to capture an Information from Webpage after giving credentials automatically. Some how I managed to do Automatic login and redirection of page. Here is my code :

HttpWebRequest req = (HttpWebRequest)WebRequest.Create("https://abcd.com.au/categories/A_dfn/sdf");
HttpWebResponse res = req.GetResponse() as HttpWebResponse;

StringBuilder sb = new StringBuilder();
byte[] buf = new byte[10000];
Stream resStream = res.GetResponseStream();
string s = null;
int c = 0;
do
{
    c = resStream.Read(buf, 0, buf.Length);
    if (c != 0) {
        s = ASCIIEncoding.ASCII.GetString(buf, 0, c);
        sb.Append(s);
    }
} while (c > 0);
string oldhead = "class=\"login_button\">";
string newhead = "class=\"login_button\">   <script type=\"text/javascript\">document.getElementById('btn').click()</script>";
sb.Replace(oldhead, newhead);

string oldbtn = "value=\"Submit\"";
string newbtn = "value=\"Submit\" id=\"btn\" ";
sb.Replace(oldbtn, newbtn);

string oldAction = "<form action=\"/login\" method=\"post\">";
string newAction = "<form action=\"https://abcd.com.au/login?orig_req_url=%2Fcategories/A_dfn/sdf\" method=\"post\">";
sb.Replace(oldAction, newAction);

string oldUsername = "<input id=\"login_email\" type=\"text\" name=\"user[email_address]\" class=\"textBox\" value=\"\">";
string newUserName = "<input id=\"login_email\" type=\"text\" name=\"user[email_address]\" class=\"textBox\" value=\"[e-mail]\">";
sb.Replace(oldUsername, newUserName);

string oldPass = "<input id=\"login_password\" type=\"password\" name=\"user[password]\" class=\"textBox\" value=\"\">";
string newPass = "<input id=\"login_password\" type=\"password\" name=\"user[password]\" class=\"textBox\" value=\"abc\">";
sb.Replace(oldPass,newPass);
Response.Write(sb);

This is show me expected output as I want by rendering page(Response.write(sb)). But, now I want to do same thing without redirecting to "https://abcd.com.au/login?orig_req_url=%2Fcategories/A_dfn/sdf" and want to do more stuff on this. I expect to get output of Response.Write(sb) in some buffer. Is it possible to do?

Here is example, that explains exactly what I want to do. I am looking for an product's qty say name : Screw 15mm, this resides in page https://abcd.com.au/%2Fcategories/A_dfn/sdf. So, I am requesting this url first, but as need login to access that page, its redirecting me to login page, filling username and password, pressing login button by using javascript,and then redirected to Originally Requested page. And on this page I want to find for that product, and return information to my web app.

All this I want to do without showing to user. I just want to show retrieved information.

Thanks.

[LastMitch]

So, I am requesting this url first, but as need login to access that page, its redirecting me to login page, filling username and password, pressing login button by using javascript,and then redirected to Originally Requested page. And on this page I want to find for that product, and return information to my web app.

This is more than just rendering on a page. The way you put it you want this to be a cycle?

Read this:

http://msdn.microsoft.com/en-us/library/ms178472%28v=vs.100%29.aspx

To find the product you need to trace it with the cycle:

http://msdn.microsoft.com/en-us/library/y13fw6we%28VS.71%29.aspx

[Hakoo]

@LastMitch: Hi, thanks for your answer. This process is not happening in asp.net. I want to do this by my web application to another website. So its like, I want to capture information of [e-mail]/products/abcProduct, but to access that, I need login to that site. So, thats what I am doing in my code. If you have look to my code, you can understand that, I am replacing some sentense which I need to login and then press button using javascript. I am getting response back, and that response is necessary to me. So, all this happening using Response.Write(sb). But I want to do all this things without rendering it on page. May be I can store all this in some buffer. Hope, you now understand my problem.

Thanks
Hakoo Desai.