I am having problem getting all the links from a webpage. I have successfully created this using just C#. I used
foreach (HtmlElement link in webBrowser1.Document.Links)
{
string linkItem = link.GetAttribute("HREF").ToString();
Apparently C# in ASP.net doesn't have System.Windows.Forms so I can't use HtmlElement. I've tried a couple of different ways but I never get all the links. It seems that links that are located in javascript sections of the won't show up. Here are a couple of ways that I've tried:
string webpage = TextBoxUrl.Text.ToString();
string source = getPageSource(webpage);
string getPageSource(string URL)
{
System.Net.WebClient webClient = new System.Net.WebClient();
string strSource = webClient.DownloadString(URL);
webClient.Dispose();
return strSource;
}
And this that I found at http://www.csharp-station.com/HowTo/HttpWebFetch.aspx
class WebFetch
{
static void Main(string[] args)
{
// used to build entire input
StringBuilder sb = new StringBuilder();
// used on each read operation
byte[] buf = new byte[8192];
// prepare the web page we will be asking for
HttpWebRequest request = (HttpWebRequest)
WebRequest.Create("http://www.mayosoftware.com");
// execute the request
HttpWebResponse response = (HttpWebResponse)
request.GetResponse();
// we will read data via the response stream
Stream resStream = response.GetResponseStream();
string tempString = null;
int count = 0;
do
{
// fill the buffer with data
count = resStream.Read(buf, 0, buf.Length);
// make sure we read some data
if (count != 0)
{
// translate from bytes to ASCII text
tempString = Encoding.ASCII.GetString(buf, 0, count);
// continue building the string
sb.Append(tempString);
}
}
while (count > 0); // any more data to read?
// print out page source
Console.WriteLine(sb.ToString());
}
}
Is this even possible to do?