Member Avatar for UNOWN

Hey guys, I am having a problem figuring out how I would extract all the links from a webpage, and then filter them to get only a certain type of link.

For Example: i want to filter all the links that start with:

How would I do this? (PS. Im using a webBrowser on my Windows Form)

  • 2 Contributors
  • 2 Replies
  • 104 Views
  • 8 Hours Discussion Span
  • Latest Post Latest Post by UNOWN

Dani AI

Generated

As requested and building on 's approach: grabbing anchors is the right idea, but there are a few common pitfalls to fix so the filter is reliable and safe.

First, wait for the full document (handle WebBrowser.DocumentCompleted and skip frame events by checking e.Url == webBrowser1.Document.Url). Always check href for null/empty and skip non-HTTP schemes like javascript: or mailto:. Second, do not compare against the literal anchor markup — compare the href string or, better, parse it into a Uri and resolve relative links against the page base. Use case-insensitive comparisons and prefer checking Uri.Host when you mean to match a domain rather than an arbitrary prefix.

Example pattern (robust, avoids the common mistakes):

private void webBrowser1_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{
    if (webBrowser1.Document == null || e.Url != webBrowser1.Document.Url)
        return;

    var matches = new List<string>();
    foreach (HtmlElement a in webBrowser1.Document.GetElementsByTagName("a"))
    {
        var href = a.GetAttribute("href");
        if (string.IsNullOrEmpty(href))
            continue;

        Uri uri;
        if (!Uri.TryCreate(href, UriKind.Absolute, out uri))
        {
            if (!Uri.TryCreate(webBrowser1.Document.Url, href, out uri))
                continue;
        }

        if ((uri.Scheme == Uri.UriSchemeHttp || uri.Scheme == Uri.UriSchemeHttps) &&
            uri.Host.StartsWith("www.googleads", StringComparison.OrdinalIgnoreCase))
        {
            matches.Add(uri.AbsoluteUri);
        }
    }

    // matches contains the filtered absolute URLs
}

Notes: if you truly need a prefix match on the full URL use uri.AbsoluteUri.StartsWith("http://www.googleads", StringComparison.OrdinalIgnoreCase). For background scraping or better HTML parsing control, consider HttpClient + HtmlAgilityPack instead of the UI WebBrowser.

Recommended Answers

All 2 Replies

Member Avatar for farooqaaa

Its pretty simple.

First you'll store all the "a" tags in an HtmlElementCollection and then check each one if it StartsWith "". If so, then add it to the links list.

List<string> links = new List<string>();
HtmlElementCollection col = webBrowser1.Document.GetElementsByTagName("a");

foreach (HtmlElement elem in col)
{
   if (elem.GetAttribute("href").StartsWith(""))
          links.Add(elem.GetAttribute("href"));
}

// Test it
foreach(string str in links)
  MessageBox.Show(str);

Thanks

Edited by farooqaaa because: n/a
Member Avatar for UNOWN

no...thank you very much, i will try this right now and let u know if it worked.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.