Member Avatar for bo0ga

Hey guys. I am new at programming and would appreciate any help I can get. I want to make a program that reads the HTML code of a web page, and writes a specific line into a document. So for example, I want my code to read the source of the www.daniweb.com homepage and write whats inbetween the <title></title> tags into a notepad file which should be "DaniWeb - Technology Publication Meets Social Media." The code below works, but returns the entire HTML source code of the page.

import java.io.*;
import java.net.MalformedURLException;
import java.net.URL;

public class UrlReadPageDemo {
    public static void main(String[] args) {
        try {
            URL url = new URL("http://www.daniweb.com");

            BufferedReader reader = new BufferedReader(new InputStreamReader(url.openStream()));
            BufferedWriter writer = new BufferedWriter(new FileWriter("data1.txt"));

            String line;
            while ((line = reader.readLine()) != "<title>") {
                System.out.println(line);
                writer.write(line);
                writer.newLine();
            }
            reader.close();
            writer.close();
        } catch (MalformedURLException e) {
            e.printStackTrace();
        }  catch (IOException e) {
            e.printStackTrace();
        }
    }
}
  • 4 Contributors
  • 5 Replies
  • 229 Views
  • 3 Days Discussion Span
  • Latest Post Latest Post by NormR1

Dani AI

Generated

Short answer: stop treating HTML as plain line-by-line text and use a real HTML parser. Line-based comparisons are brittle (different casing, extra whitespace, tags split across lines, character encoding, entities, redirects, etc.). correctly flagged a loop/condition bug in the original code, and ’s substring idea can work for simple cases — but as noted it breaks when the <title> and </title> are on different lines or when the page uses unusual formatting.

Recommended — use jsoup (lightweight, handles encodings and entities, and gives you the title directly):

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import java.io.FileWriter;

Document doc = Jsoup.connect("http://www.daniweb.com").get();
String title = doc.title();
try (FileWriter fw = new FileWriter("title.txt")) {
    fw.write(title);
}

If you cannot add a dependency, read the whole response into one string and search with a DOTALL, case‑insensitive regex (still fragile, but handles tags on different lines):

StringBuilder sb = new StringBuilder();
String line;
while ((line = reader.readLine()) != null) sb.append(line).append('\n');

Pattern p = Pattern.compile("(?is)<title>(.*?)</title>");
Matcher m = p.matcher(sb.toString());
if (m.find()) {
    String title = m.group(1).trim();
    // write title to file
}

Practical tips: trim the result, unescape HTML entities if needed, and set a user-agent/timeout when fetching remote pages. Prefer the parser approach for reliability; use regex only for quick one-off scripts. For more on jsoup, see jsoup.org.

Recommended Answers

All 5 Replies

Member Avatar for anuj_sharma

new

Try using this:

 while (!(line = reader.readLine()).equals("<title>"))

Not sure that the logic you are using will get the desired results. Try thinking over it again.

Edited by anuj_sharma
Member Avatar for bo0ga

anyone?

Member Avatar for NormR1

A simple way would be to read the input stream until the starting tag is found and then save what is read until the ending tag is found.

Member Avatar for ColmSmith 
String line;
String outLine;

while ((line = reader.readLine()) != null) {

if (line.contains("<title>")){
    outLine = line.substring(line.indexOf("<title>")+7, line.indexOf("</title>") );
    writer.write(outLine);
    writer.newLine();
    System.out.println(outLine);
    }
Member Avatar for NormR1

What if the tags are on different lines?

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.