Member Avatar for Benkyou

Hi there,

I was wandering if anyone can give me a brief explanation on how to connect my C application to the Internet. What i would like to do in the end is to access a web page trough the application and have means to read the html code behind it. Please bear in mind i have no idea how to do this or even if its practical. Im on Ubuntu by the way.

If any of you know of a good website to study this subject, i would be very grateful.

Thanks in advance.

  • 3 Contributors
  • 6 Replies
  • 359 Views
  • 1 Day Discussion Span
  • Latest Post Latest Post by sree_ec

Dani AI

Generated

Short, focused supplement that builds on ’s socket suggestion and the thread comments from and .

Learning raw BSD sockets (getaddrinfo, socket, connect, send, recv) is valuable for understanding TCP and DNS, but HTTP adds many protocol details (request line and headers, Host, persistent connections, redirects, chunked transfer encoding, compression, and TLS for HTTPS). For a C program whose goal is “fetch a web page and read the HTML,” using a mature HTTP client library is far more practical and robust than rolling everything from scratch.

A recommended, minimal path on Ubuntu:

  • Install the libcurl development package (package name can vary; common names include libcurl4-openssl-dev or libcurl4-gnutls-dev).
  • Compile with the curl library (for example: gcc fetch.c -o fetch -lcurl).
  • libcurl handles DNS, redirects, TLS, cookies and compressed responses.

Simple libcurl outline (prints the page to stdout):

#include <curl/curl.h>
#include <stdio.h>

size_t write_cb(void *ptr, size_t size, size_t nmemb, void *userdata) {
    return fwrite(ptr, size, nmemb, stdout);
}

int main(void) {
    CURL *c = curl_easy_init();
    if (!c) return 1;
    curl_easy_setopt(c, CURLOPT_URL, "http://example.com/");
    curl_easy_setopt(c, CURLOPT_WRITEFUNCTION, write_cb);
    curl_easy_perform(c);
    curl_easy_cleanup(c);
    return 0;
}

If the goal is educational (implement HTTP over sockets), keep a checklist: use getaddrinfo and non-blocking or timed I/O, send a correct HTTP/1.1 request with Host: and CRLF line endings, parse Content-Length and Transfer-Encoding: chunked, follow 3xx redirects, and use OpenSSL/GnuTLS for HTTPS. Common pitfalls include partial reads/writes, failing to handle chunked encoding, and certificate verification errors.

Quick troubleshooting tips: verify the URL with the command-line curl or wget first; enable verbose output for TLS issues (curl -v); for compile/link errors add -lcurl; use tcpdump/wireshark or openssl s_client to inspect traffic and TLS handshakes. For production or complex pages, prefer libcurl or a higher-level parser (libxml2/Gumbo) rather than ad-hoc string parsing.

Recommended Answers

All 6 Replies

Member Avatar for Benkyou

Ok, going to check it out. Thanks.

Off topic: i think its yours, http://eternallyconfuzzled.com/. Used this site to study hash functions and hash tables. Very well written, can't thank you enough.

Member Avatar for sree_ec

Ok, going to check it out. Thanks.

Off topic: i think its yours, http://eternallyconfuzzled.com/. Used this site to study hash functions and hash tables. Very well written, can't thank you enough.

Oh Is it? I use that to study datastructures.. Whoever is the Author, I am greatly thankful to him/her....

Member Avatar for Narue

I use that to study datastructures..

If you'd like to see anything in particular added, just send me a PM or email. I've had trouble updating the site due to time constrains these last couple of years though, so I can't make any guarantees. It takes quite a while to write a proper tutorial.

Whoever is the Author, I am greatly thankful to him/her....

:)

Member Avatar for Benkyou

Finished reading that guide by beej last night, his little jokes along the text made it so easy to read trough the whole thing... altough at some times i did feel like i was a kid being explained that 2+2=4. Still good a guide i was able to get what i needed working.

Member Avatar for sree_ec

If you'd like to see anything in particular added, just send me a PM or email. I've had trouble updating the site due to time constrains these last couple of years though, so I can't make any guarantees. It takes quite a while to write a proper tutorial.


:)

Ofcourse, But I would like to finish reading through whatever you have written untill now before requesting you for more, because whatever I look to study is there in that site... :)
One request is you can put all the tutorials you have written in that site so everything is available under one roof. I have read a small tutorial you have written about Assembly language programming, which is not there. I dont know how many of such tutorials are yet to reach the site..Keep on writing..We appreciate that :)

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.