Member Avatar for yazooney

hey,

I want to read internet data through a C++ program. I am told that the way to do this is by using a proxy server and spitting out the data through the proxy to a file which then can be read.

How exactly do you setup a proxy server which preforms this task (i.e. all the internet data going through it is sent to a file). Can someone help me out or show me where to look to get the information.

Thanks

  • 5 Contributors
  • 5 Replies
  • 932 Views
  • 8 Months Discussion Span
  • Latest Post Latest Post by jen140

Dani AI

Generated

As explained, the real task is not “how to use curl” but identifying what protocol and endpoint the Java program actually talks to, then choosing the simplest interception method. ’s suggestion (direct HTTP pulls) is the easiest when it applies; ’s sniffing tip is the right first diagnostic; and is correct that a proxy/firewall is the usual interception point.

A practical short workflow:

  1. Discover the endpoint and protocol. Use a packet capture (Wireshark/tcpdump) or OS tools (netstat/ss/TCPView) to see remote host, port and whether the traffic is HTTP, HTTPS, WebSocket, raw TCP, etc.
  2. If it is plain HTTP, configure the Java app to use a local proxy and log the requests. Many Java programs respect system proxy properties; for example:
java -Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=8888 \
     -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=8888 \
     -jar app.jar

You can also use -DsocksProxyHost/-DsocksProxyPort for SOCKS proxies.

If traffic is HTTPS you must either MITM TLS (install the proxy CA into the JVM/OS trust store) or capture decrypted data inside the process. If the protocol is non‑HTTP or binary, capture + reverse‑engineer or run a small TCP proxy that logs bytes and forwards them.

For a C++ solution there are three common options: write a minimal logging proxy (HTTP/SOCKS/TCP), use libpcap/WinPcap to capture and reassemble streams, or—if you control the Java process—attach a Java agent to log data before it hits the network. Start by sniffing (as recommended); if you find a clean feed URL, pull it directly (as hinted). Note: intercepting traffic or installing MITM certificates can violate terms or law—only do this on systems you control or with permission, and test on an isolated machine.

Recommended Answers

All 5 Replies

Member Avatar for Agni

what do you mean by 'read internet data' ? If it means making an HTTP 'get' request to some URL and getting the data that i think you can try using the 'curl' library http://curl.haxx.se/

Member Avatar for yazooney

Hmmm i dont think thats what i mean. I downloaded java program which has live stock prices updated second by second. The prices are updated via the program conncecting to the internet.

What I want to do is capture these prices from the program and store them within my own program. I spoke to someone who said one way of doing this is by routing the internet feed the program uses (not sure if this is the right terminology) through a proxy server and saving the data files sent to the program. If the program receivces the internet data via xml files then it can be possible to capture the prices from the data sent via the internet to the program which displays the data.

What I want to know is how do I set up and use a proxy to intercept the data files sent and then transfer the data to a file so it can be read?

Hope that clears it up

Member Avatar for Bladtman242

Agni: the lib you are refering to, does it only work in linux or am i mistaken?

Member Avatar for tux4life

What I want to know is how do I set up and use a proxy to intercept the data files sent and then transfer the data to a file so it can be read?

Intercepting data, that's actually the base of a firewall :) ...

Edit:: The following post is maybe what you're looking for (if you're using Windows): http://www.daniweb.com/forums/post784786-3.html
Edit:: This thread might be also useful for you :) ...

Member Avatar for jen140

You could use a network sniffer , to see what packets and to where are being sent/recieved . And after ,try recreating the same in c++ using sockets .

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.