Search DaniWeb - scrape

Re: How would we poison AI web crawls? 2 Months Ago by rproffitt … style reimaginings of your upstream pages, poisoning any LLMs that scrape your content. @mike 🔗 https://codeberg.org/MikeCoats/poison-the-wellms… Scrape page content? 14 Years Ago by brianjoe … auto updated on my site. How is this possible, to scrape theese few lines from another site with PHP? Thanks in… Re: Scrape page content? 14 Years Ago by brianjoe I know that I can make a page refresh, but that doesnt get me anywhere if I dont have imported the content from the other site to my own. How do I import that? It's not my own site I want the content from, it's from another site (maybe located in US? I dont know).. I heard that I could do a page scrape, but how? Scrape/Pull/Or do "XXX" to extract data 12 Years Ago by stevanity … a code in ruby (my project is in Rails) to scrape the data from the website iterating through each url for… scrape the image 11 Years Ago by DjFumon hello, i need help to scrape the image of the movies from http://www.yaske.to/… cURL won't scrape this page. Why? 10 Years Ago by Ryujin Greetings. Trying to scrape data from search results in [a library catalog](http://pilot.… the issue in this case. The script won't even scrape the URL of the catalog's 'home page,' the first…=23"; echo "The URL we'd like to scrape is " . $url . "<br />"; $results_page = curl… need help with c# screen scrape 9 Years Ago by someoneelse I need help with .net c# screen scrape. I can't seem to find an example in c# …; <html> <body> <h1>Screen Scrape of www.aspmessageboard.com</h1> <p>… Re: how to scrape the data from 5 hours backward? 12 Years Ago by mark103 …; My current time is 4:00pm and I want to scrape the data that is in the same row as 11…. If my current time is 5:00am, I want to scrape the data in the same row as 12:00am. The… Re: how to scrape the data from 5 hours backward? 12 Years Ago by LastMitch … guide website is come from usa. Those are words not scrape. Scrape means unwanted/weird characters. You trying to remove dupilcutle sentences… Re: need help with c# screen scrape 9 Years Ago by tobyITguy all i see in the VB code is that your downloading a webpage, saving it as a string and then displaying it in a paragraph. Is that what you mean by screen scrape? Is that what you wanna do in c#? Re: need help with c# screen scrape 9 Years Ago by someoneelse … object... I just need a working exampke of a screen scrape in c# so i can pull info of one page… screen scrape 19 Years Ago by Techscrip Can any one please help me find an sample of screen scrape. I need a sample where content are extracted from a given start point to end point. Thanks in Advance. Screen Scrape remove spaces/line breaks between specified tags 16 Years Ago by webfort Hi, I'm doing a screen scrape of a web page, which works with out any problems … Re: Screen Scrape remove spaces/line breaks between specified tags 16 Years Ago by selvaganapathy Hi, you specify, what method you are using to Scrape the Page . Have you heard Regex class? how to scrape the data from 5 hours backward? 12 Years Ago by mark103 … 5 hours backward time is 5pm so i want to scrape the data that is on the same row as the… Re: how to scrape the data from 5 hours backward? 12 Years Ago by LastMitch … of the page. This is the first time I heard scrape data from a time frame of 5 hours. I mean… How to scrape a table off a website and put results in datagridview.. 8 Years Ago by TheGuy831 Hi im trying to Scrape a **table** of a website and list it in a **… Re: How to scrape a table off a website and put results in datagridview.. 8 Years Ago by rproffitt Let's change that word scrape to parse. I see that if I do that I can research like this: https://www.google.com/search?q=parsing+html+in+vb.net Parsing is something we get taught almost from the beginning in programming classes. So you already scraped the HTML and now you need to parse it. Re: Scrape page content? 14 Years Ago by yoge911 hi, well, how frequently does the content change? Just a thought ..... may be u can set the approximate time interval for the page refresh using meta tag or an ajax request to refresh the page every X seconds. Re: Scrape page content? 14 Years Ago by mschroeder You have to first get the contents of that page. Some options for this would be cURL, file_get_contents, sockets, fopen/fread/fclose, etc. Once you have the content you have a few options for extracting that particular piece of information. You can try to use a regular expression with the preg functions to match on the tags around it. You can … Re: Scrape page content? 14 Years Ago by brianjoe So it would be something like this? $html = file_get_html('http://www.google.com/'); But how do I use xpath (which I have never heard of) to get the specific data from that site? Re: Scrape page content? 14 Years Ago by mschroeder In this example I load the html from the daniweb homepage and than query on ALL div's with id='stats', which should only be one. See the link below for some good tutorials on crash course xpath. [CODE] <?php $html = file_get_contents('http://www.daniweb.com'); $dom = new DOMDocument('1.0', 'iso-8859-1'); //Suppress any warnings from … Re: Scrape page content? 14 Years Ago by brianjoe Ah okay, I want to read more about it thanks! BUT, I think there is a problem.. The content I want to grab is not inside a div, but a table. This is the lines I want: [CODE]<script type="text/javascript"> var flashvars = { 'feedback_url':'http://site.com/test.php?args=558065,31,1298047277:… Re: Scrape page content? 14 Years Ago by mschroeder You can navigate to any node anywhere in an xml based (html) document with XPath. But, the path to get to that script node depends entirely on the rest of the document it is contained within. For example if it was the first script tag in the head: [ICODE]/html/head/script[1][/ICODE] Once you have your selection narrowed down to that node you can … Re: Scrape page content? 14 Years Ago by brianjoe Thanks for your help, but I dont seem to get it. But I guess I now know how difficult it would be. Fx. this is not working, not displaying anything: [CODE]<?php $html = file_get_contents('http://www.daniweb.com'); $dom = new DOMDocument('1.0', 'iso-8859-1'); //Suppress any warnings from invalid html markup @$dom->loadHTML( $html ); $xpath… Re: Scrape page content? 14 Years Ago by mschroeder You need to read about Xpath if that is the route you want to take, it really isn't difficult but your Xpath expression up there is wrong, and it is not the one I posted previously. Walk through the simple tutorials I posted earlier: [url]http://www.w3schools.com/xpath/default.asp[/url] Re: Scrape page content? 14 Years Ago by brianjoe Thanks man, really appreciate it ! Re: Scrape page content? 14 Years Ago by smantscheff You could also use a regular expression to extract the desired content: [CODE]if (preg_match( '~feedback_url\':\s*\'([^\']+)\',\s\'file\':\s*([^\']+)'~, $sourcecode, $match )) { $feedback_url = $match[1]; $file = $match[2]; } [/CODE] Re: Scrape/Pull/Or do "XXX" to extract data 12 Years Ago by pritaeas Wouldn't it be easier to request some other method of accessing this data? Re: Scrape/Pull/Or do "XXX" to extract data 12 Years Ago by stevanity > Wouldn't it be easier to request some other method of accessing this data? it would be rather expensive... :) cany u give any idea :)