So, can you explain to me, why your website/host is running a website scraper abusing the server and resources of another online community that is (unlike your own) providing a free AND ad-free service to its users?
Every hour at x:25 your server is requesting a page hosted on our server.
Is that the kind of behaviour you want to "teach" the people here? To use online scrapers that ignore the robots.txt?
Maybe I should contact your hoster and file an abuse claim.
afsdfasdf654654
dtpp commented: pics/logs or go away..... +0
Recommended Answers
Jump to PostHi,
Firstly, you can choose to disable ads from within your member settings.
Secondly, we don't have any cron jobs running 25 after the hour that attempt to request any external pages.
We do have a cron job that attempts to fetch external pages that people link to within posts …
Jump to PostWe've been going back and forth. Seems to be a bug on my side but I can't get to the bottom of it.
Jump to PostIf you cant find the cause, can you maybe just remove the url from that post? That would at least solve the issue for our site.
From my perspective, the issue related to your site is already [temporarily] resolved: you added our IP address to your firewall, and …
All 11 Replies
afsdfasdf654654
169.55.25.105 - - [03/Oct/2015:07:25:09 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 12677
169.55.25.105 - - [03/Oct/2015:08:25:04 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 43943
169.55.25.105 - - [03/Oct/2015:09:25:04 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 12910
169.55.25.105 - - [03/Oct/2015:10:25:14 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 35234
169.55.25.105 - - [03/Oct/2015:12:25:08 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 34194
169.55.25.105 - - [03/Oct/2015:13:25:04 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 19884
169.55.25.105 - - [03/Oct/2015:14:25:09 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 38838
169.55.25.105 - - [03/Oct/2015:15:25:04 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 14583
169.55.25.105 - - [03/Oct/2015:16:25:04 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 45777
169.55.25.105 - - [03/Oct/2015:17:25:09 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 11875
169.55.25.105 - - [03/Oct/2015:18:25:04 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 35265
169.55.25.105 - - [03/Oct/2015:19:25:04 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 19448
169.55.25.105 - - [03/Oct/2015:20:25:09 +0000] "HEAD <path_removed>?show=calendar HTTP/1.1" 200 20 "-" "-" "-" "-" - - 62902
Dani 4,675 The Queen of DaniWeb Administrator Featured Poster Premium Member
Hi,
Firstly, you can choose to disable ads from within your member settings.
Secondly, we don't have any cron jobs running 25 after the hour that attempt to request any external pages.
We do have a cron job that attempts to fetch external pages that people link to within posts to ensure that the link is valid (so that we aren't linking to broken pages). However, it certainly doesn't re-check the same link every hour, and like I said, none of the cron jobs we have related to links run 25 after the hour.
Can you please send me a private message with the full URL so that I can investigate this further please?
afsdfasdf654654
No, I cant send you a pm. Because my account is sandboxed or whatever the reason is.
And 169.55.25.105 is clearly your www4 host. Which requested the page in question again at 21:25 and 22:25 UTC.
I also searched for our domain on your site and only found 1 match from 2 years ago. Which wasn't a link anway.
dtpp commented: www4 means its virtual host +0
Dani 4,675 The Queen of DaniWeb Administrator Featured Poster Premium Member
I've sent you a private message. You should be able to reply to it.
diafol
@your-site-sucks - this really is a PM issue, not general feedback. Please conduct this off-forum with Dani as requested. Thanks.
Dani 4,675 The Queen of DaniWeb Administrator Featured Poster Premium Member
We've been going back and forth. Seems to be a bug on my side but I can't get to the bottom of it.
afsdfasdf654654
Still havent found anything?
Dani 4,675 The Queen of DaniWeb Administrator Featured Poster Premium Member
I still haven't, I'm really sorry. I can't find anything in our logs that is showing that we are attempting to retrieve your site, and the databse query is not picking up on your URL either.
afsdfasdf654654
If you cant find the cause, can you maybe just remove the url from that post? That would at least solve the issue for our site.
Reverend Jim 5,225 Hi, I'm Jim, one of DaniWeb's moderators. Moderator Featured Poster
Perhaps one or both of you could add a blocking rule to your respective firewalls or a HOSTS entry to redirect to 127.0.0.1
Dani 4,675 The Queen of DaniWeb Administrator Featured Poster Premium Member
If you cant find the cause, can you maybe just remove the url from that post? That would at least solve the issue for our site.
From my perspective, the issue related to your site is already [temporarily] resolved: you added our IP address to your firewall, and the problem is mitigated for the time being. I am much more concerned with why it is happening, how widespread the issue is, and potentially how many other websites are being affected by this bug.
Be a part of the DaniWeb community
We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.