Member Avatar for shouvik.d

What exactly is this crawler special about, and why didn;t this take off. Recently Wikia acquired this.
http://grub.org/

Sorry if this is a wrong thread to have posted

  • 4 Contributors
  • 3 Replies
  • 103 Views
  • 6 Days Discussion Span
  • Latest Post Latest Post by x3mario
Member Avatar for cooluks

I think it is justs an add-ons... To access wiki...

Member Avatar for JoetjeF

Grub is a distributed web crawler. It maintains an index of url's including status information (changed/not-changed/unavailable). The problem is, it uses your cpu and bandwidth and they sell the information to other search engines.

Member Avatar for x3mario

Nice post JoejeF. That's a great tip for newbies.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.