here it goes. I have decided no more f*&&$*in around. I am finishing this thing today !
The crawler has been proving itself better as a chron job then a manual file. Please don’t ask me why – i am not...
the crawler worked up 3800 listings before showing signs of hatred towards man kind. It began to error out when it was flipping categorys, but as far as i could tell, it did it fine before on all the sites, so i was tota...
So a little about the current project, which is on its last bend to be completed; the exxxcavate crawler. The crawler is intended to crawl data from sites ( inputed into the database ) and get listings from sites like re...
yet again, another freaking blog. The last davidkilmer.com had a auto-blogged plugin in it, and i had it set to scrape a million things. Before i knew it, i had over three thousand blog posts and 98% of them were not min...