Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

As someone who's done a lot of data mining, I'm always surprised to find that sites with lots of data don't do this. Your output generator just needs to randomize some class names and throw a couple of pointless elements around the place to make it MUCH more difficult.

There's one site I've scraped that has code like this: <div class="name"><span class="first_name">...</span> <span class="last_name">...</span></div><div class="street_address">...</div>



How is this marginally difficult to parse?

A CSS3 selector could easily just get this - in jQuery:

$('div span').attr('class')

will easily give you the expression that returns all the classnames - easily mapped to whatever output you want.


He was actually giving example of page that is ridiculous easy to mine.




Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: