Web scraping with YQL, jQuery and JSONP

Using screenr, I’ve just recorded a new sub-five minute screencast detailing a web scraping example using YQL, jQuery and JSONP.

In the video, I quickly introduce YQL‘s web scraping capability by returning a the contents of an HTML element on the JustGiving website. The data is obtained using jQuery’s getJSON method via JSONP for a cross-domain request. It’s short, but simple, and aims to give a brief example of what’s possible using YQL and jQuery.

  • Guy

    Thanks! It helped.

  • http://paulgeraghty.posterous.com PaulG

    Excellent! Been following YQL for ages and never really “got it” till your screenr presy.

    Thanks for doing this!

  • http://www.monitojoomla.com/ Jmarreros

    It’s interesting, thanks

  • http://jacobdubail.com Jacob

    Wow. That was the easiest to follow YQL tutorial I’ve seen to date. Thanks for posting the link over at NetTuts!

  • tommy

    Thank you for the video!

    Any idea why:

    select * from html where url=”http://news.ycombinator.com/item?id=1665999″ and xpath=’/html/body/center/table/tbody/tr[3]/td/table/tbody/tr/td[2]/a’

    does return null?

  • http://www.freshclickmedia.com Shane

    Hi tommy,

    best thing is to grab what you need in Firebug and select the XPath (as I did in the XPath). Although you can do it by expanding the html, it’ll be a lot easier with Firebug’s help.

    Hope that helps.

  • http://www.apashley.co.uk Alex

    Saw the link from nettuts comments to this video, have to say YQL is extremely powerful, thanks for taking the time to make this video :)

  • mustafa

    nice short tutorial, thanks! i’m wondering if I could use YQL to query for binary files and return them to a client?

    • admin

      Hi Mustafa – my simple example returned a text element… when you say binary files, do you mean a JPEG for example? If you wanted to grab a JPEG file, you could look for a particular img tag, and grab the src attribute value… is that what you mean?