Using Yahoo! Pipes to filter an RSS feed

Pipes

I’m currently working on a redesign of the University’s research homepage which involves a requirement to pull in three RSS feeds:

  1. Recent research outputs (from the PURE research portal, which hasn’t gone public yet)
  2. Recent research activities (from the PURE research portal)
  3. Research news (from the main University news page. But this third feed needs to pull in only those stories that are related to research.)

Using PHP SimpleXML

I’m using the PHP SimpleXML extension to pull in the news feeds, extract the data that I want and then output it onto the Web page.  For the first two feeds that was simple, I created a PHP function to which I would pass two parameters, the URL and the number of posts I wanted to display on the page:

getNews($url, $postsRequired)

As the name suggests SimpleXML was very easy to use. SimpleXML converts the XML document into an object which you can then iterate through like a collection of arrays and objects.  It’s great if you already know the structure of the XML document (which we do because a) it follows the RSS 2.0 standard and b) we generated it).

So to load  a new XML document you use the simplexml_load_file() function:

$feed = simplexml_load_file('http://www.st-andrews.ac.uk/rss/news/index.xml');

And you then gain access to the different elements as you would using ordinary object properties. So for an RSS 2.0 file which contains this data:

<rss version="2.0">
  <channel>
      <title>News feed from The University of St Andrews</title>
      <link>http://www.st-andrews.ac.uk/rss/news/index.xml</link>
      ...

I could use this code to grab the feed title and URL:

$feedTitle = $feed->channel->title; // get name of RSS feed
$feedURL = $feed->channel->link;    // get URL of RSS feed

In this case $feedTitle would equal “News feed from The University of St Andrews” and $feedURL would equal “http://www.st-andrews.ac.uk/rss/news/index.xml&#8221;.

But I only want Research news

The third feed required a bit of thinking because I don’t want to simply pull in all the latest news, I only want the latest news articles that relate to research.

To help us the RSS specification allows content creators to add any number (zero or more) of <category> tags to each news item, e.g.

<category>News</category>
<category>Research</category>

When a content creator adds a new news item to our content management system they have an option of selecting a category from a drop-down list; this populates a category tag in the RSS feed.

I realised that I then had two options:

  1. I could write into my PHP code a script that would filter out any posts that were not tagged with the Research category.
  2. Use an external, third-party application to do this for me.

Yahoo! Pipes

In the end I opted for Yahoo! Pipes simply because it was much quicker to have Pipes filter the news RSS feed for me than it was for me to write a PHP parser to do the same.  (“I’m a PHP coder of very little brain”, as Winnie the Pooh might say!)

But I then realised that an attractive side-effect of this was that I then had a new RSS feed that not only I could use but that other users could subscribe to it as well.

Screenshot of Yahoo! Pipes fetching an XML feed, filtering it and then outputing the results

Screenshot of Yahoo! Pipes fetching an XML feed, filtering it and then outputing the results

Yahoo! Pipes was really simple to use and the feed filter took less than a minute to set up.  It just involved three stages:

  1. Fetch the feed.
  2. Filter the feed, permitting only items that matched the rules I set.
  3. Output the results.
Advertisements

6 thoughts on “Using Yahoo! Pipes to filter an RSS feed

  1. Once you’ve created your pipe (as far as I can see, having just logged into our Yahoo! Pipes account, you don’t even need to publish it) you have various options on how to use it including: Get as RSS, Get as JSON, Results by Email or Phone, Get as PHP.

    Obviously, here I wanted RSS so I selected that and grabbed the URL which I could then use in my PHP script.

    Does that help? You can very easily sign up for a Yahoo! Pipes account and start playing about with it. I’ve barely scratched the surface of its functionality so far, but even in doing this it’s pretty useful.

  2. Hello,

    I took the article from the feed, but the content contains links encoded (base64), how do I do to decode this links in yahoo pipes? Thanks

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s