• Shortcuts : 'n' next unread feed - 'p' previous unread feed • Styles : 1 2

» Publishers, Monetize your RSS feeds with FeedShow:  More infos  (Show/Hide Ads)


Date: Monday, 10 Dec 2012 23:59

Earlier this year http://babelfish.yahoo.com was shut down. The Pipes translate module relied on this service and has also been deprecated.

Pipe developers have been looking for a solution to this gap. One of the available free solutions we’ve been using is the Microsoft Translate API with Pipes.

First you need to sign up with Microsoft to get your translate key and username.

When filling out their form you can enter your domain or pipes.yahoo.com in the “Redirect URI” field. (E.g. http://pipes.yahoo.com)

The only 2 things you need to remember is the Client ID (which you assign) and the Client secret (which they generate).

You can do that here: https://datamarket.azure.com/developer/applications/

After you set your clientid and get your client secret, clone this Pipe: http://pipes.yahoo.com/pipes/pipe.info?_id=c33dfcb6228fcf820afd1bac96833d4c and add it to your favorites by “staring” it.

When building your Pipe you can access it via the Favorites folder in the Pipes Editor.

You can see from this Pipes example how to use the Translate Pipe to fine tune what you want your Pipe to translate: http://pipes.yahoo.com/pipes/pipe.edit?_id=ac325bbbc6c5603c8c8e33dd26fe3fc3


Here i’m putting the Bing Translate Pipe into a Loop and translating the RSS feed titles from Spanish to English.

You can clone and then modify the Pipe to your liking.

Remember the free version of Microsoft Translator will give you 2 million characters of translation per month.

For developers familiar with YQL there is also a YQL Microsoft Translate table available here. As with Pipes, you’ll need to sign up for the client ID and client secret from Microsoft.

Author: "Administrator" Tags: "news"
Comments Send by mail Print  Save  Delicious 
Date: Tuesday, 24 Apr 2012 20:58

We are introducing a new module, the XPATH Fetch Page.

We are also going to deprecate the Fetch Page module at the end of June. So please convert your existing Pipes that use the Fetch Page module to the XPATH Fetch Page module.

To use the XPATH Fetch Page module, first enter the URL of the site you want. By default, the module will output the DOM elements as items in the preview pane. You can optionally use the “Emit items as string” checkbox if you need the html as a string.

You can use the “Extract using XPATH” field to fine tune what you need from the HTML Page. For example, if I want all the links in the page I can simply use “//a” to grab all links. If I want all the images in the html I can do “//img”. Read more on XPATH. You can also find XPATH statements using firebug/developer tools to target data that you want in a HTML page.

Previously with the older Fetch Page module you would have to wrangle with regex, splits and other complicated methods to get the data you wanted. The new XPATH Fetch Page module is more powerful, easier and more inline with todays standards.

Currently this module will extract the page and fix malformed tags using Tidy. You have the option to run the parser using support for HTML4 (by default) or checking the “Use HTML5 parser” checkbox to use the HTML5 parser. We recommend using the HTML5 parser when using this module for most cases.

Click here for an example Pipe using the new XPATH Fetch Page module.

You can also use Pipes special variable substitution method (e.g ${<dom node path here>}) to construct new content from the dom nodes. For example:

${td.0.span.0.a.content} will pull content from that dom path, viewed in the preview pane.

and a longer example:

Company name / Ticker: ${td.0.a.content} ${td.0.p}<br>Underwriter: ${td.1.p.content.0} ${td.1.p.content.1}<br>Price Range: ${td.2.p.content}<br>Shares: ${td.3.p}<br>Pricing Date: ${td.4.p}

You can use the Regex module to help you build this new content. View an example that uses this method here.

Note on usage: The module will only fetch HTML pages under 1.5MB and the page must also be indexable (e.g. allowed by the site’s robots.txt file.) If you do not want your page made available to this module, please add it to your robots.txt file, or add the following tag into the page’s element:

<meta name=”robots” content=”noindex”>

Author: "Administrator" Tags: "news"
Comments Send by mail Print  Save  Delicious 
Date: Thursday, 15 Mar 2012 21:16

In this talk from YUIConf 2011, YQL engineer Paul Donnelly demonstrates the features of the Yahoo! Pipes editor and explains how you can use Pipes and YQL to power your web apps, create mashups, and more.

(original posting here: http://www.yuiblog.com/blog/2012/01/11/video-yuiconf2011-donnelly/)

Author: "Administrator" Tags: "news"
Comments Send by mail Print  Save  Delicious 
Date: Friday, 10 Feb 2012 18:55
  • Frontend Changes
    • String builder, Item builder and URL builder modules now have a max of 40 item per module, increased from 10.
    • In the Pipes Editor, in folders “My Pipes” and “Favorites”, the sort order is now based on last modified Pipe.
  • Engine Changes
    • Date Formatter Module can use Java SimpleDateFormat Patterns (http://y.ahoo.it/qSAA+)
    • SubPipe using default text bug fixed. (http://y.ahoo.it/8O7kH)
    • Some atom feeds not getting y:published emitted – fixed (note: if building own data (item builder, fetch data) and you want y:published emitted – run data through the “Create RSS” Module or “RSS Item Builder” module. y:published is needed to emit a valid RSS pubDate element in final Pipes RSS output)
Author: "Administrator" Tags: "changelog"
Comments Send by mail Print  Save  Delicious 
Date: Tuesday, 20 Dec 2011 23:50
  • Renamed Module “Fetch Site Feed” to “Find First Site Feed“. Changed due to confusing name, which resulted in users using “Fetch Site Feed” instead of “Fetch Feed” – http://y.ahoo.it/uaPda
  • Removed Alert icon in all modules that access a url. Confused users thinking the feed was invalid or unreachable. – http://y.ahoo.it/IoPXn
  • Fixed Namespaced elements issues – http://y.ahoo.it/7x6PZ
  • Fixed Subsequent Split module increases Truncate value – http://y.ahoo.it/pDyu0
  • Fixed Pipe name alias redirect issue: http://y.ahoo.it/yoEsT
Author: "Administrator" Tags: "changelog"
Comments Send by mail Print  Save  Delicious 
Date: Tuesday, 13 Dec 2011 21:56

Currently our cloud DB service has an outage that effects writes to the service. Reads are operational and working.

This means creating new pipes or editing existing Pipes will not work. But existing Pipes will still run and produce output as normal.

You can access you Pipe output by entering:

http://pipes.yahoo.com/pipes/pipe.run?_id=<yourpipeid>& _render=rss/json/kml…etc.

e.g. http://pipes.yahoo.com/pipes/pipe.run?_id=ZKJobpaj3BGZOew9G8evXg&_render=json

Author: "Administrator" Tags: "news"
Comments Send by mail Print  Save  Delicious 
Date: Monday, 24 Oct 2011 21:29

Last week, we released upgraded Pipes Badges to be compatible with YUI3. However, there was a slight hiccup and the versions of the List Badge, Map Badge, and Image Badge script files that were produced are problematic. If you recently updated or added badges to your personal site and are experiencing issues, please look at your source code and make sure you are not using the following versions of these files:

  • http://l.yimg.com/a/i/us/pps/imagebadge_1.4.js
  • http://l.yimg.com/a/i/us/pps/listbadge_1.5.js
  • http://l.yimg.com/a/i/us/pps/mapbadge_1.4.js

If you are using any of those versions, please update them to be one of these instead:

  • http://l.yimg.com/a/i/us/pps/imagebadge_1.5.js
  • http://l.yimg.com/a/i/us/pps/listbadge_1.6.js
  • http://l.yimg.com/a/i/us/pps/mapbadge_1.5.js

We fixed the pipe.info page, so you should be getting proper badge code snippets moving forward.

Author: "Administrator" Tags: "changelog"
Comments Send by mail Print  Save  Delicious 
Date: Wednesday, 19 Oct 2011 21:13


Author: "Administrator" Tags: "changelog"
Comments Send by mail Print  Save  Delicious 
Date: Monday, 12 Sep 2011 21:17

With the next release(TBD) we will require any Pipe that uses the Flickr Module to use an API key. You can obtain the Flickr API key here: http://www.flickr.com/services/apps/create/apply/. It’s easy to get a key and please get it ready for when this change happens. You can also use the Private String Module to make sure your API key is not viewable to the public as described in the screenshot below.

Flickr Module

Author: "Administrator"
Comments Send by mail Print  Save  Delicious 
Date: Wednesday, 31 Aug 2011 17:43

We recently integrated webplayer.yahoo.com into the pipe.info pages when viewing the “List” view.

For more info on webplayer and Pipes please read: http://developer.yahoo.com/blogs/ydn/posts/2011/08/yahoo-pipes-webplayer-awesome/

Author: "Administrator"
Comments Send by mail Print  Save  Delicious 
Date: Friday, 26 Aug 2011 19:22

A few weeks ago we rolled back the V2 release. Today all Pipes are now running on the V2 engine!

Author: "Administrator"
Comments Send by mail Print  Save  Delicious 
Date: Friday, 05 Aug 2011 08:36

Yesterday we migrated all Pipes to the V2 engine. We experienced some minor delays in the completion of the full migration.

We expect to migrate all Pipes to the V2 engine sometime over the next week or two. V1 Pipes will still run as V1 for the time being. Please take this opportunity to further test your V1 Pipes as V2 by appending &_v2=true to any V1 Pipe URL.

Thanks! -Pipes Team

Author: "Administrator" Tags: "Uncategorized"
Comments Send by mail Print  Save  Delicious 
Date: Thursday, 04 Aug 2011 23:27

It’s been a little over a year since we first announced the V2 engine. After many months of work and help from the Pipes developer community, all Pipes now are running off our new V2 engine! Special thanks to hapdaniel and libra_sun for reporting issues and making V2 a better engine.

We also improved several things with our Date Modules.

In the Date Formatter:

- Added new format token: “%K”, produces RFC 822 compliant date (shortcut for “%a’,’ %d %b %G %H’:'%M’:'%S ‘GMT’”)

- Added new Timezone input field – which allows you to specify a target time zone. This must be one of the time zone IDs supported by the Java platform, or a custom time zone ID using the format defined in http://download.oracle.com/javase/6/docs/api/java/util/TimeZone.html

In the Date Builder:

- Added new output fields: day_name, day_ordinal_suffix, and month_name

We are continually updating the V2 engine and will be adding new features. If your (previous) V1 Pipes still do not work, please report the issue and include your Pipe ID in your post. Community members and staff will look at the issue. Please also read about the new V2 limits.

(if you get a blank page on the Pipe Editor please clear your browsers cache)

Author: "Administrator"
Comments Send by mail Print  Save  Delicious 
Date: Thursday, 21 Jul 2011 08:11

Here at Pipes, we’ve been working hard to prepare the V2 Engine for full deployment in August 2011. We realize that this is a huge change for most of you and your Pipes, so we are trying to make the transition as seamless as possible by giving you incremental updates and opportunities to test your Pipes with V2 functionality. Throughout this process, you’ve been giving us your feedback, and we’ve been listening. So, we’d like to take a moment to both reassure you about the V2 Pipes Engine and provide the previously-lacking information regarding limitations and how to best mitigate pain points in the migration process.

Many of you have been asking why V2? Why can’t we just keep the Pipes V1 Engine? Well, simply put, change is intrinsic to software. Products need to be maintained and updated on a regular basis to fix bugs, make use of improved technologies, and provide cool, new features to you — our awesome users. The Pipes V1 engine is nearly 4 years old and was originally built quickly to provide a proof-of-concept. No one could have predicted how hugely successful Pipes would become! And we are grateful to all our devoted users for years of commitment to the product. But now to prove our commitment to you, we are moving to the V2 Engine to provide an even greater Pipes product.

Now on the nitty gritty…

The V2 engine isn’t vastly “buggy” or “broken”. It’s just different. It uses the same engine as the YQL product. We wouldn’t be releasing V2 if we didn’t have high confidence in the engine. After comparing thousands of Pipes running in V1 versus V2, we’ve found only minor incompatibilities and have been working diligently to update V2 where it makes sense. But no process is perfect, and so we’ve been relying on you to report issues and findings.

There are however some explicit differences with the V2 Engine. As many of you have probably already noticed, we’ve imposed a few limitations on Pipes that previously did not exist in V1. Why? Primarily to reduce abuse, improve performance, and to scale. These limitations are in no way intended to prevent you from building feature-full Pipes. Rather, we just ask you to think differently in certain cases about how you create your Pipe. Anything that you were able to do with V1 you should be able to do in a slightly different way with V2.

Here’s an outline of the new V2 limitations in detail:

* 3 2 Subpipes (nesting level of 3): Pipe nesting using subpipes has been limited to a nesting depth of 3. For a single pipe, you can drag as many subpipe modules as you like on the editor. However, opening a subpipe and including another subpipe module can only be done up to a depth of 3. That is, we only allow pipe -> subpipe1 -> subpipe2. Subpipes are meant to be used just like functions in programming. If you keep nesting functions, the trace becomes so long and complex that it essentially defeats the purpose of modularizing functionality. With subpipes, nesting too deep actually negatively impacts the performance of your Pipe. This limit will affect both existing and new Pipes. If you get this error when running your pipe, you’ll need to edit it and remove any/all subpipe modules at level 3 (the third tab in the editor).

* 10 Module Inputs: Many modules have dynamic inputs. Previously you could add as many inputs as you would like for feeds, parameters, etc. Now, we limit it to 10 fields per input parameter, with the exception of the URL Builder Module. This change is independent of the engine and was a feature we wanted to include with V1 but never got around to. Simply put, having too many input fields makes a module appear unruly. If you want to fetch many feeds for example, you can use multiple Fetch Feed modules and Union them together. Existing Pipes if left untouched will not be affected by this limit. However, if you begin editing a module with more than 10 inputs, the restriction will be enforced.

* 30sec Execution Timeout: All Pipes must run within a 30 second execution timeout. This is a restriction put in place by the underlying YQL engine. In the current implementation, we allot 27 seconds to fetching your external feeds and sources, and 3 seconds for the V2 engine to perform the Pipe processing on top of those feeds. Any feeds that take longer than 27 seconds will be cut off. Because the V2 engine is so powerful, 3 seconds is plenty of processing time. So if your V2 Pipes are hitting the timeout limit, chances are there are one ore more sources that are slow to fetch. In situations like those, you may want to take into consideration the quality of the source itself. For example, if a feed takes longer than 27 seconds to fetch, it’s likely a poor feed and should perhaps be replaced with a different source if possible.

* 1.5MB Feed Size: YQL enforces a maximum response size of 1.5MB, so Pipes is subject to this as well. That is, we can currently only fetch 1.5MB of data per feed. We will continue to investigate configuring this value and potentially adjusting it for Pipes.

So there you have it. We know it’s late in the game, but we hope we have helped clarify some core aspects of the Pipes V2 Engine. After the full migration to V2 in August, we will continue with regular, incremental updates to provide further improvements and features. If you have any more questions, concerns, encounter problems as you are upgrading your Pipes, or have feedback on the limitations, please let us know. All we ask is that you help maintain the integrity and relevance of this forum by only posting specific Pipes V2 related questions and issues. And as always, please include your pipe id so we can help you more quickly!

- The Pipes Team

Author: "Administrator" Tags: "Uncategorized"
Comments Send by mail Print  Save  Delicious 
Date: Monday, 27 Jun 2011 19:03

We are starting phase 1 for the Pipes V2 engine migration this Wednesday June 29th.

Phase 1 – All newly created Pipes and clones will use the V2 engine and you will no longer be able to revert to V1.

Existing V1 Pipes will still continue to run as V1 until August 1st. On August 1st all V1 Pipes will be auto upgraded to V2.

Please post any issues to the V2 message board here: http://tech.groups.yahoo.com/group/pipes-engine2users/

Thanks – Pipes Team

Author: "Administrator"
Comments Send by mail Print  Save  Delicious 
Date: Friday, 10 Jun 2011 18:55

The V2 engine will be phased in 2 steps.

June 28, 2011 – All new Pipes will use the V2 engine and you will no longer be able to revert to V1. Existing V1 Pipes will still continue to run as V1.

August 1, 2011 – *All* Pipes will be upgraded to the new V2 engine.

Please post any V2 issues here: http://tech.groups.yahoo.com/group/pipes-engine2users/post

And please read this previous post about the V2 engine: http://blog.pipes.yahoo.net/2010/06/09/yahoo-pipes-v2-engine/

Thanks! – The Pipes Team

Author: "Administrator"
Comments Send by mail Print  Save  Delicious 
Date: Tuesday, 25 Jan 2011 21:16

If you use a modern browser (FF3.5+,Chrome 3+,Safari 4+,IE8+) you can now access Pipes data using AJAX from a different domain.

You can use YUI3 IO (use: ‘native’) to make the request for you or you can hand craft your own.

Some sample code:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
        <title>Pipes CORS test</title>
        <script src="https://github.com/douglascrockford/JSON-js/raw/master/json2.js">
        </script>
    </head>
    <body>
        <div id="output">
        </div>
        <script>
            function createCORSRequest(method, url){
                var xhr = new XMLHttpRequest();
                if ("withCredentials" in xhr) {
                    xhr.open(method, url, true);
                } else
                    if (typeof XDomainRequest != "undefined") {
                        xhr = new XDomainRequest();
                        xhr.open(method, url);
                    } else {
                        xhr = null;
                    }
                return xhr;
            }

            var request = createCORSRequest("get", "http://pipes.yahoo.com/pipes/pipe.run?_id=ZKJobpaj3BGZOew9G8evXg&_render=json&ticker=YHOO,GOOG");

            if (request) {
                request.onload = function(){
                    var r = JSON.parse(request.responseText);
                    var holder = "";
                    for (var x = 0, rl = r.value.items.length; x < rl; x++) {
                        holder += r.value.items[x].title + "<br>";
                        holder += r.value.items[x].description;
                        holder += "<hr>";
                    }
                    document.getElementById("output").innerHTML = holder;
                };
                request.send();
            }
        </script>
    </body>
</html>
Author: "Administrator"
Comments Send by mail Print  Save  Delicious 
Date: Wednesday, 09 Jun 2010 13:46

Some background: The current Pipes engine is nearly 3 years old and was created quite quickly to see if the Pipes idea would be successful. It was. Wildly so. Its been a good stretch for that software and the system design, but the time has come to put something else in its place. During the last year the Pipes team has been developing a second generation engine, which is used in the YQL product (developer.yahoo.com/yql). Its much more stable, performs far better that the current engine, and we’ve been working on migrating the old Pipes-specific functionality out of the v1 engine to this new engine. We’re at a “beta” stage now, where most of the big bugs are out, and we’re shaking down the system to see what we’ve missed. Some things to note:

  • strict equality between the output of Pipes 1 and Pipes 2 is not a requirement. However, the Pipe should run and give output that makes sense.
  • there will be bugs, this isn’t ready for primetime.
  • the mechanism to run Pipes on the new engine is given to you in confidence.
  • the Pipes 2 engine is a new “baseline” for new functionality, not the end goal. It gives us a scalable platform to build features and capabilities which we’ve been unable to add to the old one, including more sophisticated quota management, Pipe stat tracking, caching and enhanced modules (like scripting).

What we’d like to get from you:

  • general feedback. It works the same, it doesn’t, etc
  • specific bugs. What doesn’t work. Please give us the Pipe ID and if you can pinpoint where things are going awry the better. Once you find a bug (for example with a specific module) it will probably be common to many Pipes so please try and avoid duplication.

Please post your feedback here.

So how do you get going?

  • Its easy – First login to Pipes, then use the upgrade links on the pipe.info pages or in the Pipes editor.
  • You can only upgrade your own Pipes.
  • If you see a Pipe you want running in V2 but do not own it, you can clone it and then upgrade it.

What’s next:

  • Auto-upgrade all Pipes that work the same in both engines.
  • After a number of months we’ll retire the old engine

Thanks for trying out the V2 engine!

The Pipes Team

Author: "Administrator"
Comments Send by mail Print  Save  Delicious 
Date: Tuesday, 13 Apr 2010 23:40

If anyone has been using YQL execute to power the Pipes webservice module, in your execute statement JSON.stringify is no longer needed.

Our previous post about using YQL to power the Pipes webservice module has been updated: http://blog.pipes.yahoo.net/2009/06/01/using-yql-execute-to-power-the-pipes-webservice-module/

Author: "donnelly" Tags: "Announcements, Bug Fixes, News"
Comments Send by mail Print  Save  Delicious 
Date: Thursday, 17 Dec 2009 20:16

We've been working with Craigslist over the last week to see how both products can better work with each other. After providing more information about how to track and ban Pipes that violate their TOS (or are considered generally abusive), they have opened Craigslist RSS feeds back to us.

Author: "donnelly" Tags: "Uncategorized"
Comments Send by mail Print  Save  Delicious 
Next page
» You can also retrieve older items : Read
» © All content and copyrights belong to their respective authors.«
» © FeedShow - Online RSS Feeds Reader