Updating OpenService for Microformats

This post details the ongoing work that Gustavo Garcia and I have been doing at http://microformats.org/wiki/OpenService_Extensions. Our stated goal is to figure out the best way to extend Microsoft's OpenService specification to allow for microformat support (and possibly other functionality). We definitely want to have more discussion on this, so if you are interested, please participate. Note that Microsoft has suggested that any changes we make be in a new name space to allow for good coexistence with IE. We're investigating that.

This post assumes you have a basic understanding of what OpenService looks like, so you might want to read up on that first. So without further ado, here are our suggested changes to OpenService.

Support for more contexts

Currently IE supports three contexts: selection, link and document. A given action is applied based on the context where the user right clicks. For microformats, we would add support for microformat names as a possible context. We would also allow those microformat names to be followed by an item in the microformat so that actions could be shown based on the availability of data in the microformat. For example,

would indicate that the action would be available for any adr.

would indicate that the action would be available for any adr that also had a postal-code.

Domain scoping of actions

Currently a given action is available on every website. There are some actions that should only be displayed for a given site. For instance, cork'd.com uses tags to reference their wines, but they really only make sense in the context of their site. To solve this, we would introduce a new attribute to activityAction called domain that would specify which website should have the given action available.

More substitution variables

The substitution variables for the actions would have to be enhanced to allow for the substitution of values in the microformat. Here's an example:



  

This action queries the country-name property directly from the adr microformat and substitutes it. If the referenced property is a plural property, all the items are put in one string separated by a space and then substituted.

Scripting Support

Unfortunately, for complex web services, just substituting variables in URLs and parameters is not enough. So to further enhance the functionality of the OpenServices XML files, we propose that scripting be allowed. These scripts would be evaluated in a sandbox with the contents of the microformat available as local variables (possibly prepended with the microformat name) with some additional helper functions added (XMLHTTPRequest, writing to a temporary directory). A simple example of where this is needed is Google Calendar. Google Calendar requires that if there is a dtstart, there must be a dtend. There's really no way to convey this with simple string substitution. For some web services, some pretty complex JavaScript is needed. Here's an example of how JavaScript might work.



 
  
   
  
 

Here we were able to use JavaScript to create some basic logic around the date and then the correct value is returned.

This idea is probably the most controversial, but I don't see a better way to get the richness that is needed to interact with certain web services. If anyone has better ideas, I'd love to hear them.

Action Discovery on Web Pages

We'd also like to see the ability for a web page to advertise that they have a service available, similar to how OpenSearch works. For OpenService, it might look something like this:



We're still working out how this would be discoverable to the user, but we think it should be in the specification.


On a completely unrelated note, if you are the praying kind, please keep the Chapman family in your prayers. They lost their 5 year old daughter yesterday in a tragic accident. My cousin Melissa works for them as their nanny, so she needs your prayers as well. She's taking this pretty hard. Thanks.

Microsoft Screwed Up Activities API Docs - It's addService NOT AddService

UPDATE: jst ROCKS! - I was able to do exactly what he said and I now have both cases of APIs working in my extension.

OK, this is really starting to bother me because Activities are starting to show up and they don't work on Firefox because Microsoft screwed up the documentation.

The correct syntax for adding an Activity (per my conversation with Microsoft) is:

window.external.addService('FindInWikipedia.xml');
as documented here
NOT

window.external.AddService('FindInWikipedia.xml');
as documented here. (I have no idea what the whitepaper says - I can't agree to the license agreement.

Note the lowercase a at the beginning. Even though both work on IE, USE THE LOWERCASE addService!

You would think people would have just cut and paste the code from the Activity Providers site and got it right.

Incidentally, if anyone knows how to create an API in Firefox that will work with either case, that would be great. Then I could work like Internet Explorer.

New Activities, Updated Operator and More

I've made version 0.6 of Firefox Activities available in the Add-ons Sandbox. If you like it, please review it so I can get it moved out of the Sandbox.

This version has a working preview for Firefox 3, as well as management of activities. It's very close to the Microsoft version. The only thing it is lacking at this point is the button that appears after you select text. I'm looking at that.

I also had to make Operator 0.9.1 available due to some translation issues and some interaction issues with Firebug.

Finally, if you are a fan of GTD, I shall shamelessly plug my friend Andy Mitchell's extension, GTDInbox which turns GMail into GTD central.

Enjoy!

An Analysis of Microsoft Activities

People seem to think that because I implemented Microsoft Activities for Firefox that I am agreeing with Microsoft's solution. That conclusion is not correct. I simply implemented Activities because I thought it would be fun. But in the process of implementing it, I've learned a bit about it, and I'm going to provide some of that information here.

Clearly when I first saw Activities, I thought "man, that looks a lot like Operator." And that motivated me to take a closer look. But what's interesting is that I believe Operator and Microsoft Activities share a common ancestor: Live Clipboard.

Microsoft introduced Live Clipboard two years ago their PDC. The basic idea was to put a little orange scissors icon on web pages and allow people to copy and paste data from those web pages to other web pages and to applications. The data that was copied was a microformat, so it would maintain its identity. So for instance, you could copy an event from a web page and paste it into Windows Live Writer. It seemed like a good idea, but it had some problems:

  • Required HTML and JavaScript hacks
  • Scissors mean cut - they don't mean copy and they don't mean paste
  • Finding places to paste the data was difficult
  • No one wants to put an orange scissors icon on their web pages

But they did get one thing very right - the use of microformats to encapsulate the data.

Really, though, the problem is that people don't want to copy and paste data; they just want to be able to send that data to the place they want it to go. That's where the idea for Operator came from. The idea was to take the microformats and create actions to allows data on a web page (microformats, RDF or other) to be sent directly to a web service or native application. This is the idea of the web as a switchboard. (See Alex Faaborg's presentation) Incidentally, Live Clipboard was one of the first actions I created for Operator!

Activities looks like it is trying to solve the same problem. This quote from the readiness page is very telling:

Activities are contextual services to quickly access a service from any webpage. Users typically copy and paste from one webpage to another. Internet Explorer 8 Activities make this common pattern easier to do.

So Microsoft ended up coming to the same conclusion that we did: you need to be able to execute an action directly on a web page, not cut and paste. Unfortunately, in creating Activities they lost the one component of Live Clipboard that was useful - the structured data (microformats).

Activities acts on three contexts in a web page, a selection, a link or the document itself.

The selection context is the least useful of all the contexts. It takes selected text and sends it to a URL, similar to what search already does. It's essentially a search with preview. It does not attempt to interpret the text it sends or validate it.

The link and document contexts both act on a URL and they are definitely more useful. The reason they are more useful is because a link implicitly provides more data than selected text. You know it is a link, therefore you can do more interesting things with it like add it to del.icio.us or digg it.

The fact that links are more useful then selected text should tell us something; the more data we have about what we are acting on, the more choices we are able to give a user that they might actually want to do. If you really want to see this in action, check out BlueOrganizer from AdaptiveBlue. This shows you the power of knowing the type of data that the user has selected.

So by leaving microformats (or any semantic markup for that matter) out of Activities, a lot of the usefulness is gone. I much prefer using Operator to right click on a calendar item and add it to my Google calendar (assuming it is marked up with microformats).

Activities has caused me to realize one thing though - we really need to work together to define an XML syntax for defining actions. I'm hoping to work with the folks at Adaptive Blue and maybe Microsoft to make that happen.

And if you are looking for an update to my Activities extension, I'll have something soon. It's working much more like the Microsoft implementation and allows you to manage the activities better.

Microsoft Activities for Firefox - New Version

I have an update for Activities that adds preview and fixes some bugs. If you just want that, skip to the end of the post.

One of the things I realized as I worked on this extension is just how amazingly easy it is to write Firefox extensions. When you combine great technology (Firefox extensions), great people (like Mark Finkle, for example), great documentation (developer.mozilla.org, XULPlanet) and open source, it's amazing what you can produce and how fast you can produce it, as noticed in Compiler from Wired.com.

Compare this to my experience with Internet Explorer.

One of the things I wanted to do with Operator was make a version for Internet Explorer. I spent a few weeks trying to find good documentation on creating a toolbar for IE, and I have to say it was sorely lacking. I got a basic HTC working, but compared to my experience working on Firefox extensions, it was just SO painful, I gave up. I realize I'm not a Windows programmer, and that might have helped, but I just couldn't believe how hard it was to create extensions for IE. It's not even called a toolbar, it's called a a toolband which started me off in the wrong direction!

In a way, that was kind of sad because I was feeling like my focus on web development and in particular Mozilla and Firefox browsers was limiting my skills so that when it came to things like Windows development, I was at a disadvantage. (I've been working in the browser space and the development of web browsers for over 12 years). Hopefully, though, what I've actually done is cultivated a set of specialized, but very useful skills so that when other opportunities come up, I'll be able to take advantage of them. Especially since it seems like my work at IBM is requiring less and less of those Mozilla/Firefox skills.

So all that to say, I've thoroughly enjoyed working on the Microsoft Activities extension and the other extensions I've done and I hope that there will be more opportunities to do that, whether inside or outside IBM. Maybe I'll have a chance to talk to some folks about that at SXSW this weekend.

On to the new release. Version 0.3 of Microsoft Activities for Firefox is available (make sure you didn't grab the 0.2 that was up for a few minutes this morning). Features include:

  • Preview mode (only in FF3)
  • Problems adding services in FF3 have been fixed
  • Issues with query parameters fixed (all current services have been tested)
  • UI is updated immediately when a new service is added
  • Activities menu identifies the context you are acting in, and activities are limited based on that context (selection, link, document)
  • Built-in link to take you to the IE8 Service Gallery
  • Verified it works on Mac OS X

Interaction with the preview window is a little quirky, but I'm working on that. Surprisingly it seems to work better than IE8.

Enjoy!

More Microsoft Activities for Firefox (Calling out Microsoft)

I've made lots of progress since yesterday. I hope to have a new version available today. I thought I'd start the day by calling out Microsoft for not even following their own spec (and you too Yahoo!). What part of 320x240 preview window do you not understand? (And yes there is stuff offscreen even if the scrollbars aren't there)

You can check out other providers (who got it right) here.

And one more thing - is it AddService or addService? The reference page says "AddService" but links on the Service Gallery say "addService." (I got an answer - it's addService)

Microsoft Activities for Firefox

After looking at Microsoft Activities in IE8 and noticing that they look suspiciously like my ideas around actions in Operator, I decided to implement them in Firefox.

For your downloading pleasure, here is a first pass at Activities support in Firefox. I haven't actually looked at IE 8, so what I did was based on the page about Activities I saw this morning. You can install activities from the Service Gallery, and then if you open a new window, those activities will be active (they don't become active in the page you download them from - working on that). To use an activity, highlight some text and right click on it.

If folks think this is interesting, I'll keep working on it. Enjoy.

Note this doesn't support previews yet - I'm working on that.