Semantic Web Austin

GeekAustin hosted the Semantic Web Austin launch party last night. I was a speaker and got a chance to talk about Operator, Activities and the Microformats support in Firefox 3. I also took the opportunity to hand out some Firefox stickers. All in all, a very fun event, and a nice chance to meet Austin internet folks. If you want to learn more, Austin 3.0 has some interviews and other info about the event.

Updating OpenService for Microformats

This post details the ongoing work that Gustavo Garcia and I have been doing at http://microformats.org/wiki/OpenService_Extensions. Our stated goal is to figure out the best way to extend Microsoft's OpenService specification to allow for microformat support (and possibly other functionality). We definitely want to have more discussion on this, so if you are interested, please participate. Note that Microsoft has suggested that any changes we make be in a new name space to allow for good coexistence with IE. We're investigating that.

This post assumes you have a basic understanding of what OpenService looks like, so you might want to read up on that first. So without further ado, here are our suggested changes to OpenService.

Support for more contexts

Currently IE supports three contexts: selection, link and document. A given action is applied based on the context where the user right clicks. For microformats, we would add support for microformat names as a possible context. We would also allow those microformat names to be followed by an item in the microformat so that actions could be shown based on the availability of data in the microformat. For example,

would indicate that the action would be available for any adr.

would indicate that the action would be available for any adr that also had a postal-code.

Domain scoping of actions

Currently a given action is available on every website. There are some actions that should only be displayed for a given site. For instance, cork'd.com uses tags to reference their wines, but they really only make sense in the context of their site. To solve this, we would introduce a new attribute to activityAction called domain that would specify which website should have the given action available.

More substitution variables

The substitution variables for the actions would have to be enhanced to allow for the substitution of values in the microformat. Here's an example:



  

This action queries the country-name property directly from the adr microformat and substitutes it. If the referenced property is a plural property, all the items are put in one string separated by a space and then substituted.

Scripting Support

Unfortunately, for complex web services, just substituting variables in URLs and parameters is not enough. So to further enhance the functionality of the OpenServices XML files, we propose that scripting be allowed. These scripts would be evaluated in a sandbox with the contents of the microformat available as local variables (possibly prepended with the microformat name) with some additional helper functions added (XMLHTTPRequest, writing to a temporary directory). A simple example of where this is needed is Google Calendar. Google Calendar requires that if there is a dtstart, there must be a dtend. There's really no way to convey this with simple string substitution. For some web services, some pretty complex JavaScript is needed. Here's an example of how JavaScript might work.



 
  
   
  
 

Here we were able to use JavaScript to create some basic logic around the date and then the correct value is returned.

This idea is probably the most controversial, but I don't see a better way to get the richness that is needed to interact with certain web services. If anyone has better ideas, I'd love to hear them.

Action Discovery on Web Pages

We'd also like to see the ability for a web page to advertise that they have a service available, similar to how OpenSearch works. For OpenService, it might look something like this:



We're still working out how this would be discoverable to the user, but we think it should be in the specification.


On a completely unrelated note, if you are the praying kind, please keep the Chapman family in your prayers. They lost their 5 year old daughter yesterday in a tragic accident. My cousin Melissa works for them as their nanny, so she needs your prayers as well. She's taking this pretty hard. Thanks.

Where are the microformats in Firefox 3?

With Firefox 3 RC1 available, I'm getting asked "Where are the microformats?" The answer is that there is a microformats API in Firefox 3, but unfortunately there is nothing available in the UI for this release. There are a couple reasons why this is the case, so I thought I would take some time to explain.

The primary reason that microformats aren't exposed in the Firefox UI is that there was never any agreement as to how to expose them. Originally the idea was to have a "rich media sidebar," but after implementing that in Operator, it was realized that it wouldn't have much content in it most of the time. Plus sidebars take up a lot of real estate. Having a toolbar in Firefox didn't make a lot of sense since, again, there would be a lot of screen real estate needed for something that the user wouldn't use very much (yet). I liked the idea of an icon on the URL bar similar to how RSS feeds work, but the consensus was that noone wanted the awesome bar to become cluttered with small icons on the right. The other option was to make them available only when you right click on microformats, but this wasn't very discoverable and the technical aspects of making it discoverable (icons on hover, etc.) were a little daunting. There was one really good idea related to adding a kind of bar on the left side that showed where the microformats were, but that idea really needed to be prototyped. So in the end, we were left with nothing in the UI.

There was also a second problem that wasn't discussed as much, but was still a big issue - how to plug in new services. Currently Operator uses the concept of user scripts to allow the installation of additional microformats and connections to web services. Unfortunately, the method I chose (raw JavaScript files) introduces security and other issues. It works for Operator, but would not be a good method to put into the core browser.

So where does that leave things? We still haven't solved the UI problem, but Microsoft has come up with a solution for the services problem that I am currently in the process of integrating into Operator. It's called OpenService. I mentioned a few posts ago that I'm extending the specification a little bit so we can define Operator web services connections using XML. Tomorrow, I'll go into some detail as to what I'm doing. Hopefully I can get some good feedback and work with Microsoft to standardize my additions to the specification.

Update on Activities, Microformats and Operator

I haven't blogged in a while, so I wanted to give everyone a quick update on som of my projects.

Activities

Version 0.7.1 of Firefox Activities is waiting patiently to be moved out of the AMO sandbox. Unfortunately my 0.7 version had a memory leak, so I missed my chance. Once Firefox 3.0b5 was released, the AMO queue got very long and 0.7.1 is lost somewhere in that queue.

That version matches the IE version of Activities feature for feature, except for the floating button that appears when you select text. Activities management is much better, and I was able to support both the uppercase and lower case version of the API (which I complained about earlier). In addition, the code has been substantially rewritten so that it doesn't affect the global namespace at all.

Microformats in Firefox 3

I thought my Microformats API for Firefox 3 had settled down, but Dmitry's Acid Test combined with Clint Talbert's great unit tests and a post on the microformats-discuss list uncovered a few more issues. Unfortunately time is short, so only the API change will make it. Hopefully we'll get the other parser fixes in a point release.

Operator

I've actually been spending quite a bit of time on Operator lately, working with Gustavo García to figure out how to move to an XML model for user scripts using Microsoft's OpenService Format as a basis. Our primary motivation for this is to allow the installation of user scripts on the fly instead of the current model where you have to download them. We've actually made quite a bit of progress in this space, and hope to have something out soon for testing. Note Operator will still support the old user scripts model, especially for scripts that need access to things like the file system.

Here's an example of what one of these new scripts might look like:




  
    Find with MapQuest

http://www.mapquest.com/favicon.ico

http://www.mapquest.com

We're also planning to allow small pieces of JavaScript to be embedded in the XML to allow more complex actions. These will be run in a sandbox to prevent security issues. If you want to be involved in the discussion of how we are going to do this, check out the microformats wiki.

An Analysis of Microsoft Activities

People seem to think that because I implemented Microsoft Activities for Firefox that I am agreeing with Microsoft's solution. That conclusion is not correct. I simply implemented Activities because I thought it would be fun. But in the process of implementing it, I've learned a bit about it, and I'm going to provide some of that information here.

Clearly when I first saw Activities, I thought "man, that looks a lot like Operator." And that motivated me to take a closer look. But what's interesting is that I believe Operator and Microsoft Activities share a common ancestor: Live Clipboard.

Microsoft introduced Live Clipboard two years ago their PDC. The basic idea was to put a little orange scissors icon on web pages and allow people to copy and paste data from those web pages to other web pages and to applications. The data that was copied was a microformat, so it would maintain its identity. So for instance, you could copy an event from a web page and paste it into Windows Live Writer. It seemed like a good idea, but it had some problems:

  • Required HTML and JavaScript hacks
  • Scissors mean cut - they don't mean copy and they don't mean paste
  • Finding places to paste the data was difficult
  • No one wants to put an orange scissors icon on their web pages

But they did get one thing very right - the use of microformats to encapsulate the data.

Really, though, the problem is that people don't want to copy and paste data; they just want to be able to send that data to the place they want it to go. That's where the idea for Operator came from. The idea was to take the microformats and create actions to allows data on a web page (microformats, RDF or other) to be sent directly to a web service or native application. This is the idea of the web as a switchboard. (See Alex Faaborg's presentation) Incidentally, Live Clipboard was one of the first actions I created for Operator!

Activities looks like it is trying to solve the same problem. This quote from the readiness page is very telling:

Activities are contextual services to quickly access a service from any webpage. Users typically copy and paste from one webpage to another. Internet Explorer 8 Activities make this common pattern easier to do.

So Microsoft ended up coming to the same conclusion that we did: you need to be able to execute an action directly on a web page, not cut and paste. Unfortunately, in creating Activities they lost the one component of Live Clipboard that was useful - the structured data (microformats).

Activities acts on three contexts in a web page, a selection, a link or the document itself.

The selection context is the least useful of all the contexts. It takes selected text and sends it to a URL, similar to what search already does. It's essentially a search with preview. It does not attempt to interpret the text it sends or validate it.

The link and document contexts both act on a URL and they are definitely more useful. The reason they are more useful is because a link implicitly provides more data than selected text. You know it is a link, therefore you can do more interesting things with it like add it to del.icio.us or digg it.

The fact that links are more useful then selected text should tell us something; the more data we have about what we are acting on, the more choices we are able to give a user that they might actually want to do. If you really want to see this in action, check out BlueOrganizer from AdaptiveBlue. This shows you the power of knowing the type of data that the user has selected.

So by leaving microformats (or any semantic markup for that matter) out of Activities, a lot of the usefulness is gone. I much prefer using Operator to right click on a calendar item and add it to my Google calendar (assuming it is marked up with microformats).

Activities has caused me to realize one thing though - we really need to work together to define an XML syntax for defining actions. I'm hoping to work with the folks at Adaptive Blue and maybe Microsoft to make that happen.

And if you are looking for an update to my Activities extension, I'll have something soon. It's working much more like the Microsoft implementation and allows you to manage the activities better.

Operator 0.9 is available

NOTE: There is a problem with the zh-TW, hr-HR, cs-CZ, pt-BR and ru-RU translations. I'll post a 0.9.1 when I get this worked out. Unfortunately this makes Operator 0.9 unusable in those languages because for some reason Babelzilla replaced the strings with blanks (not what I asked for).

Operator 0.9 is available. This was mainly done so I can get my t-shirt, although it has been overdue.

I've talked about what is in this release before, but to repeat:

This version includes:

  • Songbird support!
  • Fixes to make sure things work in Firefox 3
  • Menus no longer magically disappear on some pages
  • hCards supports nested menus to access actions (more logical than trying to use adr for multiple addresses)
  • adr is no longer on the menu by default (you can add it, but it has no default actions)
  • Nested microformats are now handled per the spec as best we can figure it out
  • Performance should be much better (and it has been in my experience)
  • If a microformat is not visible, it doesn't appear in Operator (You can turn on a preference if you want to see all microformats)
  • Actions for MapQuest and Amazon.com
  • A new RDFa parser based on the new spec (pages will need to be updated)

Unfortunately, I don't have more detail on the RDFa situation. Elias sent me a new parser, and I included it. Please bug him to know what needs to change on pages. And I apologize to the existing RDFa users out there that I have broken.

Also, there were some performance problems with the release candidate that were definitely addressed. Hopefully things should be speedy.

Enjoy!

Operator 0.9g (Release Candidate) available

I'm making a release candidate for Operator available hoping that I can shake out any last bugs in it. It's available here. Note the release candidate is English only. I have it on Babelzilla at this moment getting translated.

First the caveat - RDFa support is not baked in this version (as in RDFa the old way doesn't work, and RDFa the new way doesn't work). That's mainly because the RDFa spec was in flux. I think it's done now and hopefully we'll have the RDFa parser before I release the official 0.9.

So what is going to be new in 0.9? Lots of bug fixes and improvements to general interpretation of how microformats work. I detailed this in a previous post, but here are some highlights:

  • Songbird support!
  • Fixes to make sure things work in Firefox 3
  • Menus no longer magically disappear on some pages
  • hCards supports nested menus to access actions (more logical than trying to use adr for multiple addresses)
  • Nested microformats are now handled per the spec as best we can figure it out
  • Performance should be much better (and it has been in my experience)

I hope some people will try this out and let me know how I did. I'm trying to get 0.9 out next week for SXSW. Thanks!

ONE MORE THING: If your microformat is hidden (via CSS or otherwise), it will no longer display in Operator by default! You need to turn on the "show hidden microformats" preferences. See this post.

One Year of Operator

It's been one year since I released Operator and I wanted to take the opportunity to thank everyone who is using it and has had good things to say about it. I haven't advanced the technology as much as I wanted to over the past year but hopefully this next year we'll see much more, especially with the inclusion of a microformats API in Firefox 3.

And now for something completely different.

If you use Operator on amazon.com, you might notice that the "Tagspaces" item is available, but when you pop it down, you get a lot of stuff that says "ref=tag_dpp_cust_itdp_t." This is because amazon (like a lot of other sites - any drupal site comes to mind) is not following the rel-tag spec. To attempt to work around this problem, I've added a hidden preference to Operator called "extensions.operator.allowInvalidTags." If you add this preference via about:config and set it to true, Operator will display the text content of the tag and use it, instead of the proper tag value. Note that although this makes some things work on amazon.com, they still is completely clueless about how to use rel-tag - checkout their Firefox tag page and take a look at the Tagspaces menu with the hidden preference turned on. You'll get things like "See all 226 tags" and "See all 25 customers." Great job amazon! Go read the spec!

And for everyone else that has screwed up tags, please go read the spec. If you don't like the spec, that's fine, but please don't add rel="tag" to anything that is an invalid tag.

Operator 0.9 beta available

I'm making available a beta of Operator 0.9, which has support for Firefox 3 and uses the built-in Firefox 3 microformats support.

The primary focus of this release has been performance, and I believe it shows. Sites like the Samsonite Store Locator are MUCH faster. Note that if you want to improve performance even more, you should remove the adr microformat from your toolbar. I've done this for new installs of Operator in the future. Since most pages with adrs have them in hCards, I've moved all the functionality that was for adrs into the hCard submenus.

Other changes include support for whether or not hidden microformats are displayed. You can try this out on the Samsonite website. Note that this functionality doesn't work on Firefox 3 because it required a change to the core Microformats support which hasn't made it into Firefox 3 yet.

Other small changes are the addition of searching for tags on amazon.com products, as well as searching for addresses on MapQuest.

One thing that is NOT in this release is modifications to support the new RDFa spec. We're still waiting for some stuff to be finalized for that.

Enjoy!