Back from the Dead

It's been way too long since I have blogged.

Most of my summer was spent on a cross country road trip which you can read about here. I worked as we traveled which went reasonably well, but didn't leave time for blogging.

The biggest news I have is that I have resumed work on Operator. In particular, I'm fixing bugs, adding a few usability enhancements and adding support for new microformat stuff like the value class/pattern for dates. I'm also considering completely removing the "Actions" toolbar and switching to interacting only with the data. I'm definitely looking for feedback on that one.

If you have anything you wish Operator did, please let me know and I'll see if I can get it in.

Note that Operator is a part of the AMO Contributions program and I'd appreciate any support you can provide. Right now I'm mixing it in with my paid work, but I'd like to spend more time on it.

The bulk of my paid work has been for Brand Thunder. We've been updating our extensions to work properly with Firefox 3.5. There's quite an eclectic mix there, so you'll probably find something that you like.

Going forward I'm going to try to start doing extension related posts, primarily to see if I can drum up some more business. So if by chance you have a question about anything I've done in the past with extensions, or should do in the future, please let me know.

UPDATE: Forgot that I've also proposed a panel for SXSW. Please vote:
Vote for my PanelPicker idea!Love AND Money: Can Fansites Pay the Bills?

Semantic Web Austin

GeekAustin hosted the Semantic Web Austin launch party last night. I was a speaker and got a chance to talk about Operator, Activities and the Microformats support in Firefox 3. I also took the opportunity to hand out some Firefox stickers. All in all, a very fun event, and a nice chance to meet Austin internet folks. If you want to learn more, Austin 3.0 has some interviews and other info about the event.

Updating OpenService for Microformats

This post details the ongoing work that Gustavo Garcia and I have been doing at http://microformats.org/wiki/OpenService_Extensions. Our stated goal is to figure out the best way to extend Microsoft's OpenService specification to allow for microformat support (and possibly other functionality). We definitely want to have more discussion on this, so if you are interested, please participate. Note that Microsoft has suggested that any changes we make be in a new name space to allow for good coexistence with IE. We're investigating that.

This post assumes you have a basic understanding of what OpenService looks like, so you might want to read up on that first. So without further ado, here are our suggested changes to OpenService.

Support for more contexts

Currently IE supports three contexts: selection, link and document. A given action is applied based on the context where the user right clicks. For microformats, we would add support for microformat names as a possible context. We would also allow those microformat names to be followed by an item in the microformat so that actions could be shown based on the availability of data in the microformat. For example,

would indicate that the action would be available for any adr.

would indicate that the action would be available for any adr that also had a postal-code.

Domain scoping of actions

Currently a given action is available on every website. There are some actions that should only be displayed for a given site. For instance, cork'd.com uses tags to reference their wines, but they really only make sense in the context of their site. To solve this, we would introduce a new attribute to activityAction called domain that would specify which website should have the given action available.

More substitution variables

The substitution variables for the actions would have to be enhanced to allow for the substitution of values in the microformat. Here's an example:



  

This action queries the country-name property directly from the adr microformat and substitutes it. If the referenced property is a plural property, all the items are put in one string separated by a space and then substituted.

Scripting Support

Unfortunately, for complex web services, just substituting variables in URLs and parameters is not enough. So to further enhance the functionality of the OpenServices XML files, we propose that scripting be allowed. These scripts would be evaluated in a sandbox with the contents of the microformat available as local variables (possibly prepended with the microformat name) with some additional helper functions added (XMLHTTPRequest, writing to a temporary directory). A simple example of where this is needed is Google Calendar. Google Calendar requires that if there is a dtstart, there must be a dtend. There's really no way to convey this with simple string substitution. For some web services, some pretty complex JavaScript is needed. Here's an example of how JavaScript might work.



 
  
   
  
 

Here we were able to use JavaScript to create some basic logic around the date and then the correct value is returned.

This idea is probably the most controversial, but I don't see a better way to get the richness that is needed to interact with certain web services. If anyone has better ideas, I'd love to hear them.

Action Discovery on Web Pages

We'd also like to see the ability for a web page to advertise that they have a service available, similar to how OpenSearch works. For OpenService, it might look something like this:



We're still working out how this would be discoverable to the user, but we think it should be in the specification.


On a completely unrelated note, if you are the praying kind, please keep the Chapman family in your prayers. They lost their 5 year old daughter yesterday in a tragic accident. My cousin Melissa works for them as their nanny, so she needs your prayers as well. She's taking this pretty hard. Thanks.

Where are the microformats in Firefox 3?

With Firefox 3 RC1 available, I'm getting asked "Where are the microformats?" The answer is that there is a microformats API in Firefox 3, but unfortunately there is nothing available in the UI for this release. There are a couple reasons why this is the case, so I thought I would take some time to explain.

The primary reason that microformats aren't exposed in the Firefox UI is that there was never any agreement as to how to expose them. Originally the idea was to have a "rich media sidebar," but after implementing that in Operator, it was realized that it wouldn't have much content in it most of the time. Plus sidebars take up a lot of real estate. Having a toolbar in Firefox didn't make a lot of sense since, again, there would be a lot of screen real estate needed for something that the user wouldn't use very much (yet). I liked the idea of an icon on the URL bar similar to how RSS feeds work, but the consensus was that noone wanted the awesome bar to become cluttered with small icons on the right. The other option was to make them available only when you right click on microformats, but this wasn't very discoverable and the technical aspects of making it discoverable (icons on hover, etc.) were a little daunting. There was one really good idea related to adding a kind of bar on the left side that showed where the microformats were, but that idea really needed to be prototyped. So in the end, we were left with nothing in the UI.

There was also a second problem that wasn't discussed as much, but was still a big issue - how to plug in new services. Currently Operator uses the concept of user scripts to allow the installation of additional microformats and connections to web services. Unfortunately, the method I chose (raw JavaScript files) introduces security and other issues. It works for Operator, but would not be a good method to put into the core browser.

So where does that leave things? We still haven't solved the UI problem, but Microsoft has come up with a solution for the services problem that I am currently in the process of integrating into Operator. It's called OpenService. I mentioned a few posts ago that I'm extending the specification a little bit so we can define Operator web services connections using XML. Tomorrow, I'll go into some detail as to what I'm doing. Hopefully I can get some good feedback and work with Microsoft to standardize my additions to the specification.

Update on Activities, Microformats and Operator

I haven't blogged in a while, so I wanted to give everyone a quick update on som of my projects.

Activities

Version 0.7.1 of Firefox Activities is waiting patiently to be moved out of the AMO sandbox. Unfortunately my 0.7 version had a memory leak, so I missed my chance. Once Firefox 3.0b5 was released, the AMO queue got very long and 0.7.1 is lost somewhere in that queue.

That version matches the IE version of Activities feature for feature, except for the floating button that appears when you select text. Activities management is much better, and I was able to support both the uppercase and lower case version of the API (which I complained about earlier). In addition, the code has been substantially rewritten so that it doesn't affect the global namespace at all.

Microformats in Firefox 3

I thought my Microformats API for Firefox 3 had settled down, but Dmitry's Acid Test combined with Clint Talbert's great unit tests and a post on the microformats-discuss list uncovered a few more issues. Unfortunately time is short, so only the API change will make it. Hopefully we'll get the other parser fixes in a point release.

Operator

I've actually been spending quite a bit of time on Operator lately, working with Gustavo García to figure out how to move to an XML model for user scripts using Microsoft's OpenService Format as a basis. Our primary motivation for this is to allow the installation of user scripts on the fly instead of the current model where you have to download them. We've actually made quite a bit of progress in this space, and hope to have something out soon for testing. Note Operator will still support the old user scripts model, especially for scripts that need access to things like the file system.

Here's an example of what one of these new scripts might look like:




  
    Find with MapQuest

http://www.mapquest.com/favicon.ico

http://www.mapquest.com

We're also planning to allow small pieces of JavaScript to be embedded in the XML to allow more complex actions. These will be run in a sandbox to prevent security issues. If you want to be involved in the discussion of how we are going to do this, check out the microformats wiki.

New Activities, Updated Operator and More

I've made version 0.6 of Firefox Activities available in the Add-ons Sandbox. If you like it, please review it so I can get it moved out of the Sandbox.

This version has a working preview for Firefox 3, as well as management of activities. It's very close to the Microsoft version. The only thing it is lacking at this point is the button that appears after you select text. I'm looking at that.

I also had to make Operator 0.9.1 available due to some translation issues and some interaction issues with Firebug.

Finally, if you are a fan of GTD, I shall shamelessly plug my friend Andy Mitchell's extension, GTDInbox which turns GMail into GTD central.

Enjoy!

An Analysis of Microsoft Activities

People seem to think that because I implemented Microsoft Activities for Firefox that I am agreeing with Microsoft's solution. That conclusion is not correct. I simply implemented Activities because I thought it would be fun. But in the process of implementing it, I've learned a bit about it, and I'm going to provide some of that information here.

Clearly when I first saw Activities, I thought "man, that looks a lot like Operator." And that motivated me to take a closer look. But what's interesting is that I believe Operator and Microsoft Activities share a common ancestor: Live Clipboard.

Microsoft introduced Live Clipboard two years ago their PDC. The basic idea was to put a little orange scissors icon on web pages and allow people to copy and paste data from those web pages to other web pages and to applications. The data that was copied was a microformat, so it would maintain its identity. So for instance, you could copy an event from a web page and paste it into Windows Live Writer. It seemed like a good idea, but it had some problems:

  • Required HTML and JavaScript hacks
  • Scissors mean cut - they don't mean copy and they don't mean paste
  • Finding places to paste the data was difficult
  • No one wants to put an orange scissors icon on their web pages

But they did get one thing very right - the use of microformats to encapsulate the data.

Really, though, the problem is that people don't want to copy and paste data; they just want to be able to send that data to the place they want it to go. That's where the idea for Operator came from. The idea was to take the microformats and create actions to allows data on a web page (microformats, RDF or other) to be sent directly to a web service or native application. This is the idea of the web as a switchboard. (See Alex Faaborg's presentation) Incidentally, Live Clipboard was one of the first actions I created for Operator!

Activities looks like it is trying to solve the same problem. This quote from the readiness page is very telling:

Activities are contextual services to quickly access a service from any webpage. Users typically copy and paste from one webpage to another. Internet Explorer 8 Activities make this common pattern easier to do.

So Microsoft ended up coming to the same conclusion that we did: you need to be able to execute an action directly on a web page, not cut and paste. Unfortunately, in creating Activities they lost the one component of Live Clipboard that was useful - the structured data (microformats).

Activities acts on three contexts in a web page, a selection, a link or the document itself.

The selection context is the least useful of all the contexts. It takes selected text and sends it to a URL, similar to what search already does. It's essentially a search with preview. It does not attempt to interpret the text it sends or validate it.

The link and document contexts both act on a URL and they are definitely more useful. The reason they are more useful is because a link implicitly provides more data than selected text. You know it is a link, therefore you can do more interesting things with it like add it to del.icio.us or digg it.

The fact that links are more useful then selected text should tell us something; the more data we have about what we are acting on, the more choices we are able to give a user that they might actually want to do. If you really want to see this in action, check out BlueOrganizer from AdaptiveBlue. This shows you the power of knowing the type of data that the user has selected.

So by leaving microformats (or any semantic markup for that matter) out of Activities, a lot of the usefulness is gone. I much prefer using Operator to right click on a calendar item and add it to my Google calendar (assuming it is marked up with microformats).

Activities has caused me to realize one thing though - we really need to work together to define an XML syntax for defining actions. I'm hoping to work with the folks at Adaptive Blue and maybe Microsoft to make that happen.

And if you are looking for an update to my Activities extension, I'll have something soon. It's working much more like the Microsoft implementation and allows you to manage the activities better.

Operator 0.9 is available

NOTE: There is a problem with the zh-TW, hr-HR, cs-CZ, pt-BR and ru-RU translations. I'll post a 0.9.1 when I get this worked out. Unfortunately this makes Operator 0.9 unusable in those languages because for some reason Babelzilla replaced the strings with blanks (not what I asked for).

Operator 0.9 is available. This was mainly done so I can get my t-shirt, although it has been overdue.

I've talked about what is in this release before, but to repeat:

This version includes:

  • Songbird support!
  • Fixes to make sure things work in Firefox 3
  • Menus no longer magically disappear on some pages
  • hCards supports nested menus to access actions (more logical than trying to use adr for multiple addresses)
  • adr is no longer on the menu by default (you can add it, but it has no default actions)
  • Nested microformats are now handled per the spec as best we can figure it out
  • Performance should be much better (and it has been in my experience)
  • If a microformat is not visible, it doesn't appear in Operator (You can turn on a preference if you want to see all microformats)
  • Actions for MapQuest and Amazon.com
  • A new RDFa parser based on the new spec (pages will need to be updated)

Unfortunately, I don't have more detail on the RDFa situation. Elias sent me a new parser, and I included it. Please bug him to know what needs to change on pages. And I apologize to the existing RDFa users out there that I have broken.

Also, there were some performance problems with the release candidate that were definitely addressed. Hopefully things should be speedy.

Enjoy!

Microsoft Activities for Firefox - New Version

I have an update for Activities that adds preview and fixes some bugs. If you just want that, skip to the end of the post.

One of the things I realized as I worked on this extension is just how amazingly easy it is to write Firefox extensions. When you combine great technology (Firefox extensions), great people (like Mark Finkle, for example), great documentation (developer.mozilla.org, XULPlanet) and open source, it's amazing what you can produce and how fast you can produce it, as noticed in Compiler from Wired.com.

Compare this to my experience with Internet Explorer.

One of the things I wanted to do with Operator was make a version for Internet Explorer. I spent a few weeks trying to find good documentation on creating a toolbar for IE, and I have to say it was sorely lacking. I got a basic HTC working, but compared to my experience working on Firefox extensions, it was just SO painful, I gave up. I realize I'm not a Windows programmer, and that might have helped, but I just couldn't believe how hard it was to create extensions for IE. It's not even called a toolbar, it's called a a toolband which started me off in the wrong direction!

In a way, that was kind of sad because I was feeling like my focus on web development and in particular Mozilla and Firefox browsers was limiting my skills so that when it came to things like Windows development, I was at a disadvantage. (I've been working in the browser space and the development of web browsers for over 12 years). Hopefully, though, what I've actually done is cultivated a set of specialized, but very useful skills so that when other opportunities come up, I'll be able to take advantage of them. Especially since it seems like my work at IBM is requiring less and less of those Mozilla/Firefox skills.

So all that to say, I've thoroughly enjoyed working on the Microsoft Activities extension and the other extensions I've done and I hope that there will be more opportunities to do that, whether inside or outside IBM. Maybe I'll have a chance to talk to some folks about that at SXSW this weekend.

On to the new release. Version 0.3 of Microsoft Activities for Firefox is available (make sure you didn't grab the 0.2 that was up for a few minutes this morning). Features include:

  • Preview mode (only in FF3)
  • Problems adding services in FF3 have been fixed
  • Issues with query parameters fixed (all current services have been tested)
  • UI is updated immediately when a new service is added
  • Activities menu identifies the context you are acting in, and activities are limited based on that context (selection, link, document)
  • Built-in link to take you to the IE8 Service Gallery
  • Verified it works on Mac OS X

Interaction with the preview window is a little quirky, but I'm working on that. Surprisingly it seems to work better than IE8.

Enjoy!