One of the biggest complaints about Operator is that it affects performance when loading pages and switching tabs. I’ve been spending the past few weeks analyzing these problems and I think I have them mostly solved. Note that page load vs. tab switching are actually two different problems; tab switching has to do with how fast I process the microformat data, whereas page load has to do with listening to DOM events. So here’s what I learned:

• My method for removing duplicate microformats was unnecessarily expensive. I was sorting in every case, and also duplicating and chopping up arrays. I went back to the way I originally wrote it and it works much better.
• The adr microformat is a real pain because it doesn’t have the concept of default text to display in the UI like other microformats. This meant that any page that has a lot of adr microformats was very expensive because of all the work I do computing what the address could be. To solve this in the future, adr will not be on the toolbar by default.
  Incidentally, the only reason I added adr in the first place was because I wanted to handle the edge case of multiple addresses in one hCard. I have since added support for actions to function on multiple items in a microformat, so the next release will move mapping back to hCard where it belongs with a submenu if there are multiple addresses.
• Some pages have lots of hidden microformats they show and hide based on interaction with the page. This can cause some performance issues. I think that by default, microformats should not be visible in Operator if they don’t show up on the page. So a page can have a thousand hidden microformats and it won’t affect performance. There will be an option to show all microformats.
• You have to be very careful not to add event listeners too early or you will get WAY too many messages. I was adding my DOM event listeners too soon so I was getting all DOM events during page load. Bad idea. Listeners should be added on each individual pageshow event, not DOMContentLoaded, and you should not try to be smart and add listeners recursively. Just add them to each target for each pageshow event.
• While we’re on the subject of DOMContentLoaded vs. pageshow, if you have an extension that updates the UI based on page content, you have to manage the interaction between these messages very carefully. The problem is that there are cases where you get just pageshow, just DOMContentLoaded, or both. Here’s the scoop: DOMContentLoaded happens on the first load of a page as long as it is not from back/forward cache. pageshow happens with the load of any page including from the back/forward cache and it happens after DOMContentLoaded. The case where DOMContentLoaded happens without pageshow is the display of error messages like when you put in a bad URL. So the trick is to disable your UI on DOMContentLoaded and do interesting stuff on pageshow.
• My testing shows that a pageshow event for a page containing frames doesn’t happen until all the child frames have received their pageshow. Because of that, I can wait to process the microformats on the page until I have received the pageshow event for the main content document since I am going to recurse into the child frames anyway. If I am wrong, someone please correct me.

Well that’s what I’ve learned so far. There should be an Operator release soon that addresses all these issues.

Earlier I made a comment about how I wished there was a better way to connect microformats to web services. I’d like to elaborate on that comment and see if I could get some discussion going around the subject. First let me summarize the problem. (Note that throughout this post I will use the term microformat, but all these problems hold true for RDF as well)

The problem is that there is no standard way (that I’ve found) to interact with various web services that deal with the types of data involved in microformats. Note that in some cases there are standard data interchange formats (like vCard and iCal), but the web services do not always accept these formats.

The way Operator solves this problem through the use of user scripts that implement actions. What these actions do is take the microformat and use the data to construct a URL (or even a POST request) that connects to the appropriate web service. For instance, if I want to add my hCard to to the Yahoo! address book, the URL might look something like this:

http://address.yahoo.com/?ln=Kaply&fn=Michael&co=IBM
&e=nospam@example.com&pu=http://www.kaply.com/weblog/&A=C

In this case, the user script has taken apart the microformat and then constructed a URL that emulates adding a Yahoo! Contact. In doing this, lots of code was involved because the Yahoo! Add Contact does not map completely to the data that was in the microformat (you can see that code in yahoo.js). (Folks might ask why I’m not using Yahoo! APIs – that’s mainly because I don’t want to mess with authentication keys and stuff – this is a web interface, so I’d rather just redirect to websites)

The problem with this code is that in order to express the mapping from a microformat to Yahoo!, I require a large chunk of code.

For Firefox 3, I want to provide some UI that connects microformat to web services, but I want that to be extendable by third parties. I want it to be similar to how feed readers work in Firefox 2 (see registerContentHandler). While my action architecture works in the context of Operator, I do not believe that it is robust enough to be a part of Firefox 3. So what I’m trying to find is some better way to describe the mapping between microformats and web services or possibly to create a new standard around interacting with these web services. It might be as simple as just encouraging all of the vendors to accept the upload of vCard and iCal. I don’t know. Here are some thoughts I (and others) have had.

Encourage people to support the submission of vCard and iCal in a standard way

The obvious thing to do here would be for services to allow the submission of vCards and iCals directly to their web services as a way to provide information to the services. 30 Boxes currently allows this, but unfortunately the iCal must be physically located on a web site, so I can’t send it to them. Other services that accept vCards use a POST mechanism that is difficult to emulate because it relies on lots of private data during the POST submission.

Server Data Broker

Setup up a web service that would essentially act as a data broker. Data could be given to this server in a standard way, and then the server would turn that data into a URL that could be used to redirect to the appropriate web service. In this way, the logic to turn the microformat data into a URL would be done on the server. The client would give some common data source to the server, telling the service where it wanted the data to be sent and the server would simply send the client a redirect URL.

OpenAction similar to OpenSearch

S. Sriram had an interesting thought in reply to my previous post. He thought about using something similar to OpenSearch where web services can have an XML description file that describes how to interact with their services and then we could use that file to determine how to map the microformat data to the web service.

Those are just a few ideas we’ve come up with. I’d love to hear any other ideas you might have…

Hey Google, Yahoo! and 30 Boxes (and others).

PLEASE create places where I can simply POST vCards or iCals or microformats or JSON or something else to your services to add contacts and calendar entries. I’m not a web app, so I can’t get an API key. And I shouldn’t have to maintain login state when the browser is doing it for me. All I want is an easy way to add stuff to a logged in users account. Is that too much to ask?

With Yahoo! calendars and contact, as well as Google Calendar, I have to resort to undocumented URL syntax. Google Contacts I can’t do anything. 30 Boxes requires that the ics file be physically located on a server, although they have a URL syntax that kind of works.

It doesn’t even have to be a POST. If you could come up with some straightforward URL syntax, that would be great. Trying to figure out the stuff you guys have put together so far is incredibly painful.

So please. Help a guy out.

The website iDOO added support for hCards and created an Operator user script to integrate with iDOO. The even plugged Operator in their announcement. Nice work!

You can get more information here.

Operator 0.8 is getting very close. I’m trying to finalize some UI stuff, specifically for testing some stuff for Firefox 3. Stay tuned. In the mean time, here’s some random things I need to post.

Operator 0.8 is going to have support for eRDF. Because of this, I’m changing “RDFa” actions to simply be “RDF” actions. So anywhere you use “RDFa” in an action, please change it to “RDF” for 0.8 final compatibility.

If you are interested in writing actions, I created an advanced article on creating actions. This is a follow up to my basic article on creating actions

The latest user scripts will always be on the Operator User Scripts page. This will be the home for any user scripts I mention in my blog and will also have links to third party user scripts. If you find bugs in user scripts, check that page to see if the scripts have been updated. I’ll mark any updates to that page with “NEW.”

And speaking of user scripts, there are new user scripts from Dan Noble for mapping hCards and hCalendar with Google Earth, as well as a user script from Thorsten Panknin for the Mister Wong social bookmarking site.

Incidentall, if you are an extension developer and want to try using the new microformats code in Firefox 3, you can download Gran Paradiso Alpha 7 and then follow these instructions to try it out.

So far, actions for Operator have focused on exporting data or connecting to an existing web service. Rosie Morales from MITRE has shown that actions can do much more interesting things, like actually create new views of the microformats by saving the data to a local file and displaying it. These actions are based on the Simile project.

Note these actions only work properly with Operator 0.8b.

The Simile Timeline action is an action that acts on all events on a web page. It creates a local file that contains a Simile Timeline representation of the data and then displays it.

The Exhibit Timeline action is an action that acts on all events on a web page. It creates a local file that contains a Exhibit Timeline representation of the data and then displays it.

The Exhibit action is an action that acts on all contacts on a web page or one contact on a web page. It creates a local file that contains a Exhibit representation of the data and then displays it.

I will be documenting how you can create an action that acts on all instances of a given microformat very soon.

One of the cool things that you can do with Operator is experiment with new microformats. The folks at Digital Bazaar and others have been working on an hAudio microformat and David Lehn has created some experimental Operator User Scripts for you to try out. hAudio.js contains the primary microformat. money.js is used to allow actions on currencies.

You can try out these microformats on the site Bitmunk, as well as WebOrganics

I think the APIs in Operator 0.8b have settled down enough that I can start to document how to write user scripts. Note these user scripts are for Operator, NOT for Firefox 3. How things will work in Firefox 3 hasn’t been decided yet. And I am working very hard to make this the final API for Operator 1.0.

What are Operator User Scripts?

Operator user scripts are user added pieces of JavaScript that can enable additional functionality within Operator like new actions for microformats and RDF, as well as adding new microformats. Examples can be found on the Operator User Scripts page. User scripts are added to Operator by opening Options and selecting the User Scripts tab. You can then select a user script that has been saved to your local hard drive. People have asked why I don’t allow user scripts to be installed directly from the web similar to Greasemonkey scripts. There are two reasons. First, the code to do that detection is licensed under GPL, so I can’t just take it and put it in Operator. Second, Operator user scripts operate with the same security privileges as an extension so they can pretty much do anything they want. That’s not something you want to install in a drive by situation.

Writing a User Scripts

I’m going to have separate pages for the user script documentation, rather than using posts. The first one is here – Creating a Microformat Action User Script (Basic). As always, I appreciate feedback, especially from folks that have already written actions. And speaking of actions, we have some new ones to add to the list:

• Tagzania User Scripts from Tony Tones Farndon

Enjoy!

I have made a beta of Operator 0.8 available. Continued thanks to Elias Torres for the RDFa support.

Operator 0.8b adds a much requested feature – an operator icon on the location bar. It’s not perfect yet, but it is there. Also, I updated the action API a little bit to allow for the naming of actions as well as having actions act on multiple properties in a microformat. For instance, in the past, the “Goto web page” action only worked for the first URL in an hCard. Now it will work for all the URLs.

The other big fix in this release was that there were cases where sorting/removing duplicates didn’t work correctly. For action developers, I added the ability to have multiple translations of an action in one action. I’ll document that soon.

All the caveats from Operator 0.8a apply, so please read them.

Please report bugs here.

Have fun.

I know I’ve been slacking on posting information on how to create actions for Operator 0.8. I’ve been tweaking the API a little bit, so that’s why I’ve been hesitant. Operator 0.8 beta will be very soon, and then I’ll post a tutorial on creating actions.

In the mean time, I have been working with a few folks to get their actions working on Operator 0.8.

• The Minimap Addon gives you a suite of built-in maps and mapping tools for your web browser. And it also supports integration with Operator.
• Egon Willighagen is using RDFa combined with an Operator action to allow for searching PubChem from the Chemical Blogspace. More information is available in his post.
• Søren Johannessen has updated his actions for the Danish Blogosphere for Operator 0.8. You can read about them and download them at microformats.dk.
• Pelle Wessman has updated his actions for the Swedish Blogosphere for Operator 0.8. You can read about them and download them at http://pelle.vox.nu/.
• Charl van Niekerk has created actions for the African Blogosphere for Operator 0.8. You can read about them and download them at http://blog.charlvn.za.net/.

And I’ve been adding/updating some of my user scripts. These include support for the license microformat, as well as some improvements to species.

Incidentally, the improvements to the action API will NOT break actions already written for Operator 0.8. The improvements include the ability to translate your actions into other languages, the ability to customize the name that appears for your action in the Operator dropdowns and the ability to have multiple items in the action menu for something that appears multiple times in a microformat. For instance, in the past, even if an hCard had multiple web pages, there was only one action that opened the first web page. Now an action can detect the multiple web pages and display unique action names for each of them.

Stay tuned!