Activity Streams Rubygem and announcing the (codename) Aidentiti project

Ruby Activity Streams Parser

Last week Ruben Fonseca released the first version of a Ruby Activity Streams parser that works with Atom and JSON formats. It’s available on Github and you are encouraged to use it and patch it. We will very much appreciate all the feedback you can provide. Technical details are available on Ruben’s blog. Ruby is pretty readable, so is Ruben’s code, so it should be easy to port it to other languages (start with the spec file).

Aidentiti – an Activity Streams and Open-Stack oriented social collaboration platform

The gem release is the first public product of a social collaboration platform project codenamed Aidentiti, that implements many Open Stack techs and protocols. Aidentiti is a spin-off project from Quodis, the company I’m a partner at, and is at protype stage. I’m not able to invite you to test it at the moment, but the release of the Rubygem is a good opportunity to share with you the objective and architecture of the platform, from a mostly tech social web perspective.

Aidentiti relies heavily on Activity Streams and OAuth to act like an “activity data hub” for web-based productivity applications. While most of the Activity Streams work and thought has been focused on personal/social activities, Aidentiti relies on the standard to acts as a glue for productivity applications by allowing all activities to be defined and stored in a standard language.

Through the OAuth platform, it is easy for developers to create apps (full-blown or micro, very very specialized, apps) that can speak to an Aidentiti instance and it is easy for users and instance admins to manage an application catalogue (something close to “urbanise their informations systems” in IT jargon). The Aidentiti API also exposes commodity methods and features such as notifications, social graph / org char (XFN + PoCo right now), a no-password, no-hassle, experience to using multiple apps (the best for each job).

Users and app developers will also benefit from a central “vocabulary”, where teams can translate their language into information filters. This makes it possible to tell Aidentiti to associate a milestone completion in, say, GoPlan, to a Github commit, a Yammer status update and social media buzz on other platforms — vocabularies are combinations of tags, applications, users and activity properties (verbs, object types, etc.). And encourages the development of micro apps that integrate into a whole (example: collaborative office amenities management).
The objective is to provide meaningful integrated visualizations of a person’s or team’s activities and enable fluid, real-time and all-time, reporting of activity data. Some visualizations will be provided by the platform, but developers can also create more visualizations of the activities through the API.

I said at the beginning that this was a “social collaboration platform” project. The product will be. But it’s really about offering more information technology liquidity to the individual. If we are to live out in the open, with invisible tools automatically registering, reporting and sharing everything we do, it just might be fundamental that all this info ends up in a repository controlled by the user and not only in tens of different apps, past employers and government DBs. The Open Stack is bound to provide all the tools to create web-wide protocols and platforms that provide just that, and Aidentiti positions itself there. If you don’t know much about these concepts and would like to know more, start with Chris Messina’s blog on anything about the open social web and surf on.

I write not to sell so much as to share and collect your thoughts. Activity Streams are a Great Thing and certainly the Aidentiti project can contribute. I can’t yet announce when Aidentiti’s code will be released so consider this an overview and opportune teaser.

Thanks to Ruben for contributing his skills to the project and releasing the Rubygem to the world as his “Hello, Aidentiti world”.

Many bots


Computer bot

All these APIs for all these excellent vertical web services. How about connecting all the bits of cloud you need and having all that information as accessible as an IM buddy?

Enter Computer bot.

It’s a framework that Ruby developers can easily extend and adapt to create IM bots that harness the APIs of the web services they rely on, for work and play.

You don’t have to open web sites and click your way through the information, you can type your commands and have it delivered to you.

I named it “Computer” after Star Treck: the way they consult the computer, and how it seems to be connected to everything.

It’s available on github and the actual version has some cool features, with the help of Ruben Fonseca.

The big picture for future development is clearly about moving beyond the bot realm and creating reliable deep-integration tools: a data hub for the cloud that you access via an API: locally, to generate rich views and apps on the server side and remotely, via XMPP, REST, etc., to render rich views and apps on the client side.

If you already fool around with Yahoo Pipes and Tarpipe, know about Microsoft’s Live Mesh and “the cloud”, I hope your eyes are sparkling.

Digg and Twitter modules are built-in at this moment.

Have fun with it, send your patches and your feedback.


Twitter edition

Are you looking for an IM Twitter client? Computer bot / Twitter edition is a full-featured Twitter bot that you might love:

  • Live twitter feed
  • Track and untrack keywords
  • Grab the latest X messages from your friends
  • Grab the latest X messages from any user
  • Search the latest X containing a given word
  • The classics: follow, unfollow, whois, post updates
  • More :)

You can download it from github. It’s pretty straight forward to setup, Ruby beginners can give it a try. It’s based on an early version of the Computer bot framework.


Prologuer bot

Last but not least, I have just pushed the latest Prologuer bot to github.

Prologuer bot is a bot for Wordpress, meant to be used in a “Wordpress-based twitter-like environement”. It was first developed right after the release of Prologue, a theme made by the Automattic team to power their own Wordpress-based twitter earlier this year.

During the The Star Tracker development sprint in November I’d added a live feed feature to the bot, extracted from Computer bot / Twitter edition. And since these guys started using the 0.2 version along a “Wordpress-based internal Twitter” setup for their company, it’s a good opportunity to push the new version.

With Prologuer bot, you can grab the latest posts from a given person or tagged with a given keyword, post to Wordpress, subscribe to the live feed and… little more at this point.

Prologuer bot wants to be a great productivity tool: replace some email with Twitter-like interaction, and make it even more convenient by making it available via IM. It’s really about real-life usage. If you use it with your team, do send your feedback.


Personal information systems

Many more bots I have written this year, like the “bigbrother bot” (link in portuguese) at Codebits that let you find people through the RFID tag infrastructure and the API. Data at your fingertips is a powerful idea, one that I’ll certainly pursue into 2009.

The more information I round up, the more this subject makes me think of information systems, and how they’re going personal. Social, and personal.

Nowadays services are designed for social scale and interaction not unlike like enterprise software is designed for enterprise scale and interaction; I say ‘not unlike’ because they’re merging. We almost expect SLA’s from social services and we expect corporate intranets to turn into facebooks, and yesterday’s company knowledge bases to become wikipedias. Application design is challeging organizational architecture (versus vice-versa).

Beyond the “social enterprise”, there is “user-centricity”. It’s not a new idea. Nor are social networks, nor were intranets; it’s a new timeframe, where old pieces form a new puzzle. User centricity rides on top of the social- and process- oriented paradigms in organizational software.

How powerful does user-centric software, designed to absorb social and organizational culture in the through their APIs, sound to you?

Quick how-to – tarpipe Developer Preview

tarpipe just launched a developer preview. Its aim is to get tarpipe out there for developers to try, test and, in the process, help detect bugs and gather early feedback. What’s tarpipe? From the tarpipe website:

tarpipe is a publishing mediation and distribution platform that simplifies regular upload activities:

* Publishing content to multiple Web locations;
* Combining different media into a single blog post or article;
* Transforming documents on-the-fly;
* Managing repeatable upload actions.

I’ve been playing around with it and it’s very cool, and promising. Since the service is alpha-ish and there’s some help content missing, I’m writing this simple how-to to guide you through the very first steps, from requesting an invite to making your first workflow.

I’ll quickly show you how to create a workflow that will post an email’s subject to Twitter, since it’s both simple and working. You’ll therefore need a Twitter account.

10 quick steps

1) To take part in the developer preview, you can request an invite.

2) Once you receive your invite code by email, you must sign-up. NB: OpenID is required.

signup - tarpipe

3) Signed-up and ready to go? Read the Developers page to have a general idea of what you can do.

4) Now, let’s create your first workflow. First, click on the accounts tab and type-in your Twitter credentials.

accounts - tarpipe

5) Now go to the workflows tab and create a new one. This is what you’ll see:

workflows - tarpipe

workflows - tarpipe

  • On the upper left hand corner, the workflow’s properties: title, description and receptor. There are two available receptors: REST API, to which you can make HTTP requests, and the Mail drop box, to which you can send emails.
  • On the bottom left hand corner, the connectors. They are the data manipulation tools that you hook together to create your workflow. Right now, there are only half a dozen connectors available but a lot more are coming.
  • The right hand part of the window is what I’ll call the “canvas”. This is where you’ll create the workflow by arranging and linking connectors.

6) To the chase: Choose a title and a description, select the “Mail drop box” receptor and click “save workflow”. Notice that an email address has appeared. This is where you’ll send your email.

workflows - tarpipe

7) Now click on the MailDecoder connector and a box will show up in the canvas. The 4 fields are pretty self-explanatory, but I’ll come back to it later.

workflows - tarpipe

8 ) Click on the TwitterUpdater connector. Drag the box to the right of the MailDecoder. There are 3 fields, of which only the first (“title”) is of concern to us for now.

workflows - tarpipe

9) Now, the very hard part (not!). On the MailDecoder connector, click on the little white/transparent ball in front of the “subject” field and drag your mouse to the “title” field on the TwitterUpdater connector. Hit save. You’re done.

workflows - tarpipe

10) Now copy the email address to your clipboard and compose a new email with your favorite email client. Paste the email address in the “To:” field and type “Trying out tarpipe, this is my first workflow!” on the “Subject:” field. Hit send!

11) To make sure it all worked, check out the activity tab in tarpipe, and of course you Twitter account. All done.

activity - tarpipe

Voilà

Now go ahead and play with the other connectors, imagine what applications you could develop on top of tarpipe, and of course what workflows would make your life easier. There are already two libs you can use to develop your own app: one in Perl and one in Python. The tarpipe API guide is available here.

Update: Libs for PHP and ruby are now available as well.

I’m really impressed with tarpipe and I’m sure it’ll draw a lot of attention very soon. Some very amazing apps can be written on top of it and it can solve a lot of problems for web power users. Hope this post was a helpful apetizer.

Now go play, report bugs and have fun ;)

Learn with hashtags

Learn with hashtags

Hashtags are controversial: not necessary and/or irritating for some, useful for others. I find them useful for three reasons: they simplify machine treatment, they reduce noise and, most importantly perhaps, they teach humans how to interact with machines.

Hashtags simplify machine treatment

Hashtags simplify machine treatment because they make tag detection, or keyword detection, straightforward. Keywords are unambiguous, from a machine’s point of view, if they’re explicitly identified as such. The alternative solution to detecting keywords in a message is to analyse the text, which involves complex, more expensive, coding and architecture. Furthermore, results aren’t really accurate. A same word can be used in different messages and only sometimes act as a keyword or tag. The context in which the word is used is what helps determine it, and context is precisely what it is tough for machines to figure out. Google’s billion-dollar operation is great at doing it, for example. But Twitter’s tracking feature, for example, isn’t.

Hashtags reduce noise (and increase signal)

In Twitterland, if I choose to track, say, “Microformats”, I’ll be alerted when “randomtwitterdude” tweets that he’s reading the microformats wiki. If I use Twemes.com or Hashtags.org, or any other, I’ll be alerted when someone decides to broadcast to people interested in Microformats (or simply archive what he’s saying in context), because these sites track hashtags, not words. I can safely say that I don’t really care about randomtwitterdude’s incursion to the microformats wiki, and that I do care when people use the #microformats hashtag. Information is put in a context, can be stocked in a library, can be delivered to me the way I choose and I can easily access it later. Randomtwitterdude probably didn’t even realise that he was broadcasting his message to others than his followers. By using hashtags, you explicitly declare your intention: machines, please act upon my message. Logically, you help reduce noise, even increase the signal.

Hashtags teach humans how to interact with machines

So we get to my last point: humans learn how to talk to machines using hashtags. Using a specific vocabulary, a # in this case, people know they’ll trigger a set of actions, enabling more uses from a simple message. This has great potential, since it means you can actually send a message to a computer, written in a slightly extended human language, and the computer will act on it. It carries a somewhat different cognitive meaning: don’t click, speak. Don’t be intimated by the buttons, tell this thing what it should do.

Hashtags are a nanoformat; there are other nanoformats, who tell machines other stuff: instead of a tag, you can explicitly mention a date, a location, a person, etc. So, while the computers that are great at speaking human languages are still expensive, relatively unreliable and dreadful to use (e.g talking to a robot on the phone), hashtags and other nanoformats are efficient enablers of more powerful human-computer interaction: inexpensive, accurate and precursors to what’s coming next.

Prologuer Bot v0.2: Cool new features

Prologuer v0.2 is a Jabber/XMPP companion bot for WordPress, meant to be used specifically with the Prologue theme.

It’s written in Ruby using Jabber::Bot, which is built on top of Jabber::Simple, itself built on top of xmpp4r. Prologuer requires a “read” access to a WordPress database to retrieve messages and uses WordPress’ XML-RPC API to post messages. These are configurable with an YAML file (as well as the bot’s Jabber/XMPP credentials).

Prologuer is a working prototype that allows you to post (push) and retrieve (pull) messages from WordPress. It is not a chat bot, since it will not automatically deliver new messages posted by others to you, and is not to be considered production-ready.

Once configured and launched, Prologuer comes online and is made available to all your blog’s registered users that have set their Jabber ID in their profile, excluding users with “subscriber” status, (i.e. with no ‘wp_user_level’ assigned to their ‘user_id’ in ‘the wp_usermeta’ table).

Prologuer speaks 2 Twitter nanoformats: #hashtags and @usernames. Users are able to post and retrieve messages using a particular syntax. Here are two examples:

post Getting ready for the #prologuer release => posts the message and tags it with “prologuer”.

q @alex #prologuer => returns a maximum of 5 messages posted by “alex” (where “alex” is a WordPress username) and tagged with “prologuer”

I’ve created a Prologuer Bot How To as a Flickr Set with 9 screenshots; they show the bot’s full feature-set, which is pretty simple for the moment.

Prologuer can easily be refactored or modified to work with systems other than WordPress, and I’m looking forward to doing just that. If you’re interested in helping develop more features and enhancing the existing (and end up with another name), please contact me.

Download Prologuer v0.2. Read the README contained in the zip file for instructions.

I’m not a trained nor a professional developer, but I need to implement an idea and Prologuer is part of it.

The Prologuer Bot

UPDATE: Prologuer v0.2 is out, with cool new features.

So, I got pretty excited when Matt Mullenweg introduced Prologue, a Twitter-inspired theme for Wordpress. It got even better when the theme was updated a little (from community requests), as to bear more resemblance with the Twitter experience.

I’ve been playing with Twitter-likes for a while (Noumba while I worked at af83, there’s this idea, coded one from scratch a few months ago, recently made a little toy I hope to publish soon enough, now this), exploring other ways to use the “micro-blogging / post-from-anywhere / push-and-pull equilibrium / social network” thing.

So I ran to write a bot that would make use of the oh-so-beautifully-simple Prologue interface combined with its WordPress goodness (read “available XML-RPC, tagging, easy to deploy, includes an admin interface, out-of-the-box!”) and give me the Twitter confort of having a easily accessible interface from where to post information.

Using Ruby’s XML-RPC client, the “made in Twitter” xmpp4r-simple gem and WP’s XML-RPC support of tags (through “mt_keywords”) it’s a piece of cake. You can even use daemons and have a start|stop|restart interface. Yawn! (almost).

What the Prologuer Bot does

Once you’ve added it to your buddy list, it’ll sit there waiting for you to tell him things. As soon as you do, it parses your message, scans it for hashtags and publishes it to your Prologue URL, with the hashtags as tags (works with more than one hashtag).

Prologuer Bot in action (1 of 2)

Prologuer Bot in action (2 of 2)

How it does it

If you’re looking for an IM (Jabber) bot that will post to Wordpress’s Prologue, you can start here.

Warning: The following code should not be used as is! It’s just an idea, and it works as is, but!! It lacks many a security feature, it won’t degrade gracefully at all, etc. I’m giving no further details as of how it works because if you don’t know it by reading the code you should consult all the links above. I’m a lousy dev. You’re warned.

Request:Please let me know of all things that could be enhanced (regex, logic, rescues, etc.)! Thanks

The code:

You can download it here: prologuer-01.rb.zip. Notice that the regexp is pretty dumb and you should adapt it to your own needs.

## Prologuer.rb
# This simple code, meant to be used with WP's Prologue theme, creates a Jabber bot that will
# 1) publish received messages to your WP account as new posts
# 2) detect hastags (e.g #tag) and tag your new post with them
# It's like having your own Twitter bot for your own Twitter
# you need the xmpp4r-simple gem (sudo gem install xmpp4r-simple --include-dependencies)
# you need a Jabber account and a WordPress install with the Prologue theme
##
 
require 'rubygems'
require 'xmlrpc/client'
require 'xmpp4r-simple'
 
## Wordpress user config
@wpuser = ''                                   #your WP username
@wppass = ''                                    #your WP password
@wpRPCurl = '' #example = 'http://yoursite.com/wordpress/xmlrpc.php
 
## Jabber user config
@jabuser = '' # Your Jabber username: 'you@jabber.org', 'you@gmail.com', etc.
@jabpass = ''           # your Jabber account password
 
 
##
@client = Jabber::Simple.new(@jabuser, @jabpass)
@server = XMLRPC::Client.new2(@wpRPCurl)
 
 
def get_messages
   @client.received_messages.each do |msg|
     desc = msg.body
     from = msg.from.strip.to_s
 
     wptags = Array.new
     hashtags = desc.scan(/#[a-zA-Z]+/i)
 
     hashtags.each do |w|
       b = w.to_s.split(/#/)
       c = b[1].to_s.downcase
       wptags.push c
     end
 
 
     newPost = {}
     newPost['description'] = desc.to_s
     newPost['mt_keywords'] = wptags.join(', ').to_s
 
     @server.call('metaWeblog.newPost','unused_blogid',@wpuser,@wppass,newPost,true)
     @client.deliver(from, 'OK')
   end
 rescue
end
 
loop do
  get_messages
  sleep(5)
end

OK, have fun!

A better world in 2008

The Davos Question

As you probably know, The Davos Question is a Youtube initiative where everyone’s invited to say, in one minute, what they think is “the one thing that will help make the world a better place in 2008”.

This week, adding up to the thousands of individuals who posted their answers on Youtube, many world leaders have stopped by the Youtube booth at the World Economic Forum to share their thought on the subject. Here are some examples, compiled for the diversity of answers. This clearly means that challenges are multiple, and that it’s up to everyone to take responsibility. Climate change is but one of the challenges; a recurring answer is, of course, education. I note the absence of the word “solidarity”; it seems to have been replaced by “individual responsibility” and “corporate citizenship”.

  • Josette Sheeran, head of the UN World Food Program talks of hunger and children and sets an objective and a program: that no child on earth goes to school hungry. “Fill the red cup, please!”. In another video, she speaks of helping small farmers as a way to fill the red cup.
  • Shimon Peres and Michael Chertoff (US Secretary of Homeland Security) both speak of the urgency to stop terrorism. Also about climate change, freedom of movement.
  • Amr Moussa, Secretary General of the League of Arab States, responds to Shimon Peres.

There is a total of 48 videos of leaders online, and thousands from individuals like you and me. In the leaders list, like in my list, women are seriously misrepresented.

In a tentative to sum up:

Respect Life. Enjoy it, admire it, understand it, preserve it, pass it on.

Smile and share, will ya! :)

Mini Big Bada Boom

A quick follow-up on a previous post, where I rumbled for better advertising with DiSo.

Via Fred Cavazza, I’ve just come across an actual example of the kind of real-life interaction advertising you could embed in blogs.

Here it is:

This one is done in Flash, but you can do it in XHTML and Javascript, microformat the content, make it portable, deliver it to people who want to see it. The latter of course is technology-agnostic.

Very promising stuff, Fred Cavazza calls it revolutionary.

Blogmesite, maker of the embedded mini-site, is apparently a French online communication company that specializes in entertainment. As a side note, I wonder how this will disrupt their business, the “blogmesite.com” URL redirects to their site.

Now expecting a mini big bada boom.

Entreprise 2.0

Il se fait un peu tard pour exprimer ce genre d’avis puisque c’est une discussion qui n’est pas née hier, mais suite à ce post de Fred Cavazza, j’ai eu envie de commenter (puis mieux vaut tard…). Et, pour référence personnelle future, je duplique mon commentaire ici. Cela servira de continuation à mon post quelque peu désarticulé sur le même sujet, et c’est un raisonnable résumé de mon point de vue sur la question.

999) les humains

Je pense que l’entreprise 2.0 existera et que, oui, elle tirera beaucoup du web 2.0.

L’ERP fait communiquer les processus et c’est certain que les forums, GEDs et sites intranet à l’ère du web 1.0 étaient déjà des efforts visant à mieux faire communiquer les individus.

L’opportunité claire du web 2.0 pour l’entreprise est, pour moi en tant que concepteur et utilisateur, de faire communiquer à la fois les processus et les individus en les mêlant de manière plus intime.

“APIs, Mashups and Conversations” = “ERP, Middleware and Data Mining”, grosso modo.

L’entreprise 2.0 c’est cela: un meilleur alignement entre les processus humains (sociaux et cognitifs, disons) qui ont toujours existé dans l’entreprise, et industriels (inputs, traitement, outputs), qui ont toujours existé chez le professionnel, doublé d’un meilleur alignement de leurs objectifs de développement (meilleure productivité, meilleure qualité, meilleur rendement). Cela passe notamment par une réduction de la friction provoquée par la multitude de logiciels déployés, grâce, par exemple, à l’amélioration des interfaces, à la fois graphiques et systémiques. Les outils similaires historiques sont bien là, mais il s’agit du renouvellement des métaphores (simplement et ambitieusement).

Quant à savoir si on a trouvé le ton juste, je ne pense pas. Encore. Le concept n’est pas clair pour tous et les outils tâtonnent encore. Mais nous nous en approchons et c’est clairement, me semble-t-il, la direction à suivre, puisque c’est celle que prennent les petites entreprises globales et c’est à ce modèle (de flexibilité, globalité, de liberté d’action et d’initiative entrepreneuriale) que nous aspirons en ce 21ème siècle, en tant qu’humains et organisations.

Comme promis en conclusion de mon autre post, il reste à discuter du design (point 666), donc de la vision de l’outil.

Want my data? State your business!

Techcrunch has a ‘maybe’ story on Plaxo being for sale.

For the last year, all we hear from Plaxo is adoption of open standards. Advocacy of open standards even.

They have a practical and useful service, although there’s a lot of room for improvement. So I went on and trusted them with all my contact and calendar information, as with my feeds and other profile information. And encouraged people I know, people who trust me, to do the same.

I felt really outraged that a company that, in such a short period of time, could win your trust with saying and doing good things (OpenID, Microformats, conversation with the community, OpenSocial, etc.) and then could go on and produce such a scary trick. Good news is Plaxo has this post about their position on the Bill of Rights for Users of the Social Web.

So there’s hope that the people buying Plaxo will respect this. I’m still wary and with reason. Why would they put themselves for sale with my data as an asset? And like themselves say, they are a true exception on this matter. What’s happening with my data in all the social silos? I have accepted Terms that state that I’m actually offering it (Facebook is but an example). But really, shouldn’t we be raising our expectations?

We should. This year, 2008, will be the year where we seriously tackle open data. This I demand: that not only I own what’s rightfully mine, but also that I know what’s happening with my data inside your silo and what will happen to it if you sell your assets to someone else. By the end of the year, if I don’t have answers to these questions, you can keep whatever I gave you but you’ll hear of me no more. Because by the end of the year we’ll have some serious alternatives to what you’re offering. Customers will be Kings, period. Want my data? State your business!

Happy new year.

Update: Robert Scoble was banned from Facebook and is joining the dataportability.org project. It seems that he was running a script, which is forbidden, aimed at sucking back all his social graph data. Facebook’s warning email is funny: delete that info that belongs to us or suffer.

Update #2: Scoble revealed what he was up to on Facebook: he was testing a new feature on Plaxo Pulse along with other bloggers and journalists (one of which he seems to suspect of alerting Facebook, which puts into perspective the automated email he received). Michael Arrington pretty much patronizes the whole story (although he did co-author the above mentioned Bill of Rights) and Scoble, reinstated in FB, answers back to criticism. The open data subject is hotter and more attention is given to dataportability.org. Moral: fight for your rights, helps to be the under the limelight? Interesting 2008 kickoff.

Update #3: Still don't know more about Plaxo being for sale. Their move with Facebook is certainly bold. The Techcrunch story followed by the editor's opinion are interesting facts. Waiting for statements on these, this all reinforces my opinion, here expressed.

Update #4, January 8th, 08: Facebook, Plaxo and Google join the dataportability.org effort :)