49 articles and counting

PyHawk – Hawk for Python

In my last post “Proposal for Securing Mozilla Web Services with Hawk”, I recommended using Hawk.

I know what your saying…

“Okay, armchair architect, I’ve read Hawk’s README, it sounds great, but there is no Django or Python port.”

PyHawk is a new Python port of the Node.js Hawk codebase.

It’s here today and seems to work. It is interoperable with the latest version of Hawk (0.13).

The following API is TBD, because I want to work with you to make it more elegant, but here is a the current usage…

Clients

A website can consume web services that are locked down with Hawk as follows:

Server

Similarly, it is fairly easy to lock down your web service APIs.

Conclusion

Hawk is pretty darn simple, right? Be sure to read the original project for caveats and best usage.

So this homely API works, but I need your help:

  • My Python is rusty, help make the code beautiful through Issues or Pull Requests
  • I need you to add it to your Python pet projects and kick the tires
  • Let’s make the API elegant
  • More/better unit tests
  • Read the docs, Travis, and other polishing

Who’s in? Let’s play in the PyHawk repo.

Proposal for Securing Mozilla Web Services with Hawk

Webdev, as well as many other teams at Mozilla, create awesome web services. I think we need to be creating even more services, so that we can isolate certain responsibilities and reuse them throughout our community and internal tools.

One missing piece is a consistent way to secure our web services.

Let’s look at the Mozillians list users API. It was written by Giorgos Logiotatidis, who is awesome and has been kicking all sorts of ass on the Mozillians codebase.

This API allows you to lock down your website based on if someone is a vouched Mozillian. You can check to see if they are in the group “staff”, etc. Awesome stuff!

How does one get to use this API? How is it secured? The API uses a very simple authentication scheme.

  1. You register an API key for your application
  2. You send your App Name and App Key in your API call
  3. You make your web service calls over HTTPS

Essentially, we’re sending the password over SSL. We can do better!

I try not to promote Golden Hammers, but Eran Hammer’s new protocol seems quite appropriate.

Eran has created an Authentication system called Hawk, that is exactly built for this use case. Erin lead the OAuth specification and is now developing a couple new authentication systems. Hawk and (the still under development) Oz.

Ben Adida suggested using Hawk in a recent bug related to PeterB‘s MozLDAP web service.

So, what is Hawk?

According to the README, Hawk is an HTTP authentication scheme using a message authentication code (MAC) algorithm to provide partial HTTP request cryptographic verification.

So Hawk is a standard way for clients to sign their requests and verify that the response is from the intended server. It allows servers a standard way to verify a request is authentic and to digitally sign it’s response.

Much like the Mozillians API, there is an App ID and a App Secret, but the secret is never sent over the wire. This makes Hawk much better than Basic Digest or an ad hoc method where the secret is exposed.

What’s this partial in “partial HTTP request cryptographic verification” business? Only certain aspects of the HTTP request / response are used in the signing process; making it compatible with HTTP proxies and the real world internet.

The cryptographic verification includes the HTTP method, request URI, host, and optionally the request payload.

Signing involves calculating a request MAC value. There are libraries that hide the complexities of signing and verifying requests.

In addition to not sending the secret, there are more defense in depth features we can use:

  • Timestamp – expires requests
  • Nonce – mitigates replay attacks

This all sounds rather complicated, but good library support makes using Hawk easy. In my next post, I’ll show examples.

At first blush, Hawk seems like a good system to standardize Mozilla web services on.

What do you think?

  • Do you think we should standardize API authentication?
  • Do you have a different golden hammer?

Taxiderm – Preview EJS Templates directly

Sometimes you have to build something, to find out if it already exist…

Last Friday I created Taxiderm.

Watch the demo…

Taxiderm Freaky Friday from on Vimeo.

The elevator pitch? Taxiderm takes the skins from your frontend and pumps them full of sawdust. Creates life-like webapps which can easily be:

  • worked on for CSS tweaks in hard to reach places
  • share links with localization community to preview their translations

Of course this idea could be generalized to any templating language, jinja2 using nunjucks, mustache templates, etc.

Does a tool like this already exist for Designers to preview their CSS without setting up a full application stack? That creates a catalog of links for all the various application states?

iwantmyname and github pages sitting in a tree, K I S S I N G

If you need to throw up a static website with lightening speed… Here is one method.

I forget these steps, and if you do them wrong… it’s a PITA to fix, but if you do them right, you have a working domain in a couple minutes.

So here is the recipe:

  1. Register a domain with iwantmyname.com
  2. Create a github repo
  3. In Settings click Automatic Page generator
  4. Clone the repo and git checkout gh-pages
  5. Add a file CNAME which contains the domain name you’ve registered
  6. Commit and git push origin gh-pages
  7. In the iwantmyname.com dashboard, find your new domain
  8. Click Install Application and find Github
  9. In the textbox, enter {username}.github.com and submit
  10. In a browser go to https://{username}.github.com/{repository-name}
  11. If it’s a 404, hit refresh after a minute or two
  12. Eventually, you will see the project page github automatically generated
  13. Try your custom domain
  14. I usually screw up step #9 by putting the project name in that input field. Don’t do that… Put your user name there.

    Iwantmyname.com will setup your DNS and give you a proper A record to github.com.

    The CNAME file allows github to do proper virtualhost resolution and serve up your site.

    DNS can take a few minutes… and then you should be in business.

    You can delete all the HTML, CSS, etc in the gh-pages branch and build your static app. After you commit and push to the gh-pages, your website will be updated in a few minutes.

    This is great for prototypes and static websites that don’t change often, usually promoting some code you’ve written and host on github.

    Have fun!

Self-Hosting my Persona Identity with HostedPersona.me

Inspired by Indie Web Camp as well as the Unhosted project, I now host my own Persona Identity. I believe that Persona protocol will be an important building block for the Federated Social Web.

What is Persona?

Persona is a the decentralized authentication method which is a competitor to Facebook Connect, Twitter sign-in, etc. It’s being built by Mozilla, but works out of the box across devices and browsers.

Self-hosted?

Persona hopes to get email providers, universities, companies, and other entities to implement the BrowserID protocol. I’ve implemented this for myself, so any email address on the ozten.com domain will use my service for authentication. All ozten.com users would basically include me and my cat :)

What does it look like?

In this video, I’m logging into The Times Crossword page via Persona using my own service. It shows my avatar and I can type in my password. Avatars are provided by Libravatar and it’s the same avatar I use across the web. Sensing a theme here?

How does Persona and my self-hosted service look across the web? In the next video I log in to my self-hosted blog’s admin page, again to the Times Crossword, and a several other sites… Notice I don’t have to re-enter my password. I can just pick an active identity and go.

On the 3rd site, I click "this is not me" to show my hostedpersona log in screen again. Then I log in to several other sites with various identities. For example on Bugzilla I use a gmail account to manage bugs, so shout@ozten.com wouldn’t be a good identity there.

Wow… so easy to reuse my self hosted identity across all these sites. Once I’m logged in, the Persona UX manages my session, so I’m not bugged for a password.

A Challenge

I challenge the organizers of next year’s IndieWebCamp:
Offer Persona Log In as a way to authenticate to http://indiewebcamp.com
(I’m happy to help)

Wanna own your own Persona?

Definitely weigh the options… self-hosting an important identity (I’ve used shout@ozten.com for 15 years) is kind of dangerous (especially for me as Mozilla web properties use it) and will become more risky as Persona gains adoption across the web.

Wanna hack without too much risk? Register a new domain name instead of using your actual personal domain. Okay, enough parenting…

Requirements

So you want to run your own service… what is required?

  • SSL Certificate

  • Crypto Code
  • Dynamic server side code for provisioning and authentication

To get an SSL cert I needed a static IP. Getting a static IP was 1 click and 3.95 per month.

Getting an SSL cert was one click, 12 hours, and 15 bucks a year from my website hosting provider. You can do it cheaper and better, but I wanted to see what this would be like for novice netizens. (Okay, and I’m lazy)

My personal website is mostly PHP and I didn’t want to re-invent the Persona crypto wheel. I’ve been writing more Node code lately, so I decided to:

  1. Write a new Node based service
  2. Host it on ec2
  3. Reuse Mozilla’s browserid-certifier server to do all the crypto

The BrowserID protocol enables a domain to delegate authority to another domain. I had ozten.com delegate authority to hostedpersona.me.

Starting projects at 4:30am can lead to lame names… which is how I registered hostedpersona.me ;) I picked this name as I might host friends’ Identities too, if they are dumb enough to trust them to me, since this is only a side-project.

Level of Difficulty

Implementing an Identity Provider service is much harder than adding Persona Log In to your website. If you haven’t played with that… do that first!

You are definitely better off trusting Mozilla or your email provider to manage your account’s security. Just say’n.

If you do want to do this… I think it’s about 10x as hard as adding Persona Log In to a website. Be sure to check out these tips.

Conclusion

So there you go, Persona has allowed me to have an easy to use, portable, self-hosted Identity on the web. I think Persona is going to be a key ingredient to the federated social web.

Next Steps

4 More Years

Oct 6th marked my 4th year with Mozilla!

I, like Obama, have gained much experience and grey hair in these last thousand+ days. It is still an honor to be an employee and I’m really excited about the work our Identity team is doing.

I was reflecting back and I can remember 20+ projects I’ve help with over these years. I’ve very proud of most of these projects, except for the ones that still plague my nightmares.

When I started there were 150 people and I’d visit building K during onsites.

Some things have changed; I would never have imagined we’d be building a phone! Our respect for web properties, web services, and mobile devices has completely shifted.

I’m still really passionate about Mozilla’s mission and consider myself lucky that Bret and morgamic took me from community lurker to active participant.

Onward! and 4 More Years, for Obama and maybe me too ;)

IUSEA and U Should too’a

I want to share a new architectural pattern for web applications and services that Mozilla has been using to empower the user and put them back in control of their data and their web.

An Example

I use’a the Gmail,
I use’a the LinkedIn,
Why they no’a work’a together? — Mario

Well Mario, if Google and LinkedIn used the IUSEA pattern, they would!

  1. Mario’s browser requests https://gmail.com/email/compose
  2. That page includes a script tag to https://linkedin.com/contacts_include.js
  3. Mario starts typing ‘Al’ in the To: field
  4. Gmail’s JavaScript calls LinkedIn.autocompleteContact('Al')
  5. If Mario’s session with LinkedIn.com has timed out, he is prompted to login in a new window. This is the same login screen as he sees everyday
  6. LinkedIn asks Mario if he wants to expose his contacts
  7. Mario clicks yes
  8. Gmail receives rich contact data for Al and Alice, including a picture, degrees of seperation from Mario, etc
  9. Gmail uses this rich data to improve their contact picker
  10. Mario clicks Alice and finishes writing his email

IUSEAIntentional User Service Exposure Architecture

If your building a web application or service, you should continue to provide REST APIs for public data, but user data should be exposed via IUSEA where possible.

Technical Details

LinkedIn’s contacts_include.js defined a autocompleteContact function. How is it implemented?

It opens a hidden iframe to https://linkedin.com and uses a postMessage based protocol to ask for contacts that have a first, last name, or email that start with ‘Al’.

LinkedIn.com hosts a page that is the iframe’s target. It authenticates the user, makes sure the user wants to expose this information, and delivers the data back Gmail via postMessage.

This pattern is ready today and works across Browsers, OS, Desktop, and Mobile. Mozilla’s jschannels library is an easy way to rock this, even on IE.

Isn’t this just OAuth?

As a developer, your thinking “Congrats, you’ve re-invented the OAuth wheel.”

OAuth is about server to server data sharing:

Value in this diagram means value for the user and the service provider, such as a user’s contacts.

IUSEA puts the user back into the center. Value resides in each server but also is controlled by the user and flows through the user at their discretion.

Business Details

Did you notice that all happened in the browser? Technically, Gmail doesn’t need to upload this new LinkedIn data to their servers, since all the user benefit was provided directly in the Gmail web UI.

Going one step further, LinkedIn could provide this API with a TOS that forbids Gmail from importing this user data… wow.

Google can fulfill it’s mission to to organize the world’s information and make it universally accessible and useful. LinkedIn can fulfill it’s mission of connecting the world’s professionals to make them more productive and successful. The user wins as they gain more control of their data, while being able to reap the benefits across the web.

IUSEA (a term I made up to help explain this concept) is the secret sauce we’re applying liberally, and you can start using this pattern in your products.

IUSEA and U should too’a.

Disclaimers, Image Credits

How the Node Based BrowserID Service Shipped 28 New Languages

Update: Ukraine translation of this article.

Our amazing community has localized the BrowserID service into 28 languages.

browserid_zh-TW

If you are a website who is using navigator.id to authenticate your users, you
don’t have to do anything special
, your users will start having a better experience… today.

Woah, that just happened.

I wanted to share a little bit about the people and the technology.

People

Mathjazz did a great job leading the l10n effort, with pike and milos also involved.

I’ve helped with 7 “from scratch” localizations at Mozilla and this was the smoothest yet. The l10n drivers have done a ton of work to make collaborating on websites and services better.

It’s not an easy task to help coordinate volunteer efforts. You can see everyone who is involved and what locales new locales are coming soon; check out our Verbatim.

browserid_el

Technology

Node.js LogoWith our launch, we’re probably unique in being a Node app at our scale with the breadth of languages supported. We use the express framework;
doing l10n on Node is totally possible ;)

Under the guidance of l10n drivers, we’ve reused all of our standard web l10n practices. That means
Gettext for strings and SVN for the locale directory (let us not speak of SVN again).

We used the following libraries and tools:

  • node-gettext – server side string lookup from .mo files
  • GNU Gettext – string extraction, merging, etc
  • Translate Toolkit – podebug for debug locale

We wrote the following bits:

  • Client side gettext library
  • Custom middleware for accept language, exposing gettext, etc
  • Various conversion scripts

We do string lookups on both the server and the client side. In the browser, JSON blobs are minified and concatenated into our usual JavaScript bundle.

Having excellent Django codebases like zamboni (AMO) and kitsune (SUMO) to reference was also really helpful.

We ran into a shocking low number of i18n bugs, given that Express and Node don’t have an officially blessed stack to do l10n on.

A Call for Help

This release also brought in many new contributors to the project porting Perl code, reporting bugs, and other new energy that is critical to the Mozilla project.

As with any release, there are things we didn’t have enough people to accomplish. We need your help!

  • CSS Help – Help support the visual design for RTL locales like Hebrew?
  • Tools Help – More robust string extraction

Thanks again to everyone involved in this release. If you are waiting to try integrating BrowserID “because it only works in English”, you’re running out of excuses :)

Go migrate your site’s authentication today. Get your users off of passwords. With an additional 14 locales in the works, tomorrow promises to bring even friendlier auth flows.

Ukraine translation by Android SDK, thanks!

Demystifying BrowserID and Mozilla’s LDAP backed webapps

IT, Operations, Security and Webdev have prepared a BrowserID adoption plan for our LDAP directory. Questions come up frequently, so I wanted to capture the basics of the plan.

In terms of BrowserID adoption, there are three classes of Mozilla websites.

  1. Many webapps that use MoCo’s LDAP instance for authentication and group permissions
  2. Mozillians.org which uses a new community LDAP instance for auth and groups
  3. Many more webapps that manage their own authentication (via MySQL or whatever)

Number 3 has nothing to do with LDAP, so we can go dogfood BrowserID on a case by case basis.

Classes 1 and 2 depend on LDAP and what that means is fairly confusing, so I wanted to clarify our current plan and where we are at.

The Cast

We all know the the Frog Prince story.

BrowserID is a lonely Princess as well as a protocol that gives the person currently using your webapp a way to disclose a verified email address.

LDAP (aka slapd) is an unloved frog that moonlights as a backend database; it authenticates people and keeps track of who is in what group. It’s got some warts, but don’t be so superficial.

sasl-browserid is the magical golden ball. Huh? Ya, blame the Brother’s Grimm.

When our princess drops the golden ball into the frog’s pond, poof, the frog transmogrifies into the princess’ long sought after life-partner. And they live happily ever after…

Okay, sasl-browserid is more of a C plugin than a mystical sphere, but whatever. This plugin BrowserID enables LDAP.

Technical Details

This will be short and painful, I promise.

Before sasl-browserid

  • You enter your email address and password into a form
  • Middleware code does a simple bind with a shared account
  • Code does a search filtering by your email address to find your DN
  • Code does a second simple bind using your DN and password, attempting to authenticate on your behalf:
    • You are a known user with a good password
    • Or not so much
  • More stuff happens.

simple bind, search, filter and DN are LDAP terms.

After sasl-browserid

  • You click a link or button and do the BrowserID dance. Frontend code gets a BrowserID assertion and sends it to middleware.
  • Middleware code does a sasl interactive bind with your assertion and audience
  • Code uses ldap’s whoami* and checks the output
    • You are a known user in the system
    • You are an unknown user in the system
  • More stuff happens

sasl interactive bind and whoami are also LDAP terms.

If that didn’t make your eyes glaze over, there is more gnarly details and an architectural diagram here.

What changed between before and after BrowserID? The authentication mechanism. What usually doesn’t change? The authorization and business logic of an application. Both sections have a "More stuff happens" section which means the goodness your app does and the permissions your users have, usually just continue to work.

We do not have to re-write our internal apps to take advantage of BrowserID.

Deployment Plan

sasl-browserid is getting a thorough review from the platform security team. In particular, David Chan has been doing a kick-ass job of finding flaws and ways to improve it.

Once it is ready, we’ll deploy it to development stage servers (next week or two).

But which app should we start with?

Our MoCo LDAP powers many, many Mozilla tools. This is beyond just web applications and bleeds into Desktop, command-line, and other uses; Reed’s biological functions are literally regulated by MoCo LDAP. It’s got both critical and sensitive information, so we have to be careful, but luckily we’ve also resently launched…

Mozillians.org also has a LDAP server backend. It has a lower risk profile, so we’ll deploy to it first. It’s lower risk for a bunch of reasons, including:

  • Smaller, newer, fresher
  • If it falls over, it won’t block Firefox development or othe critical Mozilla processes
  • It has a dedicated team to monitor, diagnose, and fix any issues sasl-browserid introduces

The current plan is to BrowserID enable mozillians.org in it’s 1.2 release (3-5 weeks out). This will give us experience with sasl-browserid. Once it has proven itself production ready…

We’ll add the plugin to MoCo LDAP. We’ll update the internal phonebook’s PHP code. We’ll test, deploy, wait and see.

If that goes well, then all the other LDAP backed webapps can follow.

Summary

and gentleman… I use the acronym LDAP with all due respect for the Fear and Loathing I’m sure it provokes in each and every one of you…

We’ve made an organizational commitment (training, hiring, etc) to improving our ability to maintain our aging LDAP infrastructure. Jabba and Corey have done a top-notch job of revamping the MoCo LDAP, so it’s a sustainable sub-system.

We can adopt BrowserID without throwing away MoCo’s LDAP backend by utilizing sasl-browserid.

Questions/Answers

  • This will not eliminate passwords from LDAP. Someone figure that part out, please.
  • This plugin doesn’t automagically convert your webapp to BrowserID. Some additional coding is required
  • Technically, this is not Single Sign On, but has some of the same useful properties
  • Yes, I’m happy to help you design/setup/whatever your BrowserID solution. Ping me in #identity.
  • Yes, this plugin might be reusable by large organizations such as schools, businesses, and pillow fight flash mobs. Please spread the word.
  • Technically sasl-browserid should work with any Cyrus SASL enabled client or server. OpenLDAP and Postfix are two known servers, are there more? Clients?

Let’s figure out what works well and where we can improve the BrowserID project. If we can do LDAP, we can do anything ;)

* Some programming languages lack whoami bindings, search can be used to emulate this check.

A Dart Daydream

Since Google released the Dart spec today, I wanted to share my speculation on the reason Dart exists: Dart is a new technology to solve a cross-organization engineer people problem, internal to Google.

I have no inside knowledge, but reading between the lines, Google had several sets of engineering efficiency problems.

This is not surprising. Google is a massive, incredible engineering organization. They are creating cutting edge solutions to some of the world’s hardest problems. I would expect them to have huge people challenges;
sudo herd cats still doesn’t work.

Classic engineering problems

  • Code reuse
  • Cost of shipping a project, etc.
  • Communication
  • Standards (Code Quality, Security, etc)

A classic mistake is to solve these things with technology first, before solving the actual people-oriented problems. Technology can help enforce standards and make code reuse easier, but tech alone will fail.

Imagined Problems

As I day dream, I tried to imagine problems that could result in Dart as a (flawed) solution…

Perhaps Google spun up 6 teams to investigate and solve 6 problems (Why isn’t project X using GWT?, Why did project Y fail? Why did project Z take 6 extra months and 30% more $$$ to ship?).

Being an awesome company, these postmortems may have bubbled up into a more core question: How can we solve this once and for all?

One of many possible analysis:

An example of flawed logic that could lead to project Dart:

  • Writing webapps on GWT is better (for our Java developers) than hacking together Javascript, so
  • All our teams should use GWT instead of JavaScript, but
  • We couldn’t get everyone on board with GWT, so
  • If Google could replace JavaScript with a new clean language, we wouldn’t have these engineering team problems, and it turns out
  • We employee some key language pioneers in enterprise languages, so
  • Google needs a ‘programming in the large’ friendly platform like Java was for the last decade.

The end goal is enticing

  • Lower power usage and data center costs, be greener
  • Ramp up new engineers quickly
  • All projects reusing standard language and APIs
  • Lower cost of engineers switching teams
  • Avoid legally tainted platforms (Java)
  • Provide puppies and rainbows for all

Some of these are real issues. Some of these are nice to haves. Just technology isn’t the right solution to all of these problems.

Perspective

Some, definitely not all, but some people at Google have this perspective:
Google hires all the smartest people of the world. We will figure out how to solve the world’s problems; you will live in our resulting products. You’re welcome.

Sometimes being the smartest person can blind oneself to the actual root problem. Having an engineering bent causes one to look at fixing tools instead of process.

Solution

Google has two tactical thrusts, to continue investing in the next generation of web standards (Yay!) and to replace JavaScript with Dart for all internal mobile and desktop app development (Yawn!).

Engineer problems like the ones I invented could have caused this new tactical decision; engineering process problems are hard to solve in any large organizations. JavaScript is a detail, not the cause of many of these types of problems.

In 5 years, various hypothetical teams across my imaginary Google will have drifted back into a Dart-based Babylon, if hypothetical Google doesn’t fix the core communication and standardization issues which may have prompted Dart’s creation.

It is possible to address root causes directly – Google’s Java programmers want GWT, but Google’s native web programmers want JavaScript… Make every employee use GWT; this fits with the only C, Go, Python, or Java languages stance. Introduce training to convert web programmers into Java oriented programmers as part of engineer on-boarding.

These are not fun decisions to make or communicate, but that is the point of project and engineering management.

Creating Dart is expensive (language, tools, runtime, etc), but may give them the social framework to communicate people oriented engineering changes (employees will all use Dart) that address the real core issues. So ultimately Dart might not be a loss for Google internally.

Dart alone will do nothing to solve the root problems that prompted Dart.

Day Dream

Again, this post is based on a hypothetical situation internal to Google. I’ve pieced together this day dream from my own “Enterprise” experiences, various Google posts over the years, etc to resolve why Google would create the Dart platform.

I hope I haven’t offended any of my friends at Google and my speculation is probably wildly inaccurate. If nothing else this post is a parable to talk about solving people problems directly before building tools.

But the real world announcement has some real world impact…

Collateral Damage from the Dart Launch

An unintended consequence of the Dart launch is that it may weaken Google’s efficiency in promoting web standards. It is a very confusing story to explain how promoting the Dart platform and working towards next generation, open web standards are not conflicting Google projects.

Example Dart versus CoffeeScript:

Early adopters in the webdev community don’t see Dart as more attractive than CoffeeScript which they see as having a superior syntax to Dart. Since both must compile down to JS, they are equally viable for experimenting with language semantics. So on first blush, Dart is a meh to CoffeeScript commentators on Hacker News and Reddit.

Aspects of CoffeeScript may inform the next generation of JavaScript standards. CoffeeScript comes from the community organically, not from ‘the one offical smart guy in the room’.

I’m probably wrong, as I don’t have much insight into Google’s engineering organizations, but it is quite possible that Dart is a facile throw at the wrong target.