Sunday, December 13, 2009

Amazon Mechanical Turk Perl Library Error

I've been playing with Amazon Mechanical Turk for a project, and I think it's really, really powerful.  So far, I've done a bunch of research and some tests using the Requester UI, but have not yet dug into the API.  But, over the past few days, I've finally started my research and experimenation using the API directly.  However, my very first time trying to use the modules that Amazon published, I got a big error:

t/01-ListOperations.t ..................... Can't locate object method "new" via package "Net::Amazon::MechanicalTurk::Transport::RESTTransport" at /Users/dviner/.cpan/build/Net-Amazon-MechanicalTurk-1.01-1kIAvQ/blib/lib/Net/Amazon/MechanicalTurk/ line 21.

This error message was repeated a few dozen times.  Also, in searching for the problem, I found this StackOverflow issue: Perl module Net::Amazon::MechanicalTurk failing tests.

I find it pretty amazing that Amazon hasn't fixed this problem.  But, after doing a bunch of searching and sniffing around, I couldn't find any solution at all.

So, I made one of my own.  Here's a short patch that will allow the Net::Amazon::MechanicalTurk module pass all it's tests.  Most will be skipped with the message:

skipped: Configure Amazon AWS Authentication to enable tests against Mechanical Turk Sandbox

If you want to try to get those to verify, you must make a file named in the current directory.  In that file, you must list your access key and secret key as well as the turk service URL and API version you are using.  Here are the sample contents:

access_key: your-access-key
secret_key: your-secret-key
service_version: 2008-08-02

Note that even with this file in place, several of the tests still fail.  I'm not an expert on the Turk API (yet), so I'm not sure how to fix the other errors that occur.

But hopefully this patch will help people.  Feel free to contact me if you find additional changes that should be made to the library.

Wednesday, December 2, 2009

jQuery, tables, and administrative interfaces

A few month ago, I posted about using Jquery to re-imagine simple things. Recently, I've been working with jQuery and jQuery UI quite a bit - in particular in building simple administrative interfaces.

Simple admin interfaces almost always require a table of stuff - users, comments, posts, actions, jobs, whatever - the "things" being administrated. jQuery has a really cool plugin called TableSorter that makes it's incredibly easy to convert a simple HTML table into a fancy, easy-to-read interface.

TableSorter even has a simple pagination capability (see the demo here). The current version (2.0.3) of the paginator requires that you have an html element of class "pagesize" which defines the number of items to display on one page. But the code doesn't actually do an error check to see if that value doesn't exist or if the value is NaN (javascript's fancy way of saying the value there is not a number). So, here's a simple patch to apply to the jquery.tablesorter.pager.js file which will perform the validation & error check.

I found one other drawback of the paginator. By default, the paginator will display something like "1/103" for the current page number and the total number of pages. But, the current code is somewhat limiting. You can change the separator ("/" by default) between the current page number and number of pages, but that's about it. In addition, the code will invoke jQuery's .val() method to display the resulting string. This is also really limiting, since not all elements support using val() to set their contents. For example, invoking .val() on a element does nothing.

So, here's another patch to apply to the jquery.tablesorter.pager.js file which will allow you to register a callback function. By using a callback function, you can control the display as you see appropriate. The callback function is passed 3 arguments:
*func (config, page_active, page_total)
The config parameter is the internal configuration information from the paginator, but the really useful parameters are the other two. page_active is the currently displayed page number. page_total is the total number of pages.

Here's a sample of how you can use the callback:

    .tablesorter({ 'widgets': ['zebra'] })
    'container': jQuery("#jq-pager"),
    'size': 20,
    'positionFixed': false,
    'updatePageDisplayFn': function (config, pg, total)
        jQuery('.pagedisplay_nice').text('Page '
            + pg + ' of ' + total);

This sample code results in displaying the pagination information like this:
Page 1 of 103
in the text area of all elements with the class "pagedisplay_nice". In my HTML page, I have a simple span element in a different place on the page, which looks like this:
<span class="pagedisplay_nice" />
This provides a massive amount of flexibility for the user of TableSorter's Pager plugin.

Hope this helps!

Visible Networking

Recently, Tony Karrer posted an article where he described a pretty interesting new idea - which he termed Visible Networking. I think this concept is really cool. I've always thought that we should use blogs to have more open debates on issues, thoughts, etc. Too often, blogs seem like a one-way broadcast about something - but I think that the most interesting ideas and thoughts are identified and refined in the course of a dialog, not a monologue.

So, I'm going to ask Tony to start visible networking with me. I'm not sure exactly what that means or how it will play out, but I think it's worth trying!

Tony, what do you think?

Tuesday, November 3, 2009

Crazy Business Idea #27

Occasionally, I think of crazy business ideas. I have no idea if they are practical, but they're fun to imagine. Today, I thought of another one.

Imagine a plugin/widget that enabled a DVR to replace ads in a program with ads from a different decade - of your choosing.

I thought of this while watching the original V miniseries on SyFy. It's from the early/mid 80s. The hair, clothes, special effects, everything is awesome. Then I thought - imagine if, instead of the crappy Levitra ads or whatever garbage is showing now, they showed commercials from the 80s that are roughly in time with the show's original airing? How cool would it be to see a Members Only ad, or a Kool-Aid ad with the giant pitcher of Kool-Aid, or one of the original Nest-e Plunge ads, or a Sunkist ad from the 80s! That would be awesome! I'd actually pay attention to the commercials just for the kitch factor.

From a business perspective, it'd be hard because you'd need to obtain the commercials from the early 80s. I have no idea if those ads are laying around in some storehouse or not. And, if they are, would the companies allow you to use them? Maybe... It's cool brand advertising for long-lived brands. So, maybe they'd pay a similar price for the 80s ad as they would for the modern ads.

From a tech perspective, there are several problems. Assuming you can find the original ads somewhere, you'd have to digitize them for delivery in modern cable signals. This is probably feasible assuming they exist. Also, there's the issue of the set-top box integration. You'd need some way to detect that an ad was about to display, and intercept the display in order to show something else.

I don't think that's possible currently, but I could be wrong.

The UI aspect wouldn't be too hard... at the start of the show, or in a persistent configuration setting you could pop up a small dialog window asking from what decade you want to see ads. Actually, I would love to see ads from the 60s. I'm too old to have ever seen the cigarette commercials on TV, but having seen some clips, they totally rock.

All in all, I doubt this is a viable business, but it would be so cool.

Friday, October 23, 2009

ImageMagick, CentOS, and PHP

Ever wonder how to get ImageMagick to work well in a PHP environment on CentOS?

I have done it a few times, and here's a simple recipe to follow. I use this recipe when I have yum managing the basic libraries, but not managing the libraries in active use on the site (like php, apache, perl, imagemagick and a few others).
% sudo yum install libX11-devel
% sudo yum install libjpeg libjpeg-devel
% sudo yum install libpng libpng-devel
% sudo yum install libtiff libtiff-devel
% # download ImageMagick from
% ./configure --with-perl=/usr/local/bin/perl
% make
% sudo make install
(make sure that /usr/local/lib is in the /etc/*.conf files)
% sudo /sbin/ldconfig

Now image-magick is installed.

To Use in PHP
% sudo /usr/local/bin/pecl install imagick
% sudo vi /usr/local/lib/php.ini
% sudo /etc/init.d/httpd stop
And that's it. Hope this helps.

Thursday, September 24, 2009

Cool event in LA for entrepreneurs

Just wanted to pass the word on a cool new event coming up. It's a
workshop put on my the DealMaker team which also brings you the
Strategy Series and Startonomics. I've been to several of these
events, and I have to say that they are really, really good - very
high signal-to-noise ration. Here's the next big thing they are
putting on here in LA.

Details are below. Enter "STARTUP LA" in the ""how did you hear about
The Workshop" and get $20 off!

Seats are limited, and from the speaker/mentor list, this event
promises to sell out fast! Check it out!

Official Info

THE WORKSHOP: October 6, 2009. 1:00pm – 6:00pm
This hands-on "workshop-style" event will allow entrepreneurs to drill
down on the key issues we face when laying the foundation to build and
fund their company.

The Workshop is your chance to get your hands on actual cap charts and
term sheets.
Ask questions, get answers.

Tuesday October 6, 2009 | 1:00pm - 6:00pm
Microsoft LA | 333 S. Grand Ave. 33rd Floor, Suite 3300 | Los Angeles, CA 90071
Parking: Joe's Auto Parts:
220 So. Hope
(Microsoft will validate)

* Charles Halloran, Attorney, KPPB
* Dan Gould, VP Technology, Fox Interactive Media
* Dave McClure, Founders Fund
* Daniel C. Burnham, Partner, Strategic Law Partners
* Frank Addante, CEO, Rubicon Project
* Jason Nazar, CEO, DocStoc
* John Babcock, Partner, Rustic Canyon Partners
* John Suh, CEO, Legal Zoom
* Kent Goldman, Principal, First Round Capital
* Jeff Tinsley, CEO, Mylife
* Mark Suster, Partner, GRP Partners
* Richard Wolpert, Partner, The Mailroom Fund
* Seth Sternberg, CEO, meebo
* William Quigley, Partner, Clearstone Ventures

SESSIONS – What You'll Learn:

Romancing the Deal: Getting to "I do" And Getting What You Want
Only fools rush in to venture money, and for good reason –
underestimate the bargaining power of your target investor, and you
may just end up with a broken heart. Before you jump in, you need to
understand the strategies and tactics of negotiating a funding round
and it starts with the first date. Get tips from successful founders,
lawyers, and VC's on the moves that lead to "Mr. or Mrs. Right Money".

Determining Need vs. Greed
We've all heard the horror stories of taking too much funding and
diluting your value. This session will help you determine how much
financing you actually need, explore alternative financing options
every entrepreneur should consider, understand the investor's math,
and protect your (ass)ets along the way.

Equity Structure
Equity structure is one of the most important and most misunderstood
factors of starting a company. With numerous legal, investment, and
strategic impacts, implementing a solid equity structure is a
prerequisite to success. How should you divide equity among employees,
and even more importantly, how can your equity structure affect your
legal rights, venture outlook and exit?

Team, Board, & Advisor Compensation
How can you determine the right combination and levels of compensation
plans for your team, board and advisers? What are the legal landmines
and loopholes you should know about? How have VCs' successful
portfolio companies structured compensation packages? We'll help you
understand the ins and outs of incentive plans and choose the right
mix for your start-up.

Term Sheet Negotiation
Think you've mastered the term sheet? Think again. In this session
you'll get to walk through an actual term sheet with a VC,
entrepreneur, and lawyer to find the quick sand and learn how to get
around it.

Working with Investors, Advisors & your Board: What to Watch Out For
In the beginning, everything is happy-go-lucky...until something goes
wrong. What could you have done in the beginning to protect against
explosions, and how much leeway to you really have? We'll discuss how
to structure investor and adviser relationships so you have maximum
control, as well as how to effectively manage your investors and
advisors in the good times, and bad.

Tickets are limited. Entrepreneurs will be given priority access to tickets.

Potential consulting work in Los Angeles: Drupal-based

A client of mine (based here in LA) is in need of Drupal expertise very soon (in the next few weeks). The project is a consumer-facing Q&A site that is in open beta now, but in need of additional work in the coming months.

Please contact me if you're interested.

Friday, February 27, 2009

jQuery and reimagining simple things

For the past several months, I've been using jQuery more and more in various ways. Every time I use it, I totally love it. It's so freeing not to use a library that is compact, flexible, powerful, cross-platform, and that doesn't require a crapload of programming paradigms or force a whole regime of programming on me.

This isn't meant to bash any other frameworks or libraries. But, I've used YUI and Prototype extensively, as well as toyed with Ext. I know that every library or framework has its strengths and weaknesses as well as its proponents and detractors. I don't intend to compare absolutely everything about these frameworks either. There are several posts which already do that sort of blow-by-blow comparison. Instead, I'd like to talk about how jQuery changed my perception of what's possible.

I've worked online for more than a decade - so I tend to think of a lot of thinking in a historical context. For example, menu navigation. Almost every site that I've built has some kind of nav bar with links. Usually, those navbars have hierarchical menus or drop down menus that let you move around. Usually I think of a simple HTML table that has a couple of boxes. Clicking on or mousing over each box opens the dropdown menu with the navigation items below. Yes, it's quite web 1.0-ish. But to be honest, it's a tried-and-true formula for getting something to work. You know it works absolutely everywhere.

After messing with jQuery and reading about it, that's all gone for me.

Want cool navigational menus? How about Kwicks? Here are just a few example pages that show you the power. Ok, so maybe you don't like that sort of boxy-bouncy type stuff. How about something a bit more subtle? Check out Bedrich Rios' demo of MooTools inspired navigational links. (Walkthrough of that demo is here.) Ok, so even that's too much? How about something really simple that even looks like the old-fashioned navbar menus? Check out this demo from kriesi. (Walkthrough of that hierarchical navbar is here.)

I like these examples because they really blow away the traditional way of thinking about menus. I had never considered a navbar that would expand horizontally when you mouse over the header you want. I hadn't really thought about small touches like the little bounce on mouseover to indicate visually that you've moused over the link. It's nice. It turns what used to be a totally hum-drum thing into something that's eye-catching and feels neat.

Moreover, I like these examples because the simplicity of the jQuery library lets these work with minimal javascript and without crazy javascript contortions. Sure, this is possible in lots of frameworks and you can (probably) drop in a single remote javascript file to handle it. But, just because you can, doesn't mean you should. Any client side programming language and framework enables you to do stuff like this. But, jQuery provides a way to do it elegantly.

Also, if you are utterly bored by the navigational menu examples, check out the 45 other examples from Smashing Magazine. Some of my navbar examples are taken from there, and some I've found elsewhere. Most importantly, take a quick look through the variety of examples there and you'll get a good sense for the flexibility and power of the jQuery library.

I can't wait to see what new cool super-charged stuff people create using libraries like jQuery.

Wednesday, January 28, 2009

Rest Exploration: Part 1. PUT versus POST

I have been working on a new set of Rest web services for and I keep coming across interesting challenges in defining a meaningful, REST-compliant API. I'll post more details as I work through the whole API.

Now, for the first challenge. I've discovered that PUT and POST are pretty complicated. Well, not complicated, but their proper use is not always obvious. I've read several articles that discuss PUT versus POST. Here's a brief list:
After all that reading, I think that I have a pretty decent understanding of the basics. Specifically, PUT is an idempotent method to create/set the full state of the resource specified at a particular URI. POST is not idempotent, nor is its operation stated so simply. Paul suggests that we think of POST as "create new URI as child of the current URI." In other words, POST is meant to add a new resource to a collection. In fact, the HTTP 1.1 spec nicely describes the distinction:
The fundamental difference between the POST and PUT requests is reflected in the different meaning of the Request-URI. The URI in a POST request identifies the resource that will handle the enclosed entity. That resource might be a data-accepting process, a gateway to some other protocol, or a separate entity that accepts annotations. In contrast, the URI in a PUT request identifies the entity enclosed with the request -- the user agent knows what URI is intended and the server MUST NOT attempt to apply the request to some other resource.

As you can see, the true meaning of POST is pretty unclear. It can do, more or less, anything that the service wants. I think that Paul suggests we use POST sparingly because it can be made to do pretty much anything we want. And, like Spiderman's learns,
"with great power comes great responsibility".

This part makes total sense to me. But, here's where I get confused. What if I want to modify only part of the state of a resource?

As I understand the "official" thinking, PUT can really only be used if I send the entire state of the resource. If I want to update just part of the state, then PUT doesn't seem like the right verb. Since POST is the more flexible verb, perhaps I could use POST to modify just the part of my state that I care about. But somehow, this seems exceedingly unsatisfying. First, it's definitely violoating Paul's suggestion of using POST for adding child resources. Second, it feels like I'm shoe-horning a portion of the PUT function into a POST.

So, let's make it a bit more concrete with the real example I'm working on. In AboutMyBaby, you create a scrapbook. A scrapbook has a bunch of state or metadata - e.g., title, owner, url, and "template" or "skin". The "template" or "skin" defines the look and feel of that particular scrapbook. There are hundreds of templates, each of which is its own resource in REST-speak. So, each scrapbook resource "has a" template resource. The API needs a way to change the template of a given scrapbook. It's really that simple.

So, what are my options?

Option 1: Use PUT in the traditional way.
Meaning: In my client, I must retrieve the entire current state of the scrapbook resource, modify the one item I care about (namely the template URI to use), and PUT it to the URI of the scrapbook.
PUT /scrapbook/1234/metadata HTTP/1.1

<amb_template>URI TO new template</amb_template>

Pros: Well, it's definitely REST and it definitely follows all the precepts people talk about.
Cons: It seems totally bizarre and overly complicated to force the client to retrieve the entire state of the scrapbook just to modify 1 part of it. This means that there must be additional work on both the client (to get the state prior to updating) and the server (to service the "get state" request).

Option 2: Use POST
Meaning: In my client, I just create a POST message that encapsulates the change I want to make.
POST /scrapbook/1234/metadata HTTP/1.1

<amb_template>URI to new template</amb_template>

Pros: The client need not create the entire state of the scrapbook metadata to alter just the template. The resulting XML message (or other format) is relatively compact - at least as compared to the message representing the full state of the resource.
Cons: It just doesn't seem RESTy. I know the precise thing I want to have happen. The operation should be idempotent. Moreover, this feels like the easy way out, and frequently, the easy way is not the architecturally correct way.

Option 3: Make a new URI and use PUT
Meaning: Instead of having the URI represent the "metadata" resource associated with a scrapbook, make a new URI that is just the template associated with the metadata associated with the resource.
PUT /scrapbook/1234/metadata/template HTTP/1.1

<amb_template>URI to new template</amb_template>

Pros: This feels more RESTy. The action is very clear-cut. The operation is definitely idempotent.
Cons: Although it is RESTy, it also seems like a slippery slope. That is, what if I want to change the title of the scrapbook? Should the title be it's own resource with its own URI? This seems like it would cause an explosion of URIs associated with each little characteristic of a scrapbook.

Option 4: Filter the URI and use PUT
Meaning: Use a URI that represents the metadata of the scrapbook, but "filter" it by using a query string. Then, the client can use PUT but only update one part of the metadata.
PUT /scrapbook/1234/metadata?template HTTP/1.1

<amb_template>URI to new template</amb_template>

Pros: This feels RESTy. The action is very clear-cut. The operation is definitely idempotent. This option prevents the runaway resource creation of option 3.
Cons: Although it seems RESTy, I'm not sure that query strings were meant for PUT operations. I think this may lead to strange results down the road because it almost feels like I'm tweaking the true meaning of PUT. Also, I defined PUT at the outset as a method which requires that you send the entire state of the resource. Clearly, in option 4, we are modifying that dictum.

As you can see, a seemingly simple operation ("update the template associated with a scrapbook") has turned into 4 potential service definitions. I am still thinking through this problem, but I would love to hear the advice, thoughts, and comments on the structure of this API.

Tuesday, January 20, 2009

Blind repetition and learning from the past

I think that this post - "The 37Signals Effect" by Louis Brandy - is a great article and really points out a useful lesson. In short, Louis argues that by simply following the precise advice of a successful company or blogger, you will achieve your own success.

I firmly believe that far too often, people blindly follow the exact words of successful predecessors. Just think about it. Someone else, who was solving some other problem, took a series of steps that helped to solve that problem. Now you, facing a different set of problems, think that you can solve your problem by doing the exact same thing? That just doesn't make any sense. Blindly repeating action items is a sure recipe for failure. Or at least, doing so doesn't really get you any closer to success. You might still become successful - just not because of these repeated steps.

On the other hand, those who don't know history are doomed to repeat it. So, to assume that there are no lessons to be learned from past experiences of others is equally foolhardy. Surely, successful people were successful for some reason. Maybe they can't articulate that reason very clearly or maybe the reason was pure chance (as Malcolm Gladwell argues in Outliers), but either way, they were successful, you want to be successful, and so it's worth considering why they succeeded. Perhaps, buried somewhere in there is a nugget that will help you succeed in your business.

With this in mind, let's revisit Louis' primary example of blind repetition.

First, he quotes Getting Real:
Do less than your competitors to beat them. Solve the simple problems and leave the hairy, difficult, nasty problems to everyone else.

Then, Louis comments that "
doing 20% of the work for an 80% product, and solving only the easy problems" leaves the consumer wanting more and lets someone else duplicate your product or service quickly and easily.

There are actually 2 issues intertwined here - first, the "do less" strategy, second, the "easy solutions are vulnerable to duplication" challenge.

I think that the "do less" strategy misses the real point. Consumer-facing businesses succeed when they solve a problem or pain that consumers experience, and solve that pain "better" than the current solutions available to the consumer. Some consumer pains are solved with simple solutions and some require more complex solutions. But the complexity of the solution doesn't matter. What matters is how well it solves the problem people experience.

However, I also think that the "do less" strategy can be useful under certain circumstances. For example, new technology can fundamentally change the possibilities for solving a certain problem. Frequently, products that have traditionally solved a problem are eclipsed by newer technologies and businesses are slow to react. Or, if existing products are not eclipsed, then the newer technologies enable new solutions that solve the problem more successfully. When this happens, the "do less" strategy can be extremely successful. New, clean, simple products can quickly replace large, unwieldy, complex products. (
Usually, newer products are simpler, but only because they are newer and don't have days/weeks/months/years of cruft built up around them.) But, the critical issue isn't the product itself or its complexity - it's how the product addresses the pain felt by users. If your product addresses the user pain better than the competitor's, then you will likely succeed. Doing less or more work than your competitor per se doesn't make a difference.

Now, the "it's easy to duplicate easy solutions" challenge is different story. I think that there is an underlying flaw in the logic of this line of thinking. It suggests that your product is somehow novel or not duplicative or won't be copied if it's complex. Well, guess what. Pretty much every idea is a copy of some other idea. Jason Nazar points this out quite dramatically in his presentation "Mistakes People Make Before Starting a Business".

Mistake 3: Thinking no one else has ever done it before
... just go to page 3 of the Google search result... It doesn't make a difference if someone else is doing it, just do it better.
I think that Jason capture the point exactly. Don't worry if someone else is going to copy you or if you are copying someone else! Being successful is only partially about the idea. And the chances are, someone else either has done it before or is doing it now. So, who really cares if your idea is easy to duplicate? You are more likely to be successful if you focus on the solution to the problem faced by users. If you execute on your product better than anyone else, then your business is likely to succeed. (Or, at least, it won't fail just because someone copies you.) Eventually someone else will copy you, so you must do it better, cheaper, faster, etc. And if Google copies you, well, take it as a complement and move on to the next idea. Or walk into the Googleplex and pitch your business to them as a better solution that their simple demo app.

So, in conclusion, don't blindly repeat someone else's steps. But don't blindly ignore the learnings buried in others' experiences. Read the blog posts and articles from successful and unsuccessful people. Think about how they apply to you and your business. Shape those lessons and mold them into something that's useful for you. If you can't figure out how to do that, then file them away as interesting, but not clearly useful. Revisit that list periodically to see if you can make sense of those ideas you can't apply immediately. Of course that's not simple. But no one said that being successful in business is easy. If it was that easy, then everybody would be successful.

Friday, January 16, 2009

What web video could be like....

My day job is the Director of Technology for 60Frames Entertainment, a digital studio making, distributing, and monetizing original video content for the web. A lot of people have told me that video online will never be more than low quality, "watch me as I set myself on fire and ride my bike around" junk. Some people have duly noted that there is that kind of low-grade fair, and then there is porn, and that's it.

I think that it's possible to do more. I took this job because I think that there are people who are really good at making original video content and I firmly believe that the future of video is online. It's not that every video will be great. I mean, look at feature films - is every film good? No way. But, there are good films. In the same way, not every web video will be good. But, I haven't really seen a lot of great web videos.

If you don't believe that web video can be high quality, watch this video. It's a pilot that 60Frames produced with John August. If you're not in the entertainment biz, you might not know who John August is. But, if you've seen Charlie and the Chocolate Factory, Charlie's Angels, Go, or Big Fish, then you've seen John's work. You can read more of his bio on IMDB or read his blog about screenwriting.

I think you'll find this video is engaging, funny, fast-paced, and actually tells a story. Hopefully, it even leaves you wanting more..

So, spend a few minutes and check out this pilot. I think you'll agree that it's really pretty damned good.

The Remnants from John August on Vimeo.

Hi world. Nice to see you again!

After much delay, I've decided it's time to blog again. I have about a zillion mini-posts in my head that I just never bothered to write down. I know they won't all be fascinating, but what blog is 100% interesting all the time?

My posts will be a mix of technology, business, politics, sociology, and god knows what else comes into my mind.

So, enough of the introductory crap... let's get on with the getting on!

By the way, if you want to see my old blog posts, you can read them at