Mon, 18 Jan 2016

Requirements in Micro-purchasing

I recently completed an 18F micro-purchase task. It turned out to be a very simple task, tweaking some CSS. I spent more time on bidding, setting up the development environment, and figuring out how to receive payment than on the actual work.

My pull request was accepted, and 18F paid me quickly, but after my changes were merged, Maya Benari pointed out that a simpler solution may have been acceptable, or perhaps even preferable.

It turns out that 18F have drafted U.S. Web Design Standards, and have already given a lot of thought to things like vertical spacing, the subject of my change.

As an independent contractor with no previous experience working with 18F, I didn't know there was a U.S. Web Design Standards document, much less its contents. Had this task been assigned to an internal staff member, they probably would have known that the page being updated should follow the standards, or that if the standards didn't adequately address the requirements for the task, that perhaps a future update to the standards should be planned.

Getting the right level of details in task requirements is often tricky. One doesn't want to spend days writing requirements for a task that can be completed in hours. But one also doesn't want what could have been an hour-long task drag on for days because the requirements were ambiguous.

When working within a long-lived team structure, there is some shared institutional knowledge that doesn't need to be made explicit in every task, like the fact that U.S. Web Design Standards exist. Usually when bringing in an outside contractor or new employee, one is starting a relationship that will last weeks, months, or years, and knowledge transfer is an expected part of the on-boarding. This investment makes sense given the size of the overall engagement.

Micro-purchasing of software development work is unique in this respect. Neither party is committing to a long-term relationship, and therefore unlikely to bear significant knowledge transfer costs. Had the acceptance criteria for my spacing fix task included compliance with the U.S. Web Design Standards, I would have had to bid more to cover the time to read and understand the document. And 18F wouldn't want to bear the cost of making me an expert on U.S. Web Design Standards if I were only going to complete a single task.

It's still early in 18F's micro-purchasing experiment, but if the experiment continues or if other organizations want to try it out, it's worth thinking about what types of tasks are best suited for micro-purchasing, and what level of detail in the requirements is optimal.

Some possible questions to ask about tasks you are considering outsourcing as micro-purchases:

As always when discussing project management, I suggest ordinal prioritization of tasks and confidence interval based estimates. If your existing tools don't support these, I recommend checking out LiquidPlanner (affiliate link).

I like the micro-purchasing model, and I hope that it succeeds and is adopted by commercial organizations.

I've gotten multiple emails from grad students doing research on Github. If you're an economics student interested in industrial organization or open source, micro-purchasing of open source software could be an interesting research project.

business | Comments | Permanent Link

Wed, 17 Sep 2014

On Instacart Pricing Transparency

Stephen Klein, a biz dev guy from Instacart, emailed me to ask why I hadn't tried it yet. Here's my response:

Hi Stephen, I was excited to learn about Instacart offering deliveries from Costco. Unfortunately, your current pricing model makes deciding whether to use Instacart too expensive. It's not that your prices are too high. It's that I don't know if your prices are too high; that is, the cost of information is too high.

When deciding whether to use Instacart, my decision is between going to Costco or having Instacart go to Costco for me. In most cases, I would prefer to have Instacart go for me. (Sometimes, I might want to browse.) But to make the decision, I need to know how much it would cost me to use Instacart.

The current pricing model is not transparent enough to determine this cost. Your prices are not the same prices as those offered as Costco, so I don't know what the total cost of using Instacart is, and rather than spend the time trying to figure it out, I simply don't use Instacart.

My recommendation would be to change your pricing model to make explicit the cost of using Instacart. You could charge a fixed percentage above the Costco price, charge a higher delivery fee, or charge a per-item delivery fee, for example; whatever model would be both transparent and profitable.


business | Comments | Permanent Link

Thu, 05 Sep 2013

Estimating Tasks

As described in the previous post about our development process at GreatVines, we use LiquidPlanner for project management. Estimating how long tasks will take is important in most software development projects (and generally any project involving more than one person).

LiquidPlanner provides a great way of estimating tasks using an 80% confidence interval, but estimating tasks like this is a new concept for most developers. Below is a transcript with a developer that recently joined our project. We were able to get from "I have no idea how long it will take" to a decent estimate in about half an hour, but it should only take a minute or two for the next task.

2013-09-05 11:16:52: <Christian G. Warden> have you guys reviewed all of the
                     tasks for the september release?
2013-09-05 11:20:44: <Ben H> yes, but many of the tickets refer to parts of the
                     code I have no experience with so I have no idea how much
                     work they will be.
2013-09-05 11:23:11: <Christian G. Warden> you can put wide estimates on tasks
                     like that.  can you give me an example, and i'll walk you
                     through it?
2013-09-05 11:25:15: <Ben H> 10631525- Make Modal Form Fields scroll-able: I
                     have no idea where contacts are viewed in the application,
                     I haven't had any reason to look at their controller/model
2013-09-05 11:25:52: <Ben H> so any time estimate I would come up with would be
                     a sheer guess
2013-09-05 11:26:38: <Christian G. Warden> ok, no problem.  what's the
                     likelihood it will take you more than 2 weeks to complete
                     the task given what you currently know about the app?
2013-09-05 11:27:21: <Ben H> darn near 0% I would figure
2013-09-05 11:27:26: <Christian G. Warden> phew :) 
2013-09-05 11:27:31: <Christian G. Warden> how about 1 week?
2013-09-05 11:28:29: <Ben H> that's still very high, so 2%?
2013-09-05 11:28:38: <Christian G. Warden> how about 3 days?
2013-09-05 11:28:40: <Ben H> given that it looks like pure GUI work
2013-09-05 11:30:03: <Jim Thompson> yeah Ben the problem is that one customer
                     has put so many fields in the FieldSet that you can't see
                     the entire modal, the Save and Cancel buttons scroll below
                     the page
2013-09-05 11:30:33: <Ben H> I guess 3-4 days (if it is pure GUI work then 1-2
                     should be more then enough, unless there's something
                     unseemly complex)
2013-09-05 11:30:45: <Jim Thompson> I am going to move this task beneath the
                     Performance items Christian
2013-09-05 11:30:47: <Jim Thompson> (and Ben)
2013-09-05 11:32:20: <Christian G. Warden> would you say there's a 10%
                     probability that it will take more than 4 days?
2013-09-05 11:33:34: <Jim Thompson> at 12-32 hours it might have to drop off
                     the list, or at least to the bottom
2013-09-05 11:34:40: <Christian G. Warden> jim, we'll go through the tasks and
                     reprioritize after the estimates are updated.
2013-09-05 11:34:57: <Jim Thompson> ok, I also moved "New Task" and "New Event"
                     to October, which is when we promised it
2013-09-05 11:35:39: <Christian G. Warden> you're interrupting my estimation
                     lesson    :) 
2013-09-05 11:36:02: <Ben H> (yeah, he's been high balling for tutorial
2013-09-05 11:36:11: <Jim Thompson> ok im out
2013-09-05 11:37:50: <Christian G. Warden> no, i want to come up with realistic
                     estimates.  if they're wide right now because of
                     uncertainty as to what it will take to complete them,
                     that's fine.  once we have estimates, we can decide
                     whether to invest time to come up with tighter estimates.
2013-09-05 11:38:44: <Christian G. Warden> so, do you think 4 days as a high
                     estimate is accurate?
2013-09-05 11:39:29: <Ben H> for this example I put 12-24, it's really wide and
                     most likely most of that will be learning about how that
                     GUI element is working and making the tests
2013-09-05 11:39:57: <Ben H> the range is more to show that it could go
                     horribly wrong if there is something there I have no idea
2013-09-05 11:40:24: <Ben H> but 12 still seems really high
2013-09-05 11:40:30: <Christian G. Warden> what's the likelihood that you will
                     complete the task in 2 hours?
2013-09-05 11:41:14: <Ben H> the likihood I'll complete it in 6 hours I would
                     meet the 10% threshold
2013-09-05 11:41:26: <Ben H> *likelihood
2013-09-05 11:41:38: <Christian G. Warden> ok, so, let's make it 6-24
2013-09-05 11:45:29: <Christian G. Warden> and you can check yourself by
                     imagining we're going to a casino.  you can choose from
                     two wagers:
                     1) roulette - there are 10 numbers on the roulette wheel.
                     if 1 or 10 comes up, you lose. if 2 through 8 comes up,
                     you win $100.
                     2) the bookmaker - if you complete the task in less than 6
                     hours, you lose. if you complete the task in more than 24
                     hours, you lose.  if you complete the task between 6 and
                     24 hours, you win $100. which wager do you want to take?
2013-09-05 11:47:25: <Ben H> bookmaker, it also means I can spend some
                     speculative time trying different approaches to find one
                     that solves the problem better
2013-09-05 11:47:34: <Ben H> if things go well
2013-09-05 11:49:16: <Christian G. Warden> so it's not really an 80% confidence
                     interval.  it might be more like a 90% confidence
                     interval.  you should make the range narrower.
2013-09-05 11:49:52: <Ben H> how then do you show the risk if (I doubt in this
                     case) that 10% could be really bad
2013-09-05 11:52:19: <Christian G. Warden> it's ok if there's an outlier
                     occasionally.  we want to get an accurate estimate across
                     all of the tasks.
2013-09-05 11:56:09: <Christian G. Warden> you should narrow the range until
                     you're ambivalent between taking the two wagers.  and
                     widen the range when you'd rather play roulette.
2013-09-05 11:58:02: <Christian G. Warden> but we've already gone from "i have
                     no idea how long it will take" to "i'm 90% confident, it
                     will take between 6 and 24 hours".
2013-09-05 11:59:40: <Ben H> I'm just worried it's misplaced confidence
2013-09-05 12:00:03: <Ben H> either way, given the range, I am (90%) confident
                     that I can get it done within the range
2013-09-05 12:07:07: <Christian G. Warden> nobody dies if you're wrong.  you're
                     expected to be wrong 20% of the time (once you narrow it
                     to an 80% confidence interval).  if we decide we need to
                     know with more certainty when something will be done, we
                     might ask you to spend an hour to do a little research so
                     you can come up with a tighter estimate, after which you
                     would adjust the high and/or low estimate.  for now,
                     though, estimate the tasks based on what you currently
2013-09-05 12:07:50: <Ben H> ok

The roulette idea comes from Douglas Hubbard's How to Measure Anything.

business | Comments | Permanent Link

Fri, 07 Sep 2007

Motorola: Terrible Customer Service

I sent the following email to the Motorola Complaint department this morning. It seems I'm not the only one who has suffered from Motorola's decision to completely isolate their customer service department from those in the company who do more than simply answer the phone. Pat Keene has set up a blog called Hello Moto. Where's my phone? detailing roughly the same experience I describe below. If your call center is in India, and your repair center is in Tennessee, it's not like the CSR can walk over to the shipping department or a technician to get an actual answer. I got the idea to copy Stu Reed from Pat.

Date: Fri, 7 Sep 2007 10:55:32 -0700
From: "Christian G. Warden" 
Subject: Customer Service Breakdown (RMA #XXXXXXXXXX)

Sir or Madam,
I sent my Moto Q in for repair at the end of July with RMA #xxxxxxxxxx
(zip code xxxxx).  It was received by Motorola on July 31.  According to
the IDEN website, the replacement was sent back to me on August 17.  The
Fedex web site confirms that it received notification of a package
shipped Overnight Priority with tracking number xxxxxxxxxxxx, but has no
record of actually receiving the package.

Having not received the package after one week, I called customer
service at 1-800-453-0920 on Friday, August 24.  I was given case number
xxxxxxxx, and was told that I would receive a call back within 24 hours
from the repair facility.  I didn't receive a call until 3 days later,
on Monday, August 27, and I was told that it was a "courtesy call", that
the caller didn't have any information about my phone, and that I would
receive a call within 24 hours.  I never received that call.

I have since called the customer service number four or five more times.
Every time, I am told that I will receive a call within 24 hours, but I
have never received another call since the courtesy call.  During one
call, I was transfered to the voice mail at repair facility.  I left a
message, but alas, did not receive a call back.

On September 1, I submitted a request through your web site (Incident
#xxxxxx-xxxxxx) explaining my situation and my inability to get through
to someone regarding the whereabouts of my phone.  I received an email
response saying that I would receive a call back.  I didn't.

On Tuesday, September 4, I called 1-800-331-6456 as suggested in the
response to by online request.  I was transfered to same call center I
had spoken with on numerous occasions prior.  I explained the situation
and was given a new case number, xxxxxxxx, and once again told that I
would receive a call within 24 hours.  I didn't receive that call.

On September 5, I received an automated response that my online request
had been closed as "SOLVED".  I responded that it was indeed not solved,
and that I still needed someone to contact me.  I received an email
response, "Once again, we apologize for any inconvenience this delay may
cause you. You will be contacted back to the contact phone number you
provided for accurate assistance with your repair."

This morning I called customer service once again, explained the
situation, and was told that they would contact the repair facility to
have them contact me.  I explained that this process obviously does not
work because the repair facility never calls me.  I asked to speak to
someone at the repair facility, and was told that no phone number was
available.  I asked to speak to the person responsible to managing the
repair facility, and was told that no such person was available.  I
asked for the phone number of the Chief Operations Officer, or a person
of a similar role in your organization, and was told that such
information was not available.  Finally, I simply asked to speak with
the supervisor in the call center.  I was told that the supervisor,
whose name the CSR ("Nicole", #351359) refused to give me, was
preoccupied and would call me within 10 minutes.  I didn't receive that
call either.

Due to the malfunctioning phone, and inability to get it replaced in a timely
manner, I have been without a phone for about six weeks, and have continued to
incur charges from my carrier, Sprint, during this period.

Your assistance in resolving this issue would be greatly appreciated.

Christian Warden
(temporary) mobile:

Update (09/23/2007): After I sent the email above, I received a call from Dorothy Hodges in the consumer advocacy department. She was very pleasant and eager to resolve my problem. She ordered me another Q, but they weren't in stock so she sent me a RAZR2 as a loaner. She also sent me a bluetooth headset as a sorry-we-screwed-up consolation token.

Upon receiving the RAZR, I checked my voicemail, and I had two messages from the management of my apartment complex. It turns out the original replacement phone was devlivered, but that Fedex didn't leave a notice, and that the tracking number Motorola recorded was incorrect. So last Monday I called Sprint with Dorothy and activated the Q. She requested a credit on my behalf since I had been without a phone for six weeks while still paying the bill. I managed to get them from their original offer of 100 minutes to $10 to $30, at which point I accepted.

business | Comments | Permanent Link

Sun, 04 Mar 2007

Fore Close

I thought it'd be a nice gesture to make up some signs for Casey. Actually, I just get a kick out of these. I'm probably the only one.

See the Housing Bubble Blog for some real news.

business | Comments | Permanent Link

Sat, 04 Nov 2006

Amazon aStore

A few months ago, Amazon introduced aStore, which allows associates to create their own Amazon store. It's a cool idea, but unfortunately, the current version is pretty limited in fuctionality. The store creator can only pick nine featured products sold by Amazon to display in the store. Categories of goods to be sold through the store can also be selected, but with only limited control over which products show up within these categories through the use of keywords.

For the nine featured products, the store builder can write a little description, but it would be nice if the products displayed the seller's product ratings, comments, etc.

Anyway, here's my store.

business | Comments | Permanent Link

Fri, 19 May 2006

Naked Shorts

I was forwarded this email:

Date: Wed, 17 May 2006 22:43:49 EDT
To: undisclosed-recipients: ;
Subject: If you invest in stocks, listen to this...from Bobby K.

Turn up your volume and set aside about 10 minutes to listen to this. Theres
a crack in the system whereby people can create bullshit shares of stock and
sell it, where in essense creating more shares than there should be and driving
a companies stock down and eventually destroying the company. Ive known about
this for awhile and so have politicians, however, if they tried to fix it,
the entire financial system of the U.S. would collapse.

Ive always said that the stock market crash we saw in 2000 that wiped out
alot of people would be a walk in the park compared to the next one that will
happen. I think within the next 1-3 years the topic of these bullshit "phantom"
shares will be the big one that will be the biggest hit the stock market has
ever seen. Dont paic, just be careful. The stock market is done. The crooks have
made billions in the last 60 years in the stock market and i think the shit
will hit the fan soon. Listen...

I wrote the following response before listening to the audio. I should have guessed that it was the Overstock dude.

The CEO of bitches about naked shorts a lot.  Read  Mark Cuban talks
about it on his blog sometimes too.  There was also a 60 minutes story
about it, but it wasn't very good as Leslie Stahl is about as
financially literate as a chihuahua.

While it's fraudulent to sell something that doesn't exist, the
purchaser of a share of stock is the one who would have a complaint if
he were unable to take delivery of said share.

The CEOs who bitch about it are the ones whose compensation is in the
form of stock options.  If they were speaking on behalf of the company,
they should be delighted that people are driving down the stock price.
The company can buy back its shares cheaply enriching the existing

Imagine a company that sells 1,000,000 shares at $10/share during an
IPO.  If short sellers drive down the stock to $1/share, and the company
can buy back all of its shares, it makes $9,000,000 without giving up
any equity (neglecting underwriting fees and the upward movement of the
stock price as the company starts to buy back a lot of shares).

Likewise, if you're a shareholder, and you think the stock is
undervalued at $1, you can buy more stock and wait for the profits to
come rolling in.

If your company isn't making any money, and you expect to make money on
the stock, well, too bad.


business | Comments | Permanent Link

Sat, 11 Mar 2006

Creative Talent Tax Exemption

I had to pay business tax to the city of Los Angeles last month since I receive 1099 income. The city took $600 from me, but it turns out that "creative artists" who earn up to $300,000 are exempt from the business tax thanks to the lobbying of the Screen Writers Guild. I should move to New Hampshire.

business » taxes | Comments | Permanent Link

Mon, 02 Jan 2006

Losing Money Vicariously Through Slate

I have the fantasy of opening up a little cafe, apparently like many other people. Michael Idov has already done it and documented the money-losing venture at Slate.

His cafe, which he opened in Manhattan, was right out of my fantasy, "where coffee was served on silver trays with a glass of water and a little chocolate cookie" — it's actually a little chocolate square in my imagination.

He goes on to describe how he lost money on every croissant sold, and how he and his wife end up working "$8-per-hour jobs slinging coffee" before going bankrupt.

The European cafes that Michael tried to emulate probably don't $4,500/month rents to pay, because I don't think they get the to-go business that makes Starbucks so profitable.

business | Comments | Permanent Link

Fri, 09 Dec 2005

In Bubble Wrap Champ

I'm an In Bubble Wrap champ! I love free books, and today I got my first free book from In Bubble Wrap, Creating Customer Evangelists.

In Bubble Wrap is an advertising program run by 800ceoread in which you can win free business stuff, mostly books, every day. Thanks go to Seth for the head's up.

In other free book news, I have Paul to thank for a free copy of The Number. I haven't finished reading it yet, but so far it's pretty good. The Number, by Lee Eisenberg, is about the amount of money you need to retire, and discusses the ways different people think about the Number. The book is targeted to those in their 40s and 50s getting closer to retirement who, for a variety of reasons that Eisenberg investigates, don't know what their Number is, but it's a worthwhile read for those of us pushing 30.

Its discussion of retirement being a rather new concept is pretty interesting. One hundred years ago, most people worked until they died. Then, people started to live longer, get kicked out of their jobs, and die in poverty. Then came Social Security. Next up, long lives requiring savings somewhere north of -1.5% during the productive years.

business | Comments | Permanent Link

The state is that great fiction by which everyone tries to live at the expense of everyone else. - Frederic Bastiat