I recently completed an 18F micro-purchase task. It turned out to be a very simple task, tweaking some CSS. I spent more time on bidding, setting up the development environment, and figuring out how to receive payment than on the actual work.
It turns out that 18F have drafted U.S. Web Design Standards, and have already given a lot of thought to things like vertical spacing, the subject of my change.
As an independent contractor with no previous experience working with 18F, I didn't know there was a U.S. Web Design Standards document, much less its contents. Had this task been assigned to an internal staff member, they probably would have known that the page being updated should follow the standards, or that if the standards didn't adequately address the requirements for the task, that perhaps a future update to the standards should be planned.
Getting the right level of details in task requirements is often tricky. One doesn't want to spend days writing requirements for a task that can be completed in hours. But one also doesn't want what could have been an hour-long task drag on for days because the requirements were ambiguous.
When working within a long-lived team structure, there is some shared institutional knowledge that doesn't need to be made explicit in every task, like the fact that U.S. Web Design Standards exist. Usually when bringing in an outside contractor or new employee, one is starting a relationship that will last weeks, months, or years, and knowledge transfer is an expected part of the on-boarding. This investment makes sense given the size of the overall engagement.
Micro-purchasing of software development work is unique in this respect. Neither party is committing to a long-term relationship, and therefore unlikely to bear significant knowledge transfer costs. Had the acceptance criteria for my spacing fix task included compliance with the U.S. Web Design Standards, I would have had to bid more to cover the time to read and understand the document. And 18F wouldn't want to bear the cost of making me an expert on U.S. Web Design Standards if I were only going to complete a single task.
It's still early in 18F's micro-purchasing experiment, but if the experiment continues or if other organizations want to try it out, it's worth thinking about what types of tasks are best suited for micro-purchasing, and what level of detail in the requirements is optimal.
Some possible questions to ask about tasks you are considering outsourcing as micro-purchases:
As always when discussing project management, I suggest ordinal prioritization of tasks and confidence interval based estimates. If your existing tools don't support these, I recommend checking out LiquidPlanner (affiliate link).
I like the micro-purchasing model, and I hope that it succeeds and is adopted by commercial organizations.
I've gotten multiple emails from grad students doing research on Github. If you're an economics student interested in industrial organization or open source, micro-purchasing of open source software could be an interesting research project.
Stephen Klein, a biz dev guy from Instacart, emailed me to ask why I hadn't tried it yet. Here's my response:
Hi Stephen, I was excited to learn about Instacart offering deliveries from Costco. Unfortunately, your current pricing model makes deciding whether to use Instacart too expensive. It's not that your prices are too high. It's that I don't know if your prices are too high; that is, the cost of information is too high.
When deciding whether to use Instacart, my decision is between going to Costco or having Instacart go to Costco for me. In most cases, I would prefer to have Instacart go for me. (Sometimes, I might want to browse.) But to make the decision, I need to know how much it would cost me to use Instacart.
The current pricing model is not transparent enough to determine this cost. Your prices are not the same prices as those offered as Costco, so I don't know what the total cost of using Instacart is, and rather than spend the time trying to figure it out, I simply don't use Instacart.
My recommendation would be to change your pricing model to make explicit the cost of using Instacart. You could charge a fixed percentage above the Costco price, charge a higher delivery fee, or charge a per-item delivery fee, for example; whatever model would be both transparent and profitable.
As described in the previous post about our development process at GreatVines, we use LiquidPlanner for project management. Estimating how long tasks will take is important in most software development projects (and generally any project involving more than one person).
LiquidPlanner provides a great way of estimating tasks using an 80% confidence interval, but estimating tasks like this is a new concept for most developers. Below is a transcript with a developer that recently joined our project. We were able to get from "I have no idea how long it will take" to a decent estimate in about half an hour, but it should only take a minute or two for the next task.
2013-09-05 11:16:52: <Christian G. Warden> have you guys reviewed all of the tasks for the september release? 2013-09-05 11:20:44: <Ben H> yes, but many of the tickets refer to parts of the code I have no experience with so I have no idea how much work they will be. 2013-09-05 11:23:11: <Christian G. Warden> you can put wide estimates on tasks like that. can you give me an example, and i'll walk you through it? 2013-09-05 11:25:15: <Ben H> 10631525- Make Modal Form Fields scroll-able: I have no idea where contacts are viewed in the application, I haven't had any reason to look at their controller/model 2013-09-05 11:25:52: <Ben H> so any time estimate I would come up with would be a sheer guess 2013-09-05 11:26:38: <Christian G. Warden> ok, no problem. what's the likelihood it will take you more than 2 weeks to complete the task given what you currently know about the app? 2013-09-05 11:27:21: <Ben H> darn near 0% I would figure 2013-09-05 11:27:26: <Christian G. Warden> phew :) 2013-09-05 11:27:31: <Christian G. Warden> how about 1 week? 2013-09-05 11:28:29: <Ben H> that's still very high, so 2%? 2013-09-05 11:28:38: <Christian G. Warden> how about 3 days? 2013-09-05 11:28:40: <Ben H> given that it looks like pure GUI work 2013-09-05 11:30:03: <Jim Thompson> yeah Ben the problem is that one customer has put so many fields in the FieldSet that you can't see the entire modal, the Save and Cancel buttons scroll below the page 2013-09-05 11:30:33: <Ben H> I guess 3-4 days (if it is pure GUI work then 1-2 should be more then enough, unless there's something unseemly complex) 2013-09-05 11:30:45: <Jim Thompson> I am going to move this task beneath the Performance items Christian 2013-09-05 11:30:47: <Jim Thompson> (and Ben) 2013-09-05 11:32:20: <Christian G. Warden> would you say there's a 10% probability that it will take more than 4 days? 2013-09-05 11:33:34: <Jim Thompson> at 12-32 hours it might have to drop off the list, or at least to the bottom 2013-09-05 11:34:40: <Christian G. Warden> jim, we'll go through the tasks and reprioritize after the estimates are updated. 2013-09-05 11:34:57: <Jim Thompson> ok, I also moved "New Task" and "New Event" to October, which is when we promised it 2013-09-05 11:35:39: <Christian G. Warden> you're interrupting my estimation lesson :) 2013-09-05 11:36:02: <Ben H> (yeah, he's been high balling for tutorial purposes) 2013-09-05 11:36:11: <Jim Thompson> ok im out 2013-09-05 11:37:50: <Christian G. Warden> no, i want to come up with realistic estimates. if they're wide right now because of uncertainty as to what it will take to complete them, that's fine. once we have estimates, we can decide whether to invest time to come up with tighter estimates. 2013-09-05 11:38:44: <Christian G. Warden> so, do you think 4 days as a high estimate is accurate? 2013-09-05 11:39:29: <Ben H> for this example I put 12-24, it's really wide and most likely most of that will be learning about how that GUI element is working and making the tests 2013-09-05 11:39:57: <Ben H> the range is more to show that it could go horribly wrong if there is something there I have no idea of 2013-09-05 11:40:24: <Ben H> but 12 still seems really high 2013-09-05 11:40:30: <Christian G. Warden> what's the likelihood that you will complete the task in 2 hours? 2013-09-05 11:41:14: <Ben H> the likihood I'll complete it in 6 hours I would meet the 10% threshold 2013-09-05 11:41:26: <Ben H> *likelihood 2013-09-05 11:41:38: <Christian G. Warden> ok, so, let's make it 6-24 2013-09-05 11:45:29: <Christian G. Warden> and you can check yourself by imagining we're going to a casino. you can choose from two wagers: 1) roulette - there are 10 numbers on the roulette wheel. if 1 or 10 comes up, you lose. if 2 through 8 comes up, you win $100. 2) the bookmaker - if you complete the task in less than 6 hours, you lose. if you complete the task in more than 24 hours, you lose. if you complete the task between 6 and 24 hours, you win $100. which wager do you want to take? 2013-09-05 11:47:25: <Ben H> bookmaker, it also means I can spend some speculative time trying different approaches to find one that solves the problem better 2013-09-05 11:47:34: <Ben H> if things go well 2013-09-05 11:49:16: <Christian G. Warden> so it's not really an 80% confidence interval. it might be more like a 90% confidence interval. you should make the range narrower. 2013-09-05 11:49:52: <Ben H> how then do you show the risk if (I doubt in this case) that 10% could be really bad 2013-09-05 11:52:19: <Christian G. Warden> it's ok if there's an outlier occasionally. we want to get an accurate estimate across all of the tasks. 2013-09-05 11:56:09: <Christian G. Warden> you should narrow the range until you're ambivalent between taking the two wagers. and widen the range when you'd rather play roulette. 2013-09-05 11:58:02: <Christian G. Warden> but we've already gone from "i have no idea how long it will take" to "i'm 90% confident, it will take between 6 and 24 hours". 2013-09-05 11:59:40: <Ben H> I'm just worried it's misplaced confidence 2013-09-05 12:00:03: <Ben H> either way, given the range, I am (90%) confident that I can get it done within the range 2013-09-05 12:07:07: <Christian G. Warden> nobody dies if you're wrong. you're expected to be wrong 20% of the time (once you narrow it to an 80% confidence interval). if we decide we need to know with more certainty when something will be done, we might ask you to spend an hour to do a little research so you can come up with a tighter estimate, after which you would adjust the high and/or low estimate. for now, though, estimate the tasks based on what you currently know. 2013-09-05 12:07:50: <Ben H> ok
The roulette idea comes from Douglas Hubbard's How to Measure Anything.
I sent the following email to the Motorola Complaint department this morning. It seems I'm not the only one who has suffered from Motorola's decision to completely isolate their customer service department from those in the company who do more than simply answer the phone. Pat Keene has set up a blog called Hello Moto. Where's my phone? detailing roughly the same experience I describe below. If your call center is in India, and your repair center is in Tennessee, it's not like the CSR can walk over to the shipping department or a technician to get an actual answer. I got the idea to copy Stu Reed from Pat.
Date: Fri, 7 Sep 2007 10:55:32 -0700 From: "Christian G. Warden"
To: email@example.com Cc: firstname.lastname@example.org Subject: Customer Service Breakdown (RMA #XXXXXXXXXX) Sir or Madam, I sent my Moto Q in for repair at the end of July with RMA #xxxxxxxxxx (zip code xxxxx). It was received by Motorola on July 31. According to the IDEN website, the replacement was sent back to me on August 17. The Fedex web site confirms that it received notification of a package shipped Overnight Priority with tracking number xxxxxxxxxxxx, but has no record of actually receiving the package. Having not received the package after one week, I called customer service at 1-800-453-0920 on Friday, August 24. I was given case number xxxxxxxx, and was told that I would receive a call back within 24 hours from the repair facility. I didn't receive a call until 3 days later, on Monday, August 27, and I was told that it was a "courtesy call", that the caller didn't have any information about my phone, and that I would receive a call within 24 hours. I never received that call. I have since called the customer service number four or five more times. Every time, I am told that I will receive a call within 24 hours, but I have never received another call since the courtesy call. During one call, I was transfered to the voice mail at repair facility. I left a message, but alas, did not receive a call back. On September 1, I submitted a request through your web site (Incident #xxxxxx-xxxxxx) explaining my situation and my inability to get through to someone regarding the whereabouts of my phone. I received an email response saying that I would receive a call back. I didn't. On Tuesday, September 4, I called 1-800-331-6456 as suggested in the response to by online request. I was transfered to same call center I had spoken with on numerous occasions prior. I explained the situation and was given a new case number, xxxxxxxx, and once again told that I would receive a call within 24 hours. I didn't receive that call. On September 5, I received an automated response that my online request had been closed as "SOLVED". I responded that it was indeed not solved, and that I still needed someone to contact me. I received an email response, "Once again, we apologize for any inconvenience this delay may cause you. You will be contacted back to the contact phone number you provided for accurate assistance with your repair." This morning I called customer service once again, explained the situation, and was told that they would contact the repair facility to have them contact me. I explained that this process obviously does not work because the repair facility never calls me. I asked to speak to someone at the repair facility, and was told that no phone number was available. I asked to speak to the person responsible to managing the repair facility, and was told that no such person was available. I asked for the phone number of the Chief Operations Officer, or a person of a similar role in your organization, and was told that such information was not available. Finally, I simply asked to speak with the supervisor in the call center. I was told that the supervisor, whose name the CSR ("Nicole", #351359) refused to give me, was preoccupied and would call me within 10 minutes. I didn't receive that call either. Due to the malfunctioning phone, and inability to get it replaced in a timely manner, I have been without a phone for about six weeks, and have continued to incur charges from my carrier, Sprint, during this period. Your assistance in resolving this issue would be greatly appreciated. Regards, Christian Warden (temporary) mobile: 310.xxx.xxxx home: 408.xxx.xxxx
Update (09/23/2007): After I sent the email above, I received a call from Dorothy Hodges in the consumer advocacy department. She was very pleasant and eager to resolve my problem. She ordered me another Q, but they weren't in stock so she sent me a RAZR2 as a loaner. She also sent me a bluetooth headset as a sorry-we-screwed-up consolation token.
Upon receiving the RAZR, I checked my voicemail, and I had two messages from the management of my apartment complex. It turns out the original replacement phone was devlivered, but that Fedex didn't leave a notice, and that the tracking number Motorola recorded was incorrect. So last Monday I called Sprint with Dorothy and activated the Q. She requested a credit on my behalf since I had been without a phone for six weeks while still paying the bill. I managed to get them from their original offer of 100 minutes to $10 to $30, at which point I accepted.
See the Housing Bubble Blog for some real news.
A few months ago, Amazon introduced aStore, which allows associates to create their own Amazon store. It's a cool idea, but unfortunately, the current version is pretty limited in fuctionality. The store creator can only pick nine featured products sold by Amazon to display in the store. Categories of goods to be sold through the store can also be selected, but with only limited control over which products show up within these categories through the use of keywords.
For the nine featured products, the store builder can write a little description, but it would be nice if the products displayed the seller's product ratings, comments, etc.
Anyway, here's my store.
I was forwarded this email:
Date: Wed, 17 May 2006 22:43:49 EDT From: email@example.com To: undisclosed-recipients: ; Subject: If you invest in stocks, listen to this...from Bobby K. Turn up your volume and set aside about 10 minutes to listen to this. Theres a crack in the system whereby people can create bullshit shares of stock and sell it, where in essense creating more shares than there should be and driving a companies stock down and eventually destroying the company. Ive known about this for awhile and so have politicians, however, if they tried to fix it, the entire financial system of the U.S. would collapse. Ive always said that the stock market crash we saw in 2000 that wiped out alot of people would be a walk in the park compared to the next one that will happen. I think within the next 1-3 years the topic of these bullshit "phantom" shares will be the big one that will be the biggest hit the stock market has ever seen. Dont paic, just be careful. The stock market is done. The crooks have made billions in the last 60 years in the stock market and i think the shit will hit the fan soon. Listen... http://www.ultimateuploads.com/audio/view.php?play=10aa902b7bb9763edd29058fb220e350
I wrote the following response before listening to the audio. I should have guessed that it was the Overstock dude.
The CEO of Overstock.com bitches about naked shorts a lot. Read http://jeffmatthewsisnotmakingthisup.blogspot.com/. Mark Cuban talks about it on his blog sometimes too. There was also a 60 minutes story about it, but it wasn't very good as Leslie Stahl is about as financially literate as a chihuahua. While it's fraudulent to sell something that doesn't exist, the purchaser of a share of stock is the one who would have a complaint if he were unable to take delivery of said share. The CEOs who bitch about it are the ones whose compensation is in the form of stock options. If they were speaking on behalf of the company, they should be delighted that people are driving down the stock price. The company can buy back its shares cheaply enriching the existing shareholders. Imagine a company that sells 1,000,000 shares at $10/share during an IPO. If short sellers drive down the stock to $1/share, and the company can buy back all of its shares, it makes $9,000,000 without giving up any equity (neglecting underwriting fees and the upward movement of the stock price as the company starts to buy back a lot of shares). Likewise, if you're a shareholder, and you think the stock is undervalued at $1, you can buy more stock and wait for the profits to come rolling in. If your company isn't making any money, and you expect to make money on the stock, well, too bad. xn
I had to pay business tax to the city of Los Angeles last month since I receive 1099 income. The city took $600 from me, but it turns out that "creative artists" who earn up to $300,000 are exempt from the business tax thanks to the lobbying of the Screen Writers Guild. I should move to New Hampshire.
I have the fantasy of opening up a little cafe, apparently like many other people. Michael Idov has already done it and documented the money-losing venture at Slate.
His cafe, which he opened in Manhattan, was right out of my fantasy, "where coffee was served on silver trays with a glass of water and a little chocolate cookie" — it's actually a little chocolate square in my imagination.
He goes on to describe how he lost money on every croissant sold, and how he and his wife end up working "$8-per-hour jobs slinging coffee" before going bankrupt.
The European cafes that Michael tried to emulate probably don't $4,500/month rents to pay, because I don't think they get the to-go business that makes Starbucks so profitable.
In other free book news, I have Paul to thank for a free copy of The Number. I haven't finished reading it yet, but so far it's pretty good. The Number, by Lee Eisenberg, is about the amount of money you need to retire, and discusses the ways different people think about the Number. The book is targeted to those in their 40s and 50s getting closer to retirement who, for a variety of reasons that Eisenberg investigates, don't know what their Number is, but it's a worthwhile read for those of us pushing 30.
Its discussion of retirement being a rather new concept is pretty interesting. One hundred years ago, most people worked until they died. Then, people started to live longer, get kicked out of their jobs, and die in poverty. Then came Social Security. Next up, long lives requiring savings somewhere north of -1.5% during the productive years.
The state is that great fiction by which everyone tries to live at the expense of everyone else. - Frederic Bastiat