Microsoft Black Boxes

Ray Ozzie and Bill Gates


Happy Belated Birthday Microsoft (who turned 37 last week)…

Having worked at Microsoft for nearly half of Microsoft’s existence as well as more than half of my career it has provided one of my broadest pools of experiences including the areas I explore here. I was struck by Ray Ozzie’s (heir to Bill Gates as technical visionary of the company) memo (going on a little while ago now) that echoed so many of perspectives about ‘Black Box Complexity’ I have been investigating…

  • But as the PC client and PC-based server have grown from their simple roots over the past 25 years, the PC-centric / server-centric model has accreted simply immense complexity…. Complexity sucks the life out of users, developers and IT. Complexity makes products difficult to plan, build, test and use. Complexity introduces security challenges. Complexity causes administrator frustration. And as time goes on and as software products mature – even with the best of intent – complexity is inescapable…Complex interdependencies and any product’s inherent ‘quirks’ will virtually guarantee that broadly adopted systems won’t simply vanish overnight….But so long as customer or competitive requirements drive teams to build layers of new function on top of a complex core, ultimately a limit will be reached.”

It’s not just technical complexity that threatens to forge an inscrutable black box around the code base. But also organisational complexity can make the equally large and intricate matrix of decision making as inscrutable and opaque.

Microsoft 2.0

Microsoft 20 Mary Jo Foley

Speaking of technology earlier this month, my summer reading finally got around to probably the most insightful writer on my professional alma mater, Mary Jo Foley and her latest book on the subject, Microsoft 2.0. I couldn’t help gleaning an embracing failure gem…

“No matter how fault-tolerant and reliable systems are, downtime and outright system failure are unavoidable. Microsoft and other vendors seem to be talking less about 99.999 percent uptime guarantees these days. Instead, they’re focusing more on ‘graceful degradation’, ‘self-restoration’ and other realities. The Microsoft Research Eclipse project is all about designing distributed/fault-tolerant systems while taking performance realities into consideration.”

Microsoft’s Corner Office

Steve Ballmer Microsoft

The New York Times’ Adam Bryant interviews Microsoft CEO Steve Ballmer about leadership. Ballmer faces constant competitive pressures, new markets, new technologies, ever changing landscape, shareholder expectations and what does he say his biggest challenge is…

Q. What’s the most challenging part of your job?

A. Finding the right balance between optimism and realism. I’m an optimist by nature, and I start from the belief that you can always succeed if you have the right amount of focus combined with the right amount of hard work. So I can get frustrated when progress runs up against issues that should have been anticipated or that simply couldn’t have been foreseen. A realist knows that a certain amount of that is inevitable, but the optimist in me always struggles when progress doesn’t match my expectations.

Balancing upside and downside is one of the core executive issues requiring leadership (more upside) and management (less downside). 

Microsoft Senior Leadership

clip_image001                    clip_image003

Last month, I attended our annual global summit for Microsoft senior management to prepare for the upcoming fiscal year. The time is a chance to reflect on results, challenges and of course the leadership we provide to our respective parts of the company.

The host of the meeting is, like Allan Leighton featured last month, another Walmart executive alumnus, COO Kevin Turner who himself invests a lot of time, energy and thought on the subject of Leadership. At his keynote and later in an internal web-cast symposium he did on the topic, he shared a few choice words many of which focused on the role of adversity…

[Referring to the building of the Windows franchise] “They just refused to fail. Windows took ten years to be profitable.”

[Referring to Sam Walton’s description of looking for the downsides to address] “Divine discontent. No matter how well we did it yesterday, we can do it better today.”

[Referring to the Microsoft culture and values] “One of our corporate values is embracing self-criticism without getting de-motivated.”

[Referring to professional development] “Improvement always requires some degree of failure. Tough times don’t make you who you are, tough times show you who you are”

Kevin talked about building on one’s strengths versus fixing weaknesses. It might seem that not focusing fixing weakness would be out of step with ‘embracing failure’, but actually it is the other way around. Fixing a weakness is rejecting that shortcoming and investing sometimes disproportionate resources to overcome it. Embracing failure is accepting it and moving on. Of course, there are limits and contexts to the application of all of these tenets. Glaring or debilitating weaknesses certainly need attention. But, many times one can manage around the weakness typically through partnership. It’s really a variation of the management adage to ‘focus on core business’ (hopefully a strength).

Kevin does demonstrate characteristics of both the ‘Leader’ and ‘Manager’ persona as this blog defines it around upside and downside. His COO role is central to meeting the business commitments and ensuring the smooth operation of the enterprise (ie. a manager averting downside). But, when we talks about leadership, he focuses very keenly on the upside especially around people. His first principle of his leadership talk was about bringing “people from where they are to where they want to be.” He talked about a question he was asked by a manager and now he asks all of his reports when he first met them, “What are your dreams?”

Microsoft Global Exchange 2008

People Ready MGX FY08

Microsoft kicks off its fiscal year with a major internal conference for field staff each July where the big execs (Ballmer, Ozzie, Liddell, Raikes, and for his farewell tour appearance – Billg) outline their strategies and vision for the company to a crowd of 10,000 numbers-crazed and demo-overdosed sales and marketing folks (check out my colleague Georgina Mitcham’s TechNet blog for an overview of highlights). In the various rousing a keynotes a few comments emerged echoing some of the leadership themes of this blog.

Senior VP of Human Resources Lisa Brummel echoed the ‘great leaders are great energizers’ entry talking about Microsoft inimitable and indomitable energy and how “our job is to keep that energy going

Chief Operating Officer Kevin Turner, really the ‘host’ and force behind the event these days had a great comment articulating the Leadership/Management balance needed in great companies: “We need to have one eye on the horizon and one eye on today.” A bit of a mixed metaphor, but it still works.


Best Management Advice ever received by UK Microsoft Execs

Today I participated in a Management Excellence forum at Microsoft which covered all sorts of discussion about Management (and its link to ‘Leadership’) which in itself prompted several thoughts for upcoming entries on that part of my blog.  But one of the interesting parts were a number of perspectives on ‘Rising Each Time You Fall.’  The perspectives underscore a theme of this ‘Turning Adversity to Advantage’ blog that Microsoft is a company that very much ‘gets’ the concept of embracing failure as a potentially positive force in business.

The UK Board of Directors were asked in a roundtable Q&A, “What is the best piece of management advice you ever received?”  Several of the responses were fine articulations of rising each time you fall:

“A man who never made a mistake, never made anything at all.” – Chris Parker, Head of Law and Corporate Affairs

“You’re going to screw up, so get used to it.” – Terry Smith, Head of Public Sector

Attribute success to others and failure to yourself.” – Matt Bishop, Head of Developer and Platform Evangelism (who in turn attributed the advice to outgoing UK Managing Director Alistair Baker.)

You need to be a participant in your own rescue.” – Gordon Frazer, UK Managing Director

What’s New versus What Works

Gapingvoid - Innovation and fear


Yesterday was a high holiday on the technophile calendar with the latest Apple launch event.  New icons for the altar of the neophiles.

So many articles focus on “innovation” and “embracing failures” is often a theme in those examinations. But innovation for innovation’s sake is not really the objective. What we are really seeking are “positive outcomes” (on a micro, tactical level), and “progress” (on a macro, strategic level). Innovation is just a tool for that progress.

Curiously enough (since he normally weighs in heavily for Leadership over Management), Seth Godin makes a compelling appeal for the importance of Management (ie. averting downside by respecting “what works”) in the face of excessive pandering to new and shiny in his post Neophilia as a form of hiding :

  • “Every once in a while someone will say to me, ‘yeah, sure, I’ve heard that before… what do you have that’s new?’  In contemporary art or movies, it makes perfect sense to be focused on the bleeding edge, on the new idea that’s never been previously contemplated.  But when we’re discussing our goals, our passion and the way we interact with the culture, it seems to me that what works is significantly more important than what’s new. Racing to build your organization around the latest social network tool or graphics-rendering technology permits you to spend a lot of time learning the new system and skiing in the fresh powder of the unproven, but it might just distract you from the difficult work of telling the truth, looking people in the eye and making a difference.  ‘I can’t describe the value we deliver, I’m too busy integrating this new technology into my workflow!’  All too often, the ones who are aggressively seeking the theory of the day don’t have a lot to show for what they did yesterday.”

I definitely confronted this syndrome at Microsoft where senior executives were constantly wanting to hear about everyone’s colourful rain dance rather than the boring mechanics of the success achieved.

Leaders seek what’s new, Managers seek what works. Both together achieve progress.

Backup, Backup, Backup

Hard drive failure


If there was one word which captures the actionable advice behind the principle of embracing failure it would be…”backup”.  Not embracing failure means assuming that everything will go more or less to plan.  Embracing failure will be assuming and building into your plan – whether it be your business plan or your life plan – that things will go wrong.  Then you need a backup. A “Plan B”. If you have embraced failure with backup, then most failures in life will be minor bumps. Unfortunately, those bumps get a bit harsher when the backups themselves fail. Getting a flat tire is annoying. Getting a flat tire and finding that the spare in your boot is flat…is a big problem.

This tenet has been hammered home to me this past month with the failure that every digital citizen dreads – the hard disk failure.  In this day and age, computers have gotten so much more reliable; one could be seduced into thinking that they never completely fall apart.  The last hard-drive failure I had was in the 90s (though a reminder of hard-drive mortality hit me last year when an old machine I kept for testing had its drive fail). I do have a bit of a complicated computer environment.  I run a “virtual computer” on Windows (7.0) inside my MacBook Pro computer which doubles the operating environments each of which have their own backup protocols and tools.

Fortunately, practicing what I preach here, I was well backed up and now more or less back to normal.  I’m not sharing the story in some sort of sanctimonious ‘I told you so’, but rather as a wakeup call for just how down the downside can be of problems.  It turns out that not only did my hard disk fail, but a whole series of failures took place one after the other.  It was only the rigour of my backup practices (and a bit of extra money for some recovery assistance) that saved me. 

The litany of failures were…

  • Hard Disk Fail – My internal hard drive failed on my computer when I dropped it (okay, duh, but sh*t happens).
  • Disk Recovery Fail – I took the failed drive to a disk recovery specialist and the data on the drive was unrecoverable even with their clever clean room tools and tricks (these guys can sometimes do wonders…but not every time).
  • Mac Remote Drive Fail – The remote drive I used for Apple’s Time Machine back up program stopped working (fortunately just after I used it to restore my MacBook Pro…close call!)
  • Mac Backup Software Fail.  Apple’s backup program, Time Machine, actually worked impressively well and fast to restore my Mac environment to the clean replacement drive I bought…BUT it turns out that it doesn’t backup virtual machines (ie. my Windows environment) so that major part of my data was completely omitted. Fortunately, I had manually backed up the VM, but the backup was a bit older than my Time Machine one that had recently run.
  • Windows Remote Drive Fail – When I went to restore my Windows environment (under Parallels virtualization software), my remote drive chose to pack up and fail as well! As it turned out, it was just and chassis failure and the disk mechanism itself was fine. A superb service outfit in Las Vegas (Century 23) swapped the drive into a new chassis and it was fine.
  • Windows Backup Software Fail – After everything had been sorted out and restored recently, I resumed my backup of the Windows environment only to get a number of reported problems. The free backup process in Windows is convenient and value-priced, but it is not the greatest backup program (as I learned troubleshooting my way through various problems it threw at me).
  • Lori Backup Drive Fail – I used the occasion of all of the backup work to do a long overdue backup for my wife Lori’s computer. Lo and behold, I plugged in the Maxtor USB drive…and it too had failed.

If backups were spare tires, then my experience was the equivalent of having 7 different spares in the boot/trunk all flat!

In the final tally, I restored my machine and paid the deep price of the effort doing do, the stress of not really knowing what would and wouldn’t be restored, a productivity hit for a few days and about 4 hours of work lost from the time of my most recent backup before the crash. Not a disaster.

But what is more intriguing for here are the upsides to the whole mess. The incident and the aftermath actually and literally shook up my whole computing set up.  In the aftermath, I perversely feel that my computer is working better than ever and better than it would have been had I not suffered the blow.  I think that the productivity gains have quickly compensated for the hours lost in the lost work and recovery process.

Learnings – The first dividends of any failure are the lessons learned.

  • Hard drives are more likely to fail (in more ways) than you might think.
  • Apple Time Machine does not back up virtual machines.

Improving Backup – This bonus area is a combination of Learning and a more thorough embrace of failure. I thought that I was pretty well set on all of my backup protocols, but the “this is not a drill” crisis and the knock-on failures that followed (see above) pointed the way to a number of enhancements to my backing up now…

  • New Backup Devices – I’ve invested in two new Western Digital terabyte drives (one for the Windows environment and one for the Mac environment).
  • Wireless Backup – Part of my loss (4 hours of work lost) was from the frequency of my backing up. I thought that a weekly back up would be adequate. And it is for averting catastrophe. But not for averting the loss of hours of work you have done that week. I do more than 4 hours of work a week, but much of it is captured online. Since the incident I thought of doing of nightly backup setting it off before bed each night, but that will be easy to get slack on. The current state-of-the-art is wireless ongoing backup. A number of products on the market connect to your wireless router and their backup programs continually back up your work over the air (so you don’t have to think about instigating it).
  • VM Backup – Now that I understand better how Apple Time Machine deals with VMs, I am changing how I approach backing up the VM.
  • Organising Active from Archive Files – One of the things getting in the way of regular backups is the sheer volume. A good backup program should be able to do incremental backups on only the stuff you’ve changed, but in practice it needs to at least look at every file. And after 30 years of computing, I have a lot of files. And some programs (eg. DeepZoom Composer) produce not just hundreds or thousands, but tens of thousands as artefacts. Going through this pile slows the whole process down. The answer is to segregate your “Archive” material (files that are not like to change like archived email, photos, videos, old documents) and “Active” material (your current documents). When I did this, I found that only 4 GB of my 90 GB were in any way “active”. Once I segregated them, now even a brute force backup of my 4 GB active files just takes minutes (and I could put all of them on an SD card to have access to on any machine) compared to hours before.

Cleaning House – The new hard drive in my MacBook Pro meant I could set it up properly from scratch. I always remember in the early days of the PC that when there was a Windows upgrade, it was often a good idea (given the awkward complexities of the platform at the time) to simply back up all of the data and format the hard drive for a completely fresh start. You would have to go through the rigmarole of re-installing all your apps and restoring all your data, but it was a worthwhile investment to get everything running the most smoothly. Now upgrade processes are so refined that this approach rarely makes sense. But that situation does mean that people now go years with lots of mess slowly accruing on their machines that becomes hard to decipher and untangle.

  • Reconfigured New Environment – When I first got my MacBook Pro 3 years ago, I was a relative novice to the new environment (after my 15 year career at Microsoft). As such, I made a number of missteps along the way of setting up and building my environment over time (eg. I set up a partition I didn’t need, I got a clutter of Identities, I set up two user profiles when I could have done one). I learned the error of my ways, but in a number of cases it was easier to kludge a work-around than roll up my sleeves for a comprehensive fix.
  • Cleaned up MacBook Pro – The Genius Bar of the Apple Stores is about the best conceived and best delivered service concept that there is. With my machine checked in to install the new drive, they also took the opportunity to replace a missing screw and pad and clean out the dust that had accumulated inside with a specialist vacuum. I even go my cracked iPhone back fixed for a modest fee (didn’t know you could do that at the Apple store).
  • Enhanced Backup Setup – I set up an entirely new Windows backup using a Western Digital MyCloud wifi system. I also changed my Mac Backup to a new Western Digital My Passport USB drive for Mac.

The moral of the story: Embrace the failure of computer hard drives. A cautionary tale on the too often taken for granted world of backing up your computer. Belt, braces and duct tape. Backup, backup, backup.  And when the catastrophic failures do happen, you can look forward to the stronger situation you will get to in the aftermath.

Praising to Fail vs. Failing to Praise



We all have lots to thank our mothers for today. Self-confidence is near the top of the list being one of the greatest assets parents (and other significant adults) can nurture in a child. We know its value and we enjoy the smiles that typically result from immediate praise. Balancing praise, however, is one of the great challenges of a leader/manager, coach, parent can face. Too much praise, with ribbons for coming in 10th, and it becomes diluted. The praise becomes ineffective like an over-prescribed anti-biotic. Either the person sees right through the pedestrian nature of the praise, or else they are set up for a bigger fall later in life. Of course, too little can also result in frustration and exasperation.

Sian Griffiths explores this dynamic in her Sunday Times piece “Praise her see her fail” (paywalled)

  • “’Admiring our children may temporarily lift our sense of self-esteem but it isn’t doing much for a child’s sense of self,’ he says. ‘Empty praise is as bad as thoughtless criticism — it expresses indifference to the child’s feelings and thoughts.’ It is the chapter ‘How praise can cause a loss of confidence’ that has been seized on by the chattering classes, for whom praising one’s children is as natural as putting them down for swimming or music lessons. In The Examined Life, Grosz cites research by the psychologists Carol Dweck and Claudia Mueller, who, as part of an experiment, asked 128 children to solve maths problems. On completing the first set of questions, some children were told, ‘You did really well — you’re so clever’; the others, ‘You did really well — you must have tried really hard.’ Both sets of children were then given more difficult maths problems. Those who had been praised for their efforts solved more problems and worried less about failing than those who had been told that they were clever. Even worse, when asked by the researchers to describe the experiment, some of the “clever’ children lied about the results: they exaggerated their own scores. ‘All it took to knock these youngsters’ confidence, to make them so unhappy that they lied, was one sentence of praise,’ writes Grosz. They felt they had to live up to the erroneously inflated opinion others seemed to have of them…When his daughter was two, Grosz and his wife, Nicola, listened to their babysitter lauding their child ‘for how she took the wrapping off a cupcake. My wife and I looked at each other, like: ‘This is insane.’ Why would someone go on a kind of babble of: ‘Oh, that’s so great — you took that wrapper off so beautifully’?’”

The undertone here is of course an embrace of failure. Letting go of the excess praise, empty victories and shallow successes in favour of richer development and growth over the long term.

Embracing the Failure of Belief

Dilbert - dinosaur myth


Scott Adams’ post “My Skeptical Journey” is autobiography of sceptical doubt that only seems to intensify and expand as his life goes on. In a transparently public confessional, he provides an open viewing to practicing what he preaches…

  • A rational mind needs regular maintenance. One of the maintenance systems I employ is to remind myself of things I used to be sure about and later discovered to be untrue. I started a list organized by the approximate ages at which I realized my errors. A healthy rational mind needs regular doses of humility. (I might need more humility than most people.) Here is the approximate age at which I stopped believing in different stuff…”

He then goes on to catalogue 42 different things he has stopped believing in from the age of 8 to 50+. Many of his failure of his confessions of failed models and notions paralleled my own – childhood fairy tale myths, open-mindedness to paranormal phenomenon, vitamin supplements, the legitimacy of the stock market. Some of his shifts I myself haven’t come to yet mostly in the area of sport (eg. “Exercising is a big help for losing weight…Stretching helps athletic performance.”). I suspect that his experience in exercising is focused on certain types and approaches where possibly these general principles don’t specifically apply as strongly.

My own confessional of beliefs that I stopped believing in include…

  • Teens – Politics is a noble career.
  • 20s – Ignorance of the devout.
  • 30s – Sales is a shallow discipline.
  • 40s – Microsoft.
  • 50s – Flying superstitions (more of a meditation as way to relax myself, but after the Skeptics in the Pub session on superstition, I decided to drop it).

One of my most fundamental disillusionments throughout my life has been about the nature of reality itself. Books such as “Godel, Escher, Bach” and “Thinking Fast and Slow” opened my eyes to how deceptive my own eyes can be.



Get every new post delivered to your Inbox.

Join 150 other followers