Man versus Machine

Like two male deer locking antlers, there is a lot of debate these days about whether the next generation of machine learning tools will take out the need for decision scientists running statistical tools under their command.  

I was exposed to Artificial Intelligence early on in my career, working on a machine-learned lending and underwriting platform for Home Savings of America in 1992.  They employed a human underwriter named Angus, who for whatever reason, had a near perfect underwriting record around loan performance.  They allowed a new tool, developed by a team at UC San Diego, sit next to Angus for 18 months while the software trained its learning on his day to day decisions.  Upon launching, the new system, actually called Angus, went on to review each loan made that year, about $2 billion in lending.  It was viewed as a success.

Fast forward to this year, and I seem to meet a new analytics company every month that promises to automate any statistically based solution.  The highest profile one is IBM’s Watson and their push to cognitive learning.  While I am very enthusiastic about these tools, I also remember that Scottish underwriter with a monster brain who taught our machine. 

I think the single greatest accelerator of these tools was the open source revolution of “free” software that could serve as building blocks to build out this next generation of software, fueled by tools like R.  What I see lacking in their sales pitch is decision scientists sometimes tackle problems that address millions and billions of dollars in these companies.   You can fire Angus if he made a bad loan decision; who do you address if your machine goes wonky?   Because the promise of machine learning is that it can go through thousands to millions of permutations, you never know what was put aside in the decision process.   You reach the final conclusion and hope it is the right one.

What I have learned from this, is that this is best served up as a parallel process.  Home Savings of America was smart to allow its machine Angus learn from the real Angus over 18 months before going live.   I myself have been testing this promise with several new algorithmic media buying tools, like Rocketfuel, linked to reducing the Cost Per Order with each impression served.  What I am finding is that it takes several months for their tool to really sing.   While running parallel control groups, we have also found that intuitive selections based on simple demography have actually outperformed the machine.

Like 2001 and HAL, we are probably not far off to having some amazing breakthroughs in how AI and machine learning will impact our daily life.  I also think we are a long way off before Man is supplanted.

Comments are closed.