Guest essay by Eric Worrall
Why pre-judging the value of potential stepping stones to a solution, stifles scientific innovation.
Kenneth Stanley, one of the world’s top artificial intelligence researchers, has produced a fascinating presentation, on why focusing on an objective can sabotage innovation.
Consider how you would teach a robot to solve a maze. The obvious thing to do, is to teach the robot to do its best to reach the end of the maze – to reward movements which bring the robot closer to the end of the maze, and punish movements which cause the robot to move away from its objective.
However, researchers quickly discovered that if you give these logical seeming instructions to a robot, something unfortunate happens.
The obsession with finding a solution, and on focusing resources on actions which seem to bring the robot closer to the solution, blinds the robot to the possible paths which actually lead to a solution.
How do you solve this dilemma? Ken’s solution to preventing this kind of hangup is incredibly simple – the way almost every ingenious breakthrough is simple. Instead of punishing the robot if it makes a move which takes it away from its goal, you reward the robot for innovating – for always doing something new. The only caveat is, the robot should try to avoid repeating itself.
Focussing only on trying something new avoids prejudging potential stepping stones to the solution. Until you know what the solution is, there is no reliable way of knowing how valuable a potential stepping stone might be. The only way to solve the maze in my example, is to take a path which leads away from the solution – a path which an entity obsessed with reaching the solution would find very difficult to take.
Or to put it another way, if 3 decades of attempts to build a model which works, has failed to deliver results, maybe its time to stop repeatedly butting your head into the same wall.
This approach to problem solving, focusing on novelty, does not prevent you from monitoring the process, to see whether a solution has been found – solving the problem is the ultimate goal. All novelty search means is that you keep your opinions to yourself – you let the robot do its thing, regardless of how silly any individual move might seem at the time, until it finds the solution on its own terms.
The following is Ken’s presentation “Novelty Search and the Myth of the Objective”. Ken’s presentation covers a lot more than my quick summary – including valuable insights into why scientific advance has stalled in a number of disciplines.
Update – Ken Stanley has published a book, available for pre-order on Amazon, which discusses his ideas in more depth.
Click here to hear an interview with Ken, about his new book.