The debate about AI and the benefit or impact it could have for society, has grown to a scale which dwarfs the most basic question – what actually is it? It has exploded to the extent that much of the thinking about AI is more like fantasy or dystopia than real analysis. Going back to a definition of what AI is would allow us to have a balanced idea of it, to not refuse its considerable advantages, and to prepare for its risks, which are not always ones we expect.
Initial attempt at a definition: “Machine production of processes similar to human cognitive processes.”
This definition has the merit of being generic enough to cover all aspects of AI. It’s important to retain the distinction between cognitive AI and connectionist AI, the former trying to model human reasoning and inference (expert systems, knowledge-based systems, fuzzy logic, Bayesian inference, etc.) and the latter using connected networks whose links are reinforced or weakened to adapt to the expected response, modelled of course on neural networks. Enthusiasm about the results obtained by the latter leads too often to considering AI as the same as NN, but this simplification should be avoided.
Our initial definition soon reveals several flaws. Firstly, AI can achieve performance comparable to that of human cognition without trying to imitate it. Most chess software essentially consists of a well optimized tree structure browsing engine and a very finely tuned scoring function. Nothing in that resembles human cognition, mostly because it involves much greater computing power than any human possesses. The final result looks like intelligence: when playing chess, the machine seems endowed with purpose and reasoning. In the background however it’s just a powerful combinatorial search.
What’s more, even AIs based on mechanisms inspired by human biology, such as neural networks, have only a distant relationship with biological neurons in the brain. NN are examples more of the importance of reinforcement and self-organisation of controller networks than any similarity with biology. The first, naive, approach to AI is to think that it is necessary to create a synthetic human, or a synthetic brain to produce cognition: in fact, cognition does not need to be anthropomorphic at all.
This brings us to:
Second attempt at a definition: “The ability of a machine to achieve performance equal to or better than certain human cognitive processes.”
This definition is based on the final outcome, without presupposing imitation of biological mechanisms. So it covers the whole range of what the machine is able to do by appearing to be intelligent, to have reasoning ability, or to be able to use a strategy in pursuit of a given outcome, without the underlying processes necessarily being modelled on these capacities.
But once again, this definition comes up against a hard to define boundary. If we are considering the level of pure performance, without including any attempt to imitate human cognition, many classic computer information processing capabilities would fit into this definition, for tasks that we would define as intelligent.
For example, cross-checking information can make the computer a Cluedo or Mastermind champion without the need for AI. We don’t even notice what progress is being made in some areas of computing because we take it for granted, for example SQL querying on large databases. All these techniques lead to a result that superficially seems to be the work of an intelligence and even far surpasses human capability in their field, without there being any question of AI.
What surprises us about AI techniques is that they manage to address a new range of problems that would be impossible to solve with conventional programming. The first AI successes relied not on cognition but on perception – image or handwriting recognition i.e. objects with very many small variations which cannot be captured by a pure combinatorial calculation.
It therefore seems awkward to define AI without reference to a measure of the complexity of the problems it can address. AI has allowed some breakthroughs in solving NP-complete problems, for example that of the Travelling Salesman. One of the fields of choice for AI is also pattern recognition within a series of continuous signals, although it’s not known whether these problems can be qualified as NP-complete in terms of their complexity.
Other than their classification on traditional scales of complexity, the problems AI is best suited to solving are those that go beyond processing by conditional expressions, those that cannot be summed up as an analysis of simple cases of if…then rules. AI can be usefully applied to “near automatic” problems: those that are routine enough to be done by a machine, but with countless variations that do not allow pure automation.
You can express this by saying that AI does not just work with declarative knowledge, but involves “procedural knowledge”: the problems AI is used on cannot be solved by writing an explicit specification, even one of several thousand pages. Part of their resolution is “lost” in the algorithm itself, in the course of the execution of the procedure, especially when it involves heuristics.
In particular, the recognition of “continuous patterns” such as writing, or the actions to be carried out by a vehicle while driving are good examples of such implicit knowledge. It is a question of distinguishing between continuous very subtle nuances of several signals. So any case study based on simple conditional rules is doomed to failure. Naive approaches to autonomous driving try to find fixed safety rules, such as safe distance limits, acceleration limits, etc. These kind of rules collapse under the weight of the complexity of the situations, because the exceptions start to multiply as soon as the rule is formulated.
In particular, setting consistent thresholds always reveals problems. The behavioural rules, the “laws of vehicle control” can only be described as the final result of the algorithm, which could itself be the result of several contradictory processes arbitrated over by a meta-rule. The driving strategies of an autonomous vehicle are increasingly described as a compromise between several contradictory assessments of the situation, involving several competing programs in a framework with a decision engine responsible for establishing the arbitration between them. The only set specification is the algorithm of each program, which does not predict which path will be taken at runtime.
This brings us to:
Third attempt at a definition: “The ability of a machine to achieve performance equal to or greater than that of certain human cognitive processes, on problems reaching either NP-complexity or where the solution cannot be written entirely in an explicit specification.”
However, the limits of this already somewhat overworked definition become apparent using more sophisticated techniques: those of statistical data analysis.
The different variants of factor analysis as well as discriminant analysis can in fact be comfortably fitted into this third definition. Discriminant analysis in particular becomes extremely difficult to distinguish from all “Machine Learning” techniques, to the extent that one may wonder whether these are not simply variations on it.
For this reason, any AI specialist should be required to have prior experience in statistical data analysis techniques. One cannot understand AI properly without first understanding what these techniques are capable of and the level of finesse they already operate at. They have existed for much longer than AI, and they consist mainly of geometrical mapping and distance definitions associated with plausibility calculations.
Data analysis enables classifications or forecasts to be made that go beyond the composition of conditional rules. Its automatic aspect can appear confusingly like intelligence: during factorial analyses, “concepts”, for example regarding sociological data, seem to emerge from nowhere. In one example, we can identify and explain the main “tribes” of French students in 2018, by overlapping tastes, behaviours, political commitment etc.. There are many situations with a degree of complexity covered by the third attempted definition, that of procedural or implicit knowledge, which have been resolved by data analysis.
Even the notion of learning, leading those who develop neural networks to believe that they are working with a living organism, is found in discriminant analysis: it may require several trials and variations before providing its classification model. Neural networks cannot prove that they pass the tests for life and cognition, but rather that categorisation activity is an important part of any act of cognition.
Some authors believe that the idea of AI is no more than a fad, and that it can be reduced to some new variants of statistical data analysis. With respect to discriminant analysis, it must be recognised that the neural networks that were developed before the invention of “deep neural networks” only display a difference in degree, not in nature.
DNN are a new technique for moving from identification to statistical analysis. Far from being a simple addition of further layers, DNNs manage to combine the identification of relevant knowledge and classification based on that knowledge. For example, the Alpha-go program managed to beat Stockfish, the best chess program based on raw combinatorial computation, by identifying by itself high level chess “concepts” such as the pair of bishops, the isolated pawn, etc. Alpha-go calculates a much smaller number of combinations than Stockfish, but manages to find the best strategies corresponding to the logic of each chess position.
If we compare their mode of operation to that of statistical data analysis, we might state that the DNNs are capable of initially extracting the principal components from an analysis of the data, to bring out the most relevant variables, then using these variables to carry out a discriminant analysis to arrive at the right decision. Not only are the two paradigms of knowledge representation and decision respected, but they are performed simultaneously and optimised to get the best result.
This is a decisive step: until recently, neural networks had to be fed “pre-chewed” data by providing them with input data which was already represented in a relevant form. The machine was not able to correctly break down the reality it was observing on its own: it had to be given the variables corresponding to what intuition and human experience would consider the most significant in order to properly analyse the problem. We know how important knowledge representation is in computer science in the field of object-oriented design, where the choice of initial classes strongly determines the effectiveness of the processing that will start from these classes. Knowing how to find the right classes is knowing how to present reality in the right way for the given problem, i.e. finding an adequate representation of reality before dealing with it. We know, for example, that Eskimos have several dozen words describing the different states of snow. They have refined their representation of reality on this point, for obvious reasons of survival that crop up much more in their daily lives than in ours.
“Supervised” learning consisted less in the fact that output data were already classified into a priori groups, as in discriminant analysis, than in the fact that it was often necessary to try several means of representation of input data before the network converged correctly, and that this kind of trial and error remained the prerogative of humans. In other words, neural networks carried out classification activity to perfection, but not prior conceptualisation.
The DNNs are taking this very important step. They no longer need to be shown the relevant concepts to solve a problem: they construct them themselves in the deep layers, and refine them according to their effectiveness in subsequent classification.
AI started to truly be worthy of the name once DNN were invented. The barrier of conceptualisation has been crossed, the machines’ “myopia” which prevented them from extracting the most effective content summaries to deal with a problem is no longer relevant. An objection may be raised that this is still a sequence of two statistical techniques, that of factorial axis extraction and that of discriminant analysis using these axes as variables. However, being able to link the two correctly, i.e. to see how the first is useful to the second, is much more than just successively applying them.
So we can propose this:
Fourth attempt at a definition: “The ability of a machine to achieve performance equal to or greater than that of certain human cognitive processes, on problems reaching either NP-complexity or whose solution cannot be written entirely in an explicit specification, by extracting the relevant representation of the input data on its own without having to have it done for it”
Have we reached a complete definition? Once over the barrier of defining the concept, there is another one, even harder. It is this that shows that AI is still very far from true human intelligence, although it now merits the name under the fourth definition, because this time it is a different issue from all the data processing techniques that preceded it.
In what respect could an AI not match our own human intelligence? The final frontier – the most difficult – is not one of concept, but of context.
The self driving car is a particularly interesting case in point. Three years ago, car manufacturers and digital giants thought it was a simple problem, at least on motorways, that would be solved by now. The recent setbacks in autonomous driving have shown that what appeared to be a fairly automatic task – what could be more routine than driving in traffic jams – revealed totally unexpected complexities. Today, the main players in autonomous driving are having to demonstrate much more humility, and are only forecasting Level 4 Delegation – where you really don’t have to pay attention to the road any more – for 2023 at the earliest.
But how is that possible? We know how to produce AIs capable of beating world chess and go champions, but they come to grief on something as down to earth as driving a car? The nature of the problem is quite different. What makes driving more difficult for a machine than the most complex strategy games is the same reason that in a foreign language, it is the language of everyday life that is the most difficult to master, far more than “business English” or “legal English”.
However complex, strategy games remain semantically closed universes. The world of driving is a semantically open universe. We may raise the objection that this is a real life situation that is sufficiently regulated as to amount to a corpus of finite and explicit rules. However just a few examples will show to what extent this underestimates the richness of reality: when a daily activity, even a specialised one, dips a toe into the real world and is no longer confined to a conventional rule-based system, it is entirely caught up in contextualisation problems.
One of the accidents that happened to an Uber occurred because a vehicle coming towards it cut across the road to turn left when it should have given way. Legally, the Uber had right of way. However, in this accident, as in others, human pragmatics adopted behaviours that can’t be written in any law. The Uber vehicle had a number of cars in front of it that obscured its view. In a situation like that, even an average driver slows down, thinking that it’s possible that a careless or impatient person might cut across in front of him on the approach to a junction.
In driving situations, human beings are constantly predicting potential trajectories of other vehicles, even vehicles or obstacles that they can’t see, but that they forecast as having a certain probability of suddenly emerging. The human driver anticipates the trajectories and objects he will encounter and in this respect turns out to be significantly better at it than the machine. This anticipation of potential events also explains why the autonomous car is not just an automation problem, unlike driving aids such as emergency braking or the speed limiter.
The mechanism of human anticipation may involve cultural factors which go way beyond the motorway context. If we see a ball suddenly appear on the road, we brake and expect we may see a child who has a high probability of running out following the ball. Even more routinely, every time we pass a slip road when we’re in the slow lane of a motorway, we anticipate the fact that the vehicles coming onto the motorway will try to insert themselves into the stream of traffic, involving them possibly suddenly entering our lane. A “driving negotiation” ensues where each driver tests the reaction of the others, begins a manoeuvre to test whether or not the other driver will give way, changes his mind, or may even use his manoeuvre as information intended to inform others of his intentions. For example, it can consist of slight pumping of the brakes and using the rear hazard lights to make the driver behind us understand that he’s too close and that we may have to brake hard or even stop because a vehicle entering from the slip road may cut in front of us. A good driver will use these small repeated manoeuvres to make others understand his intentions, while a novice driver would tend to brake abruptly or not anticipate the actions of the driver trying to enter the lane.
Could an AI grasp all of this? We could of course see it as just a set of behavioural rules for continuous analog signals, i.e. the trajectories of other vehicles, an issue broadly similar to the handwriting recognition. Even anticipatory behaviours should be able to be learned by a good neural network. But it’s not that simple. Because it is no longer just a matter of recognizing complex continuous forms: we are also participants in the scenario, and any physical action on our part is also interpreted as an intention by others. The same applies to driving as to quantum mechanics: there is no observation and certainly no neutral action. The infinite reflection of our anticipation of others’ actions and ours by them is very quickly taken into account. The most effective models of driving negotiation therefore borrow a lot from game theory. We are therefore no longer in the situation of optimisation of an objective decision like in chess or go, but in the intertwined anticipations of each parties’ subjective interpretations.
Worse still, the expectations of behaviour on the road may involve cultural or regional factors: driving is not the same in France, Italy or Russia or even between different regions within France. These are not just quirky regionalisms: there are fewer real accidents when everyone correctly anticipates other people’s reactions. And what is considered as normal expected behaviour is no longer objective, but a subjective cultural convention. However, this has very concrete consequences: when everyone’s expectations are synchronised, accidents are avoided because the anticipated behaviour is the one that occurs: there is an intersubjective construction in the avoidance of road accidents, not only the observance of objective safety rules.
This intersubjective factor takes on huge importance when there is snow on the road. Road markings can be completely obscured. In which case, the lanes that should be followed are no longer those delimited by the road markings, but those that the lines of cars have formed spontaneously, possibly transforming a 4-lane motorway into three lanes. The true road becomes in this case one which has been spontaneously formed by intersubjective organisation.
One last example showing the difficulties of contextual factors: when we cross an area of road works, we do not realise how easily a human driver can answer the simple question “Where is the road, where does it carry on now?” However this question is far from trivial. One cannot rely on perception alone: the boundary between the carriageway and the roadworks is sometimes entirely unclear. In the blink of an eye, the human driver processes sensory, logical and even cultural factors to understand the situation and see where the road continues. Logic will allow him to rule out absurd assumptions about the direction the carriageway would take. And cultural knowledge will permit him to distinguish between construction machinery and personal vehicles – even if they are sometimes exactly the same model – by their behaviour, the task they are performing or their paintwork.
All this, one may well say, should be easily achievable by Machine Learning: after all, aren’t very subtle variations in context its field of expertise? That would be to underestimate an obstacle encountered by all AI techniques, perhaps the most fundamental and the most serious.
All the examples cited about autonomous driving would be manageable if they were each taken individually and distinguished by unambiguous criteria. AI still does not know the degree of contextuality of the situation it faces. In particular, it does not know whether it is overfitting, i.e. whether it is solving the problem posed by ad hoc considerations or whether it has attained a real capacity for generalisation. The separation of data between the learning base and the base used to test the generalisation capacity of the machine remains a human operation, the last area of supervision required by the machine.
Thrown in at the deep end of a complex contextual situation, the machine does not yet have the capability to assess by itself what level of generalisation it will require. Human beings already have the ability to know for themselves whether they are facing a problem with a limited context or with a broader context, and can self-programme to deal with it: they will adjust their strategies to avoid overfitting, or after trial and error will assess that they have reached a capacity for generalization that is insufficient to fully respond to the intrinsic complexity of the problem and will embark on more ambitious learning to be able to resolve it. In limited contexts, human beings voluntarily “robotise” themselves and know how to return themselves to a state of sufficient flexibility if the context expands.
Finally, the human being knows how to use the data banks of his previous experience, even when this has only a distant connection with the problem posed: for example, to differentiate the construction machinery from other industrial vehicles, he will call upon memories which he may have acquired in a very different context from that of the road, in his professional experience in a business, or in types of behaviour which he will have noticed while watching a film. As soon as the universe is semantically open, even a specialised context like the road attracts to it all the other contexts of everyday life and makes them participate in the correct interpretation of the situation. In everyday life, we are able to swiftly mobilise a great deal of other knowledge that may be very old and ostensibly far removed from the problem to be solved and are able to judge for ourselves whether we have adjusted ourselves to the complexity of the problem to be faced.
It is no surprise that the final frontier for AI is the one that has just been described. It may correspond to what we usually call “consciousness”. But rather than floundering in the vague and almost mystical surrounds of this ill-defined notion, the difficulty that AI encounters in approaching it gives us a much clearer image. Consciousness is perhaps only the ability to assess for ourselves how deep we will have to go into the immense collection of our memories to confront all facets of a problem, and adjust ourselves accordingly. It is a more concrete definition of consciousness, closer to the one that some Asian philosophies cultivate: to be conscious is to measure the extent of the depth of our relationship with the whole universe, with the cosmos, and to know how to utilise them ourselves in each situation of our lives.
Machines are capable of forging their own concepts, but not yet capable of being aware by themselves of the degree of contextualisation of the problem they have to face. If one day a machine were to possess this faculty, we could begin to refer to an AI that would threaten the preserves of our humanity. For as long as this is not the case, AI remains a powerful technology for statistical analysis, potentially dangerous if misused, but still far from being equivalent to human intelligence.
This post is also available in: FR (FR)DE (DE)