“AI may quickly point out a corrupt official, but it is not very good at explaining the process it has gone through to reach such a conclusion,” the researcher said. “Although it gets it right in most cases, you need a human to work closely with it.”
And that's the crux of the issue with AI being used in any law enforcement situation.
If we allow decisions and conclusions being drawn by the AI without a clear explanation of how it got there, we're just creating a monster that will advise -maybe replace- the judgment of law enforcement professionals who won't have the means to question these decisions.
Catching corrupt officials is a laudable goal anywhere and I'd like to see it applied to highlight potential irregularities that may require a second look but the danger here is that an unprovable AI be used to make claims or start being used as sufficient evidence to ruin people's lives.
This reminds me of Klaatu's speech in "The Day The Earth Stood Still." They created a race of robots to prevent aggression.
"The test of any such higher authority is, of course, the police force that supports it. For our policemen, we created a race of robots. Their function is to patrol the planets—in space ships like this one—and preserve the peace. In matters of aggression, we have given them absolute power over us; this power can not be revoked. At the ?rst sign of violence, they act automatically against the aggressor. The penalty for provoking their action is too terrible to risk. The result is that we live in peace, without arms or armies, secure in the knowledge that we are free from aggression and war—free to pursue more pro?table enterprises. Now, we do not pretend to have achieved perfection, but we do have a system, and it works."[1]
Fabulous movie. The 1950s one anyway. We'll not speak of the 00's awfulness.
If we continue to blindly stumble towards that future we are likely to set 21st century (USA) biases in stone. Racial, national, sexual and corporate biases etc will be baked in. We've seen they usually are in the systems we have already. Yet it will be done mostly innocently, and unaware from accidental bias in training data.
There's a rather obvious gap between calling someone names and throwing nuclear weapons at them.
As for controlling thoughts - that's the easiest thing in the world today. There are entire industries devoted to making sure people think as they're told to, and act as they're told to.
Of course they don't work on everyone. But if you can fool more than half of the people when you need to, you can do more or less anything you want.
My point is there are effective ways to destroy people without using overt violence.
> There are entire industries devoted to making sure people think as they're told to, and act as they're told to.
I'm talking about punishing people for the use of certain words with the idea that it will reshape their thoughts, which is not the same thing as persuasion techniques.
They're solving the wrong problem. Instead of writing "an AI that determines guilt" (e.g. a classifier) they should be trying to write "an AI that compiles a list of evidence against the accused".
This list of evidence would then be subjected to the same scrutiny that a list of evidence from a department of human police officers would be subjected to. If the AI cannot make the case against somebody in a way that's persuasive to a human prosecutor, then the case goes nowhere.
If the AI cannot provide rational clear and understandable justification for it's assertion, then it's not ready to replace humans who can be expected to provide justifications for their beliefs. Police officers are expected to write police reports, and so should any AI that's meant to do their job. Right now they've just got the AI equivalent of a cop who says "I feel in my gut that he's guilty, and I'm not going to explain why." But because it's "AI" I guess some politicians thought that was good enough.
> Instead of writing "an AI that determines guilt" (e.g. a classifier) they should be trying to write "an AI that compiles a list of evidence against the accused".
The problem there is selection bias. If you have a list of fifteen things, three make it slightly more likely they've committed a crime. Seven make it slightly less likely. Two make it significantly more likely and three make it significantly less likely. If you consider them all, the probability they've committed a crime is in the "probably not" range. But if you take the list and prune everything that makes it less likely, leaving only the factors that make it more likely, you get a distorted result. This is what prosecutors do for juries in general, and it's problematic.
Now suppose instead of a list of fifteen you start with a list of fifteen million and do the same thing.
Legal systems avoid this problem by imposing a high degree of confidence on the persecutors, just like some areas of physics require 5 sigma on their p-values and thus can ignore all the problems with p-value hacking and repeating experiments.
Of course, criminal law works on a much lower confidence level than physics, so it needs a trial where the defense is in charge of gathering that exonerating evidence when the persecutors get a false positive. It could work better, but this one problem is already accounted for.
"Beyond a reasonable doubt" accounts for only disclosing the five most incriminating factors out of fifteen.
If you instead got the thousand most incriminating factors out of millions, it looks like a damning mountain of evidence when it's really just the first page of the list of millions of factors sorted by how bad they look.
It completely destroys anyone's credibility scale because seeing a dozen one-in-a-million matches intuitively feels impossible to be a coincidence, but that's less than the result you get by random chance when you attempt to match fifteen million factors.
I don't think you'll find a persecutor willing to take a laundry list of low probability evidence and go after it, whatever the size of the list. Almost certainly, the longer you make the list, the less people you'll find willing to use it.
It's not about the length of the resulting list -- just the opposite. It's that you can take a list of a million and prune it to a thousand and suddenly everything bad happens with 1000 times the expected probability because you excluded the 999 things that didn't for every one that did.
If you take the list of a million and prune it to ten it's even worse. You can find ten 1:100,000 probability events that occur randomly in a list of a million. It's this:
That's no different from how regular law enforcement works. It's the standard to which any autonomous law enforcement should be implemented.
If you want to build better prosecutors that understand statistics, that should be considered a separate project. One AI that assembles the best case against the accused, and another AI that critically examines that case and discards it if it fails to meet some threshold for statistical relevance, or passes it on to human prosecutors if it does.
One is that the selection process is still opaque. If you have a corrupt prosecutor you're screwed either way. But if you have an honest prosecutor and you give them a collection of evidence selected for a bias in favor of guilt, they have no independent way of knowing that. So now you not only need an honest prosecutor, you need an honest AI, and we're back to the opaqueness issues.
The second problem is that in principle right now if the prosecutor and the defense attorney spend an equal amount of resources, they each get proportional results. If you create this database which the prosecutor can use, can the defense also use it? Do they get access to the data to run their own queries and algorithms against? If so it seems a rather large privacy problem, since the database would contain sensitive information about everyone, but if not then you're handing an asymmetric advantage to the prosecution.
I believe the data being compiled in the first place is the privacy problem - especially when it is in the hands of government officials with the power to jail, seize, and execute.
Transparent at least allows everyone to go through it and undermine it and the system throughly. "The judge has more corruption data points than the accused sentenced, and this newborn with a Han name has a far higher rating than this infamous connected embezzler!"
Explainable isn't the same as justifiable. An expert system may be able to readily explain that it determined the chance of recidivism is high due to factors x, y, and z. However, there is no guarantee that those factors correlate with recidivism at all since expert systems were simply built to emulate (human) expert decisions.
While it's very easy to do machine learning _incorrectly_ it is also possible to reasonably attribute factors to outcomes based on large quantities of data. You can also look a LIME/SHAP and other factor contribution metrics. This seems like a significant improvement over expert systems.
> Explainable isn't the same as justifiable. An expert system may be able to readily explain that it determined the chance of recidivism is high due to factors x, y, and z. However, there is no guarantee that those factors correlate with recidivism at all since expert systems were simply built to emulate (human) expert decisions.
Re-reading the article now, it may very well be an expert system. The "big data" mentioned isn't training data, but pre-existing databases like bank account info, salaries, contracts, property ownership, etc. FTA:
> Disciplinary officials need to help scientists train the machine with their experience and knowledge accumulated from previous cases. For instance, disciplinary officials spent many hours manually tagging unusual phenomenon in various types of data sets to teach the machine what to look for.
As far as explaining the results, it's hard to draw any conclusion without knowing exactly what "not very good at explaining the process it has gone through " means.
And that's the crux of the issue with AI being used in any law enforcement situation.
If we allow decisions and conclusions being drawn by the AI without a clear explanation of how it got there, we're just creating a monster that will advise -maybe replace- the judgment of law enforcement professionals who won't have the means to question these decisions.
Catching corrupt officials is a laudable goal anywhere and I'd like to see it applied to highlight potential irregularities that may require a second look but the danger here is that an unprovable AI be used to make claims or start being used as sufficient evidence to ruin people's lives.