Topic: Machine Learning and the Death of Accountability (Read 385 times)

Maybe last millennium.      In the early 2000s, computer scientists took what we know about human learning, and created Machine Learning (in the thread title too).   Now, with each exam analyzed anywhere by any system, the model and the computer learns - and gets better.  By 2050 there should be nothing a computer cannot do better than our best humans.
 

If this is only your concern, then simply put code to log every decision path, and you will be able to read the "accountability."

On the flipside, ML can actually be used to help us detect unfairness and bring accountability where we as humans fail to.

If ML decides that person A shouldn't be entitled to a credit card or a mortgage, person B shouldn't be considered for a job role, person C should be investigated by the police... then we only see these outputs, we have no way of unravelling how the machine reached these decisions, and so no way of determining whether the decisions were justified. That's what I meant by the death of accountability.

I feed your raw data into an ML black box. The computer decides to block your access to all gambling sites. Because this is ML, there is no human-devised algorithm that we can consult to determine why you were blocked.

I think we need the ability to detect unfairness and bias. Racial profiling for example can be insidious, and difficult to determine. The more control over our lives that we grant to machine-based reasoning, the more we relinquish the ability to question that control. Again, I'm not a Luddite, I think that ML can bring us huge benefits. It's just that I'm also a fan of transparency and accountability.

~

I don't know how email works and I use it everyday. Same sort of thing with a large amount of the pieces of tech we use daily.

I see the problem is more about the formulation phase (or called fundamental flaws), why they decided to use such mechanism, i.e., AI for predicting the exam results, and take the prediction like the real results.

too much gambling problem

This presents other issues. For example, the connection dropping, or someone sitting just out of shot feeding the pupil the correct answers.

How much understanding do you really need or want? To use a banal example, most of us don't know how cars work yet we drive them and even occasionally kill other people with them without questioning their basic functions.

OTOH we tend to be so extremely risk-averse these days that I'm quite certain any AI will be held to a much higher standard than humans are, so for example if an AI black box kills someone it will likely be crippled/limited/banned until some additional safeguards are built into it, even if statistically it performs better than humans.


I have zero clue what this is about so I must be way older or way younger than you... I'd bet on the former

I'm quite certain any AI will be held to a much higher standard than humans are, so for example if an AI black box kills someone it will likely be crippled/limited/banned

How much understanding do you really need or want? To use a banal example, most of us don't know how cars work yet we drive them and even occasionally kill other people with them without questioning their basic functions.

I have zero clue what this is about so I must be way older or way younger than you... I'd bet on the former

I don't know. I want to be convinced, but there's an argument that we don't and can't know whether these ML debuggers are fair. We get into a 'who watches the watchmen'* kind of argument. If we are saying that the way to understand ML 'thought' patterns is to use more ML, I don't think it's the answer. I appreciate that this sort of thing will be used, but I can't see that it will open the black box and make everything understandable to us.

*I didn't, but then I'm not a fan of superhero movies. The X-Men ones were okay, but steadily deteriorated as the sequels progressed. And I have no idea how Kamala became Sansa Stark. A question for a different thread.

I want to be convinced, but there's an argument that we don't and can't know whether these ML debuggers are fair. We get into a 'who watches the watchmen'* kind of argument. If we are saying that the way to understand ML 'thought' patterns is to use more ML, I don't think it's the answer. I appreciate that this sort of thing will be used, but I can't see that it will open the black box and make everything understandable to us.

...humans can and do make fucked up irrational decisions all the time and good luck explaining those.

Then we would have ML models testing and debugging each other perhaps?

I'm pretty sure we can still test and debug ML models, so deliberately not doing that and evading responsibility wouldn't be much different from screwing up human-written code.

I was trying to say (but not being very clear about it) that in this example, there was no machine-learning, the algorithm was devised entirely by humans, which meant that when people started to question the results, the algorithm was looked at and found to contain inbuilt bias (for info, the algorithm is below). But when, instead of this, we are using machine-devised algorithms and machine-learning, and there is no possibility of unpicking the algorithm (because we won't understand how the computer has reached its conclusions), then how do we determine unfairness or accountability? Machine-learning is black box reasoning, and impenetrable to humans. Whose fault is it when it outputs unfair results? And how do we prove that the results are unfair?

I couldn't agree more. If we can predict the future - fuck accountability, let's just all buy winning lottery tickets.

OTOH, the above post is a good example of how far we still need to go with simple things like language translation or whatever world-salad generator this person was using, so the ML apocalypse is probably still quite far away.

Machine learning or ML-related skills are at the top in terms of IT skills required in the present age. Experts say that in the future there is immense potential for things like machine learning and artificial intelligence. it is important to acquire skills in such technology sector there is no need for simple accountability to make all the decisions if you can predict its future in the right way. At present the demand for machine learning engineers is increasing in big technology institutes as well as general institutes.

In response to this, and as a mild qualifier to my response to c_atlas's point, I'd argue that yes, we can to an extent, and in certain simple situations. But if we are talking about complex neural nets, and taking AG Zero as an example of where the technology is headed, I'd say this isn't always the case now, and most certainly won't be the case in the future. Taking the point to an absurd extreme, if we consider the eventual aim of sentient AI, then there's no way that humans have the ability to simply debug it. I do think that the 'black box' problem of ML is real.

AlphaGo Zero knew the game state before and after it made a "1 in 10,000" move, so the programmers knew the success probability before and after that move.

I'm pretty sure we can still test and debug ML models

...the point I'm making is all about the distinction between machines that arrive at conclusions based on initial rules and conditions supplied by humans, and machines that arrive - through impenetrable reasoning - at their own conclusions....

...But then we look at the later version, AlphaGo Zero. This version was entirely self-taught, with no input from human games at all.

I'm just saying that we reach a point with ML where it is impossible for humans to understand how the conclusions were arrived at... and that once pure-ML (rather than human-devised) algorithms start to have a serious impact on human lives, then accountability becomes a huge issue, which the human programmers can convincingly sidestep.

I'm just saying that we reach a point with ML where it is impossible for humans to understand how the conclusions were arrived at... and that once pure-ML (rather than human-devised) algorithms start to have a serious impact on human lives, then accountability becomes a huge issue, which the human programmers can convincingly sidestep.

Machine learning can definitely create issues if used by incompetent dolts but on the other hand - used responsibly it can solve massive problems that are simply unsolvable otherwise. I think we'll find a balance where we will use proven well-tested models as building blocks for more complex systems and we'll learn for figure out whom to blame... just like we don't blame Microsoft when we write buggy code in C# or feed garbage data to well-intended code.

At first it sounds like a good idea, until you notice that the machine learning technology is HEAVILY reliant on teacher input / past performance to come to a conclusion.
[...]
This WONT be the norm in education though, people aren't happy when you give them a bad grade and then you try to blame a computer.....lol

Synopsis
The examination centre provided a list of teacher predicted grades, called 'centre assessed grades' (CAGs)
The students were listed in rank order with no ties.
For large cohorts (over 15)
With exams with a large cohort; the previous results of the centre were consulted. For each of the three previous years, the number of students getting each grade (A* to U) is noted. A percentage average is taken.
This distribution is then applied to the current years students-irrespective of their individual CAG.
A further standardisation adjustment could be made on the basis of previous personal historic data: at A level this could be a GCSE result, at GCSE this could be a Key Stage 2 SAT.
For small cohorts, and minority interest exams (under 15).
The individual CAG is used unchanged

The formulas
for large schools with n>=15
P_kj=(1-r_j)C_kj+r_j(C_kj+q_kj-p_kj)
for small schools with n<15
P_kj=CAG

The variables
n is the number of pupils in the subject being assessed
k is a specific grade
j indicates the school
C_kj is the historical grade distribution of grade at the school (centre) over the last three years, 2017-19.
That tells us already that the history of the school is very important to Ofqual. The grades other pupils got in previous years is a huge determinant to the grades this year’s pupils were given in 2020. The regulator argues this is a plausible assumption but for many students it is also an intrinsically unfair one: the grades they are given are decided by the ability of pupils they may have never met.
q_kj is the predicted grade distribution based on the class’s prior attainment at GCSEs. A class with mostly 9s (the top grade) at GCSE will get a lot of predicted A*s; a class with mostly 1s at GCSEs will get a lot of predicted Us.
p_kj is the predicted grade distribution of the previous years, based on their GCSEs. You need to know that because, if previous years were predicted to do poorly and did well, then this year might do the same.

r_j is the fraction of pupils in the class where historical data is available. If you can perfectly track down every GCSE result, then it is 1; if you cannot track down any, it is 0.
CAG is the centre assessed grade.
P_kj is the result, which is the grade distribution for each grade k at each school j.