A CRASH course on how to not build technology
The objectification of humans in the age of AI: It’s not looking good, bruv.
In an unsettling 6 minute video released on the internet on June 24, 2024, one that could be easily mistaken for a futuristic fiction movie trailer, Germany based Hashem Al-Ghaili unveiled the world’s next nightmare– Cognify: a technology that fast-tracks retribution for prisoners by implanting fake experiences of the crime they committed perceived from the victim’s perspective. Dubbed as ‘the prison of the future’ it aims to replace jail-time by a 4 step treatment-procedure including (1) gaining consent from a prisoner, (2) producing a digital map of the prisoner’s neural pathways through high resolution brain scanning, (3) seamlessly integrating traumatic artificial memories in their mind and (4) effective real time monitoring of the prisoner’s condition for effective optimization. A day’s worth of procedures and voila– here's your prisoner, fresh out of treatment, cured of all criminal tendencies. Rehabilitation explained in a 4 step process is rehabilitation made simple. Or is it?
To begin with, such retribution aims to fast track punishment by tampering with the prisoner’s memories to induce trauma from a victim’s standpoint. However, with no proven way to assert that this will result in a reduction in recidivism rates of criminals, this technology feeds directly into the vicious cycle of violence, a principle that states that individuals with a history of maltreatment are 29% more likely to engage in criminal behavior, achieving not correctional punishment but a sort of juvenile revenge, with an underestimation of the ethical implications embedded within the concept of justice.
Secondly, the concept of tampering with an individual's memory--prisoner or not, should scare us all. In times of increasingly questionable democratic freedom and internet movements like cancel-culture (that pervarde the online and offline worlds) threatening freedom of individual expression, technology that can tamper our memories may just be the thing that extremists and propagandists desire-- an antidote to all forms of opposition and a means to enforce strict conformity. In powerful hands, this may just be the final blow to the already crippling democracies of the world. Additionally, in times torn by war, where the value of human lives seems to be plummeting faster than prices of commodities, allowing technology to infuse human memories with artificial ones is tantamount to consenting to compromise the sanctity of the human experience even further.
And of course, we must consider the inevitable effect of all modern day technology-- the disproportionate effect on marginalized groups. In the context of prisons, take the results of a report by the US Registry of Exoneration,
“The worst miscarriages of justice any system can produce – the conviction or death sentence of an innocent person – are largely driven by racism in the U.S."
Clearly, the implication of the worst just got worse. With innocent people from marginalized communities sentenced to prisons more often, the effects of this technology may compound the unjust marginalization they face even further. Even when discounting factors of marginalization, what of the prisoners (irrespective of their race) who have been wrongly accused and incarcerated. Scared to waste away their lives in prison for a crime they did not commit, they are susceptible to consent to this technology-- freedom from prison in exchange of a day of false memory implants and medical procedures. Sounds like a bargain, too.
Well, of course, it isn't. Apart from the obvious invasion of a human psyche, there stands the obstacle of applicability of this technology in other areas and potential for misuse. Once this pandora’s box is opened, it is very likely to seep into other sectors, citing just like the developers of this technology, the absolute need to tamper with human memories for the sake of ‘convenience’, dismissing questions of morality behind a future-forward, innovation-driven marketing ploy.
And just like Al-Ghaili predicted, ‘Society won’t be able to stop it.’
This isn’t the first instance of AI integration in the judiciary system, either. COMPAS is a machine learning based predictive technology for risk assessment that was used to predict reoffending tendencies of defendants while making a judgment, which wrongly classified almost twice as many African Americans defendants as compared to white defendants. One would assume that past failures of technology integration in the judicial system would teach the world the importance of ethical technology, but all notions of ethics seem lost beneath the dazzle of novel technology.
The developers of this technology, like many others, speak the commercial language of optimization– reducing costs in prisoner management and increasing efficiency in rehabilitation of prisoners, citing ‘the reduced costs of maintenance of prisoners and timely integration of prisoners back to the society and contribution to the workforce’ as potential benefits of the technology.
But beneath their clinical optimism lies a chilling disregard for the sanctity of the human mind, a gross dehumanization of criminals–nay, prisoners, out of which researchers estimate that up to 5% may be wrongly accused. The invasion of a human’s memories– the very thing that makes them human, a recollection of a lifetime, a salute to the ups and downs of life. Crossing a line that, once crossed may diminish us all under the questionable guise of ‘consent’.
Consent, by definition, promises choice. Free will. How does one choose between freedom of life and freedom of mind? What happens when the cost of a free life is, well, freedom?
Bibliography
Widom, Cathy S., and Michael G. Maxfield. An Update on the Cycle of Violence. National Institute of Justice, Feb. 2001, https://www.ojp.gov/pdffiles1/nij/184894.pdf.
Sweeney, Kieran. "Cognify: The Prison of the Future." Wired, 25 Jan. 2022, https://wired.me/technology/cognify-prison-of-future/.
McKenzie, Tessa. "Rising Number of False Convictions Shows Stark Racial Patterns." Reuters, 27 Sept. 2022, https://www.reuters.com/legal/government/rising-number-false-convictions-shows-stark-racial-patterns-2022-09-27/.
Shukla, Raghav. "The Lost Right to Compensation of Wrongfully Convicted Victims: A Critique." Lawctopus, 15 Mar. 2023, https://www.lawctopus.com/academike/the-lost-right-to-compensation-of-wrongfully-convicted-victims-a-critique/.
留言