Regulating High-Risk AI: Insights from ILPC Seminar

Regulating High-Risk AI: Insights from ILPC Seminar

Table of Contents

1. Introduction
2. The AI Act: Overview and Scope
3. The Prohibited Systems
   1. Face Recognition and Smart Surveillance
   2. Real-Time Law Enforcement Biometric Identification
4. High-Risk AI Systems
   1. Data Quality and Governance
   2. Human Oversight
5. Critiques of the AI Act
   1. Lack of Qualitative Criteria for Risk Assessment
   2. Discretionary Enforcement and Lack of Oversight
   3. Limited Focus on Data Subjects
6. Impact on Migration and Border Control
   1. Proliferation of Technology in the Migration Management Ecosystem
   2. Experimental Technologies and Power Asymmetries
7. Recommendations for Amendments to the AI Act
   1. Inclusion of Other Risky Practices
   2. Ban on AI Lie Detector Machines
   3. Ban on Predictive Analytics in Migration Management
   4. Prohibition on Biometric Identification in Public Spaces
   5. Further Oversight and Inclusion of Big Data Databases
8. Conclusion

Article

The Impact of the EU AI Act on Migration and Border Control

Introduction

The EU AI Act, proposed in April 2021, aims to regulate the development and deployment of artificial intelligence (AI) systems in Europe. While the act covers various areas, one key aspect is its impact on migration and border control. This article will provide an overview of the AI Act, its provisions related to high-risk AI systems, and evaluate its implications for migration and border control.

The AI Act: Overview and Scope

The AI Act introduces a risk-Based approach to the regulation of AI systems. It categorizes AI systems into four risk levels: prohibited, high risk, limited risk, and minimal risk. Prohibited systems are completely banned, while high-risk systems are subject to stringent requirements. The act primarily focuses on high-risk systems, which include various domains such as law enforcement, administration of justice, education, and employment.

The Prohibited Systems

Among the prohibited systems, face recognition and smart surveillance technologies play a significant role in migration and border control. These technologies have raised concerns regarding their discriminatory potential and impact on individuals' rights, particularly when used in real-time law enforcement biometric identification. However, the act's ban on these systems is limited to law enforcement in publicly accessible spaces.

High-Risk AI Systems

The AI Act sets out essential requirements for high-risk AI systems. Two crucial requirements are data quality and governance and human oversight. Data quality and governance address the issues of biased training data and transparency in AI system development. Human oversight ensures that AI systems are controllable and monitored by the downstream users or deployers.

Critiques of the AI Act

The AI Act faces several critiques, including the lack of qualitative criteria for risk assessment. It has been argued that the act fails to provide clear guidelines on how the risks associated with AI systems should be assessed, leaving room for arbitrary decision-making. Another concern is the limited enforcement mechanisms and oversight, mainly relying on self-certification and data protection authorities that are often under-resourced and may lack technical expertise. Additionally, the act's focus on the provider and deployer overlooks the rights and participation of the data subjects affected by high-risk AI systems.

Impact on Migration and Border Control

The proliferation of technology in the migration management ecosystem has resulted in the deployment of various AI systems at different stages of the migration Journey. From automated decision-making to drone surveillance, these technologies have raised ethical and human rights concerns. The AI Act's provisions on high-risk systems should address these concerns and provide safeguards against discriminatory practices and lack of accountability.

Recommendations for Amendments to the AI Act

To ensure robust regulation of AI in migration and border control, several recommendations have been made. These include expanding the list of prohibited practices to address individualized risk assessments and banning predictive analytics used for interdiction or curtailing migration. Strengthening oversight and including big data databases within the scope of the AI Act are also suggested. Furthermore, a prohibition on biometric identification in public spaces, including at borders, is recommended to safeguard privacy and human rights.

Conclusion

The EU AI Act has the potential to Shape the regulation of AI systems in Europe, including migration and border control technologies. While it introduces valuable ex-ante requirements for high-risk systems, the act faces critiques regarding its risk assessment criteria, enforcement mechanisms, and lack of focus on data subjects. To effectively address the challenges in migration and border control, amendments to the AI Act are necessary, incorporating the recommendations Mentioned above. By doing so, it is possible to strike a balance between innovation and safeguarding fundamental rights in the Context of AI in migration and border control.

Highlights

• The EU AI Act introduces a risk-based approach to regulate AI systems.
• Face recognition and smart surveillance are prohibited in law enforcement, but with limitations.
• High-risk AI systems require data quality and human oversight.
• The AI Act lacks qualitative criteria for risk assessment and enforcement mechanisms.
• Deployed AI systems in migration Raise ethical and human rights concerns.
• Recommendations include prohibiting predictive analytics and expanding oversight.
• Amendments to the AI Act are necessary to safeguard fundamental rights in migration.