If you are handling data and running a business it is almost a foregone conclusion that at some point you will be using a system that has been buzzed-up and termed AI. What that means is a debate for a different post, but what I am going to take it to mean here is you are conducting some form of automated processing of data. This can be by a sophisticated agentic framework and large language model, a data pipeline that delivers a bespoke service for each customer/client based on their chosen preferences, or just the profiling and categorisation based on attributes in a machine derived algorithm.
You’re going to be using a little silicon buddy to do your leg work, and maybe that “plastic pal who’s fun to be with” will also get to automatically determine an outcome. “Share and Enjoy”.(1)
What this means for your business activities, is that you have another vector for potential data legislation headaches. That can be in the form of a security threat; or it could also be data processing that is out-of-scope (or in breach of rights or consent); or even uncertainty about where your data goes and what actions are performed on it. You may end up with a Subject Access Request (SAR) that is generated by an AI assking what your AI does with data and to a very fine level of detail. Do you even have an understanding of how to answer that?
These are just a few of the potential issues and for a business that is unprepared, or worse uncomprehending to the oint of absolute ignorance, it can lead to all manner of issues.(3)
Thankfully, for the UK at least,(4) the ICO has produced a Data Protection Risk Toolkit(5) to help you identify, assess, and mitigate the risks. It doesn’t replace a Data Protection Impact Assessments (DPIAs) for high-risk instances. If you have a risk that can be of severe impact you need to do a DPIA and have a plan for any changes, breaches or issues. But the toolkit can demonstrate compliance with the law.
By breaking down the evaluations you might consider into six subsections we can cover four of the “AI lifecycle stages” here.(6) These are:
- business requirements and design;
- data acquisition and preparation;
- training and testing;
- deployment and monitoring.
Under each of these areas are evaluations that you can score yourself. You score them using the categories of: ‘high’, ‘medium’, ‘low’, and ‘non-applicable’ to your organisation; from that you build up a hierarchy of risk.
Accountability & Governance
• Do a DPIA before deploying any high-risk systems.
• Appoint an owner to drive accountability and assign roles.
• Have procedures and guidance for data protection in any automated system.
• Consult with those likely to be affected by your use.
Lawfulness & Transparency
• Identify, and document, your base for processing, including any conditions for using special category data.
• Provide clear privacy notices using plain language about how data is used and the basis for use.
• Give explanations for automated decisions.
Data Acquisition & Minimisation
• Complete a data mapping exercise.
• Apply de-identification techniques or privacy-enhancing technologies before it is extracted.
• Remove features likely to result in systems tuned too closely to specific training that can introduce bias or increase exceptions.
Fairness & Bias Mitigation
• Assess training datasets, ensure they are representative, reliable, and do not reflect discrimination.
• Establish clear labelling, involve multiple human labellers for consistency and to mitigate unfair outcomes.
• Implement technical systems to reduce bias test the system for similar outcomes across different demographics.
Security Measures
• Carry out security testing, subscribe to advisories for updates.
• Enforce least privilege, ensure minimum access, maintain strict access controls.
• Maintain an inventory of all systems for accurate incident response.
Deployment & Monitoring
• Ensure human review for solely automated decisions that have legal or significant effects, grant reviewers the training and authority to override automated decisions.
• Have metrics for “model drift”, regularly retrain the model if accuracy degrades.
• Index personal data so it can be easily retrieved for any requests.
• Schedule reviews for processing documentation , ensuring systems are not being used beyond its initial agreed purpose without consent.
This can be a daunting prospect, or one you don’t have the time or resources to perform. Often it just takes a systematic approach to go through this checklist and address the concerns. My advice is to start by running a very quick assessment and defining an iniaital score for each evaluation item. Then you can look at your “high” risk areas and start to address them first.
As always, we at Shadowcat would be delighted to help. Want to talk about this more? Drop us a comment or use the contact form on our website (https://www.shadowcat.co.uk/#contact) to get in touch.
(1) thanks to my co-workers for loading me up with the HitchHikers Guide to the Galaxy references, they can “go stick their heads in a pig”.(2)
(2) yes that is another HHGTTG reference. See: https://hitchhikers.fandom.com/wiki/Share_and_Enjoy
(3) Yes the fines and penalties for this are as severe as any other under the UK’s GDPR. The poroblem is compounded for UK businesses by the changing of legislative rules in the recent legislation. See: https://www.shadowcat.co.uk/2026/06/14/using-ai-and-data-protection-some-risks-to-think-about-and-mitigate/
(4) Though organisations globally would not be harmed by following the same processes stated here.
(5) https://regulations.ai/regulations/RAI-GB-NA-IGADPXX-2023
(6) https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/artificial-intelligence/guidance-on-ai-and-data-protection/annex-a-fairness-in-the-ai-lifecycle/?search=human+review
No responses yet