contents.md

The Institute for Ethical AI & ML

A practical framework for Responsible ML

Alejandro Saucedo

@AxSaucedo
in/axsaucedo

[NEXT]

The Institute for Ethical AI & ML

A practical framework for Responsible ML

 Alejandro Saucedo

<br> Chairman <br> <a style="color: cyan" href="http://ethical.institute">The Institute for Ethical AI & ML</a> <br> <br> Chief Engineer & Scientist <br> <a style="color: cyan" href="http://e-x.io">Exponential</a> <br> <br> AI Fellow / Member <br> <a style="color: cyan" href="#">The RSA & EU AI Alliance</a> <br> <br> Advisor <br> <a style="color: cyan" href="http://teensinai.com">TeensInAI.com initiative</a> <br> <br> </td>

[NEXT]

A practical framework for Responsible ML

Motivations

Overview of Institute

Responsible AI Framework

Next steps!

[NEXT]

#LetsDoThis

[NEXT SECTION]

1. Motivations

[NEXT]

Traditional data science generalised in two workflows

• Model Development
• Model Serving

[NEXT]

If we have a small team or a small/simple project...

We can cope with the issues

[NEXT]

• Small number of models to maintain
• Data scientists have knowledge of models in their head
• They each have their methods for tracking their progress

### It all works relatively well!

[NEXT]

However

As our data science requirements grow...

We face new issues

[NEXT]

Increasing complexity in flow of data

• Large number of data processing workflows
• Data is modified without stardardised trace
• Managing complexity of flows and scheduling becomes unmanagable


[NEXT]

Each data scientist has their own set of tools

• Some ♥ Tensorflow
• Some ♥ R
• Some ♥ Spark


### Some ♥ all of them

[NEXT]

Serving models becomes increasinly harder


• Different model versions running in different environments
• Deploying and reverting models gets increasingly complex

[NEXT]

When stuff goes wrong it's hard to trace back


• Data scientists say it's a bug in the pipelines
• Data engineers say it's something wrong in the models
• Becomes a cat-and-mouse game

[NEXT]

Luckily for us

Many fellow colleagues have faced these issues for a while

and an active problem that many people are trying to address

[NEXT]

Data Scientists

In charge of development of models

Data Engineers

In charge of development of data pipelines

DevOps / DataOps / MLOps Engineers

In charge of productionisation of models, data pipelines & products

[NEXT]

As your technical functions grow...

[NEXT]

So should your infrastructure

[NEXT]

EuroSciPy Talk 2018

MLOps / DataOps

a curated list of frameworks to scale
your machine learning capabilities

bit.ly/awesome-mlops

[NEXT SECTION]

2. The ML Principles

[NEXT]

The Institute for Ethical AI & ML

http://ethical.institute

[NEXT]

The moral-consciousness matrix

	Conscious	Unconscious
Moral	🤩	🤨
Immoral	👹	🤪

Moral === Wants to do good

Conscious === Knows how to

[NEXT]

IEML phased rollout plan

• Phase 1 - Responsible ML by pledge
  • For technologists to implement
    
    
• Phase 2 - Responsible ML by process
  • For technology leaders to introduce
    
    
• Phase 3 - Responsible ML by certification
  • For industries to raise the bar
    
    
• Phase 4 - Responsible ML by regulation
  • For economies to thrive

[NEXT]

Phase 1

Our 3 core streams

• The 8 Machine Learning Principles
• Open source contributions (i.e. MLOps List)
• The Ethical ML Network of technologists

[NEXT]

The 8 ML Principles

ethical.institute/principles.html

<style> .fragment.visible.fade-out.current-fragment { display: none !important; height:0px; line-height: 0px; font-size: 0px; } </style>

[NEXT]

1. Human augmentation

Assess impact of incorrect predictions

and design with human-in-the-loop review

where reasonable

• Sentence prediction
• Fraud detection
• Temporary augmentation

[NEXT]

2. Bias evaluation

Non-trivial decisions have inherent societal bias. We should identify, document bias and implications

• Data bias
• Feature importance
• Equity vs Equality

[NEXT]

3. Explainable by design

Explainability through domain knowledge, together with feature importance analysis

• Accuracy-explainability tradeoff
• Modularisation of elements
• Domain knowledge as features

[NEXT]

4. Reproducible systems

Abstracting computations to improve reproducibility in development and production

[NEXT]

5. Displacement strategy

Identifying and documenting impact of technology towards workers being displaced

• Reducing impact
• Jevons paradox
• Business change strategies

[NEXT]

6. Practical accuracy

Taking a pragmatic approach towards accuracy and cost metrics

• Going beyond accuracy
• Domain specific metrics
• In development and production

[NEXT]

7. Trust beyond the user

Build processes to use and protect user data & privacy, and make sure they are communicated

• Privacy at the right level
• Metadata via personal data
• Communicating when reasonable

[NEXT]

8. Data Risk Awareness

Develop processes and infrastructure to ensure data and model security are taken into consideration

• Security breaches due to human error
• Adversarial attacks
• Social engineering new processes

[NEXT]

Join the Ethical ML Network

ethical.institute#network

[NEXT]

Next steps

Applying this thinking into your actual projects

#LetsDoThis

[NEXT SECTION]

3. Wrapping up

[NEXT]

A practical framework for Responsible ML

Motivations

Overview of Institute

Responsible AI Framework

Next steps!

[NEXT]

The ML Principles

ethical.institute/principles.html

Awesome MLOps List

bit.ly/awesome-mlops

Thank you

Questions? [email protected]