Is this the business side asking?
They're implicitly giving you product guidance here.
I'm also in the DS/ML space, and there is an absolute dearth of explainability in our models. It's atrocious given the decision that are based on these models to not be able to explain why they come up with what they did.
Get back to them and tell them you'll add explainability to your models. Even if it's just something simple like LIME (vary each input to a model, measure the change it produces - works for any model).
I'm also in the DS/ML space, and there is an absolute dearth of explainability in our models. It's atrocious given the decision that are based on these models to not be able to explain why they come up with what they did.
Get back to them and tell them you'll add explainability to your models. Even if it's just something simple like LIME (vary each input to a model, measure the change it produces - works for any model).