Approaches for a more secure Azure KeyVault
Azure Key Vault is a cloud service that safeguards encryption keys and secrets (such as certificates, connection strings, passwords) for your cloud applications. Since this data is sensitive and business critical, you want to secure access to your key vaults so that only authorized applications and users can access key vault.
Microsoft has some great guidance you really need to read.
In the end, I encourage you to consider creating a separate subscription just for your Key Vault if it is going to hold particularly sensitive secrets. There is no reason why your normal DevOps / CloudOps personnel need to have control of that infrastructure, and you want to prevent things like Azure DevOps from having access to it unless really necessary. Use AzureAD and Managed Service Identities (MSI) to create access policies with the least amount of privs required for the application (ie: only GET permissions on Secrets to a single Key Vault) and have the application fetch the secrets as needed.