AWS lets you assign an IAM role to, e.g., and EC2 VM that grants that VM access to cloud resources like S3. See https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html
Concretely, https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-metadata-security-credentials.html documents how applications running on a VM can get the actual credentials (ID, Secret, Token) needed to auth to other services. It relies on making some HTTP requests to an IMDS endpoint:
We were able to manually simulate this using curl to get tokens
# Step 1: get a session token
TOKEN=$(curl -s -X PUT "http://169.254.169.254/latest/api/token" \
-H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
# Step 2: use the token to query metadata
curl -s -H "X-aws-ec2-metadata-token: $TOKEN" \
http://169.254.169.254/latest/meta-data/iam/security-credentials/
The second call returns the IAM role name attached to the instance. To get the actual credentials:
curl -s -H "X-aws-ec2-metadata-token: $TOKEN" \
http://169.254.169.254/latest/meta-data/iam/security-credentials/<role-name>
and then set them as the relevant environment variables.
We'd need to figure out the API to provide for users to indicate that we want to use this authorization method. But when they do kvikio itself would make the HTTP requests to the IMDS endpoint and cache the tokens for reuse (the response includes an Expiry that we can use for invalidation).
cc @ncclementi
AWS lets you assign an IAM role to, e.g., and EC2 VM that grants that VM access to cloud resources like S3. See https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html
Concretely, https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-metadata-security-credentials.html documents how applications running on a VM can get the actual credentials (ID, Secret, Token) needed to auth to other services. It relies on making some HTTP requests to an IMDS endpoint:
We were able to manually simulate this using curl to get tokens
and then set them as the relevant environment variables.
We'd need to figure out the API to provide for users to indicate that we want to use this authorization method. But when they do kvikio itself would make the HTTP requests to the IMDS endpoint and cache the tokens for reuse (the response includes an Expiry that we can use for invalidation).
cc @ncclementi