You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 33 Next »

On 2020-08-02, the Cornell Shibboleth IdP was updated to v4.01. That update broke the awscli-login package for use at Cornell. See Help! I got a 500 Server Error below.

Thank you to user-af308 and Hong Ye for troubleshooting and identifying the problem!

 


Use Case

This document shows how to setup and use the awscli-login tool to retrieve temporary AWS access keys using your Cornell netid credentials and Duo (i.e., Cornell Two-Step Login). Using temporary access keys associated with an AWS role to authenticate to the AWS Command Line Interface (CLI) is much safer than using fixed AWS access keys tied to an IAM user. Now that this option is available to Cornell AWS users, we recommend that fixed access keys no longer be used for humans using the AWS CLI.

Prerequisites

 

Linux/Mac OS
$ python --version
Python 2.7.14
$ python3 --version
-bash: python3: command not found
$ aws --version
aws-cli/1.15.83 Python/2.7.14 Linux/4.14.77-70.59.amzn1.x86_64 botocore/1.10.82

 

In the above example, Python 2.x is installed and the AWS CLI is installed, but using Python 2.x. That's exactly what we don't want. It is beyond the current scope of this article to describe how to install Python 3.5+, and the AWS CLI, ensuring that the CLI is using your Python 3.5+ installation. Please note that virtualenv may make it easier to get the AWS CLI installed and using Python 3.5+.  

Install and Configure awscli-login

 

Don't bother trying to install or use the awscli-login tool until your installed version of the AWS CLI reports that it is using Python 3.5+. You will also need to ensure that pip is using Python 3.5+. That may mean you will need to use pip3 in the commands below, instead of plain pip.

$ pip install --upgrade https://github.com/CU-CommunityApps/awscli-login/tarball/master#egg=awscli-login
...
$ aws configure set plugins.login awscli_login
...
$ aws login configure
ECP Endpoint URL [None]: https://shibidp.cit.cornell.edu/idp/profile/SAML2/SOAP/ECP
Username [None]: <YOUR NETID>
Enable Keyring [False]:
Duo Factor [None]:
Role ARN [None]:
$ 

You may ask...

Why is the Cornell fork version 0.1.0a5 while the "official" version at PyPi is version 0.1.0a6?

We're not sure. The master branch in the original git repo is labeled 0.1.0a5, so the master branch of our fork is too. We are looking into this, but our fork of version 0.1.0a5 seems to work.

Using awscli-login

Login using defaults setup above:

$ aws login
Password: ********
Factor: push
# Provided second factor out of band
Please choose the role you would like to assume:
     Account: 000011112222
         [ 0 ]: shib-admin
     Account: 777788889999
         [ 1 ]: shib-admin
         [ 2 ]: shib-cs
         [ 3 ]: shib-dba
Selection: 1
$ aws sts get-caller-identity
{
    "UserId": "AROAICCPMY7VALLFYHWPA:pea1@cornell.edu",
    "Account": "777788889999",
    "Arn": "arn:aws:sts::777788889999:assumed-role/shib-admin/pea1@cornell.edu"
}

The options for "Factor" are "push", "sms", "phone", "auto"

Advanced Use

 

$ aws --profile foo login configure
ECP Endpoint URL [None]: https://shibidp.cit.cornell.edu/idp/profile/SAML2/SOAP/ECP
Username [None]: pea1
Enable Keyring [False]:
Duo Factor [None]: auto
Role ARN [None]: arn:aws:iam::111111111111:role/shib-admin

$ aws --profile bar login configure
ECP Endpoint URL [None]: https://shibidp.cit.cornell.edu/idp/profile/SAML2/SOAP/ECP
Username [None]: pea1
Enable Keyring [False]: 
Duo Factor [None]: auto
Role ARN [None]: arn:aws:iam::222222222222:role/shib-admin
 
$ aws --profile foo login
Password: **************
# Provided second factor out of band
$ aws --profile foo sts get-caller-identity
{
    "Arn": "arn:aws:sts::111111111111:assumed-role/shib-admin/pea1@cornell.edu",
    "Account": "111111111111",
    "UserId": "XXXXICCPMY7VALLFXXXX:pea1@cornell.edu"
}

 
$ aws --profile bar login
Password: **************
# Provided second factor out of band
$ aws --profile bar sts get-caller-identity
{
    "Arn": "arn:aws:sts::222222222222:assumed-role/shib-admin/pea1@cornell.edu",
    "Account": "222222222222",
    "UserId": "YYYYICCPMY7VALLFYYYY:pea1@cornell.edu"
}

 
 

Troubleshooting

Help! I got a 401 Client Error

$ aws login
Password: **************
401 Client Error: Unauthorized for url: https://shibidp.cit.cornell.edu/idp/profile/SAML2/SOAP/ECP

This situation occurs when you provide an incorrect password.

If you have setup your awscli-login config to save your password (i.e., enable_keyring = true) then the plugin will happily save an incorrect password in the keyring. In this situation you will never be prompted for a password and you will immediately get a 401 error. To work through this edit your ~/.aws-login/config file to set enable_keyring = false, to be prompted for a password again.

 

Help! I got a 504 Server Error

$ aws login
Password: **************
504 Server Error: GATEWAY_TIMEOUT for url: https://shibidp.cit.cornell.edu/idp/profile/SAML2/SOAP/ECP

This situation occurs when you fail to provide your Duo second factor in time.

 

Help! I got a 500 Server Error

$ aws login
Password: 
Traceback (most recent call last):
  File "/home/ec2-user/environment/python37venv/lib/python3.7/site-packages/awscli_login/__main__.py", line 175, in main
    profile.cookies,
  File "/home/ec2-user/environment/python37venv/lib/python3.7/site-packages/awscli_login/saml.py", line 149, in refresh
    soap = saml_login(url, jar)
  File "/home/ec2-user/environment/python37venv/lib/python3.7/site-packages/awscli_login/saml.py", line 92, in saml_login
    r.raise_for_status()
  File "/home/ec2-user/environment/python37venv/lib/python3.7/site-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://shibidp.cit.cornell.edu/idp/profile/SAML2/SOAP/ECP

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ec2-user/environment/python37venv/lib/python3.7/site-packages/awscli_login/__main__.py", line 123, in wrapper
    f(profile, session)
  File "/home/ec2-user/environment/python37venv/lib/python3.7/site-packages/awscli_login/__main__.py", line 180, in main
    profile.cookies, *creds)
  File "/home/ec2-user/environment/python37venv/lib/python3.7/site-packages/awscli_login/saml.py", line 118, in authenticate
    soap = saml_login(url, jar, username, password, headers)
  File "/home/ec2-user/environment/python37venv/lib/python3.7/site-packages/awscli_login/saml.py", line 92, in saml_login
    r.raise_for_status()
  File "/home/ec2-user/environment/python37venv/lib/python3.7/site-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: 500 for url: https://shibidp.cit.cornell.edu/idp/profile/SAML2/SOAP/ECP
500 Server Error: 500 for url: https://shibidp.cit.cornell.edu/idp/profile/SAML2/SOAP/ECP

This situation occurs when you use the https://github.com/techservicesillinois/awscli-login (original) version of awscli-login instead of the Cornell fork of that tool at https://github.com/CU-CommunityApps/awscli-login

To fix this, uninstall your current awscli-login version and use the Cornell fork:

$ pip uninstall awscli-login
Found existing installation: awscli-login 0.1.0a6
Uninstalling awscli-login-0.1.0a6:
  Would remove:
    /home/ec2-user/environment/python37venv/lib/python3.7/site-packages/awscli_login-0.1.0a6.dist-info/*
    /home/ec2-user/environment/python37venv/lib/python3.7/site-packages/awscli_login/*
    /home/ec2-user/environment/python37venv/lib/python3.7/site-packages/tests/*
Proceed (y/n)? y
  Successfully uninstalled awscli-login-0.1.0a6
 
$ pip install --upgrade https://github.com/CU-CommunityApps/awscli-login/tarball/master#egg=awscli-login
Collecting awscli-login
  Using cached https://github.com/CU-CommunityApps/awscli-login/tarball/master
Requirement already satisfied, skipping upgrade: awscli in ./testvenv/lib/python3.7/site-packages (from awscli-login) (1.18.110)
Requirement already satisfied, skipping upgrade: boto3 in ./testvenv/lib/python3.7/site-packages (from awscli-login) (1.14.33)
Requirement already satisfied, skipping upgrade: botocore in ./testvenv/lib/python3.7/site-packages (from awscli-login) (1.17.33)
Requirement already satisfied, skipping upgrade: daemoniker in ./testvenv/lib/python3.7/site-packages (from awscli-login) (0.2.3)
Requirement already satisfied, skipping upgrade: keyring in ./testvenv/lib/python3.7/site-packages (from awscli-login) (21.3.0)
Requirement already satisfied, skipping upgrade: lxml in ./testvenv/lib/python3.7/site-packages (from awscli-login) (4.5.2)
Requirement already satisfied, skipping upgrade: psutil in ./testvenv/lib/python3.7/site-packages (from awscli-login) (5.7.2)
Requirement already satisfied, skipping upgrade: requests in ./testvenv/lib/python3.7/site-packages (from awscli-login) (2.24.0)
Requirement already satisfied, skipping upgrade: docutils<0.16,>=0.10 in ./testvenv/lib/python3.7/site-packages (from awscli->awscli-login) (0.15.2)
Requirement already satisfied, skipping upgrade: colorama<0.4.4,>=0.2.5; python_version != "3.4" in ./testvenv/lib/python3.7/site-packages (from awscli->awscli-login) (0.4.3)
Requirement already satisfied, skipping upgrade: rsa<=4.5.0,>=3.1.2; python_version != "3.4" in ./testvenv/lib/python3.7/site-packages (from awscli->awscli-login) (4.5)
Requirement already satisfied, skipping upgrade: s3transfer<0.4.0,>=0.3.0 in ./testvenv/lib/python3.7/site-packages (from awscli->awscli-login) (0.3.3)
Requirement already satisfied, skipping upgrade: PyYAML<5.4,>=3.10; python_version != "3.4" in ./testvenv/lib/python3.7/site-packages (from awscli->awscli-login) (5.3.1)
Requirement already satisfied, skipping upgrade: jmespath<1.0.0,>=0.7.1 in ./testvenv/lib/python3.7/site-packages (from boto3->awscli-login) (0.10.0)
Requirement already satisfied, skipping upgrade: python-dateutil<3.0.0,>=2.1 in ./testvenv/lib/python3.7/site-packages (from botocore->awscli-login) (2.8.1)
Requirement already satisfied, skipping upgrade: urllib3<1.26,>=1.20; python_version != "3.4" in ./testvenv/lib/python3.7/site-packages (from botocore->awscli-login) (1.25.10)
Requirement already satisfied, skipping upgrade: importlib-metadata; python_version < "3.8" in ./testvenv/lib/python3.7/site-packages (from keyring->awscli-login) (1.7.0)
Requirement already satisfied, skipping upgrade: SecretStorage>=3; sys_platform == "linux" in ./testvenv/lib/python3.7/site-packages (from keyring->awscli-login) (3.1.2)
Requirement already satisfied, skipping upgrade: jeepney>=0.4.2; sys_platform == "linux" in ./testvenv/lib/python3.7/site-packages (from keyring->awscli-login) (0.4.3)
Requirement already satisfied, skipping upgrade: idna<3,>=2.5 in ./testvenv/lib/python3.7/site-packages (from requests->awscli-login) (2.10)
Requirement already satisfied, skipping upgrade: chardet<4,>=3.0.2 in ./testvenv/lib/python3.7/site-packages (from requests->awscli-login) (3.0.4)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in ./testvenv/lib/python3.7/site-packages (from requests->awscli-login) (2020.6.20)
Requirement already satisfied, skipping upgrade: pyasn1>=0.1.3 in ./testvenv/lib/python3.7/site-packages (from rsa<=4.5.0,>=3.1.2; python_version != "3.4"->awscli->awscli-login) (0.4.8)
Requirement already satisfied, skipping upgrade: six>=1.5 in ./testvenv/lib/python3.7/site-packages (from python-dateutil<3.0.0,>=2.1->botocore->awscli-login) (1.15.0)
Requirement already satisfied, skipping upgrade: zipp>=0.5 in ./testvenv/lib/python3.7/site-packages (from importlib-metadata; python_version < "3.8"->keyring->awscli-login) (3.1.0)
Requirement already satisfied, skipping upgrade: cryptography in ./testvenv/lib/python3.7/site-packages (from SecretStorage>=3; sys_platform == "linux"->keyring->awscli-login) (3.0)
Requirement already satisfied, skipping upgrade: cffi!=1.11.3,>=1.8 in ./testvenv/lib/python3.7/site-packages (from cryptography->SecretStorage>=3; sys_platform == "linux"->keyring->awscli-login) (1.14.1)
Requirement already satisfied, skipping upgrade: pycparser in ./testvenv/lib/python3.7/site-packages (from cffi!=1.11.3,>=1.8->cryptography->SecretStorage>=3; sys_platform == "linux"->keyring->awscli-login) (2.20)
Building wheels for collected packages: awscli-login
  Building wheel for awscli-login (setup.py) ... done
  Created wheel for awscli-login: filename=awscli_login-0.1.0a5-py3-none-any.whl size=37905 sha256=8ae34faf84b11a8237f0de5e3472465b22b2790997c3fafcfdf8cd0ccf6373a6
  Stored in directory: /tmp/pip-ephem-wheel-cache-8234a34d/wheels/f5/2d/15/dada127d400de6504a958268b535444a9bd985f839b3c5bdfe
Successfully built awscli-login
Installing collected packages: awscli-login
Successfully installed awscli-login-0.1.0a5
 
  • No labels