TIL: boto3 can't parse env variables properly

So in today's saga of fighting the Internet, I bring this salient point to your attention:

boto3 can't parse environment variables for AWS credentials properly

and it will error kind of like this:

 ValueError: a Credential=WHACKADOODLEGGET5Q\r/20180627/us-east-1/sts/aws4_request

or possibly like this:

caught_exception\nValueError: Invalid header value '

    AWS4-HMAC-SHA256 Credential=WHACKADOODLET5Q\\r/20180627/us-east-1/ec2/aws4_request, SignedHeaders=content-type;host;x-amz-date, Signature=8f84f8d811f4dcb45da5f8fadf90ae8390d5d358b4024bf0d964090032dea1c3'\n", 

    "module_stdout": "", 

    "msg": "MODULE FAILURE", 

This appears to be an error in parsing the  \r portion of the URI. (Based on a few google queries.)

Using ~/.aws/credentials seems to do the trick (with the exact same key values):

[default]

aws_access_key_id=WHACKADOODLEGGGET5Q

aws_secret_access_key=bn+++krandometyperyforwhackadoodle1YjKMWqzv

The environment variables I was using were of course:

export AWS_ACCESS_KEY_ID=WHACKADOODLEGGGET5Q

export \ AWS_SECRET_ACCESS_KEY=bn+++krandometyperyforwhackadoodle1YjKMWqzv

Find your libvirt instances...

I'm at Red Hat these days and working on cloud manageability which you will see me write as m11y more often than not.

I recently had a need to utilize Ansible for a demo and created a slug of VMs to use in my ansible inventory. However, there is no way (no obvious way?) I can find with virsh or virt-manager to determine what IP got assigned to a VM. A bit of digging shows that when using the cloud-images, dhcp is used to acquire an IP address and that dnsmasq manages those IP addresses.

In my case, these VMs were attached to virbr0 and it is that instance of dnsmasq that I needed to inspect.

Like most of my very short blog posts, I'm just writing this down so I can find it myself the next time I need it. In this case the file that "issued" the IP addresses is:

/var/lib/libvirt/dnsmasq/virbr0.status

That shows the MAC and IP address associations (but no indication of "which" libvirt domain they were actually associated.) A further exercise for me and the reader will be to tie the domain back to the IP address (if possible.)

One other related note: If you use "virt-clone" to clone a cloud-image that has been previously booted, it will typically boot fine but WITHOUT an IP address. This is due to cloud-init not properly re-running. Blowing away some of the first run cloud-init stuff takes care of that.

sudo rm -rf /var/lib/cloud
sudo rm -rf /var/log/cloud-init*

(This may be overkill to "purify" a cloud-instance but it certainly does the ttrick.)

You need to re-run cloud-init at this point (either directly or a quick reboot) and you should find that your instance has an IP address now.

It's a pretty quick and easy step to convirt virbr0.status into an inventory...

sudo cat /var/lib/libvirt/dnsmasq/virbr0.status |grep ip-address |awk '{ print $NF }' |sed  -e 's/"//' -e 's/",//'

or you could turn your inventory into a dynamic inventory based on that file, exercise also left to the reader (but be sure to make the output valid JSON dict.)