Transforming AWS CLI JSON With JQ

While creating a EC2 user data script to copy tags from an EC2 instance to EBS volumes, I got to learn a bit more about using the jq tool.

I needed to be able to transform JSON output from the aws ec2 describe-tags AWS CLI command to be usable as input for the aws ec2 create-tags command, so I can copy tags from auto scaled EC2 instances to EBS volumes.¹

The Output I Have

The JSON output from the aws ec2 describe-tags --instance-id i-xxxxxxx command, looks like:

{
  "Tags": [
    {
      "ResourceType": "instance",
      "ResourceId": "i-026ee8ba50631fe87",
      "Value": "test",
      "Key": "billing_category"
    },
    {
      "ResourceType": "instance",
      "ResourceId": "i-026ee8ba50631fe87",
      "Value": "asgtest",
      "Key": "Name"
    },
    {
      "ResourceType": "instance",
      "ResourceId": "i-026ee8ba50631fe87",
      "Value": "asgtest",
      "Key": "aws:autoscaling:groupName"
    }
  ]
}

The JSON I Need

You can get the JSON input which the aws ec2 create-tags --cli-input-json command uses , by running aws ec2 create-tags --generate-cli-skeleton. That looks like:

{
  "DryRun": true,
  "Resources": [
    ""
  ],
  "Tags": [
    {
      "Key": "",
      "Value": ""
    }
  ]
}

The Job Of The JQ Filter

I needed a jq filter which:

• Removes entries which are AWS tags starting with the string aws:, such as aws:autoscaling:groupName. Tags beginning with aws: are only able to be created by AWS services, not by customers.
• Only include the JSON keys KEy and Value, removing other JSON keys such as ResourceType and ResourceId which are invalid input to the aws ec2 create-tags AWS CLI command.

After consulting the jq manual and trial and error, I ended up with this jq filter:

{ Tags: [ .Tags[] | select(.Key | test("^aws:"; "i") | not ) | {Key: .Key  , Value: .Value} ] }

This constructs JSON in the format I want, while «selecting» only the records I want (without aws:... tags).

The Transformed JSON

The aws ec2 describe-tags JSON output now looks like this after being piped through the above jq filter:

{
  "Tags": [
    {
      "Key": "billing_category",
      "Value": "test"
    },
    {
      "Key": "Name",
      "Value": "asgtest"
    }
  ]
}

That JSON can now be fed into aws ec2 create-tags!