How to publish images ===================== To publish a VMDK file as an image (AMI) on AWS EC2 with `awspub`, a config file is required. A basic config file (say ``config.yaml``) looks like: .. literalinclude:: ../config-samples/config-minimal.yaml :language: yaml To publish the image with the name `my-custom-image`, run: .. code-block:: shell awspub create config.yaml [snipped output] { "images": { "my-custom-image": { "ap-south-2": "ami-015fa46e6ec690c8e", "ap-south-1": "ami-0fd9238a64ea231d0", "eu-south-1": "ami-0cbb4771743cc81fe", "eu-south-2": "ami-0067ee557befd09c2", "me-central-1": "ami-023fa019e0ce98e91", "il-central-1": "ami-092d3f2a7677b8cf1", "ca-central-1": "ami-0d2e897cd1ebecc45", "eu-central-1": "ami-0b9ed498e040c69e2", "eu-central-2": "ami-0fb0f61690e55ab8e", "us-west-1": "ami-069c013403cc15c2f", "us-west-2": "ami-06f9d32912a83571b", "af-south-1": "ami-0371f67e8905c045a", "eu-north-1": "ami-00710b821b31f5c78", "eu-west-3": "ami-08b74828e79d0a405", "eu-west-2": "ami-0f6f9c073bdb7b731", "eu-west-1": "ami-0a07629b25777bf07", "ap-northeast-3": "ami-07d680c934126a92b", "ap-northeast-2": "ami-01fa9f4862d957b59", "me-south-1": "ami-0827faef233b14a29", "ap-northeast-1": "ami-0d119806827c3af22", "sa-east-1": "ami-07f8dfef0a8855f06", "ap-east-1": "ami-047cb2feb00bfc834", "ca-west-1": "ami-061003b943c2d6be8", "ap-southeast-1": "ami-0a2ca6ffb79999bb5", "ap-southeast-2": "ami-0a74f3afdd309dbf2", "ap-southeast-3": "ami-091f9d0adaa612bfb", "ap-southeast-4": "ami-0ccc7ff1fcaf16948", "us-east-1": "ami-0c470d0e3eaf16e67", "us-east-2": "ami-02a7417ff5d866f4b" } }, "images-by-group": {} } The output shows the published image IDs for each region. Since those images are not public, they can only be used from within the same account. .. note:: The command can be run again without publishing anything new as long as the source path file and the config itself doesn't change. Multiple images ~~~~~~~~~~~~~~~ It's possible to publish multiple images based on the same VMDK file. The configuration looks like: .. literalinclude:: ../config-samples/config-multiple-images.yaml :language: yaml Running `awspub` using this config file will publish two images in two different regions. .. code-block:: shell awspub --log-file awspub.log create config.yaml { "images": { "my-custom-image": { "eu-central-1": "ami-0b9ed498e040c69e2" }, "my-custom-image-2": { "eu-central-2": "ami-03889118047373658" } }, "images-by-group": {} } Parameter substitution ~~~~~~~~~~~~~~~~~~~~~~ There are cases where parts of the configuration file need to be dynamic. To support that `awspub` provides basic template substitution (based on Python's `string.Template class `_) . .. literalinclude:: ../config-samples/config-with-parameters.yaml :language: yaml In the config file shown above, the identifier `$serial` which will be replaced with a value that is defined in another YAML file. This YAML file (say ``config-mapping.yaml``) contains a mapping structure (dict in python) that maps the identifiers. .. literalinclude:: ../config-samples/config-with-parameters.yaml.mapping :language: yaml Using both of these config files, the command for `awspub` becomes: .. code-block:: shell awspub --log-file awspub.log create config.yaml --config-mapping config-mapping.yaml { "images": { "my-custom-image-20171022": { "eu-central-1": "ami-0df443d5919e31d1b" } }, "images-by-group": {} } Image groups ~~~~~~~~~~~~ There might be cases were the different commands (e.g. `awspub create` or `awspub publish`) should only be applied on a subset of the defined images. That's possible with the `groups` config option: .. literalinclude:: ../config-samples/config-minimal-groups.yaml :language: yaml Use the ``--group`` parameter to filter the images that the `awspub` command should operate on: .. code-block:: shell awspub --log-file awspub.log create config.yaml --group group1 { "images": { "my-custom-image-1": { "us-west-1": "ami-09461116d07dd6604" } }, "images-by-group": { "group1": { "my-custom-image-1": { "us-west-1": "ami-09461116d07dd6604" } } } } awspub --log-file awspub.log create config.yaml --group group2 { "images": { "my-custom-image-2": { "us-east-1": "ami-018539227554e51fe", "ca-central-1": "ami-071d3602417c28201" } }, "images-by-group": { "group2": { "my-custom-image-2": { "us-east-1": "ami-018539227554e51fe", "ca-central-1": "ami-071d3602417c28201" } } } } The first command is applied only to images defined in `group1`, while the second one is applied only to images defined within `group2`. .. note:: If no `--group` parameter is given, the different commands operate on **all** defined images. Publish images ~~~~~~~~~~~~~~ To make images public, the configuration needs to have the `public` flag set for each image that needs to be public. .. literalinclude:: ../config-samples/config-minimal-public.yaml :language: yaml The image needs to be created and then published: .. code-block:: shell awspub create config.yaml awspub publish config.yaml Sharing images ~~~~~~~~~~~~~~ Images can be shared with other AWS accounts. For that, the account IDs of the other accounts are needed. .. literalinclude:: ../config-samples/config-minimal-share.yaml :language: yaml In the above example, the image `my-custom-image` will be shared with the account `1234567890123` when `awspub` runs in the commercial partition (``aws``, the default). It'll be shared with the account `456789012345` when `awspub` runs in the the china partition (``aws-cn``). AWS Marketplace ~~~~~~~~~~~~~~~ It's possible to publish to `AWS Marketplace `_ if an entity of a `Single-AMI product `_ already exists, an access role ARN is available and an AMI exists in the `us-east-1` region. .. literalinclude:: ../config-samples/config-minimal-marketplace.yaml :language: yaml The image needs to be created first and the `publish` command will request a new Marketplace version for the given entity: .. code-block:: shell awspub create config.yaml awspub publish config.yaml SSM Parameter Store ~~~~~~~~~~~~~~~~~~~ It's possible to push information about published images to the `SSM Parameter Store `_. That's useful e.g. to have a common way to get the latest image ID on different regions. To push image information to the parameter store, the ``ssm_parameter`` configuration for each image must be filled: .. literalinclude:: ../config-samples/config-minimal-ssm.yaml :language: yaml along with a corresponding mapping file: .. literalinclude:: ../config-samples/config-minimal-ssm.yaml.mapping :language: yaml Create the image and use the `publish` command to publish the image and also push information to the parameter store: .. code-block:: shell awspub create config.yaml --config-mapping config.yaml.mapping awspub publish config.yaml --config-mapping config.yaml.mapping SNS Notification ~~~~~~~~~~~~~~~~ It's possible to publish messages through the `Simple Notification Service (SNS) `_. Delivery to multiple topics is possible, but the topics need to exist in each of the regions an image gets published. To notify image information to users, the ``sns`` configuration for each image must be filled: .. literalinclude:: ../config-samples/config-minimal-sns.yaml :language: yaml along with a corresponding mapping file: .. literalinclude:: ../config-samples/config-minimal-sns.yaml.mapping :language: yaml Currently, the supported protocols are ``default`` and ``email`` only, and the ``default`` key is required to send notifications. The ``default`` message will be used as a fallback message for any protocols. Create the image and use the `publish` command to publish the image and also notify the published images to users: .. code-block:: shell awspub create config.yaml --config-mapping config.yaml.mapping awspub publish config.yaml --config-mapping config.yaml.mapping Resource tags ~~~~~~~~~~~~~ The different AWS resources (S3 objects, snapshots and AMIs) can have tags associated with them. `awspub` defines some base tags which are prefixed with `awspub:`. In addition to those tags, there's a `tags` config where tags for all resources can be defined: .. literalinclude:: ../config-samples/config-minimal-tags.yaml :language: yaml This config will add the tag(s) defined to all resources. It's also possible to define image specific tags: .. literalinclude:: ../config-samples/config-minimal-image-tags.yaml :language: yaml "my-custom-image-1" would have the common tag "tag-key" plus the image specific tag "key1". "my-custom-image-2" would have the common tag "tag-key" but the value would be overwritten with "another-value" because image specific tags override the common tags.