憤Ⓘ鬥Ⓣ屎: AWS

顯示具有 AWS 標籤的文章。顯示所有文章

2019年8月8日星期四

AWS CLI Tips

Finding Owner AMIs ID

aws ec2 describe-images --owners self --filters "Name=tag:Description,Values=Base Ubuntu Image" --query 'Images[*].{ID:ImageId}' --output 'text'

Reference:

2019年7月28日星期日

Packer build image on AWS - Part II

System Information

# sw_vers
ProductName:    Mac OS X
ProductVersion: 10.14.5
BuildVersion:   18F132

Pre-requirement

Install VirtualBox

# brew cask install virtualbox

Install Packer

# brew install packer

Package	Version	CLI
VirtualBox	6.0.6r130049	VBoxManage -v
Packer	1.4.1	packer --version

Create Packer Directory

Create Packer Directory Structure like below

Packer
├── ISO_Ubuntu_Server_xenial_16.04.6.json
├── http
│   └── preseed.cfg
└── output

Create Ubuntu Pressed.cfg file in http directory

### Localization
d-i debian-installer/language string en d-i debian-installer/locale string en_US.UTF-8
d-i debian-installer/country string TW

### Keyboard selection.
d-i console-setup/ask_detect boolean false
d-i keyboard-configuration/xkb-keymap select us
d-i netcfg/choose_interface select auto

### Setting hostname and domain assigned from dhcp server.
d-i netcfg/get_hostname string ubuntu
d-i netcfg/get_domain string unassigned-domain

### Mirror settings
d-i mirror/country string manual
d-i mirror/http/hostname string archive.ubuntu.com
d-i mirror/http/directory string /ubuntu
d-i mirror/http/proxy string

### To create a normal user account.
d-i passwd/user-fullname string Ubuntu User
d-i passwd/username string ubuntu

### Normal user's password, either in clear text
d-i passwd/user-password password ubuntu
d-i passwd/user-password-again password ubuntu

### Create the first user with the specified UID instead of the default.
d-i passwd/user-uid string 1000

### Allow installer use weak password
d-i user-setup/allow-password-weak boolean true

### The user account will be added to some standard initial groups.
d-i passwd/user-default-groups string adm cdrom plugdev lpadmin sambashare sudo ubuntu

### Set to true if you want to encrypt the first user's home directory.
d-i user-setup/encrypt-home boolean false

### Clock and time zone setup
d-i clock-setup/utc boolean true
d-i time/zone string Asia/Taipei

### Partitioning
d-i partman-auto/disk string /dev/sda
d-i partman-auto/method string lvm
d-i partman-lvm/device_remove_lvm boolean true
d-i partman-md/device_remove_md boolean true
d-i partman-lvm/confirm boolean true
d-i partman-lvm/confirm_nooverwrite boolean true
d-i partman-auto-lvm/guided_size string max
d-i partman-auto/choose_recipe select atomic

### This makes partman automatically partition without confirmation.
d-i partman-partitioning/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman/confirm boolean true
d-i partman/confirm_nooverwrite boolean true

### Apt setup
d-i apt-setup/services-select multiselect security
d-i apt-setup/security_host string security.ubuntu.com
d-i apt-setup/security_path string /ubuntu
d-i debian-installer/allow_unauthenticated boolean true

### Package selection
tasksel tasksel/first multiselect none

### Individual additional packages to install
d-i pkgsel/include string openssh-server build-essential net-tools chrony vim wget curl

### Policy for applying updates.
d-i pkgsel/update-policy select none

### When installer has finished and the system's locate database will be updated.
d-i pkgsel/updatedb boolean true

### Makes grub install automatically to the MBR
d-i grub-installer/only_debian boolean true
d-i grub-installer/with_other_os boolean true
d-i grub-installer/bootdev  string /dev/sda

### Avoid that last message about the install being complete.
d-i finish-install/reboot_in_progress note

### Running custom commands during the installation
d-i preseed/late_command string chroot /target sh -c "/bin/echo 'ubuntu ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers; \
/bin/echo 'Hello World' >> /tmp/Hello.txt

Create ISO_Ubuntu_Server_xenial_16.04.6.json in Packer directory

{
  "variables":{
  "vm_description": "Ubuntu Server Image",
  "vm_version": "0.0.1",
  "cpus": "2",
  "memory": "2048",
  "disk_size": "40960",
  "vm_name": "ubuntu",
  "iso_url": "http://ftp.ubuntu-tw.org/mirror/ubuntu-releases/16.04.6/ubuntu-16.04.6-server-amd64.iso",
  "iso_checksum": "16afb1375372c57471ea5e29803a89a5a6bd1f6aabea2e5e34ac1ab7eb9786ac",
  "iso_checksum_type": "sha256",
  "ssh_username": "ubuntu",
  "ssh_password": "ubuntu",
  "s3_bucket_name": "packer-images"
  },
  "builders": [
    {
      "type": "virtualbox-iso",
      "output_directory": "builds",
      "format": "ova",
      "guest_os_type": "Ubuntu_64",
      "iso_url": "{{user `iso_url`}}",
      "iso_checksum": "{{user `iso_checksum`}}",
      "iso_checksum_type": "{{user `iso_checksum_type`}}",
      "ssh_username": "{{user `ssh_username`}}",
      "ssh_password": "{{user `ssh_password`}}",
      "ssh_port": 22,
      "ssh_wait_timeout": "1000s",
      "disk_size": "{{user `disk_size`}}",
      "keep_registered": "true",
      "shutdown_command": "echo {{user `ssh_password`}} | sudo -S shutdown -P now",
      "vboxmanage": [
        ["modifyvm", "{{.Name}}", "--cpus", "{{user `cpus`}}"],
        ["modifyvm", "{{.Name}}", "--memory", "{{user `memory`}}"]
      ],
      "http_directory": "./http/",
      "boot_wait": "10s",
      "boot_command": [
        "<enter><wait><f6><esc><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>",
        "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>",
        "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>",
        "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>",
        " /install/vmlinuz<wait>",
        " noapic<wait>",
        " preseed/url=http://{{ .HTTPIP }}:{{ .HTTPPort }}/preseed.cfg auto<wait>",
        " locale=en_US<wait>",
        " kbd-chooser/method=us<wait>",
        " keyboard-configuration/modelcode=pc105<wait>",
        " keyboard-configuration/layout=US<wait>",
        " keyboard-configuration/variant=US<wait>",
        " netcfg/get_hostname=ubuntu<wait>",
        " fb=false <wait>",
        " debconf/frontend=noninteractive<wait>",
        " console-setup/ask_detect=false<wait>",
        " initrd=/install/initrd.gz -- <wait>",
        "<enter><wait>"
      ]
    }
  ],
  "post-processors": [
    {
      "type": "amazon-import",
      "keep_input_artifact": true,
      "s3_bucket_name": "{{user `s3_bucket_name`}}",
      "ami_name": "ubuntu-16.04.6",
      "license_type": "BYOL",
      "tags": {
        "Description": "Packer Import "
      }
    }
  ]
}

Check `ISO_Ubuntu_Server_xenial_16.04.6.json` is validated via Packer

Check the file is validate via Packer CLI

# packer validate ISO_Ubuntu_Server_xenial_16.04.6.json
Template validated successfully.

Now we can build images

# packer build ISO_Ubuntu_Server_xenial_16.04.6.json

if your want to see more information, please use Debug Mode

# PACKER_LOG=1 packer build ISO_Ubuntu_Server_xenial_16.04.6.json

Check S3 bucket

aws s3 ls s3://packer-images --recursive --summarize --human-readable

Check import task

# aws ec2 describe-import-image-tasks

Check import task status

aws ec2 describe-import-image-tasks --import-task-ids import-ami-xxxxxxxxxxxxxxxx

2019年7月25日星期四

Packer build image on AWS - Part I

Pre-reqirement

Create S3 bucket

# aws s3api create-bucket --bucket packer-images

Display S3 bucket content

# aws s3 ls s3://packer-images --recursive --summarize --human-readable

Delete S3 bucket

# aws s3api delete-bucket --bucket packer-images

Create Role - vmimport

Create trust-policy.json

{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Effect": "Allow",
         "Principal": { "Service": "vmie.amazonaws.com" },
         "Action": "sts:AssumeRole",
         "Condition": {
            "StringEquals":{
               "sts:Externalid": "vmimport"
            }
         }
      }
   ]
}

Create a role named vmimport and give VM import/Export access.

Ensure that your full path of trust-policy.json file, and that prefix file:///your/full/path/trust-policy.json

# aws iam create-role --role-name vmimport --assume-role-policy-document "file:///tmp/packer/trust-policy.json"

Check Role stting is correct

aws iam get-role --role-name vmimport

Create Policy = role-policy.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetBucketLocation",
                "s3:GetObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::packer-images",
                "arn:aws:s3:::packer-images/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "ec2:ModifySnapshotAttribute",
                "ec2:CopySnapshot",
                "ec2:RegisterImage",
                "ec2:Describe*"
            ],
            "Resource": "*"
        }
    ]
}

Create a policy and attach policy to the role.

Ensure that your full path of role-policy.json file. And that prefix file:///your/full/path/role-policy.json

aws iam put-role-policy --role-name vmimport --policy-name vmimport --policy-document "file:///tmp/packer/role-policy.json"

Check vmimport policy setting is correct.

aws iam get-role-policy --role-name vmimport --policy-name vmimport-policy

Reference:

2019年7月24日星期三

Install boto3 on MAC

System Information

# sw_vers
ProductName:    Mac OS X
ProductVersion: 10.14.5
BuildVersion:   18F132

Check pip version

# pip --version
pip 19.1.1 from /Library/Python/2.7/site-packages/pip-19.1.1-py2.7.egg/pip (python 2.7)

Install boto3

Pre-requirement Package

if you see an error like below

ERROR: Could not install packages due to an EnvironmentError: [Errno 1] Operation not permitted: '/System/Library/Frameworks/Python.framework/Versions/2.7/man'

and then you have one parameter missing needs add --user

# sudo pip install matplotlib --user

Install boto3

if you see an error like below

ERROR: Cannot uninstall 'six'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

and then you should add the parameter --ignore-installed six

# sudo pip install boto3 --ignore-installed six

Now you can use boto3 call AWS API

Reference:

2019年7月22日星期一

AWS CLI 設定

System info

Check Mac OS version

# sw_vers
ProductName:    Mac OS X
ProductVersion: 10.14.5
BuildVersion:   18F132

Install the AWS CLI on macOS Using pip

Check pip3 version

# pip3 --version

Install the AWS CLI on macOS Using pip
1. Download and install the latest version of Python from the download page of Python.org
2. Download and run the pip3 installation script.
```
# curl -O https://bootstrap.pypa.io/get-pip.py
# python3 get-pip.py --user
```
3. Use pip3 to install the AWS CLI.
```
# pip3 install awscli --upgrade --user
```
4. Verify that the AWS CLI is installed correctly.
```
# aws --version
aws-cli/1.16.203 Python/3.7.3 Darwin/18.6.0 botocore/1.12.193
```
5. To upgrade to the latest version, run the installation command again.
```
# pip3 install awscli --upgrade --user
```
Add the AWS CLI Executable to Your macOS Command Line Path
1. Find out AWS CLI installation path
```
# which aws
/Users/nobody/Library/Python/3.7/bin/aws
```
2. To modify your PATH variable
```
# echo 'export PATH=/Users/nobody/Library/Python/3.7/bin:$PATH >> ~/.zshrc
```
3. Load the updated profile into your current session.
```
# source ~/.zshrc
```

Configuration and Multiple Profiles for AWS

The aws configure command is the fastest way to set up your AWS CLI installation

# aws configure --profile "Your Name"
AWS Access Key ID [None]: "Your Access Key ID"
AWS Secret Access Key [None]: "Your Secret Access Key"
Default region name [None]: us-east-1
Default output format [None]: table

Output format:
- json
- text
- table
List .aws directory

# tree ~/.aws
/Users/nobody/.aws
├── config
└── credentials

and your can see the config in ~/.aws/config and ~/.aws/credentials

Setting environment variable to get AWS config

# export AWS_DEFAULT_PROFILE=nobody

# echo 'export AWS_DEFAULT_PROFILE=nobody' >> ~/.zshrc
# source ~/.zshrc

List aws profiles

# aws configure list --profile nobody

Get Current Region

# aws configure get nobody.region

Reference:

訂閱：文章 (Atom)

憤Ⓘ鬥Ⓣ屎

2019年8月8日星期四

AWS CLI Tips

Finding Owner AMIs ID

`aws ec2 describe-images --owners self --filters "Name=tag:Description,Values=Base Ubuntu Image" --query 'Images[*].{ID:ImageId}' --output 'text'`

Reference:

describe-images

Finding an AMI ID Using AWS CLI in AWS Marketplace

尋找 Linux AMI

2019年7月28日星期日

Packer build image on AWS - Part II

System Information

`# sw_vers ProductName: Mac OS X ProductVersion: 10.14.5 BuildVersion: 18F132`

Pre-requirement

Install VirtualBox

`# brew cask install virtualbox`

Install Packer

`# brew install packer`

Package Version CLI

VirtualBox 6.0.6r130049 VBoxManage -v

Packer 1.4.1 packer --version

Create Packer Directory

Check `ISO_Ubuntu_Server_xenial_16.04.6.json` is validated via Packer

2019年7月25日星期四

Packer build image on AWS - Part I

Pre-reqirement

2019年7月24日星期三

Install boto3 on MAC

System Information

`# sw_vers ProductName: Mac OS X ProductVersion: 10.14.5 BuildVersion: 18F132`

Check pip version

`# pip --version pip 19.1.1 from /Library/Python/2.7/site-packages/pip-19.1.1-py2.7.egg/pip (python 2.7)`

Install boto3

2019年7月22日星期一

AWS CLI 設定

System info

Check Mac OS version

`# sw_vers ProductName: Mac OS X ProductVersion: 10.14.5 BuildVersion: 18F132`

Install the AWS CLI on macOS Using pip

Configuration and Multiple Profiles for AWS

2019年8月8日 星期四

AWS CLI Tips

Finding Owner AMIs ID

aws ec2 describe-images --owners self --filters "Name=tag:Description,Values=Base Ubuntu Image" --query 'Images[*].{ID:ImageId}' --output 'text' Reference: describe-images Finding an AMI ID Using AWS CLI in AWS Marketplace 尋找 Linux AMI

2019年7月28日 星期日

Packer build image on AWS - Part II

System Information

# sw_vers ProductName: Mac OS X ProductVersion: 10.14.5 BuildVersion: 18F132

Pre-requirement

Install VirtualBox # brew cask install virtualbox Install Packer # brew install packer PackageVersionCLI VirtualBox6.0.6r130049VBoxManage -v Packer1.4.1packer --version

Create Packer Directory

Check ISO_Ubuntu_Server_xenial_16.04.6.json is validated via Packer

2019年7月25日 星期四

Packer build image on AWS - Part I

Pre-reqirement

2019年7月24日 星期三

Install boto3 on MAC

System Information

# sw_vers ProductName: Mac OS X ProductVersion: 10.14.5 BuildVersion: 18F132

Check pip version

# pip --version pip 19.1.1 from /Library/Python/2.7/site-packages/pip-19.1.1-py2.7.egg/pip (python 2.7)

Install boto3

2019年7月22日 星期一

AWS CLI 設定

System info

Check Mac OS version # sw_vers ProductName: Mac OS X ProductVersion: 10.14.5 BuildVersion: 18F132

Install the AWS CLI on macOS Using pip

Configuration and Multiple Profiles for AWS

2019年8月8日星期四

`aws ec2 describe-images --owners self --filters "Name=tag:Description,Values=Base Ubuntu Image" --query 'Images[*].{ID:ImageId}' --output 'text'`

Reference:

describe-images

Finding an AMI ID Using AWS CLI in AWS Marketplace

尋找 Linux AMI

2019年7月28日星期日

`# sw_vers ProductName: Mac OS X ProductVersion: 10.14.5 BuildVersion: 18F132`

Install VirtualBox

`# brew cask install virtualbox`

Install Packer

`# brew install packer`

Package Version CLI

VirtualBox 6.0.6r130049 VBoxManage -v

Packer 1.4.1 packer --version

Check `ISO_Ubuntu_Server_xenial_16.04.6.json` is validated via Packer

2019年7月25日星期四

2019年7月24日星期三

`# sw_vers ProductName: Mac OS X ProductVersion: 10.14.5 BuildVersion: 18F132`

`# pip --version pip 19.1.1 from /Library/Python/2.7/site-packages/pip-19.1.1-py2.7.egg/pip (python 2.7)`

2019年7月22日星期一

Check Mac OS version

`# sw_vers ProductName: Mac OS X ProductVersion: 10.14.5 BuildVersion: 18F132`