IT Cloud

changes, it is recommended to add version = "…" constraints to the

corresponding provider blocks in configuration, with the constraint strings

suggested below.

* provider.google: version = "~> 2.8"

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see

any changes that are required for your infrastructure. All Terraform commands

should now work.

If you ever set or change modules or backend configuration for Terraform,

rerun this command to reinitialize your working directory. If you forget, other

commands will detect it and remind you to do so if necessary.

Add a virtual machine:

essh @ kubernetes-master: ~ / node-cluster $ cat main.tf

provider "google" {

credentials = "$ {file (" kubernetes_key.json ")}"

project = "node-cluster-243923"

region = "europe-north1"

}

resource "google_compute_instance" "cluster" {

name = "cluster"

zone = "europe-north1-a"

machine_type = "f1-micro"

boot_disk {

initialize_params {

image = "debian-cloud / debian-9"

}

}

network_interface {

network = "default"

access_config {}

}

essh @ kubernetes-master: ~ / node-cluster $ sudo ./terraform apply

An execution plan has been generated and is shown below.

Resource actions are indicated with the following symbols:

+ create

Terraform will perform the following actions:

# google_compute_instance.cluster will be created

+ resource "google_compute_instance" "cluster" {

+ can_ip_forward = false

+ cpu_platform = (known after apply)

+ deletion_protection = false

+ guest_accelerator = (known after apply)

+ id = (known after apply)

+ instance_id = (known after apply)

+ label_fingerprint = (known after apply)

+ machine_type = "f1-micro"

+ metadata_fingerprint = (known after apply)

+ name = "cluster"

+ project = (known after apply)

+ self_link = (known after apply)

+ tags_fingerprint = (known after apply)

+ zone = "europe-north1-a"

+ boot_disk {

+ auto_delete = true

+ device_name = (known after apply)

+ disk_encryption_key_sha256 = (known after apply)

+ source = (known after apply)

+ initialize_params {

+ image = "debian-cloud / debian-9"

+ size = (known after apply)

+ type = (known after apply)

}

}

+ network_interface {

+ address = (known after apply)

+ name = (known after apply)

+ network = "default"

+ network_ip = (known after apply)

+ subnetwork = (known after apply)

+ subnetwork_project = (known after apply)

+ access_config {

+ assigned_nat_ip = (known after apply)

+ nat_ip = (known after apply)

+ network_tier = (known after apply)

}

}

+ scheduling {

+ automatic_restart = (known after apply)

+ on_host_maintenance = (known after apply)

+ preemptible = (known after apply)

+ node_affinities {

+ key = (known after apply)

+ operator = (known after apply)

+ values = (known after apply)

}

}

}

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?

Terraform will perform the actions described above.

Only 'yes' will be accepted to approve.

Enter a value: yes

google_compute_instance.cluster: Creating …

google_compute_instance.cluster: Still creating … [10s elapsed]

google_compute_instance.cluster: Creation complete after 11s [id = cluster]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Add a public static IP address and SSH key to the node:

essh @ kubernetes-master: ~ / node-cluster $ ssh-keygen -f node-cluster

Generating public / private rsa key pair.

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in node-cluster.

Your public key has been saved in node-cluster.pub.

The key fingerprint is:

SHA256: vUhDe7FOzykE5BSLOIhE7Xt9o + AwgM4ZKOCW4nsLG58 essh @ kubernetes-master

The key's randomart image is:

+ – [RSA 2048] – +

| .o. +. |

| o. o. =. |

| * + o. =. |

| = *. … ... + o |

| B +. … S * |

| = + oo X +. |

| o. =. + = + |

| . = .... … |

| ..E. |

+ – [SHA256] – +

essh @ kubernetes-master: ~ / node-cluster $ ls node-cluster.pub

node-cluster.pub

essh @ kubernetes-master: ~ / node-cluster $ cat main.tf

provider "google" {

credentials = "$ {file (" kubernetes_key.json ")}"

project = "node-cluster-243923"

region = "europe-north1"

}

resource "google_compute_address" "static-ip-address" {

name = "static-ip-address"

}

resource "google_compute_instance" "cluster" {

name = "cluster"

zone = "europe-north1-a"

machine_type = "f1-micro"

boot_disk {

initialize_params {

image = "debian-cloud / debian-9"

}

}

metadata = {

ssh-keys = "essh: $ {file (" ./ node-cluster.pub ")}"

}

network_interface {

network = "default"

access_config {

nat_ip = "$ {google_compute_address.static-ip-address.address}"

}

}

} essh @ kubernetes-master: ~ / node-cluster $ sudo ./terraform apply

Let's check the SSH connection to the server:

essh @ kubernetes-master: ~ / node-cluster $ ssh -i ./node-cluster essh@35.228.82.222

The authenticity of host '35 .228.82.222 (35.228.82.222) 'can't be established.

ECDSA key fingerprint is SHA256: o7ykujZp46IF + eu7SaIwXOlRRApiTY1YtXQzsGwO18A.

Are you sure you want to continue connecting (yes / no)? yes

Warning: Permanently added '35 .228.82.222 '(ECDSA) to the list of known hosts.

Linux cluster 4.9.0-9-amd64 # 1 SMP Debian 4.9.168-1 + deb9u2 (2019-05-13) x86_64

The programs included with the Debian GNU / Linux system are free software;

the exact distribution terms for each program are described in the

individual files in / usr / share / doc / * / copyright.

Debian GNU / Linux comes with ABSOLUTELY NO WARRANTY, to the extent

permitted by applicable law.

essh @ cluster: ~ $ ls

essh @ cluster: ~ $ exit

logout

Connection to 35.228.82.222 closed.

Install packages:

essh @ kubernetes-master: ~ / node-cluster $ curl https://sdk.cloud.google.com | bash

essh @ kubernetes-master: ~ / node-cluster $ exec -l $ SHELL

essh @ kubernetes-master: ~ / node-cluster $ gcloud init

Let's choose a project:

You are logged in as: [[e-mail]].

Pick cloud project to use:

[1] agile-aleph-203917

[2] node-cluster-243923

[3] essch

[4] Create a new project

Please enter numeric choice or text value (must exactly match list

item):

Please enter a value between 1 and 4, or a value present in the list: 2

Your current project has been set to: [node-cluster-243923].

Let's choose a zone:

[50] europe-north1-a

Did not print [12] options.

Too many options [62]. Enter "list" at prompt to print choices fully.

Please enter numeric choice or text value (must exactly match list

item):

Please enter a value between 1 and 62, or a value present in the list: 50

essh @ kubernetes-master: ~ / node-cluster $ PROJECT_I = "node-cluster-243923"

essh @ kubernetes-master: ~ / node-cluster $ echo $ PROJECT_I

node-cluster-243923

essh @ kubernetes-master: ~ / node-cluster $ export GOOGLE_APPLICATION_CREDENTIALS = $ HOME / node-cluster / kubernetes_key.json

essh @ kubernetes-master: ~ / node-cluster $ sudo docker-machine create –driver google –google-project $ PROJECT_ID vm01

sudo export GOOGLE_APPLICATION_CREDENTIALS = $ HOME / node-cluster / kubernetes_key.json docker-machine create –driver google –google-project $ PROJECT_ID vm01

// https://docs.docker.com/machine/drivers/gce/

// https://github.com/docker/machine/issues/4722

essh @ kubernetes-master: ~ / node-cluster $ gcloud config list

[compute]

region = europe-north1

zone = europe-north1-a

[core]

account = [e-mail]

disable_usage_reporting = False

project = node-cluster-243923

Your active configuration is: [default]

Let's add copying the file and executing the script:

essh @ kubernetes-master: ~ / node-cluster $ cat main.tf

provider "google" {

credentials = "$ {file (" kubernetes_key.json ")}"

project = "node-cluster-243923"

region = "europe-north1"

}

resource "google_compute_address" "static-ip-address" {

name = "static-ip-address"

}

resource "google_compute_instance" "cluster" {

name = "cluster"

zone = "europe-north1-a"

machine_type = "f1-micro"

boot_disk {

initialize_params {

image = "debian-cloud / debian-9"

}

}

metadata = {

ssh-keys = "essh: $ {file (" ./ node-cluster.pub ")}"

}

network_interface {

network = "default"

access_config {

nat_ip = "$ {google_compute_address.static-ip-address.address}"

}

}

}

resource "null_resource" "cluster" {

triggers = {

cluster_instance_ids = "$ {join (", ", google_compute_instance.cluster. *. id)}"

}

connection {

host = "$ {google_compute_address.static-ip-address.address}"

type = "ssh"

user = "essh"

timeout = "2m"

private_key = "$ {file (" ~ / node-cluster / node-cluster ")}"

# agent = "false"

}

provisioner "file" {

source = "client.js"

destination = "~ / client.js"

}

provisioner "remote-exec" {

inline = [

"cd ~ && echo 1> test.txt"

]

}

essh @ kubernetes-master: ~ / node-cluster $ sudo ./terraform apply

google_compute_address.static-ip-address: Creating …

google_compute_address.static-ip-address: Creation complete after 5s [id = node-cluster-243923 / europe-north1 / static-ip-address]

google_compute_instance.cluster: Creating …

google_compute_instance.cluster: Still creating … [10s elapsed]

google_compute_instance.cluster: Creation complete after 12s [id = cluster]

null_resource.cluster: Creating …

null_resource.cluster: Provisioning with 'file' …

null_resource.cluster: Provisioning with 'remote-exec' …

null_resource.cluster (remote-exec): Connecting to remote host via SSH …

null_resource.cluster (remote-exec): Host: 35.228.82.222

null_resource.cluster (remote-exec): User: essh

null_resource.cluster (remote-exec): Password: false

null_resource.cluster (remote-exec): Private key: true

null_resource.cluster (remote-exec): Certificate: false

null_resource.cluster (remote-exec): SSH Agent: false

null_resource.cluster (remote-exec): Checking Host Key: false

null_resource.cluster (remote-exec): Connected!

null_resource.cluster: Creation complete after 7s [id = 816586071607403364]

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

esschtolts @ cluster: ~ $ ls / home / essh /

client.js test.txt

[sudo] password for essh:

google_compute_address.static-ip-address: Refreshing state … [id = node-cluster-243923 / europe-north1 / static-ip-address]

google_compute_instance.cluster: Refreshing state … [id = cluster]

null_resource.cluster: Refreshing state … [id = 816586071607403364]

Enter a value: yes

null_resource.cluster: Destroying … [id = 816586071607403364]

null_resource.cluster: Destruction complete after 0s

google_compute_instance.cluster: Destroying … [id = cluster]

google_compute_instance.cluster: Still destroying … [id = cluster, 10s elapsed]

google_compute_instance.cluster: Still destroying … [id = cluster, 20s elapsed]

google_compute_instance.cluster: Destruction complete after 27s

google_compute_address.static-ip-address: Destroying … [id = node-cluster-243923 / europe-north1 / static-ip-address]

google_compute_address.static-ip-address: Destruction complete after 8s

To deploy the entire project, you can add it to the repository, and we will upload it to the virtual machine by copying the installation script to this virtual machine and then launching it:

Moving on to Kubernetes

In the minimal version, creating a cluster of three nodes looks like this:

essh @ kubernetes-master: ~ / node-cluster / Kubernetes $ cat main.tf

provider "google" {

credentials = "$ {file (" ../ kubernetes_key.json ")}"

project = "node-cluster-243923"

region = "europe-north1"

}

resource "google_container_cluster" "node-ks" {

name = "node-ks"

location = "europe-north1-a"

initial_node_count = 3

}

essh @ kubernetes-master: ~ / node-cluster / Kubernetes $ sudo ../terraform init

essh @ kubernetes-master: ~ / node-cluster / Kubernetes $ sudo ../terraform apply

The cluster was created in 2:15, and after I added europe-north1-a two additional zones europe-north1 -b , europe-north1-c and set the number of created instances in the zone to one, the cluster was created in 3:13 seconds , because for higher availability, the nodes were created in different data centers: europe-north1-a , europe-north1-b , europe-north1-c :

provider "google" {

credentials = "$ {file (" ../ kubernetes_key.json ")}"

project = "node-cluster-243923"

region = "europe-north1"

}

resource "google_container_cluster" "node-ks" {

name = "node-ks"

location = "europe-north1-a"

node_locations = ["europe-north1-b", "europe-north1-c"]

initial_node_count = 1

}

Now let's split our cluster into two: the control cluster with Kubernetes and the cluster for our PODs. All clusters will be distributed over three data centers. The cluster for our PODs can auto scale under load up to 2 on each zone (from three to six in total):

essh @ kubernetes-master: ~ / node-cluster / Kubernetes $ cat main.tf

provider "google" {

credentials = "$ {file (" ../ kubernetes_key.json ")}"

project = "node-cluster-243923"

region = "europe-north1"

}

resource "google_container_cluster" "node-ks" {

name = "node-ks"

location = "europe-north1-a"

node_locations = ["europe-north1-b", "europe-north1-c"]

initial_node_count = 1

}

resource "google_container_node_pool" "node-ks-pool" {

name = "node-ks-pool"

cluster = "$ {google_container_cluster.node-ks.name}"

location = "europe-north1-a"

node_count = "1"

node_config {

machine_type = "n1-standard-1"

}

autoscaling {

min_node_count = 1

max_node_count = 2

}

}

Let's see what happened and look for the IP address of the cluster entry point:

essh @ kubernetes-master: ~ / node-cluster / Kubernetes $ gcloud container clusters list

NAME LOCATION MASTER_VERSION MASTER_IP MACHINE_TYPE NODE_VERSION NUM_NODES STATUS

node-ks europe-north1-a 1.12.8-gke.6 35.228.20.35 n1-standard-1 1.12.8-gke.6 6 RECONCILING

essh @ kubernetes-master: ~ / node-cluster / Kubernetes $ gcloud container clusters describe node-ks | grep '^ endpoint'

endpoint: 35.228.20.35

essh @ kubernetes-master: ~ / node-cluster / Kubernetes $ ping 35.228.20.35 -c 2

PING 35.228.20.35 (35.228.20.35) 56 (84) bytes of data.

64 bytes from 35.228.20.35: icmp_seq = 1 ttl = 59 time = 8.33 ms

64 bytes from 35.228.20.35: icmp_seq = 2 ttl = 59 time = 7.09 ms

–– 35.228.20.35 ping statistics –

2 packets transmitted, 2 received, 0% packet loss, time 1001ms

rtt min / avg / max / mdev = 7.094 / 7.714 / 8.334 / 0.620 ms

By adding variables, which I selected in a separate file just for clarity, which parameterize our config for different uses, we can use it, for example, to create test and production clusters. Variables can be added as var.name_value , and inserted into the text similarly to JS: $ {var.name_value} , as well as path.root .

essh @ kubernetes-master: ~ / node-cluster / Kubernetes $ cat variables.tf

variable "region" {

default = "europe-north1"

}

variable "project_name" {

type = string

default = ""

}

variable "gce_key" {

default = "./kubernetes_key.json"

}

variable "node_count_zone" {

default = 1

}

They can be passed through the -var switch , for example: sudo ./terraform apply -var = "project_name = node-cluster-243923" .

essh @ kubernetes-master: ~ / node-cluster / Kubernetes $ cp ../kubernetes_key.json.

essh @ kubernetes-master: ~ / node-cluster / Kubernetes $ sudo ../terraform apply -var = "project_name = node-cluster-243923"

Our project in the folder is not only a project, but also a module ready to use:

essh @ kubernetes-master: ~ / node-cluster / Kubernetes $ cd ..

essh @ kubernetes-master: ~ / node-cluster $ cat main.tf

module "Kubernetes" {

source = "./Kubernetes"

project_name = "node-cluster-243923"

}

essh @ kubernetes-master: ~ / node-cluster $ sudo ./terraform apply

Or upload to the public repository:

essh @ kubernetes-master: ~ / node-cluster / Kubernetes $ git init

Initialized empty GIT repository in /home/essh/node-cluster/Kubernetes/.git/

essh @ kubernetes-master: ~ / node-cluster / Kubernetes $ echo "terraform.tfstate" >> .gitignore

essh @ kubernetes-master: ~ / node-cluster / Kubernetes $ echo "terraform.tfstate.backup" >> .gitignore

essh @ kubernetes-master: ~ / node-cluster / Kubernetes $ echo ".terraform /" >> .gitignore

essh @ kubernetes-master: ~ / node-cluster / Kubernetes $ rm -f kubernetes_key.json

essh @ kubernetes-master: ~ / node-cluster / Kubernetes $ git remote add origin https://github.com/ESSch/terraform-google-kubernetes.git

essh @ kubernetes-master: ~ / node-cluster / Kubernetes $ git add.

essh @ kubernetes-master: ~ / node-cluster / Kubernetes $ git commit -m 'create a k8s Terraform module'

[master (root-commit) 4f73c64] create a Kubernetes Terraform module

3 files changed, 48 insertions (+)

create mode 100644 .gitignore

create mode 100644 main.tf

create mode 100644 variables.tf

essh @ kubernetes-master: ~ / node-cluster / Kubernetes $ git push -u origin master

essh @ kubernetes-master: ~ / node-cluster / Kubernetes $ git tag -a v0.0.2 -m 'publish'

essh @ kubernetes-master: ~ / node-cluster / Kubernetes $ git push origin v0.0.2

After publishing in the module registry https://registry.terraform.io/, having met the requirements such as having a description, we can use it:

essh @ kubernetes-master: ~ / node-cluster $ cat main.tf

module "kubernetes" {

# source = "./Kubernetes"

source = "ESSch / kubernetes / google"

version = "0.0.2"

project_name = "node-cluster-243923"

}

essh @ kubernetes-master: ~ / node-cluster $ sudo ./terraform init

essh @ kubernetes-master: ~ / node-cluster $ sudo ./terraform apply

On the next creation of the cluster, I got the error ZONE_RESOURCE_POOL_EXHAUSTED "does not have enough resources available to fulfill the request. Try a different zone, or try again later" , indicating that there are no required servers in this region. For me this is not a problem and I do not need to edit the module's code, because I parameterized the module with the region, and if I just pass the region region = "europe-west2" to the module as a parameter , terraform after the update and initialization command ./terrafrom init and the application command ./terraform apply will transfer my cluster to the specified region. Let's improve our module a little by moving the provider from the Kubernetes child module to the main module (the main script is also a module). Having brought it to the main module, we will be able to use one more module, otherwise the provider in one module will conflict with the provider in another. Inheritance from main module to child modules and their transparency applies only to providers. The rest of the data for transferring from child to parent will have to use external variables, and from parent to child – to parameterize the parent, but this is later, when we will create another model. Also, moving the provider to the parent module will be useful when creating the next module that we will create, since it will create Kubernetes elements that do not depend on the provider, and thus we can untie the Google provider from our module and can be used with other providers supporting Kubernetes. Now we don't need to pass the project name in the variable – it is set in the provider. For the convenience of development, I will use the local connection of the module for now. I created a folder and file for a new module:

essh @ kubernetes-master: ~ / node-cluster $ ls nodejs /

main.tf

essh @ kubernetes-master: ~ / node-cluster $ cat main.tf

// module "kubernetes" {

// source = "ESSch / kubernetes / google"

// version = "0.0.2"

//

// project_name = "node-cluster-243923"

// region = "europe-west2"

//}

provider "google" {

credentials = "$ {file (" ./ kubernetes_key.json ")}"

project = "node-cluster-243923"

region = "europe-west2"

}

module "Kubernetes" {

source = "./Kubernetes"

project_name = "node-cluster-243923"

region = "europe-west2"

}

module "nodejs" {

source = "./nodejs"

}

essh @ kubernetes-master: ~ / node-cluster $ sudo ./terraform init

essh @ kubernetes-master: ~ / node-cluster $ sudo ./terraform apply

Now let's transfer data from the Kubernetes infrastructure module to the application module:

essh @ kubernetes-master: ~ / node-cluster $ cat Kubernetes / outputs.tf

output "endpoint" {

value = google_container_cluster.node-ks.endpoint

sensitive = true

}

output "name" {

value = google_container_cluster.node-ks.name

sensitive = true

}

output "cluster_ca_certificate" {

value = base64decode (google_container_cluster.node-ks.master_auth.0.cluster_ca_certificate)

}

essh @ kubernetes-master: ~ / node-cluster $ cat main.tf

// module "kubernetes" {

// source = "ESSch / kubernetes / google"

// version = "0.0.2"

//

// project_name = "node-cluster-243923"

// region = "europe-west2"

//}

provider "google" {

credentials = file ("./ kubernetes_key.json")

project = "node-cluster-243923"

region = "europe-west2"

}

module "Kubernetes" {

source = "./Kubernetes"

project_name = "node-cluster-243923"

region = "europe-west2"

}

module "nodejs" {

source = "./nodejs"

endpoint = module.Kubernetes.endpoint

cluster_ca_certificate = module.Kubernetes.cluster_ca_certificate

}

essh @ kubernetes-master: ~ / node-cluster $ cat nodejs / variable.tf

variable "endpoint" {}

variable "cluster_ca_certificate" {}

To check the balancing of traffic from all nodes, start NGINX, replacing the standard page with the hostname. We'll replace it with a simple command call and resume the server. To start the server, let's look at its call in the Dockerfile: CMD ["Nginx", "-g", "daemon off;"] , which is equivalent to calling Nginx -g 'daemon off;' at the command line. As you can see, the Dockerfile does not use BASH as an environment for launching, but starts the server itself, which allows the shell to live in the event of a process crash, preventing the container from crashing and re-creating. But for our experiments, BASH is fine:

essh @ kubernetes-master: ~ / node-cluster $ sudo docker run -it Nginx: 1.17.0 which Nginx

/ usr / sbin / Nginx

sudo docker run -it –rm -p 8333: 80 Nginx: 1.17.0 / bin / bash -c "echo \ $ HOSTNAME> /usr/share/Nginx/html/index2.html && / usr / sbin / Nginx – g 'daemon off;' "

Now let's create our PODs in triplicate with NGINX, which Kubernetes will try to distribute to different servers by default. Let's also add the service as a balancer:

essh @ kubernetes-master: ~ / node-cluster $ cat nodejs / main.tf

terraform {

required_version = "> = 0.12.0"

}

data "google_client_config" "default" {}

provider "kubernetes" {

host = var.endpoint

token = data.google_client_config.default.access_token

cluster_ca_certificate = var.cluster_ca_certificate