Configuring for Red Hat Virtualization

You can configure OKD for Red Hat Virtualization by creating a bastion virtual machine and using it to install OKD.

Creating the bastion virtual machine

Create a bastion virtual machine in Red Hat Virtualization to install OKD.

Procedure

  1. Log in to the Manager machine by using SSH.

  2. Create a temporary bastion installation directory, for example, /bastion_installation, for the installation files.

  3. Create an encrypted /bastion_installation/secure_vars.yaml file with ansible-vault and record the password:

    1. # ansible-vault create secure_vars.yaml

  4. Add the following parameter values to the secure_vars.yaml file:

    1. engine_password: <Manager_password> (1)
    2. bastion_root_password: <bastion_root_password> (2)
    3. rhsub_user: <Red_Hat_Subscription_Manager_username> (3)
    4. rhsub_pass: <Red_Hat_Subscription_Manager_password>
    5. rhsub_pool: <Red_Hat_Subscription_Manager_pool_id> (4)
    6. root_password: <OpenShift_node_root_password> (5)
    7. engine_cafile: <RHVM_CA_certificate> (6)
    8. oreg_auth_user: <image_registry_authentication_username> (7)
    9. oreg_auth_password: <image_registry_authentication_password>
    1Password for logging in to the Administration Portal.
    2Root password for the bastion virtual machine.
    3Red Hat Subscription Manager credentials.
    4Pool ID of the Red Hat Virtualization Manager subscription pool.
    5OKD root password.
    6Red Hat Virtualization Manager CA certificate. The engine_cafile value is required if you are not running the playbook from the Manager machine. The Manager CA certificate’s default location is /etc/pki/ovirt-engine/ca.pem.
    7If you are using an image registry that requires authentication, add the credentials.

  5. Save the file.

  6. Obtain the Red Hat Enterprise Linux KVM Guest Image download link:

    1. Navigate to Red Hat Customer Portal: Download Red Hat Enterprise Linux.

    2. In the Product Software tab, locate the Red Hat Enterprise Linux KVM Guest Image.

    3. Right-click Download Now, copy the link, and save it.

      The link is time-sensitive and must be copied just before you create the bastion virtual machine.

  1. Create the /bastion_installation/create-bastion-machine-playbook.yaml file with the following content and update its parameter values:

    1. ---
    2. - name: Create a bastion machine
    3.   hosts: localhost
    4.   connection: local
    5.   gather_facts: false
    6.   no_log: true
    8.   roles:
    9.     - oVirt.image-template
    10.     - oVirt.vm-infra
    11.   no_log: true
    13.   vars:
    14.     engine_url: https://_Manager_FQDN_/ovirt-engine/api (1)
    15.     engine_user: <admin@internal>
    16.     engine_password: "{{ engine_password }}"
    17.     engine_cafile: /etc/pki/ovirt-engine/ca.pem
    19.     qcow_url: <RHEL_KVM_guest_image_download_link> (2)
    20.     template_cluster: Default
    21.     template_name: rhelguest7
    22.     template_memory: 4GiB
    23.     template_cpu: 2
    24.     wait_for_ip: true
    25.     debug_vm_create: false
    27.     vms:
    28.     - name: rhel-bastion
    29.       cluster: "{{ template_cluster }}"
    30.       profile:
    31.         cores: 2
    32.         template: "{{ template_name }}"
    33.         root_password: "{{ root_password }}"
    34.         ssh_key: "{{ lookup('file', '/root/.ssh/id_rsa_ssh_ocp_admin.pub') }}"
    35.         state: running
    36.       cloud_init:
    37.         custom_script: |
    38.           rh_subscription:
    39.             username: "{{ rhsub_user }}"
    40.             password: "{{ rhsub_pass }}"
    41.             auto-attach: true
    42.             disable-repo: ['*']
    43.             # 'rhel-7-server-rhv-4.2-manager-rpms' supports RHV 4.2 and 4.3
    44.             enable-repo: ['rhel-7-server-rpms', 'rhel-7-server-extras-rpms', 'rhel-7-server-ansible-2.7-rpms', 'rhel-7-server-ose-3.11-rpms', 'rhel-7-server-supplementary-rpms', 'rhel-7-server-rhv-4.2-manager-rpms']
    45.           packages:
    46.             - ansible
    47.             - ovirt-ansible-roles
    48.             - openshift-ansible
    49.             - python-ovirt-engine-sdk4
    50.   pre_tasks:
    51.     - name: Create an ssh key-pair for OpenShift admin
    52.       user:
    53.         name: root
    54.         generate_ssh_key: yes
    55.         ssh_key_file: .ssh/id_rsa_ssh_ocp_admin
    57.   roles:
    58.     - oVirt.image-template
    59.     - oVirt.vm-infra
    61. - name: post installation tasks on the bastion machine
    62.   hosts: rhel-bastion
    63.   tasks:
    64.     - name: create ovirt-engine PKI dir
    65.       file:
    66.         state: directory
    67.         dest: /etc/pki/ovirt-engine/
    68.     - name: Copy the engine ca cert to the bastion machine
    69.       copy:
    70.         src: "{{ engine_cafile }}"
    71.         dest: "{{ engine_cafile }}"
    72.     - name: Copy the secured vars to the bastion machine
    73.       copy:
    74.         src: secure_vars.yaml
    75.         dest: secure_vars.yaml
    76.         decrypt: false
    77.     - file:
    78.         state: directory
    79.         path: /root/.ssh
    80.     - name: copy the OpenShift_admin keypair to the bastion machine
    81.       copy:
    82.         src: "{{ item }}"
    83.         dest: "{{ item }}"
    84.         mode: 0600
    85.       with_items:
    86.         - /root/.ssh/id_rsa_ssh_ocp_admin
    87.         - /root/.ssh/id_rsa_ssh_ocp_admin.pub
    1FQDN of the Manager machine.
    2<qcow_url> is the download link of the Red Hat Enterprise Linux KVM Guest Image. The Red Hat Enterprise Linux KVM Guest Image includes the cloud-init package, which is required by this playbook. If you are not using Red Hat Enterprise Linux, download the cloud-init package and install it manually before running this playbook.

  2. Create the bastion virtual machine:

    1. # ansible-playbook -i localhost create-bastion-machine-playbook.yaml -e @secure_vars.yaml --ask-vault-pass

  3. Log in to the Administration Portal.

  4. Click Compute Virtual Machines to verify that the rhel-bastion virtual machine was created successfully.

Installing OKD with the bastion virtual machine

Install OKD by using the bastion virtual machine in Red Hat Virtualization.

Procedure

  1. Log in to rhel-bastion.

  2. Create an install_ocp.yaml file that contains the following content:

    1. ---
    2. - name: Openshift on RHV
    3.   hosts: localhost
    4.   connection: local
    5.   gather_facts: false
    7.   vars_files:
    8.     - vars.yaml
    9.     - secure_vars.yaml
    11.   pre_tasks:
    12.     - ovirt_auth:
    13.         url:      "{{ engine_url }}"
    14.         username: "{{ engine_user }}"
    15.         password: "{{ engine_password }}"
    16.         insecure: "{{ engine_insecure }}"
    17.         ca_file:  "{{ engine_cafile | default(omit) }}"
    19.   roles:
    20.     - role: openshift_ovirt
    22. - import_playbook: setup_dns.yaml
    23. - import_playbook: /usr/share/ansible/openshift-ansible/playbooks/prerequisites.yml
    24. - import_playbook: /usr/share/ansible/openshift-ansible/playbooks/openshift-node/network_manager.yml
    25. - import_playbook: /usr/share/ansible/openshift-ansible/playbooks/deploy_cluster.yml

  3. Create a setup_dns.yaml file that contains the following content:

    1. - hosts: masters
    2.   strategy: free
    3.   tasks:
    4.     - shell: "echo {{ ansible_default_ipv4.address }} {{ inventory_hostname }} etcd.{{ inventory_hostname.split('.', 1)[1] }} openshift-master.{{ inventory_hostname.split('.', 1)[1] }} openshift-public-master.{{ inventory_hostname.split('.', 1)[1] }} docker-registry-default.apps.{{ inventory_hostname.split('.', 1)[1] }} webconsole.openshift-web-console.svc registry-console-default.apps.{{ inventory_hostname.split('.', 1)[1] }} >> /etc/hosts"
    5.       when: openshift_ovirt_all_in_one is defined | ternary((openshift_ovirt_all_in_one | bool), false)

  4. Create an /etc/ansible/openshift_3_11.hosts Ansible inventory file that contains the following content:

    1. [workstation]
    2. localhost ansible_connection=local
    4. [all:vars]
    5. openshift_ovirt_dns_zone="{{ public_hosted_zone }}"
    6. openshift_web_console_install=true
    7. openshift_master_overwrite_named_certificates=true
    8. openshift_master_cluster_hostname="openshift-master.{{ public_hosted_zone }}"
    9. openshift_master_cluster_public_hostname="openshift-public-master.{{ public_hosted_zone }}"
    10. openshift_master_default_subdomain="{{ public_hosted_zone }}"
    11. openshift_public_hostname="{{openshift_master_cluster_public_hostname}}"
    12. openshift_deployment_type=openshift-enterprise
    13. openshift_service_catalog_image_version="{{ openshift_image_tag }}"
    15. [OSEv3:vars]
    16. # General variables
    17. debug_level=1
    18. containerized=false
    19. ansible_ssh_user=root
    20. os_firewall_use_firewalld=true
    21. openshift_enable_excluders=false
    22. openshift_install_examples=false
    23. openshift_clock_enabled=true
    24. openshift_debug_level="{{ debug_level }}"
    25. openshift_node_debug_level="{{ node_debug_level | default(debug_level,true) }}"
    26. osn_storage_plugin_deps=[]
    27. openshift_master_bootstrap_auto_approve=true
    28. openshift_master_bootstrap_auto_approver_node_selector={"node-role.kubernetes.io/master":"true"}
    29. osm_controller_args={"experimental-cluster-signing-duration": ["20m"]}
    30. osm_default_node_selector="node-role.kubernetes.io/compute=true"
    31. openshift_enable_service_catalog=false
    33. # Docker
    34. container_runtime_docker_storage_type=overlay2
    35. openshift_docker_use_system_container=false
    37. [OSEv3:children]
    38. nodes
    39. masters
    40. etcd
    41. lb
    43. [masters]
    44. [nodes]
    45. [etcd]
    46. [lb]

  5. Obtain the Red Hat Enterprise Linux KVM Guest Image download link:

    1. Navigate to Red Hat Customer Portal: Download Red Hat Enterprise Linux.

    2. In the Product Software tab, locate the Red Hat Enterprise Linux KVM Guest Image.

    3. Right-click Download Now, copy the link, and save it.

      Do not use the link that you copied when you created the bastion virtual machine. The download link is time-sensitive and must be copied just before you run the installation playbook.

  1. Create the vars.yaml file with the following content and update its parameter values:

    1. ---
    2. # For detailed documentation of variables, see
    3. # openshift_ovirt: https://github.com/openshift/openshift-ansible/tree/master/roles/openshift_ovirt#role-variables
    4. # openshift installation: https://github.com/openshift/openshift-ansible/tree/master/inventory
    5. engine_url: https://<Manager_FQDN>/ovirt-engine/api (1)
    6. engine_user: admin@internal
    7. engine_password: "{{ engine_password }}"
    8. engine_insecure: false
    9. engine_cafile: /etc/pki/ovirt-engine/ca.pem
    11. openshift_ovirt_vm_manifest:
    12.   - name: 'master'
    13.     count: 1
    14.     profile: 'master_vm'
    15.   - name: 'compute'
    16.     count: 0
    17.     profile: 'node_vm'
    18.   - name: 'lb'
    19.     count: 0
    20.     profile: 'node_vm'
    21.   - name: 'etcd'
    22.     count: 0
    23.     profile: 'node_vm'
    24.   - name: infra
    25.     count: 0
    26.     profile: node_vm
    28. # Currently, only all-in-one installation (`openshift_ovirt_all_in_one: true`) is supported.
    29. # Multi-node installation (master and node VMs installed separately) will be supported in a future release.
    30. openshift_ovirt_all_in_one: true
    31. openshift_ovirt_cluster: Default
    32. openshift_ovirt_data_store: data
    33. openshift_ovirt_ssh_key: "{{ lookup('file', '/root/.ssh/id_rsa_ssh_ocp_admin.pub') }}"
    35. public_hosted_zone:
    36. # Uncomment to disable install-time checks, for smaller scale installations
    37. #openshift_disable_check: memory_availability,disk_availability,docker_image_availability
    39. qcow_url: <RHEL_KVM_guest_image_download_link> (2)
    40. image_path: /var/tmp
    41. template_name: rhelguest7
    42. template_cluster: "{{ openshift_ovirt_cluster }}"
    43. template_memory: 4GiB
    44. template_cpu: 1
    45. template_disk_storage: "{{ openshift_ovirt_data_store }}"
    46. template_disk_size: 100GiB
    47. template_nics:
    48.   - name: nic1
    49.     profile_name: ovirtmgmt
    50.     interface: virtio
    52. debug_vm_create: false
    53. wait_for_ip: true
    54. vm_infra_wait_for_ip_retries: 30
    55. vm_infra_wait_for_ip_delay: 20
    57. node_item: &node_item
    58.   cluster: "{{ openshift_ovirt_cluster }}"
    59.   template: "{{ template_name }}"
    60.   memory: "8GiB"
    61.   cores: "2"
    62.   high_availability: true
    63.   disks:
    64.     - name: docker
    65.       size: 15GiB
    66.       interface: virtio
    67.       storage_domain: "{{ openshift_ovirt_data_store }}"
    68.     - name: openshift
    69.       size: 30GiB
    70.       interface: virtio
    71.       storage_domain: "{{ openshift_ovirt_data_store }}"
    72.   state: running
    73.   cloud_init:
    74.     root_password: "{{ root_password }}"
    75.     authorized_ssh_keys: "{{ openshift_ovirt_ssh_key }}"
    76.     custom_script: "{{ cloud_init_script_node | to_nice_yaml }}"
    78. openshift_ovirt_vm_profile:
    79.   master_vm:
    80.     <<: *node_item
    81.     memory: 16GiB
    82.     cores: "{{ vm_cores | default(4) }}"
    83.     disks:
    84.       - name: docker
    85.         size: 15GiB
    86.         interface: virtio
    87.         storage_domain: "{{ openshift_ovirt_data_store }}"
    88.       - name: openshift_local
    89.         size: 30GiB
    90.         interface: virtio
    91.         storage_domain: "{{ openshift_ovirt_data_store }}"
    92.       - name: etcd
    93.         size: 25GiB
    94.         interface: virtio
    95.         storage_domain: "{{ openshift_ovirt_data_store }}"
    96.     cloud_init:
    97.       root_password: "{{ root_password }}"
    98.       authorized_ssh_keys: "{{ openshift_ovirt_ssh_key }}"
    99.       custom_script: "{{ cloud_init_script_master | to_nice_yaml }}"
    100.   node_vm:
    101.     <<: *node_item
    102.   etcd_vm:
    103.     <<: *node_item
    104.   lb_vm:
    105.     <<: *node_item
    107. cloud_init_script_node: &cloud_init_script_node
    108.   packages:
    109.     - ovirt-guest-agent
    110.   runcmd:
    111.     - sed -i 's/# ignored_nics =.*/ignored_nics = docker0 tun0 /' /etc/ovirt-guest-agent.conf
    112.     - systemctl enable ovirt-guest-agent
    113.     - systemctl start ovirt-guest-agent
    114.     - mkdir -p /var/lib/docker
    115.     - mkdir -p /var/lib/origin/openshift.local.volumes
    116.     - /usr/sbin/mkfs.xfs -L dockerlv /dev/vdb
    117.     - /usr/sbin/mkfs.xfs -L ocplv /dev/vdc
    118.   mounts:
    119.     - [ '/dev/vdb', '/var/lib/docker', 'xfs', 'defaults,gquota' ]
    120.     - [ '/dev/vdc', '/var/lib/origin/openshift.local.volumes', 'xfs', 'defaults,gquota' ]
    121.   power_state:
    122.     mode: reboot
    123.     message: cloud init finished - boot and install openshift
    124.     condition: True
    125. cloud_init_script_master:
    126.   <<: *cloud_init_script_node
    127.   runcmd:
    128.     - sed -i 's/# ignored_nics =.*/ignored_nics = docker0 tun0 /' /etc/ovirt-guest-agent.conf
    129.     - systemctl enable ovirt-guest-agent
    130.     - systemctl start ovirt-guest-agent
    131.     - mkdir -p /var/lib/docker
    132.     - mkdir -p /var/lib/origin/openshift.local.volumes
    133.     - mkdir -p /var/lib/etcd
    134.     - /usr/sbin/mkfs.xfs -L dockerlv /dev/vdb
    135.     - /usr/sbin/mkfs.xfs -L ocplv /dev/vdc
    136.     - /usr/sbin/mkfs.xfs -L etcdlv /dev/vdd
    137.   mounts:
    138.     - [ '/dev/vdb', '/var/lib/docker', 'xfs', 'defaults,gquota' ]
    139.     - [ '/dev/vdc', '/var/lib/origin/openshift.local.volumes', 'xfs', 'defaults,gquota' ]
    140.     - [ '/dev/vdd', '/var/lib/etcd', 'xfs', 'defaults,gquota' ]
    1FQDN of the Manager machine.
    2<qcow_url> is the download link of the Red Hat Enterprise Linux KVM Guest Image. The Red Hat Enterprise Linux KVM Guest Image includes the cloud-init package, which is required by this playbook. If you are not using Red Hat Enterprise Linux, download the cloud-init package and install it manually before running this playbook.

  2. Install OKD:

    1. # export ANSIBLE_ROLES_PATH="/usr/share/ansible/roles/:/usr/share/ansible/openshift-ansible/roles"
    2. # export ANSIBLE_JINJA2_EXTENSIONS="jinja2.ext.do"
    3. # ansible-playbook -i /etc/ansible/openshift_3_11.hosts install_ocp.yaml -e @vars.yaml -e @secure_vars.yaml --ask-vault-pass

  3. Create DNS entries for the routers, for each infrastructure instance.

  4. Configure round-robin routing so that the router can pass traffic to the applications.

  5. Create a DNS entry for the OKD web console.

  6. Specify the IP address of the load balancer node.