Setup CI/CD Pipeline

Now we need to setup the pipline stages in the .gitlab-ci.yml file. This file acts as the control file for defining all CI/CD jobs.

We also need to define the test cases that will be used to validate changes to the staging and production environments.

Test cases

When managing your network with NaC/IaC, adding good test cases is critical. Your test code should be equal to or better than the production code. The main objective for is to validate the code and configuration on a staging environment before deploying it to the production environment. This validation can reduce the chance of an outage dramatically.

In this lab, we will use Ansible validation playbooks for the following:

• Verify that all VRFs are deployed and in the correct state
• Verify that all Networks are deployed and in the correct state

Step 1 - Create An Ansible Validation Playbook For VRFs and Networks

touch ~/workspace/ndfclab/ansible/verify_fabric.yml
cat << EOF > ~/workspace/ndfclab/ansible/verify_fabric.yml
---

- name: Verify Fabric on NDFC
  hosts: ndfc
  gather_facts: false

  tasks:
    - name: verify | check if all VRFs are deployed
      block:
        - name: verify | query all VRFs from {{ fabric_settings.FABRIC_NAME }}
          cisco.dcnm.dcnm_rest:
            path: "/appcenter/cisco/ndfc/api/v1/lan-fabric/rest/top-down/fabrics/{{ fabric_settings.FABRIC_NAME }}/vrfs"
            method: GET
          register: result
        - name: verify | check if status is DEPLOYED
          assert:
            that:
              - item.vrfStatus != "OUT-OF-SYNC"
            quiet: true
          loop: "{{ result.response.DATA }}"

    - name: verify | check if all Networks are deployed
      block:
        - name: verify | query all Networks from {{ fabric_settings.FABRIC_NAME }}
          cisco.dcnm.dcnm_rest:
            path: "/appcenter/cisco/ndfc/api/v1/lan-fabric/rest/top-down/fabrics/{{ fabric_settings.FABRIC_NAME }}/networks"
            method: GET
          register: result
        - name: verify | check if status is DEPLOYED
          assert:
            that:
              - item.networkStatus != "OUT-OF-SYNC"
            quiet: true
          loop: "{{ result.response.DATA }}"
EOF

Examine the playbook above

We are using the cisco.dcnm.dcnm_rest module from the collection to query the VRF's and Networks on NDFC and then using Ansible ansible.builtin.assert to make sure they are deployed properly and fail if not.

Step 3 - Create Ansible Inventory File for Leaf Devices

touch ~/workspace/ndfclab/ansible/leaf_devices.stage.yml
cat << EOF > ~/workspace/ndfclab/ansible/leaf_devices.stage.yml
---
# Connection Information For Staging Fabric
#
# This file defines how Ansible will connect to the leaf1
nxos:
  children:
    stage:
      hosts:
        10.15.1.12:
          ansible_connection: ansible.netcommon.network_cli
          ansible_httpapi_use_ssl: true
          ansible_httpapi_validate_certs: false
          ansible_python_interpreter: auto_silent
          ansible_network_os: cisco.nxos.nxos
          ansible_user: admin
          ansible_password: cisco.123
EOF
touch ~/workspace/ndfclab/ansible/leaf_devices.prod.yml
cat << EOF > ~/workspace/ndfclab/ansible/leaf_devices.prod.yml
---
# Connection Information For Production Fabric
#
# This file defines how Ansible will connect to the leaf1
nxos:
  children:
    prod:
      hosts:
        10.15.1.19:
          ansible_connection: ansible.netcommon.network_cli
          ansible_httpapi_use_ssl: true
          ansible_httpapi_validate_certs: false
          ansible_python_interpreter: auto_silent
          ansible_network_os: cisco.nxos.nxos
          ansible_user: admin
          ansible_password: cisco.123
EOF

Step 4 - Define the CI/CD pipeline stages

CI/CD pipeline

The pipeline defines the stages of the CI/CD workflow. Some stages should be triggered when a PR is created, and some should be triggered when the code is merged. Different VCSs have different methods to define the pipeline. Since we are using Gitlab, the pipeline stages are defined in file .gitlab-ci.yml in the project root folder.

Step 5 - Create The GitLab CI/CD Pipeline Job Control File

touch ~/workspace/ndfclab/ansible/.gitlab-ci.yml

Step 6 - Open GitLab CI/CD Pipeline Job Control File

code-server -r ~/workspace/ndfclab/ansible/.gitlab-ci.yml

Step 7 - Copy and Paste The Following Into the GitLab Job Control File

---
variables:
  environ: "staging"
stages:
  - lint
  - deploy
  - verify

ansible lint:
  stage: lint
  image:
    name: "cytopia/ansible-lint:5"
    entrypoint: [""]
  rules:
    - if: '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "stage"'
  before_script:
    - export ANSIBLE_PERSISTENT_COMMAND_TIMEOUT=1000
    - export ANSIBLE_PERSISTENT_CONNECT_TIMEOUT=1000
    - export no_proxy="localhost,127.0.0.1,172.25.74.47"
    - python3 -m pip install requests
  script:
    - ansible-galaxy -vvv collection install cisco.dcnm
    - ansible-lint -p roles/create_fabric
    - ansible-lint -p group_vars/all/overlay.yml

deploy_on_stage:
  stage: deploy
  image:
    name: "cytopia/ansible"
    entrypoint: [""]
  rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event' && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "main"
  before_script:
    - export ANSIBLE_PERSISTENT_COMMAND_TIMEOUT=1000
    - export ANSIBLE_PERSISTENT_CONNECT_TIMEOUT=1000
    - python3 -m pip install requests
    - python3 -m pip install jmespath
  script:
    - ansible-playbook --version
    - ansible-galaxy -vvv collection install cisco.dcnm
    - ansible-playbook -i hosts.stage.yml build_fabric.yml

verify_on_stage:
  stage: verify
  image:
    name: "cytopia/ansible"
    entrypoint: [""]
  rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event' && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "main"
  before_script:
    - export ANSIBLE_HOST_KEY_CHECKING=False
    - python3 -m pip install requests
    - python3 -m pip install paramiko
  script:
    - sleep 10
    - ansible-playbook --version
    - ansible-galaxy -vvv collection install cisco.dcnm
    - ansible-playbook -i hosts.stage.yml verify_fabric.yml

deploy_on_prod:
  stage: deploy
  image:
    name: "cytopia/ansible"
    entrypoint: [""]
  rules:
    - if: '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "main"'
  before_script:
    - export ANSIBLE_PERSISTENT_COMMAND_TIMEOUT=1000
    - export ANSIBLE_PERSISTENT_CONNECT_TIMEOUT=1000
    - python3 -m pip install requests
    - python3 -m pip install jmespath
  script:
    - ansible-playbook --version
    - ansible-galaxy -vvv collection install cisco.dcnm
    - ansible-playbook -i hosts.prod.yml build_fabric.yml

verify_on_prod:
  stage: verify
  image:
    name: "cytopia/ansible"
    entrypoint: [""]
  rules:
    - if: '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "main"'
  before_script:
    - export ANSIBLE_HOST_KEY_CHECKING=False
    - python3 -m pip install requests
    - python3 -m pip install paramiko
  script:
    - sleep 10
    - ansible-playbook --version
    - ansible-galaxy -vvv collection install cisco.dcnm
    - ansible-playbook -i hosts.prod.yml verify_fabric.yml

After successfully populating the file above, save the file using Ctrl+s on the Windows keyboard or by clicking File then Save.

The pipeline file above defines 3 stages.

• lint stage
• deploy stage
• verify stage

Lint Stage

The lint stage above will only run when you commit or push code to the stage branch. The ansible-lint tool runs against the build_fabric.yml and group_vars/all/overlay.yml files in our repository. Ansible Lint is a command-line tool for linting playbooks, roles and collections aimed towards any Ansible users. Its main goal is to promote proven practices, patterns and behaviors while avoiding common pitfalls that can easily lead to bugs or make code harder to maintain. This CI stage can be extended to lint any additional files as required.

Deploy Stage

The deploy stage above will only run when a pull request or merge request is opened against the main branch or when code is committed or pushed to the main branch.

• When a merge request is opened from the stage branch against the main branch, the build_fabric.yml playbook will run against the staging environment.
• When the code from the merge request is pushed from the stage branch into the main branch, the build_fabric.yml playbook will run against the production environment.

Verify Stage

The verify stage above will only run when a pull request or merge request is opened against the main branch or when code is committed or pushed to the main branch.

• When a merge request is opened from the stage branch against the main branch, the verify_fabric.yml playbook will run against the staging environment.
• When the code from the merge request is pushed from the stage branch into the main branch, the verify_fabric.yml playbook will run against the production environment.

All of these stages run in a docker container that are built with the ansible-lint, ansible-playbook and ansible-galaxy binaries.

We will tigger the various pipeline stages in the next section.