• Nexus Dashboard
  • USER: admin
  • PASS: cisco.123
  • ND Web interface
  • Switch Login
  • USER: admin
  • PASS: cisco.123
  • staging-spine1: 10.15.5.11
  • staging-leaf1: 10.15.5.12
  • staging-leaf2: 10.15.5.13
  • staging-leaf3: 10.15.5.14
  • staing-ext-rtr: 10.15.5.15
  • prod-spine1: 10.15.5.18
  • prod-leaf1: 10.15.5.19
  • prod-leaf2: 10.15.5.20
  • prod-leaf3: 10.15.5.21
  • prod-ext-rtr: 10.15.5.22
  • GitLab Server
  • USER: pod05
  • PASS: c1sco.123
  • Gitlab
  • Test VM Login
  • USER: cisco
  • PASS: cisco.123
  • staging-server1: 10.15.5.16
  • staging-server2: 10.15.5.17
  • prod-server1: 10.15.5.23
  • prod-server2: 10.15.5.24
  • Code-Server
  • address: https://10.15.0.165:8443/?folder=/home/pod05/workspace
Pipeline Setup
CI/CD

Setup CI/CD Pipeline

Now we need to setup the pipline stages in the .gitlab-ci.yml file. This file acts as the control file for defining all CI/CD jobs.

We also need to define the test cases that will be used to validate changes to the staging and production environments.

Test cases

When managing your network with NaC/IaC, adding good test cases is critical. Your test code should be equal to or better than the production code. The main objective for is to validate the code and configuration on a staging environment before deploying it to the production environment. This validation can reduce the chance of an outage dramatically.

In this lab, we will use Ansible validation playbooks for the following:

  • Verify that all VRFs are deployed and in the correct state
  • Verify that all Networks are deployed and in the correct state

Note

The validation we do as part of this lab is just a simple example. The validation needed in a real staging and production environment should include as many tests as required in order to ensure that what you are deploying to the production environment will not break your system.

Step 1 - Create An Ansible Validation Playbook For VRFs and Networks

touch ~/workspace/ndfclab/ansible/verify_fabric.yml
cat << EOF > ~/workspace/ndfclab/ansible/verify_fabric.yml
---

- name: Verify Fabric on NDFC
  hosts: ndfc
  gather_facts: false

  tasks:
    - name: verify | check if all VRFs are deployed
      block:
        - name: verify | query all VRFs from {{ fabric_settings.FABRIC_NAME }}
          cisco.dcnm.dcnm_rest:
            path: "/appcenter/cisco/ndfc/api/v1/lan-fabric/rest/top-down/fabrics/{{ fabric_settings.FABRIC_NAME }}/vrfs"
            method: GET
          register: result
        - name: verify | check if status is DEPLOYED
          assert:
            that:
              - item.vrfStatus != "OUT-OF-SYNC"
            quiet: true
          loop: "{{ result.response.DATA }}"

    - name: verify | check if all Networks are deployed
      block:
        - name: verify | query all Networks from {{ fabric_settings.FABRIC_NAME }}
          cisco.dcnm.dcnm_rest:
            path: "/appcenter/cisco/ndfc/api/v1/lan-fabric/rest/top-down/fabrics/{{ fabric_settings.FABRIC_NAME }}/networks"
            method: GET
          register: result
        - name: verify | check if status is DEPLOYED
          assert:
            that:
              - item.networkStatus != "OUT-OF-SYNC"
            quiet: true
          loop: "{{ result.response.DATA }}"
EOF
YAML

Examine the playbook above

We are using the cisco.dcnm.dcnm_rest module from the collection to query the VRF's and Networks on NDFC and then using Ansible ansible.builtin.assert to make sure they are deployed properly and fail if not.

Step 3 - Create Ansible Inventory File for Leaf Devices

touch ~/workspace/ndfclab/ansible/leaf_devices.stage.yml
cat << EOF > ~/workspace/ndfclab/ansible/leaf_devices.stage.yml
---
# Connection Information For Staging Fabric
#
# This file defines how Ansible will connect to the leaf1
nxos:
  children:
    stage:
      hosts:
        10.15.5.12:
          ansible_connection: ansible.netcommon.network_cli
          ansible_httpapi_use_ssl: true
          ansible_httpapi_validate_certs: false
          ansible_python_interpreter: auto_silent
          ansible_network_os: cisco.nxos.nxos
          ansible_user: admin
          ansible_password: cisco.123
EOF
touch ~/workspace/ndfclab/ansible/leaf_devices.prod.yml
cat << EOF > ~/workspace/ndfclab/ansible/leaf_devices.prod.yml
---
# Connection Information For Production Fabric
#
# This file defines how Ansible will connect to the leaf1
nxos:
  children:
    prod:
      hosts:
        10.15.5.19:
          ansible_connection: ansible.netcommon.network_cli
          ansible_httpapi_use_ssl: true
          ansible_httpapi_validate_certs: false
          ansible_python_interpreter: auto_silent
          ansible_network_os: cisco.nxos.nxos
          ansible_user: admin
          ansible_password: cisco.123
EOF
YAML

Step 4 - Define the CI/CD pipeline stages

CI/CD pipeline
The pipeline defines the stages of the CI/CD workflow. Some stages should be triggered when a PR is created, and some should be triggered when the code is merged. Different VCSs have different methods to define the pipeline. Since we are using Gitlab, the pipeline stages are defined in file .gitlab-ci.yml in the project root folder.

Step 5 - Create The GitLab CI/CD Pipeline Job Control File

touch ~/workspace/ndfclab/ansible/.gitlab-ci.yml
Bash

Step 6 - Open GitLab CI/CD Pipeline Job Control File

code-server -r ~/workspace/ndfclab/ansible/.gitlab-ci.yml
Bash

Step 7 - Copy and Paste The Following Into the GitLab Job Control File

---
variables:
  environ: "staging"
stages:
  - lint
  - deploy
  - verify

ansible lint:
  stage: lint
  image:
    name: "cytopia/ansible-lint:5"
    entrypoint: [""]
  rules:
    - if: '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "stage"'
  before_script:
    - export ANSIBLE_PERSISTENT_COMMAND_TIMEOUT=1000
    - export ANSIBLE_PERSISTENT_CONNECT_TIMEOUT=1000
    - export no_proxy="localhost,127.0.0.1,172.25.74.47"
    - python3 -m pip install requests
  script:
    - ansible-galaxy -vvv collection install cisco.dcnm
    - ansible-lint -p roles/create_fabric
    - ansible-lint -p group_vars/all/overlay.yml

deploy_on_stage:
  stage: deploy
  image:
    name: "cytopia/ansible"
    entrypoint: [""]
  rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event' && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "main"
  before_script:
    - export ANSIBLE_PERSISTENT_COMMAND_TIMEOUT=1000
    - export ANSIBLE_PERSISTENT_CONNECT_TIMEOUT=1000
    - python3 -m pip install requests
    - python3 -m pip install jmespath
  script:
    - ansible-playbook --version
    - ansible-galaxy -vvv collection install cisco.dcnm
    - ansible-playbook -i hosts.stage.yml build_fabric.yml

verify_on_stage:
  stage: verify
  image:
    name: "cytopia/ansible"
    entrypoint: [""]
  rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event' && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "main"
  before_script:
    - export ANSIBLE_HOST_KEY_CHECKING=False
    - python3 -m pip install requests
    - python3 -m pip install paramiko
  script:
    - sleep 10
    - ansible-playbook --version
    - ansible-galaxy -vvv collection install cisco.dcnm
    - ansible-playbook -i hosts.stage.yml verify_fabric.yml

deploy_on_prod:
  stage: deploy
  image:
    name: "cytopia/ansible"
    entrypoint: [""]
  rules:
    - if: '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "main"'
  before_script:
    - export ANSIBLE_PERSISTENT_COMMAND_TIMEOUT=1000
    - export ANSIBLE_PERSISTENT_CONNECT_TIMEOUT=1000
    - python3 -m pip install requests
    - python3 -m pip install jmespath
  script:
    - ansible-playbook --version
    - ansible-galaxy -vvv collection install cisco.dcnm
    - ansible-playbook -i hosts.prod.yml build_fabric.yml

verify_on_prod:
  stage: verify
  image:
    name: "cytopia/ansible"
    entrypoint: [""]
  rules:
    - if: '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "main"'
  before_script:
    - export ANSIBLE_HOST_KEY_CHECKING=False
    - python3 -m pip install requests
    - python3 -m pip install paramiko
  script:
    - sleep 10
    - ansible-playbook --version
    - ansible-galaxy -vvv collection install cisco.dcnm
    - ansible-playbook -i hosts.prod.yml verify_fabric.yml
YAML

After successfully populating the file above, save the file using Ctrl+s on the Windows keyboard or by clicking File then Save.

Warning

Be sure to save your file! Not saving will result in your code not executing.

The pipeline file above defines 3 stages.
  • lint stage
  • deploy stage
  • verify stage

Lint Stage

The lint stage above will only run when you commit or push code to the stage branch. The ansible-lint tool runs against the build_fabric.yml and group_vars/all/overlay.yml files in our repository. Ansible Lint is a command-line tool for linting playbooks, roles and collections aimed towards any Ansible users. Its main goal is to promote proven practices, patterns and behaviors while avoiding common pitfalls that can easily lead to bugs or make code harder to maintain. This CI stage can be extended to lint any additional files as required.

Deploy Stage

The deploy stage above will only run when a pull request or merge request is opened against the main branch or when code is committed or pushed to the main branch.
  • When a merge request is opened from the stage branch against the main branch, the build_fabric.yml playbook will run against the staging environment.
  • When the code from the merge request is pushed from the stage branch into the main branch, the build_fabric.yml playbook will run against the production environment.

Verify Stage

The verify stage above will only run when a pull request or merge request is opened against the main branch or when code is committed or pushed to the main branch.
  • When a merge request is opened from the stage branch against the main branch, the verify_fabric.yml playbook will run against the staging environment.
  • When the code from the merge request is pushed from the stage branch into the main branch, the verify_fabric.yml playbook will run against the production environment.

All of these stages run in a docker container that are built with the ansible-lint, ansible-playbook and ansible-galaxy binaries.

We will tigger the various pipeline stages in the next section.