Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

How to Run Argo Workflows in K3s

Argo Workflows is a powerful Kubernetes-native workflow engine used to create, manage, and orchestrate parallel jobs in your Kubernetes clusters.

Argo also works well with K3s, a lightweight, simplified version of Kubernetes designed for IoT and edge computing. In this tutorial, you will learn how to deploy Argo Workflows to a Kubernetes cluster in K3s.

About Argo Workflows

Argo Workflows offers a number of key features:

  • Web-based UI 
  • Native artifact support (MinIO, S3, Artifactory, HDFS, OSS, HTTP, Git, GCS, raw)
  • Templating and cron workflows
  • Workflow Archive
  • REST API, Argo CLI

The `Workflow` resource is used to define the execution of a workflow as well as its storage state. Workflows consist of instructions that operate like functions, known as templates in Argo. Templates detail the steps of execution in the workflow. 

The `spec` is the most important part of the `Workflow` manifest file. It contains two properties:

  • templates: This defines the types of templates that you want to use in your workflow. 
  • entrypoint: This determines which template will be executed first.

There are multiple types of templates available, such as:

  • Container: The most common template type, this schedules a container. Its spec is identical to that of a Kubernetes container spec.
  • Script: This is a convenience wrapper around the container. The spec is the same as the container, but it has a `source` field that allows you to define a script. The script is saved to a file and executed from there.
  • Resource: This performs create, read, update, and delete (CRUD) operations directly on resources in the cluster.
  • Suspend: This suspends the execution of a workflow either for a specified duration or indefinitely.
  • DAG: This allows you to define the tasks in a workflow as a graph of dependencies. 
  • Steps: This allows you to define the tasks in your workflow as sequential steps. It consists of inner and outer lists; inner lists run in parallel, while outer lists run one after the other.

Prerequisites

In order to follow this tutorial, you’ll need the following:

Getting Started with K3s

First download the installation script and start up the Kubernetes server with the below command: 

You can check the status of your single-node cluster by running `sudo k3s kubectl get node`. By default, the kubeconfig file is written to `/etc/rancher/k3s/k3s.yaml` with privileged permissions. However, you can use kubectl to communicate with your cluster instead by either copying the content of the default `k3s.yml` configuration file to your main kubeconfig file, which is typically located at `~/.kube/config`, or updating the KUBECONFIG environment variable path.

K9s can also be used as a complementary tool for cluster visibility:

argo workflows
K9s cluster pods

Installing Argo CLI

The next step will be to install the Argo CLI. You can use either the latest version or select a previous one from the Argo CLI releases GitHub page. The commands you run will vary depending on your operating system. The two code blocks below are for Mac and Linux, respectively. If you’re using Windows, you can download the relevant executable from the Assets section on the Argo releases GitHub page. 

For Mac:

For Linux:

Installing Argo Controller and UI

Before installing the Argo Workflows resources, you need to create an `argo` namespace:

When the Workflow Controller and the Argo Server have successfully rolled out, you can follow the steps below to access the Argo Workflows UI.

argo workflows
Argo Resources

You can access the UI in multiple ways, but for this tutorial, use the port-forwarding method

Open your browser and go to https://127.0.0.1:2746. You will be redirected to a page for authentication.

argo workflows
Argo workflows login

To log in to the Argo Workflows server, you will need to generate a `ServiceAccount` access token that you will use to manage your workflows. For this, you need to create a `Role` and a `RoleBinding`, and then a `ServiceAccount`

Role: The `Role` resource is used to determine a set of permitted operations on certain Kubernetes resources in a given namespace.

Below is an example of a `Role` manifest: 

RoleBinding:`RoleBinding` is used to determine which users or `ServiceAccounts` are authorized to carry out specific operations on certain resources in a given namespace. The details of the permissions are outlined in the `Role` that the `RoleBinding` is attached to.

Below is an example of a `RoleBinding` manifest:

ServiceAccount: A `ServiceAccount` is used to authenticate machine-level processes to gain access to your Kubernetes cluster. The API server in the control plane manages authentication for the processes running in the Pod.

Below is an example of a `ServiceAccount` manifest:

You can create a manifest file for each of these resources using the code snippets detailed above. Alternatively, you can create the resources directly using the kubectl CLI.

Create resource with kubectl and manifest files:

Create resources with kubectl CLI and parameter options:

Next create an access token and store it in an environment variable (such as `ARGO_TOKEN`):

Copy the printed Bearer token from your terminal and paste it in the text area of the Argo Workflow UI login.

argo workflows
Argo workflows popup

Now you can explore the sidebar menu and select Workflows.

argo workflows
Workflows page

Executing Workflows

In this section, you will create both a basic and an advanced workflow example. 

Secret Template Example

This type of workflow template supports the same secret syntax and mechanisms as the Kubernetes pod specification. You can access a secret that serves as an environment variable or volume by using this template.

First you'll deploy a secret workflow template, which works the same way and supports the same syntax as the Pod specification accessing a secret in Kubernetes. You can access the secret as either an environment variable or a volume. 

Start by creating a secret in your cluster with the following command:

Next you'll create a manifest file for the workflow that will access this secret. 

Now you can use the Argo CLI tool to create the workflow. Argo uses the existing kubeconfig configurations to know which cluster to interact with. To create your first workflow, run the following command:

You can review it in the Argo Workflow UI.

Top secret workflow example

argo workflows
Top secret workflow review

Top secret logs

Volume Example

Next you’ll deploy a workflow that is used to manage volumes. In the example below, a volume will be dynamically created and used in a multi-step workflow.

Create the following manifest file:

As you did before, create this workflow using the Argo CLI:

Once the workflow has completed, you can review the results in the UI and delve into the logs to see the results of the generated message in the mounted volume.

argo workflows
Completed volume workflow example

Multi-step volume workflow

argo workflows
Mounted volume logs

Generated message

Conclusion

In this tutorial, you learned the fundamental concepts around Argo Workflows and how to use it with a single-node K3s cluster. As demonstrated, this workflow engine operates well even in a lightweight version of Kubernetes. 

In an enterprise context, however, your Kubernetes jobs could become more complex. Pipekit allows you to manage these jobs at scale. The control plane configures Argo Workflows for you in your infrastructure, enabling you to optimize multi-cluster workloads while reducing your cloud spend.

Subscribe for Pipekit updates.

Get the latest articles on all things Pipekit & data orchestration delivered straight to your inbox.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Follow Pipekit

Subscribe for the latest Argo content and news from Pipekit

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

More

Tutorials

How to Pass Key-Values between Argo Workflows Part 2

7 min read
Tutorials

How to Configure an Artifact Repo for Argo Workflows

10 min read
Tutorials

How to Use Argo Workflows with Spark

7 min read