CI/CD Catalog goes GA: No more building pipelines from scratch

GitLab's CI/CD Catalog becomes generally available in 17.0 (May 16, 2024), enabling all GitLab users to discover, reuse, and contribute CI/CD components easily. The CI/CD Catalog boosts collaboration and efficiency when creating pipeline configurations by allowing access to a treasure trove of pre-built components, ready to seamlessly integrate into DevSecOps workflows. Enterprises can use the CI/CD Catalog's centralized platform to standardize workflows across the whole organization.

With the CI/CD Catalog, GitLab is introducing several key capabilities that are also generally available.

Discover the future of AI-driven software development with our GitLab 17 virtual launch event. Watch today!

Components and inputs

The CI/CD Catalog draws its strength from two fundamental features: components and inputs. These capabilities form the backbone of the catalog, enabling developers and DevSecOps teams to streamline their pipeline development. Let’s dive into each of these features:

Components

What are components?

Components are reusable, single-purpose building blocks that abstract away the complexity of pipeline configuration. Think of them as Lego pieces for your CI/CD workflows. By using components, you can assemble pipelines more efficiently without starting from scratch each time.

Types of components

• Template-type components: These components resemble CI templates and come with predefined input definitions. They are organized within a specific directory structure, which you can easily plug into your pipelines.
• CI Steps (upcoming): This new type of component, which is available as an experimental feature, will become a first-class object in the CI/CD Catalog, so stay tuned for this exciting addition.

Inputs

What is Inputs Interpolation?

Inputs Interpolation is a powerful feature that allows you to define input parameters for includable configuration files. By using the spec: inputs keyword within your component configuration, you can dynamically replace almost any keywords within components with parameters. This flexibility extends to adjusting stages, scripts, or job names, supporting various data types making the component fully flexible to your needs.

Scoped and effective

Importantly, inputs are scoped exclusively to the included configuration. This prevents unintended effects on the rest of your pipeline. With Inputs Interpolation, you can declare and enforce constraints seamlessly, ensuring smooth integration of components.

Whether you’re a seasoned DevOps pro or just starting out, the CI/CD Catalog, components, and Inputs Interpolation will transform your pipeline development experience.

How to access CI/CD Catalog components

The CI/CD Catalog is a powerful resource for developers and DevOps teams. It allows you to share and discover pre-built components, streamlining your pipeline development. Here’s how it works:

1. Components are standalone building blocks that simplify pipeline configuration. You can create custom components tailored to your needs. But how do you make them available to others? That’s where the CI/CD Catalog comes in.

2. How to publish to the CI/CD Catalog

   • To share your components with the community, follow these steps:
     • Use a simple CI job to publish your component and make it discoverable in the CI/CD Catalog.
     • Whether it’s a reusable script, a deployment template, or any other pipeline element, the CI/CD Catalog is the perfect place to contribute. Components released to the CI/CD Catalog should be tagged with a semantic version using three digits.
   • By sharing your components, you contribute to a growing library of resources that benefit the entire community.

3. Catalog index page

   • The main page of the CI/CD Catalog (also known as the index page) provides an overview of available projects with published components. Anyone can access the catalog and search for a component that suits their needs.
   • The index page features two tabs:
     • All: Displays all component projects that have been published and visible to you.
     • Your groups: Shows components published within a namespace you’re part of.

4. Catalog details page

• Upon clicking on one of the projects in the CI/CD Catalog, you will be redirected to the details page where you can view the available components in that project.

  • Note that there could be multiple components in a single project.

• The details page features two tabs:

Readme: Displays the readme.md of the project that was previously configured by the user.

Components: Displays the detailed information for each component such as inputs table syntax to use and more. This information is generated and displayed automatically to help keep it up to date.

Using a component

To use a component from the CI/CD Catalog, simply copy the suggested snippet to your pipeline configuration. For example:

include: 
  - component:   gitlab.com/google-gitlab-components/cloud-run/[email protected]

Note that the snippet contains the fully qualified domain name of the component, so if you moved or clone the component to a different location, you should make sure the FQDN is accurate. You can use the $CI_SERVER_FQDN variable instead of hardcoding the FQDN in your pipeline configuration.

A component can be referenced using the following:

• a commit SHA, for example, e3262fdd0914fa823210cdb79a8c421e2cef79d. We highly recommend using this with $CI_COMMIT_SHA variable in your .gitlab.ci.yml file to test a component before publishing it to the CI/CD Catalog.
• a branch name, for example, main
• a tag, for example 1.0.0
• shorthand abbreviation 1.0, which will provide you the latest patched 1.0.x version or 1, which will provide you the latest 1.x.x minor version. This is why it is recommended to use the best practices of semantic versioning and always reference a specific version (minor, major, or a specific patch).
• ~latest, which always points to the latest semantic version published in the CI/CD Catalog. Use ~latest only if you want to use the absolute latest version at all times, which could include breaking changes., so please use it with caution.

Understanding the CI/CD Catalog across GitLab deployments

The CI/CD Catalog and components offer different flavors to cater to various needs and use cases.

Private and public components

Public components

• Public components are hosted in public repositories and are accessible to everyone.
• When a public component is published from GitLab.com to the main catalog, it becomes discoverable and available for consumption by all users.
• We encourage users to contribute their best components to the public catalog, helping us build a thriving community.

Private components

• Private components are hosted in private repositories.
• Visibility based on permissions: Users who access the catalog can also see and search for private components if they have permission to view the repository where the component is hosted.
  • Private catalog option: In GitLab.com, organizations can publish private components to the main catalog in GitLab.com, thereby creating a “private catalog” with content accessible only to authorized users.

GitLab.com vs. Self-managed

• The “public” catalog in GitLab.com: The main catalog is the one that is hosted on GitLab.com and can be accessible to anyone by going to gitlab.com/explore/catalog. The CI/CD Catalog is:

  • Open access: The catalog hosted on GitLab.com is available for anyone to view.
  • Contribute and grow: By sharing components, users around the world contribute to a growing library of resources that benefits the entire community.

• Self-managed customers: The CI/CD Catalog is also available for self-managed customers however it has several differences:

  • Empty catalog: For self-managed customers, the catalog initially appears empty since it doesn't contain any available components.
  • Organizational catalog: Each organization is responsible for its own catalog, where it can create and maintain its own library of components within this flavor.
  • Using a component from GitLab.com: If you want to use a component from the main catalog in GitLab.com, clone the project locally and publish it to your organizational catalog. Keep in mind that upstream updates will require mirroring to receive the latest changes. You can learn more about how to do that in our CI/CD Components documentation.

What’s next?

The CI/CD Catalog is only the first step in revolutionizing the way you build and display your available pipelines. Here is a glimpse of what we plan to offer to our users in the upcoming milestones.

CI Steps

Steps are reusable and composable pieces of a job that can be referenced in your pipeline configuration. Each step defines structured inputs and outputs that can be consumed by other steps. Steps can come from local files, GitLab.com repositories, or any other Git source.

In GitLab, we think of steps as another type of component. We are going to make sure CI Steps will become a first-class object in the CI/CD Catalog, where users can publish, unpublish, search, and consume steps in the same way as they are using components today.

Securing your catalog workflows

We aim to empower central administrators to manage component creation, usage, and publication within their organizational catalog. We are committed to ensuring the publishing process seamlessly integrates with the organization's standards and existing workflow. We want to enable the platform administrators with the capabilities to secure and govern the CI/CD Catalog and component workflows. More information can be found in this epic.

Analytics

Our goal is to empower users with seamless control over component management across pipelines, ensuring optimal version control and project alignment. This addresses the challenge of users currently lacking visibility into component usage across various project pipelines. Our objective is to provide users with the capability to swiftly identify outdated versions and take prompt corrective actions as needed. This enhancement will foster an environment where users can efficiently manage and update components, promoting both version control precision and project alignment. Read more in this issue.

Get started with the CI/CD Catalog

The introduction of the CI/CD Catalog revolutionizes pipeline development by offering a vast array of pre-built components. Users don't have to start building pipelines from scratch because the CI/CD Catalog provides an access point to search components and pipeline configurations. The CI/CD Catalog's availability makes accessing and sharing components effortless, fostering collaboration and community growth. Whether utilizing public or private repositories, users can leverage these resources to enhance their pipeline development experience. Moreover, while GitLab.com users benefit from an open-access catalog, self-managed customers can establish organizational catalogs tailored to their needs.

Get to know the CI/CD Catalog with a free 30-day trial of GitLab Ultimate.

Learn more about the CI/CD Catalog and components:

• A CI/CD component builder's journey

• FAQ: GitLab CI/CD Catalog

• Documentation: CI/CD components and CI/CD Catalog

• Introducing CI/CD components and how to use them in GitLab