GitLab
vs
Codefresh
vs
Decision Kit
Decision Kit
Summary
Codefresh is a CI/CD tool designed for containers and Kubernetes. Codefresh features a GitLab integration that allows you to use GitLab for version control and run Codefresh pipelines on your code. Codefresh has some features that make it more mature than GitLab for running pipelines across multiple projects. But it lacks the benefits of a single application.
Codefresh charges for builds per month, as well as concurrent builds. GitLab has no such limitations with the ability to elastically scale Runners to handle as many concurrent builds as needed on demand and then scale down so you aren’t paying to keep inactive Runners up.
Codefresh only offers a self-managed option for Enterprise pricing. Free, Basic, and Pro tiers are for SaaS-only. GitLab offers self-managed and SaaS options at every price point.
Resources
Comments/Anecdotes
- Codefresh makes some claims in their blog comparing themselves to GitLab that are not really accurate.
- “GitlabCI isn’t designed for micro-services since everything is tied to a single project”
- Although we can improve our microservices support, (and plan to) this claim is not true.
- Today, GitLab has multi project pipelines and can trigger pipelines for mulit-projects via API. In fact, The CI working group for CNCF chose GitLab to run their multi-project multi-cloud pipelines. Read more in the CNCF case study.
- “GitlabCI isn’t designed for micro-services since everything is tied to a single project”
- We are missing some features identified in this post that would bring us on par with Codefresh
- The ability to define multiple pipelines
- Support for monorepos with the ability to run pipelines only on specific directories - UPDATE: Shipped in Gitlab 11.4
- Group level Docker registry browser
- Group level Kubernetes clusters - UPDATE: Shipped in GitLab 11.6
- Make container building first class
- Integrated Helm repository
Additional CodeFresh Capabilities
Global Helm repo configuration
Zero configuration caching/artifacts using docker volumes
Distributed Docker layer caching
Feature Comparison
| FEATURES |
|
|
|---|---|---|
|
Built-in Container Registry
GitLab Container Registry is a secure and private registry for Docker images. It allows for easy upload and download of images from GitLab CI. It is fully integrated with Git repository management. (Codefresh will be ending their support for private docker registries as of May 1, 2020 |
|
|
|
Preview your changes with Review Apps
With GitLab CI/CD you can create a new environment for each one of your branches, speeding up your development process. Spin up dynamic environments for your merge requests with the ability to preview your branch in a live environment. Review Apps support both static and dynamic URLs. |
|
|
|
CI/CD Horizontal Autoscaling
GitLab CI/CD cloud native architecture can easily scale horizontally by adding new nodes if the workload increases. GitLab Runners can automatically spin up and down new containers to ensure pipelines are processed immediately and minimize costs. |
|
|
|
CI/CD Pipelines Dashboard
Visualize the history and current status of pipelines across projects and groups all in a single dashboard that can be customized for each user. Learn more about Cross-Project Pipelines in the Operations Dashboard |
|
|
|
Group-level Docker registry browser
A single UI view into images across multiple repositories. |
|
|
|
Helm chart repository support
Supports storage and retrieval of Helm charts. |
|
|
|
Comprehensive pipeline graphs
Pipelines can be complex structures with many sequential and parallel jobs. To make it a little easier to see what is going on, you can view a graph of a single pipeline and its status. |
|
|
|
Scheduled triggering of pipelines
You can make your pipelines run on a schedule in a cron-like environment. |
|
|
|
Run CI/CD jobs on Windows
GitLab Runner supports Windows and can run jobs natively on this platform. You can automatically build, test, and deploy Windows-based projects by leveraging PowerShell or batch files. |
|
|
|
Run CI/CD jobs on macOS
GitLab Runner supports macOS and can run jobs natively on this platform. You can automatically build, test, and deploy for macOS based projects by leveraging shell scripts and command line tools. |
|
|
|
Run CI/CD jobs on Linux ARM
GitLab Runner supports Linux operating systems on ARM architectures and can run jobs natively on this platform. You can automatically build, test, and deploy for Linux ARM based projects by leveraging shell scripts and command line tools. |
|
|
|
Run CI/CD jobs on FreeBSD
GitLab Runner supports FreeBSD and can run jobs natively on this platform. You can automatically build, test, and deploy for FreeBSD-based projects by leveraging shell scripts and command line tools. |
|
|
|
Show code coverage rate for your pipelines
GitLab is able to parse job output logs and search, via a customizable regex, any information created by tools like SimpleCov to get code coverage. Data is automatically available in the UI and also as a badge you can embed in any HTML page or publish using GitLab Pages. Learn how to generate and show code coverage information in GitLab |
|
|
|
Details on duration for each command execution in GitLab CI/CD
Other CI systems show execution time for each single command run in CI jobs, not just the overall time. We’re reconsidering how job output logs are managed in order to add this feature as well. |
|
|
|
Auto DevOps
Auto DevOps brings DevOps best practices to your project by automatically configuring software development lifecycles by default. It automatically detects, builds, tests, deploys, and monitors applications. |
|
|
|
Protected Runners
Protected Runners allow you to protect your sensitive information, for example deployment credentials, by allowing only jobs running on protected branches to access them. |
|
|
|
Easy integration of existing Kubernetes clusters
Add your existing Kubernetes cluster to your project, and easily access it from your CI/CD pipelines to host Review Apps and to deploy your application. |
|
|
|
GitLab Kubernetes Agent
Manage the deployments and connection to your Kubernetes clusters in a secure and compliant way, driven by code. |
|
|
|
Support for multiple Kubernetes clusters
Easily deploy different environments, like Staging and Production, to different Kubernetes clusters. This allows to enforce strict data separation. |
|
|
|
Integrated cluster cost management
Get a peek at the cost of running your applications in Kubernetes to help with your budgeting requirements. Read through the documentation on setting up cluster cost management |
|
|
|
Automatic Retry for Failed CI Jobs
You can specify a retry keyword in your .gitlab-ci.yml file to make GitLab CI/CD retry a job for a specific number of times before marking it as failed. |
|
|
|
Pipelines security
The ability of running CI/CD pipelines on protected branches is checked against a set of security rules that defines if you’re allowed or not. It includes creating new pipelines, retrying jobs, and perform manual actions. |
|
|
|
Include external files in CI/CD pipeline definition
You can include external files in your pipeline definition file, using them as templates to reuse snippets for common jobs. |
|
|
|
Static Application Security Testing
GitLab allows easily running Static Application Security Testing (SAST) in CI/CD pipelines; checking for vulnerable source code or well known security bugs in the libraries that are included by the application. Results are then shown in the Merge Request and in the Pipeline view. This feature is available as part of Auto DevOps to provide security-by-default. |
|
|
|
Secret Detection
GitLab allows you to perform Secret Detection in CI/CD pipelines; checking for unintentionally committed secrets and credentials. Results are then shown in the Merge Request and in the Pipeline view. This feature is available as part of Auto DevOps to provide security-by-default. |
|
|
|
Dependency Scanning
GitLab automatically detects well known security bugs in the libraries that are included by the application, protecting your application from vulnerabilities that affect dependencies that are used dynamically. Results are then shown in the Merge Request and in the Pipeline view. This feature is available as part of Auto DevOps to provide security-by-default. |
|
|
|
Dynamic Application Security Testing
Once your application is online, GitLab allows running Dynamic Application Security Testing (DAST) in CI/CD pipelines; your application will be scanned to ensure threats like XSS or broken authentication flaws are not affecting it. Results are then shown in the Merge Request and in the Pipeline view. This feature is available as part of Auto DevOps to provide security-by-default. |
|
|
|
Interactive Application Security Testing
IAST combines elements of static and dynamic application security testing methods to improve the overall quality of the results. IAST typically uses an agent to instrument the application to monitor library calls and more. GitLab does not yet offer this feature. |
|
|
|
Container Scanning
When building a Docker image for your application, GitLab can run a security scan to ensure it does not have any known vulnerability in the environment where your code is shipped. Results are then shown in the Merge Request and in the Pipeline view. This feature is available as part of Auto DevOps to provide security-by-default. |
|
|
|
Browser Performance Testing
Easily detect performance regressions for web apps and pages prior to merging into master. Browser Performance Testing is included in Auto DevOps, providing automatic performance analytics of the root page with zero configuration. |
|
|
|
Load Performance Testing
Easily detect performance regressions for APIs prior to merging into master. |
|
|
|
Automated Accessibility scanning of Review Apps
Performing accessibility testing is important in order to ensure you’re serving all the users who use your products. In GitLab you can generate Accessibility reports automatically prior to merging into master. |
|
|
|
Virtual registries
A virtual registry is a collection of local, remote and other virtual registries accessed through a single logical URL. GitLab Epic detailing the issues required to add this functionality. |
|
|
|
Forward requests for packages not found in GitLab to npmjs.com
By default, when an NPM package is not found in the GitLab NPM Registry, the request is forwarded to npmjs.com |
|
|
|
Conan (C/C++) Repository
Conan is an open source, decentralized and multi-platform C/C++ Package Manager for developers to create and share native binaries. |
|
|
|
Maven (Java) Repository
GitLab’s Maven repository makes it easier to publish and share Java libraries across an organization, and ensure dependencies are managed correctly. It is fully integrated with GitLab, including authentication and authorization. |
|
|
|
NPM (node) Registry
GitLab’s NPM repository makes it easier to publish and share NPM packages across an organization, and ensure dependencies are managed correctly. It is fully integrated with GitLab, including authentication and authorization. |
|
|
|
NuGet (.NET) Repository
GitLab’s NuGet Repository allows C#/.NET developers to create, publish and share packages using the NuGet client or visual studio. |
|
|
|
PyPI (Python) Repository
Python developers can set up GitLab as a remote PyPI repository and build, publish, and share packages using the PyPI client or GitLab CI/CD. |
|
|
|
RPM (Linux) Repository
This planned feature will enable Linux developers to build, publish and share RPM packages alongside their source code and pipelines. Check out the issue for additional details on implementation and timing |
|
|
|
Debian (Linux) Repository
This planned feature will enable Linux developers to build, publish and share Debian packages alongside their source code and pipelines. Check out the issue for additional details on implementation and timing |
|
|
|
RubyGems (Ruby) Repository
This planned feature will enable Ruby developers to setup GitLab as a remote RubyGems repository and to build, publish and share packages using the command line or GitLab CI/CD. This will also be a valuable feature for GitLab and help with dogfooding Check out the issue for additional details on implementation and timing |
|
|
|
Go Proxy
This feature helps Go developers to publish and share their packages right alongside their source code and pipelines. This will also be a valuable feature for GitLab and help with dogfooding |
|
|
|
Composer (PHP) Repository
This feature helps PHP developers to build, publish and share their packages right alongside their source code and pipelines. |
|
|
|
Use the Package Registry through REST API
Enables support for automation and integration of the GitLab Package Registry through a REST API. |
|
|
|
Explicit support for monorepos
The ability to execute jobs only/except when there are changes for a given path or file support monorepos where many microservices are contained in a single repo. |
|
|
|
Global Docker registry browser
A single UI view into images across multiple repositories. |
|
|
|
Instance-level Kubernetes cluster configuration
A single UI view into Kubernetes cluster configuration for a GitLab instance. |
|
|
|
First class container building
The ability to specify that a container should be built during a CI/CD job without needing to specify the implementation details. |
|
|
|
Group-level kubernetes clusters
Create group-level Kubernetes cluster that can be used for all projects contained within the group or sub-groups. |
|
|
|
Instance-level kubernetes clusters
Create instance-level Kubernetes clusters that can be used for all groups and projects contained within the same GitLab instance. |
|
|
|
On-demand Dynamic Application Security Testing
There’s no reason to wait for the next CI pipeline run to find out if your site if vulnerable or to reproduce a previously found vulnerability. GitLab offers scanning your running application with On-demand Dynamic Application Security Testing (DAST), independent of code changes or merge requests. |
|
|
|
Site and Scanner profiles for On-demand DAST scans
Reuse configuration profiles quickly with on-demand DAST scans, instead of reconfiguring scans every time you need to run one. Mix different scan profiles with site profiles to quickly conduct scans that cover different areas or depths of your application and API. |
|
|
|
DAST Configuration UI
Enabling DAST is now as simple as three clicks. This guided configuration experience makes it easier for non-CI experts to get started with GitLab DAST. The tool helps a user create a merge request to enable DAST scanning while leveraging best configuration practices like using the GitLab-managed |
|
|
Git is a trademark of Software Freedom Conservancy and our use of 'GitLab' is under license
