GitLab CI monorepo setup

GitLab CI/CD is a powerful continuous integration/delivery tool. It currently offers 400 free pipeline minutes per month on public and private repositories.

Recently, I've split one of my projects into multiple modules (client, server, docs) using the monorepo architecture. In this article, I'll explain how to adjust the GitLab CI configuration to support such workflows.

GitLab CI configuration

Let's start with a basic .gitlab-ci.yml configuration file. I am using a JavaScript project with yarn, but the principle can be applied to any other use case:

image: node:10
stages:
  - test
before_script:
  - yarn install --pure-lockfile --prefer-offline --cache-folder .yarn
test:
  stage: test
  script:
    - yarn lint
    - yarn test
    - yarn build
cache:
  paths:
    - node_modules/
    - .yarn

The CI pipeline consists of a single test stage that runs a linter, checks unit tests and makes sure a build doesn't fail. You might consider splitting those tasks into multiple jobs and/or stages. However, while lint, test and build can be run in parallel, the CI setup time may lead to an overall slower pipeline execution.

The node_modules and the yarn cache are stored between jobs. Notice that I'm passing the yarn cache folder directly to the install command (instead of using yarn config set cache-folder .yarn) because of an open yarn defect.

Multirepo configuration

A basic multirepo has the following structure:

├─ 📁 client
|  ├─ 📁 src
|  └─ 📄 package.json
├─ 📁 server
|  ├─ 📁 src
|  └─ 📄 package.json
└─ 📄 .gitlab-ci.yml

We still have only a single .gitlab-ci.yml file in the project root, but all sources are split into multiple directories.

To run the CI for both modules, we'll create two independent jobs:

image: node:10

stages:
  - test

client:
  stage: test
  before_script:
    - cd client
    - yarn install --pure-lockfile --prefer-offline --cache-folder .yarn
  script:
    - yarn test
    - yarn build
  cache:
    key: client
    paths:
      - client/.yarn
      - client/node_modules/

server:
  stage: test
  before_script:
    - cd server
    - yarn install --pure-lockfile --prefer-offline --cache-folder .yarn
  script:
    - yarn test
    - yarn build
  cache:
    key: server
    paths:
      - server/.yarn
      - server/node_modules/

Some things to consider:

• All paths are relative to the project root, not the subfolder.
• Before every job we have to cd into the corresponding subdirectory.
• Every job defines a custom cache by using different key values.

Improving maintainability

The config file works fine, but there's room for improvement:

1. Remove code duplicates with extend and variables.
2. Split config into multiple files with include.
3. Run every module with different node versions to ensure compatibility. I'm using dedicated jobs per node version, but as of GitLab 13.3, a job matrix is supported natively.

• .gitlab-ci.yml:

stages:
  - test

.job:
  image: node:10
  stage: test
  before_script:
    - cd $DIR
    - yarn install --pure-lockfile --prefer-offline --cache-folder .yarn
  script:
    - yarn test
    - yarn build
  cache:
    key: $DIR
    paths:
      - $DIR/.yarn
      - $DIR/node_modules/

include:
  - local: "/client/.gitlab-ci.yml"
  - local: "/server/.gitlab-ci.yml"

• client/.gitlab-ci.yml:

client-node-10:
  image: node:10
  variables:
    DIR: client
  extends: .job

client-node-12:
  image: node:12
  variables:
    DIR: client
  extends: .job

• server/.gitlab-ci.yml:

server-node-10:
  image: node:10
  variables:
    DIR: server
  extends: .job

server-node-12:
  image: node:12
  variables:
    DIR: server
  extends: .job

Keep in mind that the configuration is always relative to project root, even if it's defined in a subfolder.

Conclusion

After running the pipeline, we'll see 4 different jobs:

In the end, there's nothing special about a monorepo when using GitLab CI, except for executing all commands from subdirectories instead of the project root.

For the full code and demo, see this GitLab repository.