Skip to content

Operations

This page documents how to operate the DBnomics platform. Its target audience is the maintainers of DBnomics. Those operations require specific administrator permissions.

Fetcher jobs

Fetcher jobs run in GitLab CI pipelines using GitLab Runner with the Docker or Kubernetes executor.

Run a pipeline for a fetcher

From the GitLab project of the fetcher, on the "pipelines" page (e.g. AMECO), click on "Run Pipeline". Note: this page can also be reached from DBnomics dashboard.

The pipeline accepts some environment variables documented here.

Change the timeout of jobs

This can be done from the CI/CD settings page of the GitLab project of the fetcher: under the "General pipelines" section, there is a "Timeout" field (e.g. AQICN)

See also: GitLab documentation

Deploy a fetcher to production

The source of truth for fetchers is fetchers.yml (management project). It is used to generate the dashboard for example.

To deploy a new fetcher to production:

  • move its source code repository to the dbnomics-fetchers GitLab group
  • ensure the repo name is {provider_code}-fetcher (so the clone url should be https://git.nomics.world/dbnomics-fetchers/{provider_code}-fetcher.git)
  • create an entry for it in fetchers.yml (management project) and push this change
  • this will trigger a CI job (accessible from here) that will (among other things):
  • configure the fetcher to run on a daily basis (using dbnomics-fetcher-ops)
  • create a commit in fetcher's repo that will add .gitlab-ci.yml file

Now it is possible to trigger a new pipeline to run this fetcher. Please read the above task for more information.