Frequently asked questions (FAQ)

How can I contact the DBnomics core team?

I like DBnomics, how can I contribute?

Well, glad you like it :)

How do you guarantee that data on DBnomics is correct?

In short, there is no formal guarantee, but...

Fetcher developers use a validation script in order to detect some errors automatically.

Of course, a program cannot verify the correctness of floating-point values. To do that, manual checks must be performed against the original source data on the provider website. The DBnomics core team includes economists who manually verify data produced by a fetcher once developers have finished writing it.

Finally, like many free software projects, DBnomics quality improves each time users contribute, and reporting problems with data is very helpful.

I've seen a problem with data on DBnomics, what should I do?

Please read: report problems with data.

When is data updated?

Updating data as soon as the provider publishes it is a major concern. Here are more explanations about how it works.

DBnomics data, distributed by its web API, is updated daily (most of the time) by fetchers.

There can be two types of delays:

the delay between the publication of new data on the provider side and the next execution of the corresponding fetcher,
and the delay due to a failure of the fetcher, requiring manual intervention.

The delay can be reduced when providers publish a log of data updates (e.g. IMF, INSEE, Eurostat) that is small enough to be checked more often (e.g. hourly). In this case, the download is triggered only when needed, and applies only to data that has been updated since the last fetcher execution. We call this the incremental mode.

Where do I find the source code of a fetcher?

Look at that GitLab group: https://git.nomics.world/dbnomics-fetchers/.

Can I have private data on DBnomics?

DBnomics is designed to aggregate and redistribute publicly available economic data from external sources — it is not a storage service for private datasets. Data that a researcher keeps private, i.e. not published anywhere publicly, cannot be integrated into DBnomics.

However, if a researcher publishes their data in a publicly accessible location (e.g. a Git repository, an institutional website), DBnomics can treat them as a data provider and fetch the data from there. Their institution can serve as the provider code. In other words, DBnomics is not meant to be the primary storage for the data: it retrieves data from wherever it is already published.

In addition, DBnomics will not redistribute paid data or data that is not licensed for redistribution: doing so would violate the provider's terms of use. This is a legal constraint, not a technical one.

Please contact us if you have questions about a specific data source — we'd be happy to discuss your needs.

Why do fetchers break?

If you look at the dashboard, you will see that many fetchers are in error status with red icons.

Fetchers download and convert data from providers, but the provider website, web API, or data can evolve in an unpredictable way.

Fetcher stability is correlated with provider stability. This includes stable infrastructure, stable API URLs, stable responses, stable data models, etc. When a provider changes something, the fetcher breaks and needs to be fixed by its developers.

Which data sources are easier to process?

Data distributed in manually formatted Excel files is more difficult to process by fetchers, mainly because data is not produced systematically.

For example, having cells with a color background to add semantics to data is quite annoying to process, as well as handling merged cells, or data tables shifted below a header, or having a footer with notes.

On the other hand, structured data files with a stable schema, such as XML, JSON, or CSV, are easier to work with because there are almost no special cases to handle.

Another aspect is the transport layer: it is easier to download static files and process them locally than to follow the pagination of a web API or scrape a website.