ESA Datalabs GraphQL API Reference
This documentation lists all the GraphQL endpoints (queries, mutations, subscriptions) that are available in the ESA Datalabs system, describes their input and output parameters and shows example queries. It also describes in detail all the data types, inputs used in the inputs and outputs.
Terms of Service
API Endpoints
# Production:
https://datalabs.esa.int/graphql
# Pre-production:
https://datalabs.datk8spre.ebe.lan/graphql
Pre-conditions for using the ESA Datalabs GraphQL API
To use the ESA Datalabs GraphQL API, the external system or query tool needs to be configured to send the requests to the URL https://datalabs.esa.int/graphql.
The user requires an ESA Cosmos CAS LDAP (https://cas.cosmos.esa.int/cas/login) account which has been granted access to the ESA Datalabs platform. For getting a user account and access to EDL platform, please submit a registration request here - https://www.cosmos.esa.int/web/datalabs/self-registration.
The user account needs to have the roles that have the permissions needed to use the specific GraphQL queries, mutations. A new regular user account that is created as a result of self-registration may not have all the necessary rights by default. For additional permissions needed for specific GraphQL API services, please contact EDL Support.
For the POST requests to be recognized and responded to, the request header needs to contain the element 'Authorization' with value 'Bearer <token, which can be found in the https://datalabs.esa.int cookie as the element 'session' value>'. To get the session token value, log into https://datalabs.esa.int with the user account and copy the session token value from the cookie in your browser local storage.
If the Query or Response example contains the a "Fragment" block with the following pattern - ...Fragment - then this block can contain attributes which are listed in the "Types" sections sub-chapter of the data type whose name is before the suffix 'Fragment'. For example ...RoleFragment attributes can be found in the Types chapter for Role etc.
There exists an interactive EDL GraphQL API browsing and query tool which is available at https://datalabs.esa.int/graphiql
Queries
bot
Description
Retrieve a single bot
- Permissions required: user-management/view(bot)
- UI contexts: not implemented in the UI
- Data returned: see listed field(s) of the response data type(s).
- Filter: none
- Input variables: id of the bot.
Response
Returns a Bot!
Arguments
| Name | Description |
|---|---|
id - ID!
|
Unique identifier of a bot. Example: 83bc47b09992c48d |
Example
Query
query Bot($id: ID!) {
bot(id: $id) {
id
name
key
}
}
Variables
{"id": "83bc47b09992c48d"}
Response
{
"data": {
"bot": {
"id": "83bc47b09992c48d",
"name": "testingBot",
"key": "f2061d02bff8a5985e1864b28b74a1bc588310fb3c4d2f82ac24a272f7d7f169"
}
}
}
bots
Description
List all bots
- Permissions required: user-management/view(bot)
- UI contexts: not implemented in UI
- Data returned: id, name, key. see listed field(s) of the response data type(s).
- Filter: none
- Input variables: no variables, just the list of attributes needed in the response (id, name, key).
Response
Returns [Bot!]!
Example
Query
query Bots {
bots {
id
name
key
}
}
Response
{
"data": {
"bots": [
{
"id": "83bc47b09992c48d",
"name": "testingBot",
"key": "f2061d02bff8a5985e1864b28b74a1bc588310fb3c4d2f82ac24a272f7d7f169"
}
]
}
}
checkDatalabSecret
Description
Check if secret corresponds to the specified datalab
Response
Returns a DatalabSecretCheck!
Example
Query
query CheckDatalabSecret(
$id: ID!,
$secret: String!
) {
checkDatalabSecret(
id: $id,
secret: $secret
) {
username
valid
}
}
Variables
{"id": "5fd91ad78d82ffd9", "secret": "c778916bdb923476"}
Response
{"data": {"checkDatalabSecret": {"username": "etestuser", "valid": true}}}
dataCentres
Description
Query is performed to populate and allow the selection between clusters where the user wants to launch datalabs or pipelines.
- Permissions required: user-management/view on target user
- UI contexts: 2.1. When user opens any EDL page. The result is populated into the cluster selection menu that can be opened from the top right menu bar.
- Data returned: see listed field(s) of the response data type(s).
- Filter: none
- Input variables: none
Response
Returns [DataCentre]!
Example
Query
query DataCentres {
dataCentres {
id
displayName
clusters {
id
displayName
displayLabel
default
dataCentreId
}
}
}
Response
{
"data": {
"dataCentres": [
{
"id": "esadc01",
"displayName": "ESA",
"clusters": [Cluster]
}
]
}
}
dataVolumes
Description
List all data volumes of the current user
- Requires permission: data-volumes/view on target data-volume
- UI contexts: 2.1. When user opens the landing page of EDL after successfully logging in.
- Data returned: see listed field(s) of the reponse data type(s).
- Filter: only data volumes accessible to the current user are returned.
- Input arguments: clusterID"
Response
Returns [DataVolume!]!
Arguments
| Name | Description |
|---|---|
clusterId - ID
|
Unique identifier of the cluster in which the data volumes of the current user need to be searched |
Example
Query
query DataVolumes($clusterId: ID) {
dataVolumes(clusterId: $clusterId) {
dvId
name
path
url
dataCentreIds
clusterIds
automount
}
}
Variables
{"clusterId": "c-m-m28vdqgf"}
Response
{
"data": {
"dataVolumes": [
{
"dvId": "a96cc3db403a7567",
"name": "JWST - Public Data",
"path": "jwst_jwst01_data",
"url": "nfs://server1.volumes.datalabs.esa.int/jwst_jwst01_data",
"dataCentreIds": ["*"],
"clusterIds": ["c-m-m28vdqgf"],
"automount": true
}
]
}
}
datalab
Description
Retrieve a single datalab
Example
Query
query Datalab($id: ID!) {
datalab(id: $id) {
id
dlp {
id
name
image
description
config
version
}
nodeUrl
containerId
name
creation_time
secret
categories
state
details
config
resources {
id
dlId
name
type
categories
specification
state
details
}
launchRef
clusterId
last_access_time
}
}
Variables
{"id": "0cc7edf6-736b-42f6-891f-f395cc7896bc"}
Response
{
"data": {
"datalab": {
"id": "0cc7edf6-736b-42f6-891f-f395cc7896bc",
"dlp": DatalabPackage,
"nodeUrl": "",
"containerId": "",
"name": "jl-pipeman",
"creation_time": "2025-01-14T12:15:30.062338",
"secret": "cb3517e95b5cbacf",
"categories": 1,
"state": "running",
"details": "",
"config": null,
"resources": [Resource],
"launchRef": null,
"clusterId": "c-m-m28vdqgf",
"last_access_time": "2025-01-14T16:18:22.071397"
}
}
}
datalabFromInterface
Description
Retrieve a single datalab from interface id
Example
Query
query DatalabFromInterface($dli_id: ID!) {
datalabFromInterface(dli_id: $dli_id) {
id
dlp {
id
name
image
description
config
version
}
nodeUrl
containerId
name
creation_time
secret
categories
state
details
config
resources {
id
dlId
name
type
categories
specification
state
details
}
launchRef
clusterId
last_access_time
}
}
Variables
{"dli_id": "39925f14ea51ac71"}
Response
{
"data": {
"datalabFromInterface": {
"id": "0cc7edf6-736b-42f6-891f-f395cc7896bc",
"dlp": DatalabPackage,
"nodeUrl": "",
"containerId": "",
"name": "jl-pipeman",
"creation_time": "2025-01-14T12:15:30.062338",
"secret": "cb3517e95b5cbacf",
"categories": 1,
"state": "running",
"details": "",
"config": null,
"resources": [Resource],
"launchRef": null,
"clusterId": "c-m-m28vdqgf",
"last_access_time": "2025-01-14T16:18:22.071397"
}
}
}
datalabPackages
Description
Search for the List of datalab packages from Helm repository setup in the cluster Used in DLD main screen If an id is provided it will search for the single package The clusterid is narrowing the search of package to the cluster You need to be logged in to access this query, no special roles, the package are filtered by mission so you may have some not showing up Inputs: id of the package for a single package retrieval qs is used to filter the list of packages by name, it will try to match the query string against the name of the package cluster_id is the id of the cluster Outputs: A list of Datalab packages, even for a single one, see the definition of the DatalabPackage for more details
Response
Returns [DatalabPackage!]!
Example
Query
query DatalabPackages(
$id: ID,
$qs: String,
$categories: Int,
$cluster_id: ID
) {
datalabPackages(
id: $id,
qs: $qs,
categories: $categories,
cluster_id: $cluster_id
) {
id
name
image
description
config
version
}
}
Variables
{"id": "jl-pipeman", "qs": "", "categories": 255, "cluster_id": "c-m-m28vdqgf"}
Response
{
"data": {
"datalabPackages": [
{
"id": "jl-pipeman",
"name": "jl-pipeman",
"image": "jl-pipeman",
"description": "Jupyter lab with pipeman python library.",
"config": null,
"version": "0.0.4"
}
]
}
}
datalabs
Description
List all instantiated datalabs.
- Requires permission: datalabs/view on target datalab
- UI contexts: 2.1. When user opens the landing page of EDL after successfully logging in.
- Data returned: see listed fields below.
- Filter: only datalabs in the authenticated user namespace are returned.
- Input arguments: See example query.
Response
Returns [Datalab!]!
Arguments
| Name | Description |
|---|---|
clusterId - ID
|
Example
Query
query Datalabs($clusterId: ID) {
datalabs(clusterId: $clusterId) {
id
dlp {
id
name
image
description
config
version
}
nodeUrl
containerId
name
creation_time
secret
categories
state
details
config
resources {
id
dlId
name
type
categories
specification
state
details
}
launchRef
clusterId
last_access_time
}
}
Variables
{"clusterId": "c-m-m28vdqgf"}
Response
{
"data": {
"datalabs": [
{
"id": "0cc7edf6-736b-42f6-891f-f395cc7896bc",
"dlp": DatalabPackage,
"nodeUrl": "",
"containerId": "",
"name": "jl-pipeman",
"creation_time": "2025-01-14T12:15:30.062338",
"secret": "cb3517e95b5cbacf",
"categories": 1,
"state": "running",
"details": "",
"config": null,
"resources": [Resource],
"launchRef": null,
"clusterId": "c-m-m28vdqgf",
"last_access_time": "2025-01-14T16:18:22.071397"
}
]
}
}
dockerImages
Description
List available docker images for pipeline steps
- Permissions required: pipeline-services/launch on target system-pipeline
- UI contexts: 2.1. When user opens the 'Docker image selection' modal window by clicking the 'Browse' button on 'Pipeline Editor' page 'Pipeline step' tab.
- Data returned: see listed field(s) of the response data type(s).
- Filter: list of docker images available in the currently selected cluster
- Input variables: none
Response
Returns [DockerImage]!
Example
Query
query DockerImages {
dockerImages {
name
versions
}
}
Response
{"data": {"dockerImages": [{"name": "step/example_base", "versions": ["v1"]}]}}
dockerRegistry
Description
Get docker registry where runner can pull images
- Permissions required: pipeline-services/launch on target system-pipeline
- UI contexts: 2.1. not used in UI.
- Data returned: docker registry name or URL
- Filter: none
- Input variables: none
Response
Returns a String!
Example
Query
query DockerRegistry {
dockerRegistry
}
Response
{"data": {"dockerRegistry": "some-value"}}
executionEngines
Description
List all deployed pipeline execution backends
- Permissions required: pipeline-services/read on target execution-engine
- UI contexts: 2.1. When user opens the Pipelines dashboard page.
- Data returned: see listed field(s) of the response data type(s).
- Filter: none
- Input variables: none.
Response
Returns [ExecutionEngine]!
Example
Query
query ExecutionEngines {
executionEngines {
name
displayName
status
}
}
Response
{
"data": {
"executionEngines": [
{"name": "CALRISSIAN", "displayName": "Calrissian", "status": "UP"}
]
}
}
extractDatalabMetadata
Description
Retrieve the metadata from the Git repository and return them to the UI It relies on a tekton pipeline to clone and read the file. Used in datalab creation flow Inputs: datalab id from which we need to extract the metadata (git repos is updated before) Outputs: a list of metadata, check MetadataEntry definition for more
Response
Returns [MetadataEntry!]!
Arguments
| Name | Description |
|---|---|
datalab - ID!
|
Example
Query
query ExtractDatalabMetadata($datalab: ID!) {
extractDatalabMetadata(datalab: $datalab) {
name
value
definition {
description
type
visibility
owner
required
sensitive
recommended
pattern
errorMessage
}
}
}
Variables
{"datalab": "cdce7d27-fc5e-44da-b326-875624b6cbac"}
Response
{
"data": {
"extractDatalabMetadata": [
{
"name": "version",
"value": "1.0.0",
"definition": MetadataEntryDefinition
}
]
}
}
getAuthenticationState
Description
Return the available backends and the current user, if one has authenticated. Representation of how the server currently recognized the client, and possible methods for authentication
- Access roles: none required
- UI contexts: When user opens the EDL login page or any other page or refreshes a page after being authenticated.
- Data returned: see listed field(s) of the reponse data type(s).
- Filter: none
- Input arguments: none
Response
Returns an AuthenticationState!
Example
Query
query GetAuthenticationState {
getAuthenticationState {
backends
username
token
timestamp
user {
username
displayName
roles {
...RoleFragment
}
}
}
}
Response
{
"data": {
"getAuthenticationState": {
"backends": "CAS",
"username": "etestuser",
"token": "eyJ1c2VyX2lkIjogInVzZXIifQ.ZkYBTw.GRLKzLezKcpdH7MJueKnUIdBl8M",
"timestamp": 1715616642,
"user": User
}
}
}
getBookmarks
Description
Retrieve the list of datalab the user has bookmarked Used in main screen You need to be logged in to perform this operation Outputs: a list of bookmark, check BuildBookmarks definition for more
Response
Returns [BuildBookmarks!]!
Example
Query
query GetBookmarks {
getBookmarks {
datalab
}
}
Response
{
"data": {
"getBookmarks": [{"datalab": "cdce7d27-fc5e-44da-b326-875624b6cbac"}]
}
}
getBuilds
Description
Retrieve all the builds regardless of the datalab Used in main screen and sharing screen of the datalab detailed view If an id is provided it will return only this build You need to be logged in to perform this operation, the build are filtered based on you role Inputs: the build id (optional) qs is used to filter based on the datalab name and description Outputs: a list of build, check DatalabBuild definition for more
Response
Returns [DatalabBuild!]!
Example
Query
query GetBuilds(
$id: ID,
$qs: String
) {
getBuilds(
id: $id,
qs: $qs
) {
id
datalabId
version
status
runtimeConfiguration {
cpuAllocation
memAllocation
}
created_at
buildRef
scanRef
logs
progress
}
}
Variables
{"id": "3d971341-136b-43e6-8298-bd8913edf77b", "qs": ""}
Response
{
"data": {
"getBuilds": [
{
"id": "3d971341-136b-43e6-8298-bd8913edf77b",
"datalabId": "b27cda74-2db7-46ec-b287-a6353a47e74a",
"version": "0.0.1-0",
"status": "PUBLISHED",
"runtimeConfiguration": RuntimeConfiguration,
"created_at": "1715696521.537044",
"buildRef": "sciapps-build-datalab-4ltcw",
"scanRef": null,
"logs": "Build started (sciapps-build-datalab-4ltcw)#!1715696521.539653!#Build ended with status BUILD_SUCCESS#!1715696617.206669!#Build marked as UNDER_MODERATION#!1715697174.958686!#Build marked as PUBLISHED#!1715697826.405907!#Approved#!1715697826.407465!#Datalab build published in the public catalog#!1715697848.135899!#Build archived#!1715773558.998771!#Build unarchived#!1715783367.895708!#Datalab build published in the public catalog#!1715783388.069154!#",
"progress": "BUILD_SUCCESS"
}
]
}
}
getDatalab
Description
Returns a single datalab by ID Used in DLD main screen You need to be logged and being the owner of the datalab (Operator can see them) Inputs: datalab_id is the id of the datalab in the dalabclass terms Outputs: a single of datalab class
Response
Returns a DatalabClass
Arguments
| Name | Description |
|---|---|
datalab_id - ID!
|
Example
Query
query GetDatalab($datalab_id: ID!) {
getDatalab(datalab_id: $datalab_id) {
id
owner {
username
displayName
roles {
...RoleFragment
}
}
is_public
created_at
modified_at
repository
branch
artefact
status
name
description
icon
runtimeConfiguration {
cpuAllocation
memAllocation
}
metadata {
name
value
definition {
...MetadataEntryDefinitionFragment
}
}
nstars
version
}
}
Variables
{"datalab_id": "0cc7edf6-736b-42f6-891f-f395cc7896bc"}
Response
{
"data": {
"getDatalab": {
"id": "deaef60e-bde6-4ca8-b284-79b03db417b2",
"owner": User,
"is_public": false,
"created_at": "1715163378.021328",
"modified_at": "1715163471.601883",
"repository": null,
"branch": null,
"artefact": null,
"status": "DRAFT",
"name": "testing-lab",
"description": "A test",
"icon": null,
"runtimeConfiguration": RuntimeConfiguration,
"metadata": [MetadataEntry],
"nstars": 0,
"version": "0.0.1"
}
}
}
getDatalabAccessList
Description
Returns the list of users that has rights to access this datalab even if not public yet The list of users with whom the datalab was shared Used in share screen of datalab details view You need to be developer to perform this operation Inputs: datalab_id the datalab for which we need the list of users having access Outputs: a list of user, check User definition (in directory.graphql) for more information
Example
Query
query GetDatalabAccessList($datalab_id: ID!) {
getDatalabAccessList(datalab_id: $datalab_id) {
username
displayName
roles {
id
description
target
permissions {
...PermissionFragment
}
cosmosLdapGroup
}
}
}
Variables
{"datalab_id": "0cc7edf6-736b-42f6-891f-f395cc7896bc"}
Response
{
"data": {
"getDatalabAccessList": [
{
"username": "etestuser",
"displayName": "EDL Test User",
"roles": [Role]
}
]
}
}
getDatalabBuilds
Description
Retrieve the list of build for a given datalab Used in main screen and testing screen of the datalab detailed view You need to be logged in to perform this operation, the build are also needed for the runtime in order to select the version to start. Inputs: datalab_id the datalab for which we need the builds Outputs: a list of build, check DatalabBuild definition for more information
Response
Returns [DatalabBuild!]!
Arguments
| Name | Description |
|---|---|
datalab_id - ID!
|
Example
Query
query GetDatalabBuilds($datalab_id: ID!) {
getDatalabBuilds(datalab_id: $datalab_id) {
id
datalabId
version
status
runtimeConfiguration {
cpuAllocation
memAllocation
}
created_at
buildRef
scanRef
logs
progress
}
}
Variables
{"datalab_id": "0cc7edf6-736b-42f6-891f-f395cc7896bc"}
Response
{
"data": {
"getDatalabBuilds": [
{
"id": "3d971341-136b-43e6-8298-bd8913edf77b",
"datalabId": "b27cda74-2db7-46ec-b287-a6353a47e74a",
"version": "0.0.1-0",
"status": "PUBLISHED",
"runtimeConfiguration": RuntimeConfiguration,
"created_at": "1715696521.537044",
"buildRef": "sciapps-build-datalab-4ltcw",
"scanRef": null,
"logs": "Build started (sciapps-build-datalab-4ltcw)#!1715696521.539653!#Build ended with status BUILD_SUCCESS#!1715696617.206669!#Build marked as UNDER_MODERATION#!1715697174.958686!#Build marked as PUBLISHED#!1715697826.405907!#Approved#!1715697826.407465!#Datalab build published in the public catalog#!1715697848.135899!#Build archived#!1715773558.998771!#Build unarchived#!1715783367.895708!#Datalab build published in the public catalog#!1715783388.069154!#",
"progress": "BUILD_SUCCESS"
}
]
}
}
getDatalabComments
Description
Retrieve the comment for a databab Used in main screen You need to be logged in and the datalab to be public Inputs: a datalab id Outputs: a list of comment, check UserComment definition
Response
Returns [UserComment!]!
Arguments
| Name | Description |
|---|---|
datalab - ID!
|
Example
Query
query GetDatalabComments($datalab: ID!) {
getDatalabComments(datalab: $datalab) {
datalab_id
build_version
modified_at
comment
user {
username
displayName
roles {
...RoleFragment
}
}
}
}
Variables
{"datalab": "cdce7d27-fc5e-44da-b326-875624b6cbac"}
Response
{
"data": {
"getDatalabComments": [
{
"datalab_id": "0cc7edf6-736b-42f6-891f-f395cc7896bc",
"build_version": "1.0.2",
"modified_at": "1715697428.874147",
"comment": "Great!",
"user": User
}
]
}
}
getDatalabs
Description
List the available datalabs in DLD terms, this list is relying on datalabs the user owns or that are publicly available, shared with him. Moderator will also get the list of undermoderation datalab It contains a list of datalab that could be under development. Used in DLD main screen You need to be logged in to see the list of datalabs Inputs: qs is a string to that must match the name/description of the datalab, used to perform search cluster_id is a string to target a specific cluster Outputs: a list of datala, see the DatalabClass definition for more information
Response
Returns [DatalabClass!]!
Example
Query
query GetDatalabs(
$qs: String,
$cluster_id: ID
) {
getDatalabs(
qs: $qs,
cluster_id: $cluster_id
) {
id
owner {
username
displayName
roles {
...RoleFragment
}
}
is_public
created_at
modified_at
repository
branch
artefact
status
name
description
icon
runtimeConfiguration {
cpuAllocation
memAllocation
}
metadata {
name
value
definition {
...MetadataEntryDefinitionFragment
}
}
nstars
version
}
}
Variables
{"qs": "", "cluster_id": "c-m-m28vdqgf"}
Response
{
"data": {
"getDatalabs": [
{
"id": "deaef60e-bde6-4ca8-b284-79b03db417b2",
"owner": User,
"is_public": false,
"created_at": "1715163378.021328",
"modified_at": "1715163471.601883",
"repository": null,
"branch": null,
"artefact": null,
"status": "DRAFT",
"name": "testing-lab",
"description": "A test",
"icon": null,
"runtimeConfiguration": RuntimeConfiguration,
"metadata": [MetadataEntry],
"nstars": 0,
"version": "0.0.1"
}
]
}