Integrating data

Note: the following examples are based on the API Stream integration concept - if you are not yet familiar with it check an overview of the Query types to understand the approach.

A simple example of integrating categories into the system from scratch:

Fetch the first page of the Category stream

{
  categoryStream(first: 1) {
    pageInfo {
      endCursor
      hasNextPage
    }
    edges {
      node {
        code
        name {
          value
          language
        }
      }
      cursor
    }
  }
}

first parameter put on limit how many categories are fetched in one result

example response:

{
  "data": {
    "categoryStream": {
      "pageInfo": {
        "endCursor": "YXJyYXljb25uZWN0aW9uOjQ5",
        "hasNextPage": true
      },
      "edges": [
        {
          "node": {
            "code": "category_name_clothing",
            "name": [
              {
                "value": "Clothing",
                "language": "en_GB"
              },
              {
                "value": "Odzież",
                "language": "pl_PL"
              }
            ]
          },
          "cursor": "YXJyYXljb25uZWN0aW9uOjQ5"
        }
      ]
    }
  }
}

Fetch the next page of the Category stream

Since we received information that the next page exists data.categoryStream.pageInfo.hasNextPage=true we should request the next resource. The query is very similar to the previous one except we are passing the appropriate cursor with the request:

{
  categoryStream(first: 1, after: "YXJyYXljb25uZWN0aW9uOjQ5") {
    ...
}

Afterthe parameter represents the appropriate cursor that identifies a last-fetched resource. Available as either cursor field of the node or the endCursor of PageInfo object.

example response:

{
  "data": {
    "categoryStream": {
      "pageInfo": {
        "endCursor": "YXJyYXljb25uZWN0aW9uOjUw",
        "hasNextPage": false
      }
      ...
    }
  }
}

we do know that there is no next page, therefore, nothing to fetch at the very moment. We can retry the next request(for the retrieved cursor) with i.e. increased interval not to waste resources.

A more complex example of integrating categories

Alternatively in some systems, we will want to only fetch the identifier of the resource using some sort of queueing system in order to distribute the consumption processes. In that case, we can just fetch our resource id(code in the case of categories) and the PageInfo object:

{
  categoryStream(first: 1) {
    pageInfo {
      endCursor
      hasNextPage
    }
    edges {
      node {
        code
      }
    }
  }
}

and fetch it in the separate consuming process via a single resource query:

{
  category(code: "category_name_clothing") {
    name {
      value
      language
    }
    code
  }
}

The rest of the process looks the same for the paginating over the stream.

Though given above content provides an example of handling categories you can use the same approach for every other resource available via streams like products, multimedia, and attributes.