Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Description of concepts behind authentication system and how to prepare your own security.
Requests between Ergonode and App are signed using JWT tokens.
These tokens allow the safe exchange of custom data(claims) between two parties.
The token contains claims describing the context of the execution - what specific installation the token is referencing etc.
The token is always available in X-APP-TOKEN HTTP header with a signature signed using HMAC SHA-256.
Note you should not be forced to implement JWT authentication on your own.
There are multiple well-developed and acknowledged open-source solutions to handle the issue.
To authenticate the token firstly, on App installation, a handshake is exchanged.
Handshake is a request to [POST] /handshake path of your App.
It contains two important pieces of information:
X-APP-TOKEN HTTP header
shared secret in the request body
This secret should be persisted, i.e. the database of choice, and kept by your App safe with information about the app installation it belongs to and the Ergonode API URL retrieved from claims:
app_installation_id
api_url
All the following requests by Ergonode to an App should be authenticated using this secret.
You should also encrypt the shared secret within the persistent storage so it is not easily retrievable.
The response status has to be 2xx to process installation appropriately.
Steps to verify whether the request is coming for a specific App installation from Ergonode
obtain the token from X-APP-TOKEN HTTP header of the request
extract the app_installation_id claim from the JWT without signature verification
retrieve shared secret persisted on the handshake
verify JWTs signature using HMAC SHA-256 algorithm and the shared secret
Steps to create an appropriate token authenticating in Ergonode API
establish the App installation ID
create claims - at the very minimum the following claims are required: app_installation_id, nbf(not before), iat(issued at), exp(expiration time)
retrieve, persisted on the handshake, shared secret
create JWT with signature signed with HMAC SHA-256 algorithm and the shared secret
send the token as X-APP-TOKEN HTTP header to Ergonode API URL from handshake
Alternatively, on the requests from the Ergonode, i.e. in synchronization context, you can reuse the token provided by the Ergonode.
Extend the configuration via dictionaries
Sometimes you need to choose from a predefined list of values already existing in the system.
In the configuration, it's not always easy as the values may be related to the configuration itself. The values may be available only after configuring the API Key in step one etc, therefore, cannot be part of the regular schema.
That's where the dictionaries come in handy. There are two sources of the configuration. The data either come from Ergonode or App.
The source of the dictionary is recognized via prefixes ergonode: and app:
List of available dictionaries
ergonode:attributes
it is possible to limit types of attributes thanks to query parameters - egonode:attributes?type=TEXT,TEXT_AREA
ergonode:categories
ergonode:languages
In App, you can define as many custom dictionaries as you require.
In Manifest you reference the dictionary by the conjunction of app: prefix and dictionary ID suffix i.e app:attributes.
App provides custom dictionaries by implementing /dictionary/{dictionary} endpoint.
For the above example, the endpoint would be /dictionary/attributes.
The endpoint response for the valid dictionary should return
The endpoint will be called only on generating the step of the configuration schema it is defined in - so you have all the earlier steps of configuration at your disposal during dictionary retrieval.
Full reference of Manifest file.
Manifest is a simple JSON file that describes your App behavior and its possibilities.
To refresh the App Manifest, it has to be changed and reapplied - just like on App registration. Ergonode does not refresh it automatically.
Remember also, on refreshment, to raise the version - otherwise, changes won't be applied as described in the restrictions description.
Contained fields:
A simple text describing under what name your app is visible in the Ergonode available Apps tab.
This value should allow easy identification of what your App does.
Restrictions:
has to exist and not be empty
the length of the name is limited to 30 chars and should be no less than 3 chars.
A simple text describing the purpose fulfilled by the App.
Restrictions:
has to exist and not be empty
the length of the name is limited to 200 chars and should be no less than 20 chars.
Restrictions:
has to exist and not be empty
when reapplying the Manifest for App update the changes may only be applied when the new version is higher than the existing ones - otherwise, changes will not take effect.
This allows Apps Engine to determine whether an App administrator should take action and reconfigure the App for the new needs - fill out previously not existing configuration fields etc.
So for instance if, on the App update, the compatible field is higher than the last version(i.e. compatible = 1.1.0 and last version = 1.0.0) of the App installations will be set in Configuration required status - implicating the need for action and disallowing following actions with an App.
Restrictions:
has to exist and not be empty
the value cannot be greater than the version field
A list of predefined features App utilizes:
Enables the ability to run synchronization via the App.
The synchronization is always run as delta via this feature - meaning only changes from the last synchronization will be sent to the App.
on first synchronization, all data state is being transferred to an App
on every following synchronization, the App receives only data that has changed from the previous execution
Enables the ability to run full synchronization via the App.
All configured within App data is synchronized via this feature disregarding delta - which means the entire data state is transferred to the App on every synchronization.
This feature is always a secondary option in the Run synchronization split button if enabled together with synchronization.
Claims that the App generates a file as the result of the synchronization - for the enabled feature every finished synchronization will receive a file download button within the right-hand menu as well as a Download last synchronization button in the main App menu
This feature enables Copy feed URL button. This URL is publicly available, with no need for authorization.
Events the App subscribes to. Available:
App installed by the user in the Ergonode interface.
App uninstalled by the user in the Ergonode interface.
Attribute created.
Currently available only in the synchronization context.
In the synchronization context, the event also contains detailed updated payloads.
Attribute updated.
Currently available only in the synchronization context.
Attribute deleted.
Currently available only in the synchronization context.
Category created.
Currently available only in the synchronization context.
In the synchronization context, the event also contains detailed updated payloads.
Category updated.
Currently available only in the synchronization context.
Category deleted.
Currently available only in the synchronization context.
Product created.
Currently available only in the synchronization context.
In the synchronization context, the event also contains detailed updated payloads. It also represents the access granted event i.e. when the product is added to a segment.
Product updated.
Currently available only in the synchronization context.
Product deleted.
Currently only available in the synchronization context.
In the synchronization context, it also represents the access revoked event i.e. when the product is removed from a segment.
The new synchronization process started.
The synchronization process ended.
Claims that the App requires the write access to fulfill its purpose.
A user is notified of the request on App installation.
When installed the App can use Ergonode API write capabilities to manipulate the data.
Restrictions:
currently the field cannot be changed after the initial App registration
Icon allows easy identification of the App within available and installed Apps.
Restrictions:
value has to be represented as a Base64 image i.e. data:image/jpeg;base64,*
value cannot be greater than 10 KB
Base URL under which the App is available. When not provided a host of the Manifest Url is taken as this parameter.
Restrictions:
a valid URL string
currently, the value of the field cannot be changed after the initial App registration
Overview of steps required to create an App
To create an App you only need basic development knowledge.
You are not limited to a specific programming language, only to the possibilities granted via Ergonode by its communication interfaces.
To develop a functional App you only need to follow a couple of simple steps.
Firstly, you need to provide a Manifest - a simple JSON file determining the basic App behavior and possibilities it is utilizing.
The file is required for the app registration process for the New App to display in your available apps tab.
Next up it's essential to keep your data safe. To achieve that you should have a functional authentication system.
The App has to be able to identify itself so that Ergonode will know it can perform the API operation it is trying to execute. It is also crucial that when the App receives a request it can determine whether it comes from a trustworthy source.
On App installation, Ergonode exchanges handshake with App with data allowing authentication of the following requests performed, assuring that no 3rd party is tempering the data.
Some Apps require dedicated configuration. Once provided by Manifest App users will be able to set up their App within Ergonode to make it work appropriately to the installation needs - App should validate and persist your configuration.
If your App needs to know about basic events create specific events handling.
It is up to you what exactly the App will do.
This step is optional and depends on your intent.
If your App is meant to generate a file, integrate the data into the eCommerce system, or has a similar purpose forcing having the long-running process to handle all of the data, rather than implementing the entire synchronization yourself, let Ergonode orchestrate the process and focus only on the development of your business logic.
Host your application on an HTTP server of choice.
Ergonode currently does not provide hosting for custom Apps.
Once all set up register your App in the available apps tab.
To refresh the App Manifest, it has to be changed and reapplied just like on App registration. Ergonode does not refresh it automatically.
Configure your App in Ergonode
If the App requires configuration and defines configuration form in it has to appropriately handle the values.
It has to be able to validate and persist the configuration and also provide the configuration for reading.
To do so 2 endpoints implementation is necessary.
Each endpoint receives an appropriate X-APP-TOKEN HTTP header with JWT that allows of the caller.
The endpoint should return all so-far persisted schemas in update configuration steps in the form of an array of data - according to your schema.
For example for a simple configuration schema
the expected response would be
Every object represents single form step/one configuration schema.
Preserving the configured objects under the same indexes as their definitions in the configuration schema is mandatory.
The endpoint accepts, validates, and persists in the configuration step.
payload
All 2xx responses will be treated as success.
The forms are validated against configuration schema but not all validation can be performed based on JSON schema.
I.e. API token can only be verified in the App.
Though the frontend application performs configuration validation on every step it is a good practice to perform full validation in the App as well to limit any potential errors.
To assign a custom error message to specific user form fields it is required to return an error response with code 422 and the payload
All other error codes will be treated with a generic error message.
Overview of how to build App configuration
App most usually requires some sort of configuration specific to its installation.
Below you can find the Ergonode CSV App configuration.
Not full JSON Schema is yet covered in Ergonode.
If you have a case of a feature that suits your needs and is a part of the JSON Schema standard we may consider extending our implementation.
Please contact the respective Ergonode representative with the details.
The reason it's a list is fairly simple - in some scenarios, at first you need to input i.e. API Token, then choose which attributes from the App you'd want to configure - a piece of information that is only accessible once authorized within the system App is handling.
So the following steps in the form(following JSON Schemas) will be built once you correctly fill the current step.
Ok, so a lot is going on in this example. Let's break it down field by field.
This is the simplest case.
It's a plain text that your user can input. It'll be displayed as a regular HTML text input.
Another frequent case is a limited of values from which a user can choose. That's where the enum comes into play. There are 2 additional special fields added here
enum - represents the actual valid value list to choose from
options.enum_titles - represents the presentational titles list to be displayed to the user in dropdown
An extended example of simple enums is when you have multiple choice allowed. Instead of using type string adjust it to array with string type elements.
Here's where things get a bit more kinky. You, again, need a predefined set of values but those values are already there in Ergonode i.e. a list of languages to choose from or an attribute.
A field options.enum_dictionary is added - in short, you tell the interpreter to render a list of languages from Ergonode via a dictionary ergonode:languages - this is resolved by Ergonode.
Another possibility is that the predefined list is not part of Ergonode data but the App or system it integrates with. The App may define multiple dictionaries with different data.
Note this time options.enum_dictionary is wrapped into items - this changes the select to multi-select in the dropdown list.
Occasionally, especially on synchronization Apps, you'll need attributes mapping from Ergonode to an external system. This is possible by combining two things.
A structure definition that can carry on the mapping
Widgets are dedicated UI components that allow a custom presentation of a specific field.
Changes displaying of the plain text in HTML text input into password input.
Presents text input in the form of textarea input.
Allows for drag-and-drop mapping of the attributes and handles validation of their types.
The widget accepts 2 options:
ergonodeDictionary - saying where from claim data - this option is mandatory
appDictionary - this field is optional - if not present the Ergonode mapping will be possible to textual values
React on events occurring in your App
There are several endpoints corresponding to events in the Manifest configuration.
Each endpoint receives an appropriate X-APP-TOKEN HTTP header with JWT that allows of the caller.
subscribed event:
example payload:
subscribed event:
example payload:
Next to App events, there are multiple Synchronization events for which there are respective endpoints to be implemented.
This doc provides an overview of Ergonode Apps framework.
Apps allow you to extend the user interface of Ergonode - thanks to them you can do more in one place, eventually making handling your data and processes easier.
The App is a small custom web application (microservice) that resolves your business-specific problem.
It can generate a CSV file, integrate your data with an external eCommerce system, or even modify the data within Ergonode to i.e. automate some of the data management processes.
The App communicates with Ergonode through its public application programming interfaces (APIs) - you don't modify Ergonode code itself but instead provide a separate web application that, together with your Ergonode instance, removes obstacles on your path.
Since the App is a standalone application you are not limited to any specific programming language - you can develop it in PHP, Java, Python, JavaScript, and many more.
It can be written by yourself, by an agency you work with, or you can use one of the native Apps provided by the Ergonode team itself.
Follow this , or contact one of our trusted partners or your development agency.
A of your manifest. It does not necessarily need to address your application version but rather the content of the Manifest file.
A that the current version of the Manifest is compatible with.
A list of to allow building simple or multi-step configuration forms.
When this feature is enabled follow the .
When this feature is enabled follow the as for synchronization_file_download feature.
For development and testing use tools like to make quicker code iterations.
Remember also, on refreshment, to raise the version - otherwise, changes won't be applied as described in the restrictions description.
Such configuration can be achieved by setting up the section in the Manifest file.
The value is a list of allowing you to build a form.
The secret is very similar to the previous simple text example but as with any other sensitive data you do not always want it visible constantly on your configuration. That's why a is added - it results in a hidden secret text.
Sometimes one-line text won't suffice. You can extend the text presentation thanks to the .
and capable of making it easier to map the data in the configuration user interface.
Overview of the data flow during synchronization process
This phase is optional based on events the App subscribes to.
Status: Preparing
This is the internal phase.
Status: Preparing
Apps engine gathers the data required for the synchronization process.
This process can take some time. Its length depends on the amount of data to process and the cache size needed to be built.
Most usually it'll be the longest on the first run of the synchronization and significantly shorter on the following one, even if it is the same full synchronization.
This is the internal phase.
Status: Preparing
In this phase synchronization plan is created resulting in entries in the history created and put in Planned status.
This phase is optional based on events the App subscribes to.
Status: Execution
Firstly, as Attributes are the base of the Ergonode data model, their data is synchronized.
This phase is optional based on events the App subscribes to.
Status: Execution
As a second Categories data is delivered to an App.
This phase is optional based on events the App subscribes to.
Status: Execution
As products utilize all of the previously synchronized resources they are the last ones to be sent.
This phase is optional based on events the App subscribes to.
Status: Execution
Lastly, relation data is being sent - variants, children assignment, and product relation attribute values.
This is to avoid circular reference problems where product A depends on B and B on A - all the products should exist in this phase already.
Relations payloads are always provided with product_updated events.
This phase is optional based on events the App subscribes to.
Status: Finished
If an App purpose it to generate a file as a result of synchronization enable any of the following features based on your needs
To allow access to Ergonode for generated files you need to implement the following endpoint.
Use synchronization_id claim from JWT to determine which file should be returned.
The endpoint should return the binary file.
The response can contain content-type and content-length HTTP headers so that those are passed to the browser on file download.
The list of recent and upcoming breaking changes
The list of recent and upcoming breaking changes At Ergonode we strive to make your integration process as fluent as possible but from time to time we discover a flaw in the design of the schema or introduce a really cool new feature that is impossible to be provided in a fully-compatible non-breaking way.
In such a situation, we will always aim to not break your integration with the new release and provide you with a transition period of approximately 3-months to adjust your consumer. Every such change shall be communicated in the following list.
Good practices and tips
Carefully approach multitenant Apps. Since Apps can handle multiple installations, even on multiple Ergonode instances, query your persistent data with that in mind to prevent data security leaks.
Get to know the authentication section well.
The App should be highly available to accept multiple requests simultaneously.
To increase the performance of the synchronization process Ergonode can send a couple of requests at one time concurrently. Not only from one instance but also during one synchronization, especially if there's a lot of data being integrated.
If your app needs extra PIM data fetched by Ergonode APIs, especially during synchronization, cache it internally for at least the length of the process. It shall significantly speed up your processing.
Detailed events containing data changes.
included with events: attribute_created
included with events: attribute_created, attribute_updated
included with events: attribute_created, attribute_updated
included with events: attribute_created, attribute_updated
included with events: attribute_updated
included with events: attribute_created, attribute_updated
included with events: category_created
included with events: category_created, category_updated
included with events: product_created, product_updated
included with events: product_created, product_updated
included with events: product_updated
included with events: product_created, product_updated
included with events: product_created, product_updated
included with events: product_updated
included with events: product_created
included with events: product_updated
included with events: product_created, product_updated
included with events: product_created, product_updated
included with events: product_updated
included with events: product_updated
included with events: product_updated
Let Ergonode orchestrate the synchronization process and worry only about your business logic
At the heart of every PIM-including ecosystem, there's data integration.
A need to keep your data well-organized thanks to PIM and easily accessible to eCommerce system(s).
We do realize that creating the synchronization process is not an easy task. At the very minimum, it most usually requires:
a queue system like RabbitMQ, Amazon SQS, etc
a worker
well-developed and robust logic to handle the process
progress state
both process and entry statuses
history
frontend application displaying all that data
persistent storage i.e. database
business logic implementation
which is a lot and usually not that easy to do, especially, in the distributed environment of multiple services.
That's why at the core of the Apps engine we created an entire synchronization process orchestration with the intent of you only being left with implementing your desired business logic and the minimum effort needed around it.
Need only an incremental export of data you have recently changed? We've got you covered.
Does it have to handle full export each time? It's possible with an App.
To start with a synchronization App enable at least one of those features in your Manifest.
synchronization - enables the basic integration. During every synchronization, you'll only receive data changed from the last iteration
synchronization_full - enables the possibility of running full integration on every execution
Both features may be enabled next to each other or you can pick just one.
If your App generates a file, enable features according to your needs
synchronization_file_download - enables file download for each synchronization
synchronization_file_download_latest - generates constant URL under last-generated file will be available for feed-like functionalities
Finally, you need to develop your business logic in synchronization endpoints and file download endpoint if your App utilizes this feature.
A place where the synchronization business logic is implemented
Synchronization endpoints have a very similar role to message handlers in a message queue system.
They receive the event(message) with data and react to its payload.
There are several endpoints corresponding to events in the Manifest configuration.
Each endpoint receives an appropriate X-APP-TOKEN HTTP header with JWT that allows authentication of the caller.
JWT for all synchronization endpoint requests is extended with an extra context claim synchronization_id.
This claim is a UUID and allows identification of the synchronization process.
Every endpoint should respond with a 2xx HTTP code to be treated as a success.
All the endpoints follow the semantic [PUT] /consume/{event}.
synchronization.events contains an array of detailed payloads containing actual changes - the full list can be found here.
Those events allow retrieving information about what has changed and reacting to that change.
This field for DELETED events is always an empty array.
subscribed event: attribute_created
example payload:
subscribed event: attribute_updated
subscribed event: attribute_deleted
example payload:
subscribed event: category_created
example payload:
subscribed event: category_updated
example payload:
subscribed event: category_deleted
example payload:
subscribed event: product_created
example payload:
subscribed event: product_updated
example payload:
subscribed event: product_deleted
example payload:
subscribed event: synchronization_started
example payload:
subscribed event: synchronization_ended
example payload:
In the above examples, you might have noticed the resource_customs field.
This mechanic allows storing small pieces of information as JSON in the engine.
The most common use case for this would be storing the ID of the eCommerce entity next to the resource. It'll be passed with every following request so it's easier for your business logic to create the requests to the eCommerce system without the need of reaching to the database.
To persist the custom in the response of the endpoint add the following payload
The resource custom will only be persisted on 2xx(non-204) HTTP responses.
Handling of the request payload may fail.
In such case, there are a couple of scenarios:
failed connection(including 502 and 503 HTTP errors) - the event will be retried a couple of times before aborting completely
5xx and 3xx HTTP responses
for a single resource, the event is not retried as of an internal error and the history entry is marked with a generic message
for a synchronization error event is going to be retried eventually leading to the failed synchronization status
4xx HTTP responses
for a single resource
a custom behavior may be applied via an appropriate response
retryable parameter determines whether the event can be retried. Otherwise, the fail and custom error message will be applied immediately
the event is not retried and the history entry is marked with a generic message
for a synchronization error event is going to be retried eventually leading to the failed synchronization status
