DVN Sample Architecture

This section illustrates a simple architecture a DVN can adopt.

The DVN will be called “HelloWorldDVN”. Even if it’s intended as a demo only, it can give a good idea on how a DVN can be built in production.

5.1. Register to Kernel

  • deploy HelloWorldDVNGateway.sol and use it as DVN Owner address

    • owned by DVN developers

    • acts as DVNCoordinator owner

    • able to register Operators with custom logic

    • not mandatory, DVN can also use an EOA

  • register to Kernel by connecting with Kernel team:

    • provide the DVN Owner address (a DVNGateway contract, an EOA, or a Safe account)

    • obtain the DVNCoordinator address

5.2. Onboard Operators

  • Operators send requests to join the DVN through the DVNCoordinator contract

  • for each Operator (which will communicate its Operator ID) call:

    • DVNCoordinator::finalizeOperatorRegistration(operatorId, true) to accept the request and get the Operator ready to execute Tasks

    • DVNCoordinator::finalizeOperatorRegistration(operatorId, false) to reject Operator’s request

  • HelloWorldDVN has now a sufficient number of Operators ready to process its tasks

5.3. Build the TaskAggregator

As illustrated above, the Task Aggregator is responsible for assigning tasks to LREs, collecting responses from LREs and determining the validity of each response.

“LRE” and “Operator” are the same synonyms in this context, but “LRE” will be used because it’s the actual piece of code in charge of handling Tasks.

With simplicity in mind, the TaskAggregator can be a server exposing the following API:

  • GET /operator/{operatorId}/task/next Returns, for a given Operator ID, the next task to execute

    • each LRE calls this endpoint using its own Operator ID

    • obtain some data like: { id: 1, input: “task input”, expiration: 6000 // response must be submitted within 6 seconds }

    • “input” field can contain anything, like:

      • a string to manipulate

      • an url pointing to an image that must be analyzed by some AI tool

      • a question to answer for prediction market purposes

      • the ticker of a token for which the price is wanted

  • POST /operator/{operatorId}/task/{taskId} An Operator sends the response it calculated for a given task to the Task Aggregator

    Post data: { response: <string|object|number|bool>, signature: <signature of the task id + response> }

    Upon receiving each response, the Task Aggregator should:

    • check the signature to verify if:

      • the response was not manipulated

      • the sender matches the operatorId url parameter

    • check if the Task is still accepting responses

      • the Operator may send its response too late in relation to “expiration”, or the Task could have already received a definitive response

    • store in a database the received response

Remembering that the main purpose is to produce a definite output for a Task after collecting responses, at some point, the Task Aggregator must decide which is the “correct” response.

A simple (too simple, but effective) model to achieve would be running the following logic when receiving a response from an Operator:

  • if (at least 80% of the registered Operators sent their response) AND (if there’s a response that has frequency >= 90%)

  • then the response with >= 90% frequency is considered the correct one and:

    • it’s stored in database as definitive response for that Task

    • every response sent by Operators matching it is marked as “correct” in database, while other responses are marked as “incorrect”

5.4. Ship the LRE

DVN is responsible for delivering a Long-Running Executable (LRE) to be executed by Operators on their servers.

Can be written in any language and adopt any installation mode.

As an example, HelloWorldDVN could publish a Docker image, easily pulled and bootstrapped on any environment.

The Docker image could include:

  • config.ts A script to be run only once, before the LRE is started. It should perform initial configuration, like storing in a database the Operator ID or securely storing some private keys if necessary

  • check.ts A script to be run anytime, checking everything is configured and the machine is ready to process Tasks. It should check connection to Task Aggregator, connection with Kernel and DVN contracts, verify minimum hardware requirements (if any)

  • lre.ts The long-running process that:

    • should be always running (tools like pm2 are recommended)

    • triggers GET /operator/{operatorId}/task/next periodically to obtain the next Task to execute

    • executes the Tasks and sends responses back to the Task Aggregator

  • Database to store information about Tasks and other data

5.5. Notes

The HelloWorldDVN is not production-ready as it lacks some important features regarding performance and security. Nevertheless the illustrated architecture can be used as reference.

Some suggested improvements:

  • Run Task Verification Asynchronously When producing the definitive response for a Task, and instead of running the logic after receiving every response, it’s better to run it asynchronously in a separate process to keep a low API latency and to able to perform also long and complex verifications.

  • Use ApiKey to identity an Operator When an Operator runs config.ts, a key pair could be created and the Task Aggregator could provide the LRE with an ApiKey. The ApiKey could be used in any API call, implementing a quick and easy way to identify an Operator.

  • Do not poll to TaskAggregator but use a push model Instead of the LRE polling periodically the Task Manager, a tool like Socket.io could be used to “push” Tasks to the LRE instead of polling periodically

Last updated