DVN Sample Architecture
Last updated
Last updated
This section illustrates a simple architecture a DVN can adopt.
The DVN will be called “HelloWorldDVN”. Even if it’s intended as a demo only, it can give a good idea on how a DVN can be built in production.
deploy HelloWorldDVNGateway.sol and use it as DVN Owner address
owned by DVN developers
acts as DVNCoordinator owner
able to register Operators with custom logic
not mandatory, DVN can also use an EOA
register to Kernel by connecting with Kernel team:
provide the DVN Owner address (a DVNGateway contract, an EOA, or a Safe account)
obtain the DVNCoordinator address
Operators send requests to join the DVN through the DVNCoordinator contract
for each Operator (which will communicate its Operator ID) call:
DVNCoordinator::finalizeOperatorRegistration(operatorId, true) to accept the request and get the Operator ready to execute Tasks
DVNCoordinator::finalizeOperatorRegistration(operatorId, false) to reject Operator’s request
HelloWorldDVN has now a sufficient number of Operators ready to process its tasks
As illustrated above, the Task Aggregator is responsible for assigning tasks to LREs, collecting responses from LREs and determining the validity of each response.
“LRE” and “Operator” are the same synonyms in this context, but “LRE” will be used because it’s the actual piece of code in charge of handling Tasks.
With simplicity in mind, the TaskAggregator can be a server exposing the following API:
GET /operator/{operatorId}/task/next Returns, for a given Operator ID, the next task to execute
each LRE calls this endpoint using its own Operator ID
obtain some data like: { id: 1, input: “task input”, expiration: 6000 // response must be submitted within 6 seconds }
“input” field can contain anything, like:
a string to manipulate
an url pointing to an image that must be analyzed by some AI tool
a question to answer for prediction market purposes
the ticker of a token for which the price is wanted
POST /operator/{operatorId}/task/{taskId} An Operator sends the response it calculated for a given task to the Task Aggregator
Post data: { response: <string|object|number|bool>, signature: <signature of the task id + response> }
Upon receiving each response, the Task Aggregator should:
check the signature to verify if:
the response was not manipulated
the sender matches the operatorId url parameter
check if the Task is still accepting responses
the Operator may send its response too late in relation to “expiration”, or the Task could have already received a definitive response
store in a database the received response
Remembering that the main purpose is to produce a definite output for a Task after collecting responses, at some point, the Task Aggregator must decide which is the “correct” response.
A simple (too simple, but effective) model to achieve would be running the following logic when receiving a response from an Operator:
if (at least 80% of the registered Operators sent their response) AND (if there’s a response that has frequency >= 90%)
then the response with >= 90% frequency is considered the correct one and:
it’s stored in database as definitive response for that Task
every response sent by Operators matching it is marked as “correct” in database, while other responses are marked as “incorrect”
DVN is responsible for delivering a Long-Running Executable (LRE) to be executed by Operators on their servers.
Can be written in any language and adopt any installation mode.
As an example, HelloWorldDVN could publish a Docker image, easily pulled and bootstrapped on any environment.
The Docker image could include:
config.ts A script to be run only once, before the LRE is started. It should perform initial configuration, like storing in a database the Operator ID or securely storing some private keys if necessary
check.ts A script to be run anytime, checking everything is configured and the machine is ready to process Tasks. It should check connection to Task Aggregator, connection with Kernel and DVN contracts, verify minimum hardware requirements (if any)
lre.ts The long-running process that:
should be always running (tools like pm2 are recommended)
triggers GET /operator/{operatorId}/task/next periodically to obtain the next Task to execute
executes the Tasks and sends responses back to the Task Aggregator
Database to store information about Tasks and other data
The HelloWorldDVN is not production-ready as it lacks some important features regarding performance and security. Nevertheless the illustrated architecture can be used as reference.
Some suggested improvements:
Run Task Verification Asynchronously When producing the definitive response for a Task, and instead of running the logic after receiving every response, it’s better to run it asynchronously in a separate process to keep a low API latency and to able to perform also long and complex verifications.
Use ApiKey to identity an Operator When an Operator runs config.ts, a key pair could be created and the Task Aggregator could provide the LRE with an ApiKey. The ApiKey could be used in any API call, implementing a quick and easy way to identify an Operator.
Do not poll to TaskAggregator but use a push model Instead of the LRE polling periodically the Task Manager, a tool like Socket.io could be used to “push” Tasks to the LRE instead of polling periodically
