Validator Devops contest for Rust-Node

Mitja · December 10, 2020, 9:07pm

UPD: the contest will run under DevOps Subgovernance. Below is the final version of the Contest.

Short description

Monitoring, Alerting, Scripts and Tooling for RUST implementation of Free TON Validator

github.com

tonlabs/ton-labs-node-tools

tools for node. Contribute to tonlabs/ton-labs-node-tools development by creating an account on GitHub.

github.com

tonlabs/rustnet.ton.dev

Contribute to tonlabs/rustnet.ton.dev development by creating an account on GitHub.

Type

Contest

Dates

Submissions - Starts: 18.01.2021 at 00:00 UTC, Ends: 31.01.2021 at 23:59 UTC

Voting - Starts: 01.02.2021 at 00:00 UTC, Ends: 14.02.2021 at 23:59 UTC

Motivation

Validators need tools and automation to insure efficient scalable and secure operations to support Free TON blockchain

General requirements:

Documentation should be provided for each tool
Provided tools should operate exactly according to its documentation
Motivation and goal for each tool should be described
All software must be released with licences supported by Free Software Foundation

DevOps Skills required:

Linux
Unix Shell scripting, javascript
Infrastructure as a code
docker/docker-compose/kubernetes
Desirable: Ansible, logstash, Prometheus/Grafana

Tasks:

Enhanced node deployment infrastructure

Need to develop enhanced deployment infrastructure for Rust validator nodes based on modern tools and techniques (docker/docker-compose, Ansible, Terraform, etc.).
Based on infrastructure as a code. Take into account deployment from scratch, upgrade, cleanup scenarios on various set of Linux distros as well as other OS (e.g. MacOS).

Intellectual log dashboard with analytics:

Need to collect log files into centralised database (e.g. logstash).
Make base analytics with some predefined patterns which can simplify failure analysis and monitoring.

Extended metrics for node monitoring

Analyse available node/validator/elector metrics and define a subset useful for further analytics and monitoring tasks.
Develop visualisation concept to simplify the analysis.
Develop alert rules for daily monitoring

Automate staking through depool (and/or direct staking to the elector) based on tonos-cli or SDK (use TON-SDK core library or any existing binding)

Write validator script for elections using tonos-cli/SDK
Use tonos-cli/SDK to run elector get-methods.
Use tonos-cli/SDK to query global configuration parameters.
Use tonos-cli/SDK to prepare, sign and send validator query to depool/elector smart contract.
Automate confirmation of multisig transactions.
Improve custodian notification mechanism for multisig transactions: notify about submitted transactions via user-friendly mechanism (e-mail, sms, etc.).
Implement tools for staking monitoring and alerting

Automate staking through depool (and/or direct staking to the elector) using R-Node Console

Write a script to manage overall staking procedure (prepare elector request, sign, send, manage bids, etc.)
Automate confirmation of multisig transactions
Implement tools for staking monitoring and alerting

Evaluation criteria and Winning conditions:

Number of completed tasks. Tasks 1 (at least for one OS) and optionally 4 or 5 are necessary . Without them submission will be rejected
Correctness of operations of the provided tool
Quality of code and description
Work as advertised, perform its stated functions

The Jury

The Jury will be selected by a sub-governance which will support this contest.

Jury rewards:

An amount equal to 5% of the sum total of all total tokens actually awarded to winners of this contest will be divided equally between all jurors who vote and provide feedback. Both voting and feedback are mandatory in order to collect this reward.

Winners:

The winners of all 5 tasks will recieve the following:

1st prize — 40,000 Tons (16 000 + 24 000 by vesting)
2nd prize — 30,000 Tons (12 000+ 18 000 by vesting)
3rd prize — 20,000 Tons (8 000 + 12 000 by vesting)
4th prize — 5,000 Tons
5th prize — 3,000 Tons
6-10th price — 1,000 Tons each

Winners receive 40% of their rewards when contest results are evaluated.
Another 60% will be vested over a 12-month period with monthly payouts.

Vesting details

Vesting is devided into 12 equal parts. The first vesting payout is paid one month after the end of the contest. To receive all 12 distributions, the solution should be supported at least for 1 year. Scripts should be updated within a reasonable timeframe - 1 month maximum, and according to rust node changes. Major GitHub Issues should be resolved within a reasonable timeframe - 1 month maximum.

Frike · December 10, 2020, 9:16pm

How many contests there should be with same tasks?

sergemedvedev · December 10, 2020, 10:07pm

Although being similar in description, those contests ain’t the same.

Each of them allows community members to gain knowledge, expertise and hands-on experience related to existing and emerging tools that are vital for Free TON.

Collective experience of that kind seems to be very important to me.

Frike · December 10, 2020, 10:46pm

First contest was created before the audience had a chance to get familiar with software and use it. It ended fast, and left no chances for tools created later during the active phase of MLudi contest.

Now - the same. I think making such contest should be started in at least 3 months AFTER rust validator release (even if we approve it in advance) with result in having much more quality works.

Mitja · December 10, 2020, 10:59pm

This is not the same contest. It is about Rust Validator and working with R-Node Console (which is not the same as working with DappServer or C++ node).
The validator contest should already have a set of tools to run the T-Node. At least that is the idea. It is not about getting ideal tooling, but something the community can improve and debug during Validators contest.

Mitja · December 10, 2020, 11:00pm

The contest will probably be run by the DevOps subgov. I published it here because it is important for many contests outside the DevOps subgov in the near future.

Stanislav · December 15, 2020, 4:26pm

Can we move date from new year? Start from January 5 for example.

Stanislav · December 15, 2020, 4:45pm

hm, memes contest was more attractive…
Lets do winners reward like in Animated stickers contest?

Winner ranking	Fixed Reward
1	30,000
2	25,000
3	20,000
4	15,000
5	14,000
6	13,000
7	12,000
8	11,000
9	10,000
10	9,000
11	8,000
12	7,000
13	6,000
14	5,000
15	4,000
16	3,000
17	2,000
18	1,000
19	1,000
20	1,000

Mitja · December 17, 2020, 6:34pm

its a problem with animated stickers contest

Stanislav · December 17, 2020, 9:10pm

DevOps works cost smaller then surf testers with several reports…
With compare with other contests, this looks not well

br3d · December 18, 2020, 8:41pm

the same thing as with the first DevOps contest. Just Mitja don’t like DevOps

jarig · December 22, 2020, 8:51pm

Contest started or delayed?

Gofman · December 23, 2020, 9:54am

Delayed due to the fact that DevOps subgovernments still in the jury selection stage. While you wait you can participate in this contest: Free TON DevOps Jury Contest
you’re very welcome

Mitja · December 30, 2020, 12:50am

I love DevOps they are my favorite. I dont like stickers and other community contests paging out too much. Hope that will be corrected soon!

Gofman · December 30, 2020, 7:48pm

Hi folks!
After contest discussion on the DevOps subgov weekly call (join as if you still not in the group - > https://t.me/freetondevops) we have two suggestions:

Extend contest rewards pool but add vesting in equal parts over a 12-month period with monthly payouts. To receive all 12 distributions, the solution should be supported for 1 year
The scripts should be updated within a reasonable timeframe - 1 month maximum, and according to rust node changes
Major GitHub Issues should be resolved within a reasonable timeframe - 1 month maximum.
Winners receive 30% of their rewards when contest results are evaluated. I propose this reward distribution table:
1st prize — 40,000 Tons (12 000 + 28 000 by vesting)
2nd prize — 30,000 Tons (9 000 + 21 000 by vesting)
3rd prize — 20,000 Tons (6 000 + 14 000 by vesting)
4th prize — 5,000 Tons
5th prize — 3,000 Tons
Next 10 runners up — 1,000 Tons each
Add minimum of completed tasks (task 1, and task 4 or 5) (without them submission can be rejected) to simplify evaluation creteria and voting. Otherwise juries will potentially compare different types of works (some will be focus on monitoring/alerting, some on installation/validation scripts) which will hard and incorrect to compare (because of different purpose)

Let’s discuss. Please agree/disagree.

br3d · December 30, 2020, 9:21pm

I want to clarify, imho
for task 1
for the minimum of the task completion will enough for one OS

Mitja · December 31, 2020, 6:27am

Good. Will amend. Should change the dates as well.

4hash · December 31, 2020, 3:05pm

Just double-checked, according to the DevOps SG proposal (voted and passed), the reward for this contest is supposed to be 200’000, not 103’000. Compared to the first C++ DevOps node contest, minimum number of tasks in this one is mandated (i. e. more work required). For the C++ node DevOps node contest, the rough equivalent of task 3 has been done by TonLabs (scripts - validator_msig.sh, validator_depool.sh), and task 4 hasn’t been there at all (would’ve been something like automating staking through depool using liteclient). Other than that, requiring MVP (minimum number of tasks completed) and vesting for the first 3 places is a good idea.

sostrovskyi · January 15, 2021, 3:55pm