Validator Contest: Devops tools [Finished]

Hey,

I’ve joined contest just yesterday and seems won’t be able to deliver working solution by 1st of June, but wanted to share architecture that I came up with and going to implement next week.

Notes:

  • Validator node doesn’t have any extra ports exposed
  • Every deployment can be scaled independently and whenever is required
  • Very flexible in controlling costs - Validator, Controller and Logstash are deployed via Docker (backed-up with docker-compose) either to bare-metal machine or VM. (Ansible can help in some maintenance later on, I’ve excluded Terraform as it’s not that good for bare-metal cases).
    At the same time monitoring can be either custom solution or one of SaaS solutions with pay-as-you-go subscriptions. The same applies for message-queue (either custom deployment or SaaS).
    With the current specs for a Validator node bare-metal machines will be the most cost-effective I believe comparing to any VM in any of cloud providers.
  • Pub/Sub layer provides good abstraction and allows to inject many type of notifications and ways to control validator(s), including safe for the validator web interfaces.
    It will be easy to integrate any kind of alerting and automatic response to those alerts.
  • Controller plan to implement as set of standalone libraries for tonos-cli, lite-client, validator-engine (re-usable for any other python apps as well) + controller logic itself with interface to message queue (so that extension to any MQ will be possible)
  • Controller will be responsible for automation of participation in elections, querying for configuration and blockchain data, help interaction with smart contracts.

Sad that tackled this TON context that late :\ But anyway will be striving to join validators group :slight_smile:

Implementation going to land here and once in working-order, some parts likely will be moved to separate repositories (ex py-tonos-cli, py-ton-lite-client, py-validator-engine).

3 Likes

Hello! I want to support validator node at least for the one year by this infrastructure. It’s just initial configuration, a lot of tests and additional functionality will be available soon.

For now I have Dockerfile for C++ node (which can be run in Openshift also)
Helm chart for Validator Node
High Availability infrastructure based on AWS
Logging system based on CloudWatch
EC2 monitoring

1 Like

Hello
Where to put our solution of this contest?

Hello!
Many Validators ask me about the graphic of slow events, and now it is ready! Graphic by slow event groups will be available later.
Of course, you can download the latest version of my telegram bot on GitHub

Best regards,
Stanislav

5 Likes

Hello
This is our solution based on Kubernetes, ELK and Zabbix with alerting to telegram
https://github.com/freeton-dreamteam/contest

2 Likes

Hi
Please check out validator contest submission from our team

1 Like

Hello everyone
We are pleased to present our ansible-freeton repository
GitHub - br3d/ansible-freeton: Ansible playbook for deploy freeton
Roles list:

  • common - preparing system and install dependencies
  • freeton - build and setup FreeTon node
  • netdata - real-time monitoring
  • prometheus-node-exporter - exporter for hardware and OS metrics exposed, also this gives opportunity get balance and diff in freeton network

Dashboard example based on prometheus-node-exporter with custom metrics from freeton network

6 Likes

wiz wiz2

  1. cli wizard for manual managing of the requests for the participation in elections with a possibility of making a bid (with checking of the validity) returning of the stake.

  2. possibility of working with many nodes in different networks: FreeTON mainnet/testnet with the calls tonos-cli, or nodes of other ton networks with the calls lite-client

  3. keeping the wallet keys on the local computer, not on the node. On the node are generated only permkey and adnlkey.

  4. Script of automatic deploying of surrounding, which receives from the sources tonos-cli, lite-client, validator-engine-console, fift, fift-scripts and delivers it across its file system.

    Setting-up is realized in /tmp, and then it erase everything
    rested after the compilation.

4 Likes

Hello!

The new category opens in the Validator tool - Info.
Step by step, I will add useful information there.
Monitoring based on this information will be later.

Let’s start

  1. Election status & validators count
2 Likes

UDP:
Init TONTgBot integration. Thanks @anvme for the bot and support .

Link to repo: https://github.com/SkySonR/freeton-infra

1 Like

Some functions, right now, can don’t work correctly in docker. I hope that we will solve this.
But, on Ubuntu, all works perfectly.

1 Like

When will be the result?

Will discuss that tomorrow on a weekly call.

3 Likes

Great news! Thanks a lot.

1 Like

UDP: Fixed all docker/kubernetes issues with TGbot integration. Works 99% of functionality.

Repository: https://github.com/SkySonR/freeton-infra

1 Like

In this submission there is no reference to any other network than the FREETON network. The toolkit is called ‘GRAM’ because it’s easy to differentiate it from the currency TONs while still being unique enough to differentiate between itself and the actual freeTON code.

Right now you must build and compile the TON C++ locally (even if you are using Docker). I’m working on replacing generate-initial-keys binary and many other improvements.

You can build docker images for a number of OS’s (probably one of the more robust docker test kits out there and the images are optimized somewhat)

It uses pm2 to manage the different processes: validator node, nodeJS API, documentation server, web server to house the Vue app toolkit.

It also uses tmux to throw up some logs so you can kind of see what’s going on at the core layer, though I recommend some of the other great tools for more advanced monitoring and joining elections, I didn’t waste my time since others have done such a great job there.

The toolkit also comes with a nodeJS API which can talk to lite-client on the node, so it makes it as easy as a POST request to send files and GET request to get account info/run methods, no pub key needed. It also serves up the config json and liteserver pub keys.

I’ve added a JSON-based block explorer to the TON C++ (which is merely a copy of the other block explorer with all of the garbage HTML removed).

It also comes with a VueJS / cordova / electron app that you can extend to your liking, and I plan much more to come there.

The documentation is nice, you can check that out by looking in the docs/README.md folder.

It does include the WASM kit for fift/func in case you want to use those.

2 Likes

Update on my Cloud bot
@TON_Validators_Bot

New features:
Added Alerts
Support for online monitoring of multiple addresses.
The bot can update the last message, which shows monitoring of all addresses.
Abbreviated record of addresses, for better usability.
And fixed some bugs.
Look here: with Alerts

1 Like

Hello!
Smart contract deployer in 20 seconds =)
Fast, secure and installation in 5 steps without brain xD

4 Likes

ftvmon has been updated, numerous performance fixes (its own cpu load is negligible now), added number of threads in validator’s process check. Please note that the process check has been moved to ExtCheck section of the config.

SuTon has minimal MVP now, in particular:

  • TonValidator deployment in a given env, execution and synchronization
  • TonControl deployment and can track for validator health and its sync status
    (automatic elections will appear soon, code is ready, but being tested)

Deployment settings are done for infra-as-code style, so your settings can be tracked using Git and easily sharable across the team.
It also supports multi-node scenarios, each having own set of settings.

All bits deployed via docker-compose, no ports exposed except ADNL one for validator.

Usage, architecture and settings details are in reamde:

1 Like