Setup
Configuration
Metrics Exported configuration is stored in the TOML file ./monitoring/metrics_exporter/metrics-exporter.toml.
It consists of 3 parts called tables (table is in the square brackets). Here is a short description of such structure:
1. table scraping
- includes 1 key called
stock_fw_scrape_interval, which has default value set on 1 minute
stock_fw_scrape_interval is a parameter defining scraping interval of the Stock FW devices
2. table ranges
- includes arbitrary number (at least 1) of keys with arbitrary names
- the key represents range(s) of the IP addresses to be scanned from the farm's network (IPv4 only)
Examples
1 IP address range - range of 100 IP addresses defined on the 4th octet as "1-100":
[ranges]
Building_A = ["10.35.1.1-100"]
1 IP address range with a wildcard *, which represents the whole range 0-255:
[ranges]
Building_B = ["10.35.2.*"]
List of IP addresses - list of simple IP addresses:
[ranges]
Building_C = ["10.35.3.1","10.35.3.20","10.35.3.38"]
Several lists of IP ranges:
[ranges]
Building_A = ["10.35.3.1","10.35.3.20","10.35.3.38"]
Building_B = ["10.35.2.*"]
Building_C = ["10.35.3.1","10.35.3.20","10.35.3.38"]
Building_D = ["10.35.4.*","10.35.5.*"]
Container_X = ["10.36.1.*","10.36.2.1-100","10.36.2.178-255"]
Based on the user preferences it can make sense to use ranges as representation of the farm's structure, in our example each range represents a building. But the name of the range has no limits (apart from the fact it does not support white spaces).
Name of the range is then present in the monitoring as a label.
3. table advanced
- includes 5 keys, which are a defined setting of the discovery process
- key
scan_interval
- interval for scanning period
- default value is 5 minutes
- key
scan_timeout
- timeout interval for scanning
- default value is 15 seconds
- key
scan_max_concurrency
- maximal number of parallel scan requests
- default value is 1,000 parallel scans
- key
stock_fw_scrape_timeout
- timeout interval for scraping stock FW devices
- default value is 5 seconds
- key
stock_fw_scrape_max_concurrency
- maximal number of parallel scrape requests for stock FW devices
- default value is 4,000 parallel scrape requests
Example of the metrics-exporter configuration file
# scraping internal for stock FW devices, scraping interval for BOS devices is defined in the monitoring/promethus/prometheus.yml (default is also 5s)
[scraping]
stock_fw_scrape_interval = "1m"
# ranges needs to be defined by user, name of the ranges is then used as Prometheus label
# naming is arbitrary
# number of ranges is also arbitrary
[ranges]
Building_A = ["1.2.0.*","1.2.*.2"]
Building_B =["1.3.0.*","1.3.*.2"]
# credentials needed only for http API, CGMiner API doesn't require credentials
# default credentials are root/root, if the user is using another credentials, it has to be edited and uncommented here
#[credentials]
#username = "root"
#password = "root"
[advanced]
# interval for scanning period
scan_interval = "5m"
# timeout interval for scanning
scan_timeout = "15s"
# maximal number of parallel scan requests
scan_max_concurrency = 1000
# timeout interval for scraping stock FW devices
stock_fw_scrape_timeout = "5s"
# maximal number of parallel scrape requests for stock FW devices
stock_fw_scrape_max_concurrency = 40000
Start monitoring
When the configuration is prepared, the monitoring can get started. Run the command in the Linux terminal docker compose up -d.
Creating metrics-exporter ... done
Creating prometheus ... done
Creating grafana ... done
To see if all the Docker containers are running, hit command docker ps to list them and check their status.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
663ded1c6ac9 grafana/grafana:8.4.0 "/run.sh" 2 weeks ago Up 2 weeks grafana
3aa848025b2e prom/prometheus:latest "/bin/prometheus --c..." 2 weeks ago Up 2 weeks prometheus
fe28840ba56f docker.ii.zone/pool/main/metrics-exporter:nightly "/usr/local/bin/metr..." 2 weeks ago Up 32 hours metrics-exporter
Restart
In case Braiins Farm Monitor needs a restart, run the command docker compose restart metrics-exporter.
docker compose restart metrics-exporter
Restarting metrics-exporter ... done
The command will restart only the metrics-exporter container. Restart of metrics-exporter is needed in case of any changes in the configuration file ./monitoring/metrics_exporter/metrics-exporter.toml. If you wish to restart another Docker container it can be done similarly by replacing metrics-exporter with the name of the container that needs a restart.
Stop
Sometimes the mining operator may want to stop Braiins Farm Monitor. It can be done in Linux terminal with the command docker compose stop metrics-exporter.
docker compose stop metrics-exporter
Stopping metrics-exporter ... done
It is needed in case of any changes in the file docker-compose.yml.
To run proxy again, run the command docker compose up -d metrics-exporter.
docker compose up -d metrics-exporter
Starting metrics-exporter ... done
If you wish to stop another Docker container it can be done similarly.
Logs
Braiins Farm Monitor is saving its logs within a Docker container. Docker is configured to store a maximum of 5 GB of logs.
Log rotation and compression is in place. The number of log files is set to be 50 and the logic is that the oldest file is dismissed and a new one is spawned. The maximum size of 1 file is 100 MB.
Here are some useful commands for investigating the logs.
For more detail see
Usage: docker logs [OPTIONS] CONTAINER
Fetch the logs of a container
Options:
--details Show extra details provided to logs
-f, --follow Follow log output
--since string Show logs since timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes)
-n, --tail string Number of lines to show from the end of the logs (default "all")
-t, --timestamps Show timestamps
--until string Show logs before a timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes)
All available logs
docker logs metrics-exporter
2022-12-20T20:14:58.012502Z INFO server: metrics_exporter::server: listening on 0.0.0.0:8081
2022-12-20T20:14:58.012800Z INFO exporter: metrics_exporter::exporter: running initial scan
2022-12-20T20:14:58.012842Z INFO exporter:scan: minerctl::scanner: scanning 1024 ip addresses...
2022-12-20T20:15:28.034861Z INFO exporter:scan: minerctl::scanner: discovered 6 miners
2022-12-20T20:15:28.039230Z INFO exporter: metrics_exporter::exporter: starting scanner
2022-12-20T20:15:28.039188Z INFO exporter: metrics_exporter::exporter: starting scraper
...
Last 5 logs
docker logs metrics-exporter --tail 5
2022-12-20T20:41:48.675465Z INFO exporter:scan: minerctl::scanner: scanning 1022 ip addresses...
2022-12-20T20:42:18.691623Z INFO exporter:scan: minerctl::scanner: discovered 6 miners
2022-12-20T20:47:18.695790Z INFO exporter:scan: minerctl::scanner: scanning 1022 ip addresses...
2022-12-20T20:47:48.706334Z INFO exporter:scan: minerctl::scanner: discovered 6 miners
2022-12-20T20:52:48.710557Z INFO exporter:scan: minerctl::scanner: scanning 1022 ip addresses...
Logs from last 20 minutes
docker logs metrics-exporter --since '20m'
2022-12-20T20:36:18.660521Z INFO exporter:scan: minerctl::scanner: scanning 1022 ip addresses...
2022-12-20T20:36:48.671659Z INFO exporter:scan: minerctl::scanner: discovered 6 miners
2022-12-20T20:41:48.675465Z INFO exporter:scan: minerctl::scanner: scanning 1022 ip addresses...
2022-12-20T20:42:18.691623Z INFO exporter:scan: minerctl::scanner: discovered 6 miners
2022-12-20T20:47:18.695790Z INFO exporter:scan: minerctl::scanner: scanning 1022 ip addresses...
2022-12-20T20:47:48.706334Z INFO exporter:scan: minerctl::scanner: discovered 6 miners
2022-12-20T20:52:48.710557Z INFO exporter:scan: minerctl::scanner: scanning 1022 ip addresses...
2022-12-20T20:53:18.721492Z INFO exporter:scan: minerctl::scanner: discovered 6 miners
Logs since timestamp
docker logs metrics-exporter --since '2022-12-20T20:52:00'
2022-12-20T20:52:48.710557Z INFO exporter:scan: minerctl::scanner: scanning 1022 ip addresses...
2022-12-20T20:53:18.721492Z INFO exporter:scan: minerctl::scanner: discovered 6 miners
Logs in time interval
docker logs metrics-exporter --since '2022-12-20T20:52:00' --until '2022-12-20T20:53:00'
2022-12-20T20:52:48.710557Z INFO exporter:scan: minerctl::scanner: scanning 1022 ip addresses...
Logs for Prometheus and Grafana containers can be queried similarly.