Here is a script to get notification from telegram. It has some comments expaining what it does and how.
Really the idea is that, only notify if a problem is detected and /or send a standar report.
At the moment the "error and report part" is not finish.. I am looking forward next week to finish that bit.
In the mean time it can be run from cron to send reports.
I don't have Nvoc installed but it should not be problematic to run the script over ubuntu.
I hope that many of you dare to deal with bash scripting and customize the script according to your needs so that I've written so many comments.
I am not a bash/linux guru but I'll be happy to help if I can.
Please read the comments on the script before you play with it and don't run it as root if possible.
Note that, I remove full path to commands in order to make it as much compatible as possible over different linux distros.
#!/bin/bash
# Telegram Info Script
# By BaliMiner et al...
# for nvOC by fullzero
# ref: http://bernaerts.dyndns.org/linux/75-debian/351-debian-send-telegram-notification
# and mod by kk003
# OS: Centos 7 (tested on ubuntu too)
# Script name: telegram-info.sh
# You can run the script from cron like this:
## $ crontab -e (press the "i" key to insert) add this line without the "##":
#
## 00 11 * * * /home/m1/telegram-info.sh >> /home/m1/telegram_rig.log
#
# Then press "ESC" key to scape the insertion mode, then ":" key, then "wq" to Write file and Quit
# The above line wil send you a report every day at 11AM
# You must save the script into root dir of user m1 (/home/m1) for the obove to work
# Don't forget to $ chmod 700 telegram-info.sh
# You must create a bot and get the Chatid + apikey.
# Fallow the link http://bernaerts.dyndns.org/linux/75-debian/351-debian-send-telegram-notification to do so
# and some goggling if you need it
# Data Telegram
CHATID=YOUR CHAT ID GOES HERE # Put here your chat ID
APIKEY="YOUR API KEY GOES HERE" # Put here your Api key
# Number of gpus I know for sure the sistem must have (change this to nº of gpus your system have)
NUMBER_GPUS_FOR_SURE=13
# String I search for to check if I am running claymore
KEY_CLAYMORE="ethdcrminer64" # Exec for Claumor 9.8. Other vesions may have a different name
# You must check if this entry is right for your
# system to get the script runnin correctly
# If claymore is running use the provided data?
USE_CLAYMORE_STATICS="YES" # YES or NO
# Aliases
# Cos telegram cut off the lines I want to make them as short as possible
# Just add the gpu name such as is showed by nvidia-smi follow by the alias you want
# Fields must be space separated
MODELS=("GeForce GTX 1060 3GB" "GF1060_3G" "GeForce GTX 1060 6GB" "GF1060_6G")
# Query arguments for nvidia-smi
# You can get a complete list of the query arguments by issuing: nvidia-smi --help-query-gpu
# Add what you want here. Args must be coma separated
NVIDIA_SMI_QUERY_ARGUMENTS="index,gpu_name,gpu_bus_id,pstate,utilization.gpu,power.draw,temperature.gpu,fan.speed,clocks.mem,clocks.gr,memory.used"
# Error exit codes (or RETURN VALUE as nvidia call them) for nvidia-smi
# I'll treat them as an array so in couples; even value is the code number and odd value is the text that explains it
NVIDIA_SMI_RETURN_VALUE=(
"0" "Success"
"2" "A supplied argument or flag is invalid"
"3" "The requested operation is not available on target device"
"4" "The current user does not have permission to access this device or perform this operation"
"6" "A query to find an object was unsuccessful"
"8" "A device is external power cables are not properly attached"
"9" "NVIDIA driver is not loaded"
"10" "NVIDIA Kernel detected an interrupt issue with a GPU"
"12" "NVML Shared Library couldn't be found or loaded"
"13" "Local version of NVML doesn't implement this function"
"14" "infoROM is corrupted"
"15" "The GPU has fallen off the bus or has otherwise become inaccessible"
"255" "Other error or internal driver error occurred"
)
LF=$'\n' # New line
function my_custom_mods ()
{
# I organize some things my way
# comment out the call to this function and you will get the standar output from nvidia-smi
###
# Beginning of block
###
# Save header and remove it from the csv file
CSV_HEADER=$(head -n 1 nvidia_smi_values.csv) # Save the header of csv file
sed -i -e "1d" nvidia_smi_values.csv # Remove the header (first line of file)
##### Changing the gpu's name for the alias (if I have it)
#
# First must check that elements in the array are a even number (Multiples of 2) and is not empty
MODELS_LENGTH=${#MODELS[@]} # Get number of elements
if [[ $MODELS_LENGTH -ne 0 ]]; then # Is not empty?
if [[ $((MODELS_LENGTH%2)) -eq 0 ]]; then # Number of elements is even?. If so I assume names and alias are well matched in array
for ((INDEX_EVEN=0; INDEX_EVEN < $MODELS_LENGTH ; INDEX_EVEN=INDEX_EVEN+2)) # First element of array have index 0
do # Even elements are the gpu names, odd elements are alias
NAME_GPU_IN_ARRAY=${MODELS[$INDEX_EVEN]} # Get the name of the gpu in array
INDEX_ODD=$(($INDEX_EVEN + 1)) # +1 to get the alias index
ALIAS_GPU_IN_ARRAY=${MODELS[$INDEX_ODD]} # Get the alias of the gpu in array
# Find and replace gpu name if exists
sed -i "s/${NAME_GPU_IN_ARRAY}/${ALIAS_GPU_IN_ARRAY}/g" nvidia_smi_values.csv
done
# I get this way out if number of elements in array are odd and I don't replace names for alias
else
echo "Number of elements in variable MODELS are odd and I don't replace gpu names for alias."
echo "Check variable MODELS if you want to replace names for alias"
fi
# I get this way out if array in variable MODELS is empty
else
echo "Variable MODELS is empty and I don't replace gpu names for alias."
echo "Check variable MODELS if you want to replace names for alias"
fi
##### END of Changing gpu's name by the alias (if I have it)
# I don't want any spaces but to align thing
sed -i 's/ //g' nvidia_smi_values.csv
# Get number of elements on NVIDIA_SMI_QUERY_ARGUMENTS variable
NVIDIA_SMI_QUERY_ARGUMENTS_LENGTH=$(echo $NVIDIA_SMI_QUERY_ARGUMENTS | sed 's/[^,]//g' | wc -c)
# Review all the elements in variable NVIDIA_SMI_QUERY_ARGUMENTS and do some mods about the output for some fields
for ((INDEX=1; INDEX <= $NVIDIA_SMI_QUERY_ARGUMENTS_LENGTH ; INDEX=INDEX+1)) # Gives me the column number
do
NVIDIA_SMI_ARGUMENT=$(echo $NVIDIA_SMI_QUERY_ARGUMENTS | cut -d, -f $INDEX) # Gives me the name of nvidia-smi argument on array
#### Arrange the output for the "index" nvidia-smi argument
#### I want to aling numbers <= 9 if I have more than 9 gpus so I add a space in front
if [[ "$NVIDIA_SMI_ARGUMENT" == "index" && $GPU_COUNT -gt 10 ]]; then
LINES_TO_MODIFY=10
for ((L=1; L <= $LINES_TO_MODIFY ; L=L+1))
do
OLD_VALUE=`sed -n -e "${L}p" nvidia_smi_values.csv | cut -d, -f$INDEX` # Extraigo el dato original
NEW_VALUE=" "$OLD_VALUE
sed -i "$L"'s/^\(\([^,]*,\)\{'"$(($INDEX -1))"'\}\)[^,]*/\1'"$NEW_VALUE"'/' nvidia_smi_values.csv
done
fi
#### END of Arrange the output for the "index" nvidia-smi argument
#### Arrange the output for the "pci.bus_id" nvidia-smi argument
#### I don't want the "00000000:" at the beginning
if [[ "$NVIDIA_SMI_ARGUMENT" == "gpu_bus_id" ]]; then
LINES_TO_MODIFY=$GPU_COUNT
for ((L=1; L <= $LINES_TO_MODIFY ; L=L+1))
do
OLD_VALUE=`sed -n -e "${L}p" nvidia_smi_values.csv | cut -d, -f$INDEX` # Extraigo el dato original
NEW_VALUE=$(echo "$OLD_VALUE" | sed "s/00000000://")
sed -i "$L"'s/^\(\([^,]*,\)\{'"$(($INDEX -1))"'\}\)[^,]*/\1'"$NEW_VALUE"'/' nvidia_smi_values.csv
done
fi
#### END of Arrange the output for the "index" nvidia-smi argument
#### Arrange the output for the "utilization.gpu" nvidia-smi argument
#### I want to aling values to the right and keep the "%"
if [[ "$NVIDIA_SMI_ARGUMENT" == "utilization.gpu" ]]; then
LINES_TO_MODIFY=$GPU_COUNT
for ((L=1; L <= $LINES_TO_MODIFY ; L=L+1))
do
OLD_VALUE=`sed -n -e "${L}p" nvidia_smi_values.csv | cut -d, -f$INDEX` # Extraigo el dato original
LEN=${#OLD_VALUE}
case $LEN in
2) NEW_VALUE=" "$OLD_VALUE
;;
3) NEW_VALUE=" "$OLD_VALUE
;;
*) NEW_VALUE=$OLD_VALUE
;;
esac
sed -i "$L"'s/^\(\([^,]*,\)\{'"$(($INDEX -1))"'\}\)[^,]*/\1'"$NEW_VALUE"'/' nvidia_smi_values.csv
done
fi
#### END of Arrange the output for the "utilization.gpu" nvidia-smi argument
#### Arrange the output for the "temperature.gpu" nvidia-smi argument
#### I want to aling values to the right and add the "ºC"
if [[ "$NVIDIA_SMI_ARGUMENT" == "temperature.gpu" ]]; then
LINES_TO_MODIFY=$GPU_COUNT
for ((L=1; L <= $LINES_TO_MODIFY ; L=L+1))
do
OLD_VALUE=`sed -n -e "${L}p" nvidia_smi_values.csv | cut -d, -f$INDEX` # Extraigo el dato original
LEN=${#OLD_VALUE}
CENTIGRADO="ºC"
case $LEN in
1) NEW_VALUE=" "$OLD_VALUE$CENTIGRADO
;;
2) NEW_VALUE=" "$OLD_VALUE$CENTIGRADO
;;
*) NEW_VALUE=$OLD_VALUE$CENTIGRADO
;;
esac
sed -i "$L"'s/^\(\([^,]*,\)\{'"$(($INDEX -1))"'\}\)[^,]*/\1'"$NEW_VALUE"'/' nvidia_smi_values.csv
done
fi
#### END of Arrange the output for the "temperature.gpu" nvidia-smi argument
done
# Insert the whole header line as the first one
sed -i "1s/^/$CSV_HEADER\n/" nvidia_smi_values.csv
###
# End of block with custom my mods
###
}
function claymore_statics ()
{
#### Arrange the output for claymore statics
if [[ "$USE_CLAYMORE_STATICS" == "YES" ]]; then
# Check if claymore is running
if pgrep -x "$KEY_CLAYMORE" > /dev/null
then
CLAYMORE_IS_RUNNING="YES"
echo
echo "It seems that Claymore is running!!"
#
# Don't know why http://localhost:3333 access forces claymore's output to show up a line like:
# "GPU #XX: GeForce GTX 1060 3GB, 3013 MB available, 9 compute units, capability: 6.1", one line x gpu in the system
# Get the latest total hashrates
LATEST_TOTAL_HASHRATES=`curl -s http://localhost:3333 | sed '/Total/!d; /Speed/!d;' | awk '{print $6}'`
echo -e "Latest Total Hashrates :\n$LATEST_TOTAL_HASHRATES" > latest_total_hashrates.txt
# Get the indivual gpus hashrates and remove some charaters and spaces I don't want
GPUS_HASHRATES=`curl -s http://localhost:3333 | sed '/ETH: GPU/!d' | awk 'NR == 3' | sed 's/[^GPU]*\(GPU.*\)/\1/' | sed 's/ M/M/g' | sed 's/ G/G/g'`
# Make the string with individual hashrates vertical replacing "," for new line "\n" and removing the "GPUX" part
echo $GPUS_HASHRATES | sed "s/,/\n/g" | sed 's/^[^ ]* //' > gpus_hashrate.txt
# Get total individual and rejected shares and for how long claymore is runing in one single string
# For 13 gpus should look something like this:
# Speed: 318.710 Mh/s, Total Shares: 15835(1233+1212+1226+1256+1216+1243+1228+1303+1220+1177+1237+1209+1222), Rejected: 0, Time: 55:32
#
SOME_GPUS_DATA_FROM_CLAYMORE=`curl -s http://localhost:3333 | sed '/ETH - Total/!d; /Speed/!d; /(/!d; /+/!d; s/[^Speed]*\(Speed.*\)/\1/;' | awk 'NR == 3'`
# Get the total shares
TOTAL_SHARES=$(echo $SOME_GPUS_DATA_FROM_CLAYMORE | cut -d"(" -f 1 | cut -d" " -f 6)
# Get the individual shares (I capture the values between the two parentheses)
# and replace the "+" signs for new line so I set the data vertical
echo $SOME_GPUS_DATA_FROM_CLAYMORE | sed 's/^.*(//; s/).*$//; s/+/\n/g' > individual_shares.txt
# Get the rejected shares and remove de ","
REJECTED_SHARES=$(echo $SOME_GPUS_DATA_FROM_CLAYMORE | cut -d" " -f 8 | sed 's/,//')
# How long claymore has been running?
TIME_CLAYMORE_IS_RUNNING=$(echo $SOME_GPUS_DATA_FROM_CLAYMORE | cut -d" " -f 10)
# Creating a vertical gpux list
# If the file exists and is not empty clear it
if [[ -s v_gpus.txt ]]; then
> v_gpus.txt
fi
LINES_TO_ADD=$(($GPU_COUNT - 1))
for ((L=0; L <= $LINES_TO_ADD ; L=L+1))
do
case "$L" in
0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 )
echo "GPU $L" >> v_gpus.txt
;;
*)
echo "GPU$L" >> v_gpus.txt
;;
esac
done
# Paste the whole vertical thing together
paste -d, v_gpus.txt individual_shares.txt gpus_hashrate.txt > claymore_statics.tmp
# Mount the header field for claymore statics
CLAYMOR_STATICS_HEADER="Index, Shares, Hashrate"
# Insert the whole header line as the first one
sed -i "1s/^/$CLAYMOR_STATICS_HEADER\n/" claymore_statics.tmp
# Insert a new line on top
sed -i "1s/^/\n/" claymore_statics.tmp
# Add the list of latest total hashrates and put it all togather
cat latest_total_hashrates.txt claymore_statics.tmp > claymore_statics.txt
# Insert the Title for claymore mining information
sed -i "1s/^/** Claymore mining Information **\n/" claymore_statics.txt
sed -i "1s/^/\n/" claymore_statics.txt
# At the end of file add
echo " " >> claymore_statics.txt
echo "Total shares :" $TOTAL_SHARES >> claymore_statics.txt
echo "Rejected shares :" $REJECTED_SHARES >> claymore_statics.txt
echo "Claymore up time :" $TIME_CLAYMORE_IS_RUNNING >> claymore_statics.txt
else
echo "Claymore does not seem to be running!!. Skiping this bit."
CLAYMORE_IS_RUNNING="NO"
fi
fi
#### END of Arrange the output for claymore statics
}
function salida ()
{
END_TIME=`date "+%Y-%m-%d %H:%M:%S"`
echo "$MSG"
echo
echo "End time: " $END_TIME
echo
echo "**************************"
echo
exit
}
START_TIME=`date "+%Y-%m-%d %H:%M:%S"`
echo "Start time: " $START_TIME
# Is the script already running?. If so exit
echo -n "Process ID :"
if pidof -x $(basename $0); then
for p in $(pidof -x $(basename $0)); do
if [ $p -ne $$ ]; then
echo "$0 already running. Exiting..."
salida # exit
fi
done
fi
#
# Collect some info from the system
#
# Clear the file's content if exists and is not empty
if [[ -s system_data.txt ]]; then
> system_data.txt
fi
# Title for system information
echo "** System information **" >> system_data.txt
# Hostname
echo -n "Hostname : " >> system_data.txt
hostname >> system_data.txt
# Ips
echo -n "Ips : " >> system_data.txt
hostname -I >> system_data.txt
#
# Up since
echo -n "Up since : " >> system_data.txt
uptime -s >> system_data.txt
#
# Up time
echo -n "Up time : " >> system_data.txt
uptime >> system_data.txt
#
# who is in the system
echo "Connected users : " >> system_data.txt
who >> system_data.txt
# Here I check if there is an open sesion on any display (like display 0). If not may be the x server is down
DISPLAY=$(who | grep "(:")
if [[ -z "$DISPLAY" ]]; then
echo "Not a user on any Display. May be X server is NOT running" >> system_data.txt
fi
echo "" >> system_data.txt
#
# Number of gpus nvidia-smi reports
#
GPU_COUNT=$(nvidia-smi --query-gpu=count --format=csv,noheader,nounits | tail -1)
echo "System is reporting : $GPU_COUNT Gpus" >> system_data.txt
echo "System should have : $NUMBER_GPUS_FOR_SURE Gpus" >> system_data.txt
if [[ $GPU_COUNT -ne $NUMBER_GPUS_FOR_SURE ]]; then
echo "MAY BE A PROBLEM; we should have $NUMBER_GPUS_FOR_SURE GPUs but nvidia-smi only reports $GPU_COUNT" >> system_data.txt
fi
echo "" >> system_data.txt
# Get the gpus's info I am interested in
nvidia-smi --query-gpu=$NVIDIA_SMI_QUERY_ARGUMENTS --format=csv -f nvidia_smi_values.csv
# I organize some things my way. Just comment out the call to the function and/or
# delete the whole function if you don't like it and you will get the standar output from nvidia-smi
my_custom_mods # Comment out this line if you don't want my custom mods
# Insert the Title for nvidia-smi report
sed -i "1s/^/** Nvidia-smi Information **\n/" nvidia_smi_values.csv
# Peek up the claymore statics if configured so
claymore_statics
# Put the System info+nvidia-smi+mining info together
if [[ "$USE_CLAYMORE_STATICS" == "YES" && "$CLAYMORE_IS_RUNNING" == "YES" ]]; then
cat system_data.txt nvidia_smi_values.csv claymore_statics.txt > report.txt
else
cat system_data.txt nvidia_smi_values.csv > report.txt
fi
CONTENT=$(cat report.txt)
MSG="$CONTENT"
curl -s -X POST --output /dev/null https://api.telegram.org/bot${APIKEY}/sendMessage -d "text=${MSG}" -d chat_id=${CHATID}
CODE_CURL=$?
# Check if the mgs went out ok
if [[ $CODE_CURL -eq 0 ]]; then
echo "Msg send ok!!"
else
echo "Got a non zero exit code from curl : $CODE_CURL . The message has probably not reached its destination"
fi
salida # exit