CheapSweep: a script for low-cost address sweeping

K1773R

legendary

Activity: 1792

Merit: 1008

/dev/null

Quote from: salfter on February 05, 2013, 05:57:00 PM

Quote from: K1773R on February 04, 2013, 05:42:18 PM

Quote from: salfter on February 04, 2013, 05:33:31 PM

Quote from: K1773R on February 02, 2013, 12:34:36 AM

i updated the script so it works with bash on linux:

I've been running it on Linux, and bash is my shell. What version of bitcoind are you running? I pulled the most recent changes from git master this morning, as it had been dying on me lately for no good reason.

had nothing to do with bitcoind, the problem was non existing escaping of [ and ", you probably dont run bash, maybe dash?

Nope...bash is a system package on Gentoo. I have v4.2_p37 on most of my boxen (including the mining/bitcoind/litecoind box).

according to bash syntax [ and " have to be escaped, seems to be some wierd bash version lol

EDIT: err, got it confused, didnt mean bash, ment GNU's sed

salfter

hero member

Activity: 651

Merit: 501

My PGP Key: 92C7689C

Quote from: K1773R on February 04, 2013, 05:42:18 PM

Quote from: salfter on February 04, 2013, 05:33:31 PM

Quote from: K1773R on February 02, 2013, 12:34:36 AM

i updated the script so it works with bash on linux:

I've been running it on Linux, and bash is my shell. What version of bitcoind are you running? I pulled the most recent changes from git master this morning, as it had been dying on me lately for no good reason.

had nothing to do with bitcoind, the problem was non existing escaping of [ and ", you probably dont run bash, maybe dash?

Nope...bash is a system package on Gentoo. I have v4.2_p37 on most of my boxen (including the mining/bitcoind/litecoind box).

Prospero

full member

Activity: 194

Merit: 100

It would probably also be a good idea to try to limit the resulting transaction size, considering the latest commit to the bitcoin source decreases the allowed size from 250KB to 100KB: http://git.io/1rNpZw.

K1773R

legendary

Activity: 1792

Merit: 1008

/dev/null

Quote from: salfter on February 04, 2013, 05:33:31 PM

Quote from: K1773R on February 02, 2013, 12:34:36 AM

i updated the script so it works with bash on linux:

I've been running it on Linux, and bash is my shell. What version of bitcoind are you running? I pulled the most recent changes from git master this morning, as it had been dying on me lately for no good reason.

had nothing to do with bitcoind, the problem was non existing escaping of [ and ", you probably dont run bash, maybe dash?

salfter

hero member

Activity: 651

Merit: 501

My PGP Key: 92C7689C

Quote from: K1773R on February 02, 2013, 12:34:36 AM

i updated the script so it works with bash on linux:

I've been running it on Linux, and bash is my shell. What version of bitcoind are you running? I pulled the most recent changes from git master this morning, as it had been dying on me lately for no good reason.

K1773R

legendary

Activity: 1792

Merit: 1008

/dev/null

i updated the script so it works with linux:

Code:

#!/bin/bash

# CheapSweep v0.1
# Scott Alfter
# [email protected]
# Donations: 1TipSAXbE6owdU24bcBDJKmL8JRxQe5Yu

BITCOIND="bitcoind"
#BITCOIND="bitcoind -testnet -rpcport=18334"

help()
{
cat <&2
Usage: $0 [options] -d destaddr addr1 addr2 ...

options: -d|--destaddr destination address (REQUIRED)
   -f|--fee fee to subtract from inputs (default: 0)
   -c|--confirm minimum confirmations to include input (default: 6)
   -n|--no-send dont send; dump the raw transaction to stdout
EOF
}

fee=0.0
minconfirm=6
OPTS=$(getopt -o d:f:c:hn --long destaddr:,fee:,confirm:,help,no-send -- "$@")
eval set -- "$OPTS"
while true; do
  case "$1" in
   -d|--destaddr) destaddr="$2"; shift 2;;
   -f|--fee) fee="$2"; shift 2;;
   -c|--confirm) minconfirm="$2"; shift 2;;
   -n|--no-send) nosend=1; shift 1;;
   -h|--help) help; exit 1;;
   --) shift; break;;
   *) echo "Internal error"; exit 1;;
  esac
done
if [ "$destaddr" == "" ]
then
  help
  exit 1
fi

addrs=$(echo "$*" | sed "s/^/\[\"/;s/ /\",\"/g;s/\$/\"\]/")
echo "$addrs";

total=$($BITCOIND listunspent $minconfirm 21000000 "$addrs" | grep amount | sed "s/.*: //;s/,//" | tr "\n" "+" | sed "s/+\$/\n/" | bc)
total=$(echo $total - $fee | bc)
echo $total;

tx=$($BITCOIND signrawtransaction $($BITCOIND createrawtransaction \[$($BITCOIND listunspent $minconfirm 21000000 "$addrs" | egrep "txid|vout" | sed "s/\"txid/{\"txid/;s/\"vout\" : $[0-9]*$,/\"vout\" : \1},/" | tr -d "\n" | tr -d " " | sed "s/,\$//")\] {\"$destaddr\":$total}) | grep \"hex\" | sed "s/.*: \"//;s/\",//")

if [ "$nosend" != "" ]
then
  echo $tx
else
  $BITCOIND sendrawtransaction $tx
fi

K1773R

legendary

Activity: 1792

Merit: 1008

/dev/null

i just found the problem: [ and ] have to be escaped

Code:

tx=$(bitcoind signrawtransaction $(bitcoind createrawtransaction \[$(bitcoind listunspent $minconfirm 21000000 $addrs | egrep "txid|vout" | sed "s/\"txid/{\"txid/;s/\"vout\" : $[0-9]*$,/\"vout\" : \1},/" | tr -d "\n" | tr -d " " | sed "s/,\$//")\] {\"$destaddr\":$total}) | grep \"hex\" | sed "s/.*: \"//;s/\",//")

K1773R

legendary

Activity: 1792

Merit: 1008

/dev/null

Quote from: Prospero on February 01, 2013, 09:10:13 PM

Quote from: K1773R on February 01, 2013, 07:16:51 PM

unfortunately i cant code in perl, are you planing to complete it?

It turns out the last statement works as is. I didn't test it before because I assumed I had to translate it also. So the only statements I needed to change are in my above post.

EDIT: err, nvm

Prospero

full member

Activity: 194

Merit: 100

Quote from: K1773R on February 01, 2013, 07:16:51 PM

unfortunately i cant code in perl, are you planing to complete it?

It turns out the last statement works as is. I didn't test it before because I assumed I had to translate it also. So the only statements I needed to change are in my above post.

K1773R

legendary

Activity: 1792

Merit: 1008

/dev/null

Quote from: Prospero on January 28, 2013, 04:32:17 PM

I think the issue is that there are many versions of sed and it's hard to maintain portability between the versions. Even replacing sed with gsed (I'm on a Mac) didn't help here. But after translating to perl I got reasonable results. I haven't translated the last call to sed yet.

Code:

# addrs=$(echo $* | sed "s/^/[\"/;s/ /\",\"/g;s/\$/\"]/")
addrs=$(echo $* | perl -ne 'printf "[%s]", join ",", map qq("$_"), split " "')
# total=$(bitcoind listunspent $minconfirm 21000000 "$addrs" | grep amount | sed "s/.*: //;s/,//" | tr "\n" "+" | sed "s/+\$/\n/")
total=$(
bitcoind listunspent $minconfirm 21000000 "$addrs" \
| perl -ne '($a)=/^\s*"amount" : (\d+\.\d+),$/; $a||next;$sum+=$a } END {print $sum'
)

unfortunately i cant code in perl, are you planing to complete it?

Prospero

full member

Activity: 194

Merit: 100

I think the issue is that there are many versions of sed and it's hard to maintain portability between the versions. Even replacing sed with gsed (I'm on a Mac) didn't help here. But after translating to perl I got reasonable results. I haven't translated the last call to sed yet.

Code:

# addrs=$(echo $* | sed "s/^/[\"/;s/ /\",\"/g;s/\$/\"]/")
addrs=$(echo $* | perl -ne 'printf "[%s]", join ",", map qq("$_"), split " "')
# total=$(bitcoind listunspent $minconfirm 21000000 "$addrs" | grep amount | sed "s/.*: //;s/,//" | tr "\n" "+" | sed "s/+\$/\n/")
total=$(
bitcoind listunspent $minconfirm 21000000 "$addrs" \
| perl -ne '($a)=/^\s*"amount" : (\d+\.\d+),$/; $a||next;$sum+=$a } END {print $sum'
)

salfter

hero member

Activity: 651

Merit: 501

My PGP Key: 92C7689C

Quote from: Prospero on January 09, 2013, 02:56:51 PM

I'm using 0.8 beta of bitcoind and the script fails to work because I think the JSON format might have changed. The script looks like it is expecting each "amount: $num" to be on a separate line, but in 0.8 the JSON format is compact so there are no newlines. I think right before `grep amount`, `perl -pe 's/,/,\n/g'` (or equivalent) should be used to add a newline after every comma.

Something must've changed in the past few weeks, as now that I've gotten a chance to test CheapSweep against a beta build, it's working properly. I made no changes to the script...just had it sweep some free coins a little bit ago. The bitcoind I'm running was built from whatever was on GitHub last Friday.

salfter

hero member

Activity: 651

Merit: 501

My PGP Key: 92C7689C

Quote from: etotheipi on January 09, 2013, 11:13:31 PM

Does bitcoind RPC interface broadcast arbitrary transactions for you?

If you create raw transactions, the usual checks AFAICT are bypassed. I've sent transactions up to about 10K or so with tiny fees (BTC0.0001-BTC0.0005). I'm not sure how big my latest zero-fee transactions have been, but they've all gone through within (usually) 1-3 hours.

K1773R

legendary

Activity: 1792

Merit: 1008

/dev/null

just tested this and sadly it dosnt work.
$addrs and $totals arent calucalted correctly, i tested it with testnet (yes i changed the script apropriately):

Code:

$ ./cheapsweep.sh -n -f 0 -d n1vCS8kqNQJSnFu3xA1ma64RafUEZ7oAtJ `bitcoind -testnet -rpcport=18334 listaddressgroupings | grep '",' | sed "s/ \"//" | sed "s/\",//" | xargs echo`
error: Error parsing JSON:c
error: Error parsing JSON:c
error: {"code":-3,"message":"Invalid amount"}
error: {"code":-1,"message":"signrawtransaction [{\"txid\":txid,\"vout\":n,\"scriptPubKey\":hex},...] [,...] [sighashtype=\"ALL\"]\nSign inputs for raw transaction (serialized, hex-encoded).\nSecond optional argument (may be null) is an array of previous transaction outputs that\nthis transaction depends on but may not yet be in the blockchain.\nThird optional argument (may be null) is an array of base58-encoded private\nkeys that, if given, will be the only keys used to sign the transaction.\nFourth optional argument is a string that is one of six values; ALL, NONE, SINGLE or\nALL|ANYONECANPAY, NONE|ANYONECANPAY, SINGLE|ANYONECANPAY.\nReturns json object with keys:\n hex : raw transaction with signature(s) (hex-encoded string)\n complete : 1 if transaction has a complete set of signature (0 if not)"}

$total is 0 and $address is "c"

Prospero

full member

Activity: 194

Merit: 100

It sounds like this ticket might be related to this: https://github.com/bitcoin/bitcoin/issues/1675

etotheipi

legendary

Activity: 1428

Merit: 1093

Core Armory Developer

Does bitcoind RPC interface broadcast arbitrary transactions for you? My understanding is that "stock" bitcoind will not accept such a 50 kB transaction without a substantial fee -- it would be DOA. Obviously other nodes might use different fee rules, but I want to understand why your own bitcoind took it?

Also, the latest version of Armory has coin-control -- but only at the address level. If your coins are split up between lots of addresses, you can do this house-cleaning, though you'll have to pay a full fee, because stock bitcoind won't accept it and forward it if you don't.

salfter

hero member

Activity: 651

Merit: 501

My PGP Key: 92C7689C

Quote from: Prospero on January 09, 2013, 02:56:51 PM

I'm using 0.8 beta of bitcoind and the script fails to work because I think the JSON format might have changed. The script looks like it is expecting each "amount: $num" to be on a separate line, but in 0.8 the JSON format is compact so there are no newlines. I think right before `grep amount`, `perl -pe 's/,/,\n/g'` (or equivalent) should be used to add a newline after every comma.

Hmm...haven't tried that version yet. I'll have to look into it.

Bitsky

hero member

Activity: 576

Merit: 514

If coincontrol would make it into the main client, users could even select which addresses should be defragmented. It shouldn't be too complex to display the fragmentation for each address.
And if the single recipient address is also in the source address list then it should be obvious that it's a coin-combination (which could be fee-free so users agree to it).

DeathAndTaxes

donator

Activity: 1218

Merit: 1079

Gerald Davis

Eventually it would be nice for scripts like this to work their way into the clients. "It appears you wallet has become fragmented. Would you like to consolidate some coins. The coins will be unavailable until confirmed".

Anything that reduces the size of the pruned database is a positive. It would be optimal if miners could detect these types of transactions (where the size of the output is smaller than the size of the input) and allow them as space allows even with no fee.

EVERYONE (including miners) benefits from reducing the size of the pruned database.

Prospero

full member

Activity: 194

Merit: 100

I'm using 0.8 beta of bitcoind and the script fails to work because I think the JSON format might have changed. The script looks like it is expecting each "amount: $num" to be on a separate line, but in 0.8 the JSON format is compact so there are no newlines. I think right before `grep amount`, `perl -pe 's/,/,\n/g'` (or equivalent) should be used to add a newline after every comma.

Topic: CheapSweep: a script for low-cost address sweeping (Read 9003 times)