overflow bug SERIOUS

seems a block at height 74638 has expoited a bug in the net. It uses an integer overflow to make a negative total transaction. The two transaction outputs are:

out Value:92233720368.54(7ffffffffff85ee0) out Value:92233720368.54(7ff ffffffff85ee0)

We need a fix asap

Edit: (satoshi) 0.3.10 patch download links here: topic 827

Until there is a better fix… after a very small amount of testing this seems to work:

--- a/main.h
+++ b/main.h
@@ -473,8 +473,12 @@ public:

         // Check for negative values
         foreach(const CTxOut& txout, vout)
+ {
             if (txout.nValue < 0)
                 return error("CTransaction::CheckTransaction() : txout.nValue negative");
+ if (txout.nValue > 21000000*COIN)
+ return error("CTransaction::CheckTransaction() : txout.nValue over-max");
+ }

         if (IsCoinBase())
         {
@@ -520,6 +524,8 @@ public:
         int64 nValueOut = 0;
         foreach(const CTxOut& txout, vout)
         {
+ if (txout.nValue > 21000000*COIN)
+ continue; // ignore over-max-value...
             if (txout.nValue < 0)
                 throw runtime_error("CTransaction::GetValueOut() : negative value");
             nValueOut += txout.nValue;

You’ll need to re-download the part of the block chain before the bad block— remove the blkindex.dat and blk0001.dat files. I started with knightmb’s blockchain snapshot.

Or the same patch from your github, http://gist.github.com/525921 which gives us the raw patch that can be applied directly, http://gist.github.com/raw/525921/fe2ad7583f0dd2444caa0b3e24d750bf45cac11b/Quick%20fix%20block%2074652

EDIT: I was unable to patch directly using this, perhaps CRLF problems.

I applied your changes manually, and uploaded the result here: http://yyz.us/bitcoin/patch.bitcoin-gavin-overflow-quick-fix

Here’s the preliminary change.  Look right?  I have more changes to make, this isn’t all of it.  Will SVN shortly.

    bool CheckTransaction() const
    {
        // Basic checks that don't depend on any context
        if (vin.empty() || vout.empty())
            return error("CTransaction::CheckTransaction() : vin or vout empty");

        // Check for negative and overflow values
        int64 nTotal = 0;
        foreach(const CTxOut& txout, vout)
        {
            if (txout.nValue < 0)
                return error("CTransaction::CheckTransaction() : txout.nValue negative");
            if (txout.nValue > 21000000 * COIN)
                return error("CTransaction::CheckTransaction() : txout.nValue too high");
            nTotal += txout.nValue;
            if (nTotal > 21000000 * COIN)
                return error("CTransaction::CheckTransaction() : txout total too high");
        }

        if (IsCoinBase())
        {
            if (vin[0].scriptSig.size() < 2 || vin[0].scriptSig.size() > 100)
                return error("CTransaction::CheckTransaction() : coinbase script size");
        }
        else
        {
            foreach(const CTxIn& txin, vin)
                if (txin.prevout.IsNull())
                    return error("CTransaction::CheckTransaction() : prevout is null");
        }

        return true;
    }

Don’t sticky the topic, nobody looks up there.  There’ll be enough posts to bump.

It would help if people stop generating.  We will probably need to re-do a branch around the current one, and the less you generate the faster that will be.

A first patch will be in SVN rev 132.  It’s not uploaded yet.  I’m pushing some other misc changes out of the way first, then I’ll upload the patch for this.

I’m afraid the community is just too big and distributed now to expect much in the way of voluntary quick action on anything, especially generation which I’m sure many have on automatic and largely unmoderated.

Looks good to me.

Can you easily hardcode a check for the bad block’s hash at startup and orphan it and subsequent blocks if they’re on the best-block chain? It’s painful to have to re-download all or most of the chain to fix this…

Looks good to me.

Can you easily hardcode a check for the bad block’s hash at startup and orphan it and subsequent blocks if they’re on the best-block chain? It’s painful to have to re-download all or most of the chain to fix this…

Or just a quick re-verification of all blocks. After all, it is 5x faster than it used t be. 😉

Once you have an update, you could download knightmb’s block chain. You’ll want one that’s old enough that it ends before block 74000 so the most recent security lockin will check it. Can someone find the link for that?

http://knightmb.dyndns.org/files/bitcoin/blocks/

I prefer to just re-download them.

Block verification after the patch but before everyone upgrades is going to be SLOW! It’ll probably cause the next difficulty adjustment to decrease significantly. Of course everyone will probably have upgraded by the time the next adjustment rolls around, so we’ll probably roll through it relatively quickly.

Edit: Hmm, I see there is an info file included which might just clue me in.

How about explaining to us stupid newbies what we would do with the blockchain once we download it?

How about explaining to us stupid newbies what we would do with the blockchain once we download it?

It means replacing files in the bitcoin data directory. It is not recommended, unless you know what you’re doing.

Easiest and safest way is to backup wallet.dat then redownload everything.

Patch is uploaded to SVN rev 132!

For now, recommended steps:

1. Shut down.
2. Download knightmb’s blk files.  (replace your blk0001.dat and blkindex.dat files)
3. Upgrade.
4. It should start out with less than 74000 blocks. Let it redownload the rest.

If you don’t want to use knightmb’s files, you could just delete your blk*.dat files, but it’s going to be a lot of load on the network if everyone is downloading the whole block index at once.

I’ll build releases shortly.

How about explaining to us stupid newbies what we would do with the blockchain once we download it?

While Bitcoin is not running, you put it in your Bitcoin data directory. ~/.bitcoin on Linux. If you want to re-download the whole chain, you just delete the file from the data directory while Bitcoin is not running.

I’m going to be move my blk00x.dat files and blkindex.dat file out of by data directory and restart the client. Those are the correct files, right?

Edit: I’ll also backup the whole directory, like I usually do before upgrades.

If you don’t want to use knightmb’s files, you could just delete your blk*.dat files, but it’s going to be a lot of load on the network if everyone is downloading the whole block index at once.

Anybody wanna volunteer to create blk*.dat for block chain <= 64637 ?

Maybe the official binaries could simply ship a known-good block chain, to save time and bandwidth?

Will the bug fix include the 4-way SSE2 patch of 0.3.9 rc2? Thanks for letting me know that it is included, theymos. Please release another release candidate when you have a moment if it is not included.

What about the transactions from 74000 to the invalid block. Are those all invalid now as well?

Only the blocks including and after the invalid block are invalid. All previous blocks are valid.

i’v got one of a w7-machine count 74602, if that helps.

[Deleted] Quote from: davidonpda on August 15, 2010, 10:41:29 PM

I already said that I have it right at 74637.

Satoshi said it needs to be before 74,000

So long as a the person who creates it has a patched client and downloads the whole chain, optimally from another computer on their local network, then it should stop at the right place automatically.

[Deleted] Quote from: davidonpda on August 15, 2010, 10:41:29 PM

Satoshi said it needs to be before 74,000

He didn’t, he said “you’ll want one”, not that it needs to be, a little different. and if ”..Only the blocks including and after the invalid block are invalid. All previous blocks are valid..”, it doesn’t need to be before 74000.

just wanted to save knightmb some time, but yours is bigger than mine anyway. 😄

Don’t update the block chain download.  When you take someone’s block chain download, you don’t want it right up to the end.  A somewhat old one is better so it can download and verify the most recent blocks.

tcatm’s 4-way SSE2 SHA-256 is in the file sha256.cpp and already uploaded a few revs ago.

I just now uploaded rev 134 which is the makefile.unix that enables building with it on Linux.  If you build rev 134 on Linux now you’ll get the -4way switch.

If you have problems building because of it, then edit makefile.unix and:

• remove -DFOURWAYSSE2
• remove obj/sha256.o from the end of these lines: bitcoin: $(OBJS) obj/ui.o obj/uibase.o obj/sha256.o bitcoind: $(OBJS:obj/%=obj/nogui/%) obj/sha256.o

The 0.3.10 linux build will have the -4way option when I build it.

Here are the patch downloads for Windows:

http://www.bitcoin.org/download/bitcoin-0.3.10-win32-setup.exe http://www.bitcoin.org/download/bitcoin-0.3.10-win32.zip

SHA1 16645ec5fcdb35bc54bc7195309a1a81105242bb bitcoin-0.3.10-win32-setup.exe SHA1 4f35ad7711a38fe8c880c6c9beab430824c426d3 bitcoin-0.3.10-win32.zip

Steps:

1. Shut down.
2. Download knightmb’s blk files and replace your blk0001.dat and blkindex.dat files. http://knightmb.dyndns.org/files/bitcoin/blocks/ http://rapidshare.com/files/413168038/BitcoinBlocks.torrent
3. Upgrade to 0.3.10.
4. It should start out with less than 74000 blocks and redownload the rest.

Or if you don’t want to mess with downloading blk files, you can just do this:

1. Shut down.
2. Delete (or move) blk*.dat
3. Upgrade to 0.3.10.
4. It redownloads all blocks, probably take about an hour.

Yeah, a little confusing 😕

So I don’t need to create one, or do I need to take what you guys already have and just throw it up on the FTP?

Can you dump block data from a windows machine onto a Linux/Mac machine?

[edit] Just saw your post, I’ll build one to less than 74,000 then, should at least save you technical people a few minutes of downloading the new chain. 😉

I can’t stand the suspense! Do any Jr. Members or higher happen to have an unofficial SVN rev 134 Linux 64-bit build compiled that they’d be willing to share? Yeah, I know the official build is about to be released in a few minutes. It’s not terribly important.

Edit: Updated the SVN version number as per satoshi’s comment a moment ago.

Ah, never mind. Since the SVN version was just updated and the Windows builds were just released, I’ll wait. 😳

[edit] Just saw your post, I’ll build one to less than 74,000 then, should at least save you technical people a few minutes of downloading the new chain.  

Just leave the old one alone!  Older is better.  What block number is it?  Anywhere from 60000-74000 is good.  The one that you’ve had available for a while has been vetted and is the best choice.

I just used your old file knightmb and on a rather old system it didn’t take me very long to catch up, so I’m inclined to say “don’t bother”.

What block number is it?

It’s around 67k. It only took me about 3 minutes to get to the current block.

I think that you should add something about this: http://bitcointalk.org/index.php?topic=259.0 There must be a label on the client that show a warning message if needed 😊 Now everyone have always to check the website, and I think that this is bad.

Cool, works for me! 😄

Starting at 67000 is perfect.  

Yeah, at the moment you’ll stop at 74638.  It should start slowly creeping up as more nodes upgrade and generate.

Linux build links below.

The Linux version includes tcatm’s 4-way SSE2 SHA-256 that makes generating faster on i5 and AMD CPU’s.  Use the “-4way” switch to enable it and check if it’s faster for you.

Download links: http://www.bitcoin.org/download/bitcoin-0.3.10-win32-setup.exe http://www.bitcoin.org/download/bitcoin-0.3.10-win32.zip http://www.bitcoin.org/download/bitcoin-0.3.10-linux.tar.gz

SHA1 16645ec5fcdb35bc54bc7195309a1a81105242bb bitcoin-0.3.10-win32-setup.exe SHA1 4f35ad7711a38fe8c880c6c9beab430824c426d3 bitcoin-0.3.10-win32.zip SHA1 e3fda1ddb31b0d5c35156cacd80dee6ea6ae6423 bitcoin-0.3.10-linux.tar.gz

I think that you should add something about this: http://bitcointalk.org/index.php?topic=259.0

There must be a label on the client that show a warning message if needed Now everyone have always to check the website, and I think that this is bad. Agree, wanted to do that for a long time, haven’t had time to do it.

For now, you could also subscribe to the bitcoin-list mailing list.  It rarely gets used except for announcements like this and major new versions.

Subscribe/unsubscribe page: http://lists.sourceforge.net/mailman/listinfo/bitcoin-list

Quote from: knightmb on August 15, 2010, 10:59:04 PM UTC

[edit] Just saw your post, I’ll build one to less than 74,000 then, should at least save you technical people a few minutes of downloading the new chain.

Just leave the old one alone! Older is better. What block number is it? Anywhere from 60000-74000 is good. The one that you’ve had available for a while has been vetted and is the best choice.

I’ve put the block number with the file, so you’ll know exactly where each stops at.

BitCoinBlocks_Linux_67309.zip BitCoinBlocks_Windows_67300.zip

I’ve leave them be. Glad to know I could help 😊

… It already is on block 74638. I assume that means that block is now a good one?

I had some confusion on this myself and got clarification in #bitcoin-dev:

The bad block was number 74638, the last good one was 74637. The numbers start at 0, so when your client shows there are 74638 blocks then that means you have up to block number 74637, the last good one.

Agree, wanted to do that for a long time, haven’t had time to do it.

For now, you could also subscribe to the bitcoin-list mailing list. It rarely gets used except for announcements like this and major new versions.

Subscribe/unsubscribe page: http://lists.sourceforge.net/mailman/listinfo/bitcoin-list

Good 😄

Anyway I usually check the website, I was thinking to a solution for newbie-users ( that they are more to come on future ) A small bug can be a big problem with a network with a lot of newbie and/or not-informed users. ( and none of them will ever give a look to a source like this: http://lists.sourceforge.net/mailman/listinfo/bitcoin-list 😛 )

( sorry for my english, I hope you are still understanding me 😁 )

my forwarded node counts 74642 as i type (36conn.), 3 other nodes are still at 74638 (8conn. each)

Ya know, in a way, this actually temporarily decreases the difficulty to generate blocks until everyone upgrades. Yeah, it’ll be slower, but that just gives each client more time to try to generate the difficult hash.

Question about fallout: I had a transaction that I submitted after the bad block, using the bad block chain.

What is the status of that transaction? From what I can tell, my (updated) sending client wallet shows the deducted amount.

Will it get reincorporated into the fixed chain, and will the recipient be able to spend it?

I did all steps, now my client is 0.3.10 and it stopped at block 74638. Is all fine?

Question about fallout:  I had a transaction that I submitted after the bad block, using the bad block chain.

What is the status of that transaction? From what I can tell, my (updated) sending client wallet shows the deducted amount.

Will it get reincorporated into the fixed chain, and will the recipient be able to spend it? Right, it will get reincorporated into the fixed chain.  The transaction won’t disappear, it’ll still be visible on both sides, but the confirmation count will jump back to 0 and start counting up again.

It’s only if you generated a block in the bad chain after block 74638 that the 50 BTC from that will disappear.  Any blocks in the bad chain wouldn’t have matured yet.

I did all steps, now my client is 0.3.10 and it stopped at block 74638. Is all fine?

If you still show 74638 blocks then you aren’t connected to any 0.3.10 nodes.  

For today, try adding these parameters:  -addnode=75.158.131.108 -addnode=99.27.237.13 -addnode=68.68.99.14

See http://bitcointalk.org/index.php?topic=828

Most people running clients are not reading this message thread. So… Silly questions:

1. How will this continue to affect version 3.8.1 (pre-catastrophe) clients with bad block chain?
2. How will this affect clients that upgrade to 3.8.10 but don’t remove their block chain files?

I added the addnode options and info about the ‘stuck at 74638 blocks’ issue at topic 823

Most people running clients are not reading this message thread.  So…  Silly questions:

1. How will this continue to affect version 3.8.1 (pre-catastrophe) clients with bad block chain?
2. How will this affect clients that upgrade to 3.8.10 but don’t remove their block chain files?
3. Once more than 50% of the node power is upgraded and the good chain overtakes the bad, the 0.3.10 nodes will make it hard for any bad transactions to get any confirmations.
4. If you didn’t remove your blk*.dat files, you’re not helping to contribute to that 50%, and you’ll still show bad transactions until the good chain overtakes the bad chain.

Ya know, in a way, this actually temporarily decreases the difficulty to generate blocks until everyone upgrades. Yeah, it’ll be slower, but that just gives each client more time to try to generate the difficult hash.

Of course, this happened right after the difficulty kicked up to 511.7, so all these new “good” blocks will have to contend with that. And a good chunk of the network grind power is off on the wrong chain, so new blocks may take a lot longer than 10min to find. At what point will the balancing figure this out and possibly kick the difficulty back down again?

The bad chain is also slowed down as more nodes upgrade.

We’ve already generated 14 blocks since 74638. The builds of 0.3.10 were uploaded about 2 and 3 hours ago. Of the nodes I’m connected to, more than half are already 0.3.10. I would say we probably already have more power than the bad chain.

I’m chucking as much CPU at this as I can. Yeah, bit of an unfair advantage I guess until everyone upgrades. 🤐 My wife’s PC already generated 2 of the new blocks (LOL), luck is on her side I guess.

… Of the nodes I’m connected to, more than half are already 0.3.10. …

That’s great news! How did you figure that out, though? I was considering writing a client specially tailored to put itself out there and try to get as many people to connect as possible, then survey their versions and write out a log (in addition to all the normal bitcoin stuff).

On Windows, findstr /c:“version message” debug.log

It looks like the bad chain was on block 74678 recently.  Can’t wait to overtake it.

On the stats at http://nullvoid.org/bitcoin/statistix.php  there’s been 5 blocks per hour in the last 3 hours.  We had a difficulty adjustment about a day ago that should have put it back to 6 blocks per hour.

The bad chain is also slowed down as more nodes upgrade.

We’ve already generated 14 blocks since 74638. The builds of 0.3.10 were uploaded about 2 and 3 hours ago. Of the nodes I’m connected to, more than half are already 0.3.10. I would say we probably already have more power than the bad chain.

I think it’d probably be a good idea still to come out with another version that rejects connections from older versions - otherwise the network might remain rather fragmented for a while. :/

4b. Currently the network has a combination of good and bad nodes (those running versions older than 0.3.10). If you only connect to nodes running old versions, you will be stuck at 74638 blocks. You can fix this by running the client with the options “-addnode=75.158.131.108 -addnode=99.27.237.13 -addnode=76.235.234.64 -addnode=74.137.15.169 -addnode=68.68.99.14”

http://www.bitcoin.org/wiki/doku.php?do=show&id=74638_nodes

Quote from: satoshi on August 16, 2010, 2:16:10 AM UTC

The bad chain is also slowed down as more nodes upgrade.

We’ve already generated 14 blocks since 74638. The builds of 0.3.10 were uploaded about 2 and 3 hours ago. Of the nodes I’m connected to, more than half are already 0.3.10. I would say we probably already have more power than the bad chain.

I think it’d probably be a good idea still to come out with another version that rejects connections from older versions - otherwise the network might remain rather fragmented for a while. :/

We want to keep the older clients connected so that when the correct chain overtakes the incorrect chain, they will switch back to the correct chain. Although I don’t know the specifics of how far back in the chain the old chain will accept a branched chain.

Quote from: bdonlan on August 16, 2010, 2:39:55 AM UTC

Quote from: satoshi on August 16, 2010, 2:16:10 AM UTC

The bad chain is also slowed down as more nodes upgrade.

We’ve already generated 14 blocks since 74638. The builds of 0.3.10 were uploaded about 2 and 3 hours ago. Of the nodes I’m connected to, more than half are already 0.3.10. I would say we probably already have more power than the bad chain.

I think it’d probably be a good idea still to come out with another version that rejects connections from older versions - otherwise the network might remain rather fragmented for a while. :/

We want to keep the older clients connected so that when the correct chain overtakes the incorrect chain, they will switch back to the correct chain. Although I don’t know the specifics of how far back in the chain the old chain will accept a branched chain.

The old clients should accept it all the way back to the last snapshot of release, so this being found so quickly and so long after the last release, it should work in theory. Here’s a good test of the theory 😁

Quote from: Ground Loop on August 16, 2010, 12:29:55 AM UTC

Question about fallout: I had a transaction that I submitted after the bad block, using the bad block chain.

What is the status of that transaction? From what I can tell, my (updated) sending client wallet shows the deducted amount.

Will it get reincorporated into the fixed chain, and will the recipient be able to spend it?

Right, it will get reincorporated into the fixed chain. The transaction won’t disappear, it’ll still be visible on both sides, but the confirmation count will jump back to 0 and start counting up again.

It’s only if you generated a block in the bad chain after block 74638 that the 50 BTC from that will disappear. Any blocks in the bad chain wouldn’t have matured yet.

Interesting.. fascinating to watch this work its way through the bowels of the system. It was block 73746 that I generated. CTxOut(nValue=50.00000000, scriptPubKey=0x8DDD5C7DADB2D43AC5F186) 08/12/10 02:35 generated 50.00

And I sent 49.00 of it to 19Nzg21hQQDAY5GTdTTuUVPA4MaE7AusXz (using the broken chain)

Now I’m waiting for that node to figure out it was received, and it’s using the new chain.

Would it be useful to have RPC command to chop off the tail of the chain manually, instead of removind and redownloading the whole chain?

So at this point, would a client that has not upgraded also have the correct chain?

So at this point, would a client that has not upgraded also have the correct chain?

Yes. The only exception would be if an unpatched client made another fake transaction and managed to verify it. They would be able to spread that fake block to a few other nodes that haven’t upgraded, but the upgraded clients seem to have more power than the unpatched clients, so the bad link would not last long and would not be spread by upgraded clients.

Un-upgraded nodes have the correct chain most of the time, but they are still trying to include the overflow transaction in every block, so they’re continually trying to fork and generate invalid blocks.  If an old version node is restarted, its transaction pool is emptied, so it may generate valid blocks for a while until the transaction gets broadcast again.  0.3.9 and lower nodes still must upgrade.

The SVN now has the code we needed to automatically reorg the block chain without having to delete the blk*.dat files manually.  I knew I couldn’t write that code fast and carefully enough yesterday, so I went with the quick manual option.