Author

Topic: [Guide] Compiling for Window (Read 7045 times)

legendary
Activity: 1148
Merit: 1018
It's about time -- All merrit accepted !!!
October 08, 2014, 09:34:42 PM
#12
a bit old but still a lot of useful information here, thank you very much
newbie
Activity: 47
Merit: 0
June 03, 2012, 10:15:39 PM
#11
Thank you @Onichan - this guide is very helpful, though it took me the whole day to accomplish the mission.

And indeed, the content of bitcoin-qt.exe and bitcoind.exe at the end is different from the released one, though the sizes are just the same.
So I don't know what to think about this...

What I can add to this is that I wasn't able to make the kvm working on VirtualBox, so I had to switch to VMWare.

And then, it's really annoying that you need to restart the system every time before doing the new "sudo bin/gbuild ...."

But anyway, you are the hero! Smiley

So I finished testing VirtualBox and I couldn't get hardware acceleration to work either so it took 3-4 hours just to compile 6.2 main source, I just used the dependencies from before. I did end up with the same installer hash so it should work for those people wanting to use VirtualBox even if they can't use kvm, but its going to take all day. I didn't spend too much time looking into fixing hardware acceleration for VirtualBox as I'm not really a fan to begin with as it doesn't have drag and drop and it doesn't run as smooth or fast as VMware.

I do agree it can be a little annoying having to reboot, but it only takes about 20 seconds for me so it's not that bad. Now you might be able to do it without rebooting, but I found that is just much easier and has better success rate then trying to ensure the gitian virtual is stopped.
newbie
Activity: 47
Merit: 0
June 03, 2012, 04:53:38 PM
#10
Thanks for checking that. Then I'm assuming you also get the same bitcoin as I do
Code:
1253937b0bb3aa899f5a9a1239349064 *bitcoin-qt.exe
a442dda7eee1d88ffa38ee34d2631f45 *bitcoind.exe
Which means: whoever releases the Windows binaries, he must do it somehow else...
Those are the same hashes I have.

Also I am not sure what the build.assert file is, but there is a /gitian/result/bitcoin-res.yml file that looks like it has sha256 hashes of all files used during compile. It's over 600 lines so I don't really wanna just paste it here, but that would probably be good to compare ours with the dev that builds the official release.


Edit: So I just built 5.5 from https://bitcointalksearch.org/topic/version-055-and-046-released-79651 and my installer hash matches Luke's so the source used for the windows installer found at http://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.6.2/ must have been different then whats tagged as 6.2 on github.
legendary
Activity: 2053
Merit: 1356
aka tonikt
June 03, 2012, 04:05:28 PM
#9
If it were written in pure Assembly there wouldn't be any compiling issues.  Grin
sure - only linking issues then... Grin
full member
Activity: 196
Merit: 100
Web Dev, Db Admin, Computer Technician
June 03, 2012, 04:03:46 PM
#8
If it were written in pure Assembly there wouldn't be any compiling issues.  Grin
legendary
Activity: 2053
Merit: 1356
aka tonikt
June 03, 2012, 03:35:46 PM
#7
Thanks for checking that. Then I'm assuming you also get the same bitcoin as I do
Code:
1253937b0bb3aa899f5a9a1239349064 *bitcoin-qt.exe
a442dda7eee1d88ffa38ee34d2631f45 *bitcoind.exe
Which means: whoever releases the Windows binaries, he must do it somehow else...
newbie
Activity: 47
Merit: 0
June 03, 2012, 03:33:04 PM
#6
I don't know where to find the bitcoin-build.assert, but here are md5 sums for the files in my "input" dir before I build the actual client:
Code:
3da3128ef77a8cb610076dfb7ff8f286  bitcoin-deps-0.0.4.zip
a2dc343f7bc7f83f8941e47ed4a18200  boost_1_47_0.tar.bz2
22b63379de8c9387e3bcf90308564c00  boost-win32-1.47.0-gitian.zip
a14a5486d6b4891d2434039a0ed4c5b7  db-4.8.30.NC.tar.gz
c740ba66cd7074ba2471b6a4ff48e1fb  libpng-1.5.9.tar.gz
88055f2d4a061cfd4cfe25a9eae22f67  miniupnpc-1.6.tar.gz
a1da58ce63baef3812004714fa302c47  openssl-1.0.1b.tar.gz
7e90615eb314abcd2eb2eab5c8155b97  qrencode-3.2.0.tar.bz2
9831cf1dfa8d0689a06c2c54c5c65aaf  qt-everywhere-opensource-src-4.7.4.tar.gz
71fb76765dc6734a1940176527b619bd  qt-win32-4.7.4-gitian.zip
618e944d7c7cd6521551e30b32322f4a  zlib-1.2.6.tar.gz
If any of the above is not the same, the final result will obviously be different as well.
Otherwise it must be something about building the actual bitcoin exe - maybe the modifications in gitian-win32.yml?

And, about building on Windows - it's not actually that bad.
I mean the build-msw.txt is quite outdated and makefile.mingw and bitcoin-qt.pro need some tweaks to work, but I can actually build the bitcoin-qt.exe (not to mention bitcoind.exe) without a virtual machine, neither Visual C. It's just that at the end it is a completely different exe, from the one that you serve at bitcoin.org - i.e. mine is 5 MB smaller, but requires MINGWM10.DLL

piotr_n your checksums match what I have so I don't think that's it. My guess it would be something in bitcoin itself, but as in the instructions I take a zip of the clean bitcoin source before any modifications to use during the compiling so that we should have the same end results. Also I compared the md5 checksum of all files in the source I have with the source that's in the official bitcoin-0.6.2-win32-setup2.exe installer and they all match. So I am still not sure what part of my build causes the difference.
legendary
Activity: 2053
Merit: 1356
aka tonikt
June 03, 2012, 02:47:48 PM
#5
I don't know where to find the bitcoin-build.assert, but here are md5 sums for the files in my "input" dir before I build the actual client:
Code:
3da3128ef77a8cb610076dfb7ff8f286  bitcoin-deps-0.0.4.zip
a2dc343f7bc7f83f8941e47ed4a18200  boost_1_47_0.tar.bz2
22b63379de8c9387e3bcf90308564c00  boost-win32-1.47.0-gitian.zip
a14a5486d6b4891d2434039a0ed4c5b7  db-4.8.30.NC.tar.gz
c740ba66cd7074ba2471b6a4ff48e1fb  libpng-1.5.9.tar.gz
88055f2d4a061cfd4cfe25a9eae22f67  miniupnpc-1.6.tar.gz
a1da58ce63baef3812004714fa302c47  openssl-1.0.1b.tar.gz
7e90615eb314abcd2eb2eab5c8155b97  qrencode-3.2.0.tar.bz2
9831cf1dfa8d0689a06c2c54c5c65aaf  qt-everywhere-opensource-src-4.7.4.tar.gz
71fb76765dc6734a1940176527b619bd  qt-win32-4.7.4-gitian.zip
618e944d7c7cd6521551e30b32322f4a  zlib-1.2.6.tar.gz
If any of the above is not the same, the final result will obviously be different as well.
Otherwise it must be something about building the actual bitcoin exe - maybe the modifications in gitian-win32.yml?

And, about building on Windows - it's not actually that bad.
I mean the build-msw.txt is quite outdated and makefile.mingw and bitcoin-qt.pro need some tweaks to work, but I can actually build the bitcoin-qt.exe (not to mention bitcoind.exe) without a virtual machine, neither Visual C. It's just that at the end it is a completely different exe, from the one that you serve at bitcoin.org - i.e. mine is 5 MB smaller, but requires MINGWM10.DLL
legendary
Activity: 1652
Merit: 2301
Chief Scientist
June 03, 2012, 02:28:32 PM
#4
I'd love to see somebody resurrect the Visual C++ makefile and build instructions (and let us know if we've accidentally added and gcc-specific code), so Windows developers didn't have to spin up a VM and cross-compile everything.

As to why your builds are different:  no idea, but you can look at the file-by-file checksums in the bitcoin-build.assert file to see exactly what is different.
legendary
Activity: 2053
Merit: 1356
aka tonikt
June 03, 2012, 01:47:59 PM
#3
Thank you @Onichan - this guide is very helpful, though it took me the whole day to accomplish the mission.

And indeed, the content of bitcoin-qt.exe and bitcoind.exe at the end is different from the released one, though the sizes are just the same.
So I don't know what to think about this...

What I can add to this is that I wasn't able to make the kvm working on VirtualBox, so I had to switch to VMWare.

And then, it's really annoying that you need to restart the system every time before doing the new "sudo bin/gbuild ...."

But anyway, you are the hero! Smiley
full member
Activity: 203
Merit: 100
June 03, 2012, 06:19:28 AM
#2
Maybe you should put this in the wiki?
newbie
Activity: 47
Merit: 0
June 02, 2012, 11:48:18 PM
#1
I am still learning and this is just a guide based on what I have found as a Windows user trying to compile for Windows. This guide is intended for people who aren't as familiar with Linux so I go into more details about a few things.

First off don’t even bother with the windows build instructions. Last I checked it's not kept up to date and hasn’t worked properly in quite some time. The way windows binary’s are compiled is using gitian builder in Ubuntu. It creates a virtual environment that compiles the sources within it so that anybody can build software and get the same file in the end. So the first thing you will need is Ubuntu 64-bit and you will need version 10 as I know 11 has issues. Now I have noticed 12 is out, but I haven’t tested it.

Anyways the version I used was the long term support ubuntu-10.04.4-desktop-amd64 http://releases.ubuntu.com/lucid/. Next you will need some kind of virtual environment such as VirtualBox or VMware. I used VMware workstation, but whatever you choose ensure you enable hardware virtualization. Which in VMware it is located under the VM settings in the processors setting and called “Virtualize Intel VT –x/EPT or AMD-V/RVI”. This can make a very significant difference in the compiling speed. Also you want to only have one core per processor and max out the processors as when I was testing I noticed gitian performed better that way.

Note: If using VMware the arrow keys are messed up in vi so to fix that I had to edit vimrc.tiny
sudo vi /etc/vim/vimrc.tiny
Go down to the line that says “set compatible” then just double press d to delete the line then press i to insert some text and type “set nocompatible” then press esc and do a :wq to save the file. You cant use the arrow keys so if you mess up just press esc then type :q! to force quit and try again.

Also if you are using VMware I know you can drag and drop files between host and VM, but I have noticed sometimes the virtual tries moving the file before its finished copying to the virtual. I just wait till it’s done copying and clicking the retry on the error popup. Also I have noticed a file has occasionally become corrupt from copying so I normally open the zips and such to see if it gives errors.


So after you have installed Ubuntu on your virtual you will need to install some applications. Open a terminal window found under the menu applications/accessories/terminal and run
Code:
sudo apt-get install python-vm-builder qemu-kvm apt-cacher ruby
During install, a package configuration window will popup. Just tab to the ok and on the next screen select no configuration. Once they are installed run
Code:
sudo service apt-cacher start
It will tell you how it’s not enabled in /etc/default/apt-cacher so run
Code:
sudo vi /etc/default/apt-cacher
and go down the the autostart line and change the 0 to a 1. This is so now it autostarts when you reboot Ubuntu.

Now you need gitian which can be found https://github.com/devrandom/gitian-builder I just download the zip of it as I prefer keeping a backup if needed, but you can install git and use that if you want.
Extract gitian onto your desktop and have it so that the main sub-folders are in a gitian folder on your desktop like /Desktop/gitian/bin/gbuild

Next download the bitcoin source you want from https://github.com/bitcoin/bitcoin/tags also I know one of the developers has some other tags at https://gitorious.org/bitcoin/bitcoind-stable/trees/0.5.x
Then extract it to the desktop and name the folder bitcoin so the folder structure is such /Desktop/bitcoin/readme.md
Note: Only the current branch which is 6 at this time is actively developed, though 4 and 5 do have bug fixes.

You will also need a clean copy of bitcoin source zipped so right click the bitcoin folder on your desktop and click compress. Name it bitcoin-6.2 and choose zip from the dropdown.
Note: If you are planning on using some modified bitcoin code you should first finish your modifications to the source before you make that zip as that zip is what will be used later during the actual compiling of bitcoin in the virtual.


Now with gitian being a virtual in a virtual it’s going to suffer some performance and because of that I have noticed you need to modify some of the gitian files. So open gitian/bin/gbuild and when asked just select display. Modify line 60 from "(1..10).each do" to
Code:
(1..100).each do
Then in the gitian/libexec folder change the copy-from-target line 50 from " -oConnectTimeout=5" to
Code:
 -oConnectTimeout=60
Then do the same thing in the copy-to-target, but for on-target I just used 30
Also for start-target I changed the " -smp ${NPROCS:-2}" to "-smp 2"
I have 4 processors on the VM noticed it was only using 1/4 of my processors so I wanted to force it to use more for better performance. Though I did find out using 3/4 would cause issues so just use half of your total.

Note: You can tab most things to auto fill the name such as typing De then tab will fill Desktop/
Now it's time to build the gitian virtual. In a new terminal window go to your gitian folder by cd Desktop/gitian/ and run
Code:
sudo bin/make-base-vm -a i386
I had to use sudo because half the time without it would cause problems. Now this will probably take a few minutes and look like it’s not doing anything, but just wait for it. While waiting you can work on the other parts.



If you open your bitcoin folder and go to /contrib/gitian-descriptors/gitian-win32.yml and in the “files:” section, which is line 17 in 6.2, you will see it list a few zip files which you will need. To get them you will need to compile them and to do that you will use those other .yml files.

The first one is qt-win32-4.7.4-gitian.zip so opening qt-win32.yml and in files section you will see this needs qt-everywhere-opensource-src-4.7.4.tar.gz which can be found at http://ftp://ftp.qt.nokia.com/qt/source/
Just make sure you are downloading that correct file, I just used ctrl+f to search for it.

Next requirement from gitian-win32.yml is boost-win32-1.47.0-gitian.zip. So opening boost-win32.yml we see it needs boost_1_47_0.tar.bz2 found at http://sourceforge.net/projects/boost/files/boost/1.47.0/

Then the final file in gitian-win32.yml is bitcoin-deps-0.0.4.zip so in deps-win32.yml we see it needs quite a few.
openssl-1.0.1b.tar.gz at http://www.openssl.org/source/
db-4.8.30.NC.tar.gz at http://www.oracle.com/technetwork/products/berkeleydb/downloads/index-082944.html (called Berkeley DB 4.8.30NC.tar.gz)
miniupnpc-1.6.tar.gz at http://miniupnp.free.fr/files/
zlib-1.2.6.tar.gz at http://sourceforge.net/projects/libpng/files/zlib/1.2.6/
libpng-1.5.9.tar.gz at http://sourceforge.net/projects/libpng/files/libpng15/older-releases/1.5.9/
qrencode-3.2.0.tar.bz2 (notice it’s the bz2 version) at http://fukuchi.org/works/qrencode/index.html.en

Ok so now we should have all the sources needed they all need to be copied/moved to a new folder called inputs in your gitian folder so its Desktop/gitian/inputs
Now your virtual should be done building and you can start compiling the dependencies. I’ll go down the list starting with qt we run
Code:
sudo bin/gbuild ../bitcoin/contrib/gitian-descriptors/qt-win32.yml
This can take quite some time.

Note: If after you enter your sudo password and it starts the VM, but then asks about some @localhost password you need to reboot and delete the /gitian/target-lucid-i386.qcow2 and /gitian/var/target.pid which you may have to do by “sudo rm target.pid” and then try again.
If you get an error about on-target or banner or some kind of timeout then try increasing the gbuild loop number number and/or the on-target timout. Also make sure you aren’t using too many processors as mine would never connect when using more than half. If you have some warning about kvm not loaded that means you don't have hardware acceleration enabled. It should still work, but will be extremely slow, many hours when I tested it, and you may have to increase the gbuild loop and connection timeouts.

Once is is done it will put the files in /gitian/build/out/ and you must copy them to your inputs folder before you build the next one as outputs is erased on each gbuild. Also after each build I normally deleted the target vm and the target.pid because I found that tend to help with any issues I would sometimes get.

So next one is boost which we run
Code:
sudo bin/gbuild ../bitcoin/contrib/gitian-descriptors/boost-win32.yml
And the final dependency is bitcoin-deps which is
Code:
sudo bin/gbuild ../bitcoin/contrib/gitian-descriptors/deps-win32.yml

Now we should have all dependencies for bitcoin, but there are a few things we need to do before compiling bitcoin. By default gitian-win32.yml will always try downloading the most recent source to use for compiling. Well I don’t want that because I want to use my source so we need to modify a few things.
In gitian-win32.yml change the “remotes:” section lines 14-16 to "remotes: []"
Then under “files:” add
Code:
- "bitcoin-6.2.zip"
Then above the now line 45 “cd bitcoin” add
Code:
  unzip bitcoin-6.2.zip
  cp -r bitcoin $OUTDIR/src
  cp -r bitcoin nsis
Next delete the now line 49 and 50 that start with mkdir and git. Then scroll down and replace now line 67 and 68 which are mkdir and git, with
Code:
mv ../nsis .
Now take that clean bitcoin-6.2.zip from the desktop and put it in the inputs folder. Then start the compiling by running
Code:
sudo bin/gbuild ../bitcoin/contrib/gitian-descriptors/gitian-win32.yml

After a while it should put the windows installer in the /gitian/build/out
I have tried this a couple times and the installer and bitcoin works, but for some reason my hashes aren't matching the developers. It could be from some of the changes I made, but I don't think that should matter because I am using the same source.

All of this should work with any source you use so you can use your own modified code, but if using older major releases I know 4 and 5 has different dependencies so you will need to find and compile them.



So I started all of this because I wanted to use a slightly modified client with the 5 branch. I need version 5 as I still test Armory and it doesn't support the new compressed wallets. So here are the few differences if you want to use version 5.
If you followed the above instructions you should already have all the dependencies needed for 5.5 so the only thing you will need to do is first take your 5.5 source, modified or not, and make a zip of it.
The zip file structure should be so that when you open it there is a folder called "bitcoin" and within it starts all the files/folder like the contrib folder. Next name it bitcoin-5.5.zip and stick it in your inputs folder.
Then modify the /Desktop/bitcoin/contrib/gitian-descriptors/gitian-win32.yml which should have your 5.5 source in that bitcoin folder. We do the same thing with remotes making it "remotes: []". Then add the
Code:
- "bitcoin-5.5.zip"
to the files section. Then the only other thing we have to do is add
Code:
  unzip bitcoin-5.5.zip
to above now line 64 which has the "  cd bitcoin". Thats it then just compile it like you would above.


If there are any suggestions or things I messed up please let me know and I'll update the guide.
Jump to: