This script requires making a fresh VM of the same OS,
Just out of curiosity: What do you use for the Raspberry Pi offline bundles? A VM/Emulator, or just another Raspberry Pi?
Topic: [HOW TO] Get the needed packages on your offline computer (Synaptic PM) (Read 3263 times)
September 11, 2014, 02:59:18 AM
Just out of curiosity: What do you use for the Raspberry Pi offline bundles? A VM/Emulator, or just another Raspberry Pi?
September 11, 2014, 01:04:37 AM
I'd think my post could be done from the command line too, as Synaptic is a GUI for apt.
It's just two different approaches I guess, one from a "developer mindset" and one from a "user mindset".
September 11, 2014, 12:14:01 AM
That's cool. I didn't know that script was available and rather than waiting for offline bundles I looked at ways to do it myself. With Synaptic offline I also like that I can add other packages or get updates to my offline box, pretty much the same way as I would on an online PC.
Yeah, I used to use the Synaptic "Generate package download script". For users doing this once, that method is probably simpler and more straightforward. But I ran into issues with it on some distros, and I always prefer a command-line method anyway
For me, I need to do this somewhat frequently, and also find that CLI scripts are less error prone. I also never published this script since I thought I was the only person needing it. I should've known others would want it. I'll add it to the repo in the extras directory.
The script can be modified to work with any packages, as long as the VM on which it is run has equal-or-less packages on it than the offline computer (I take a snapshot of the VM on the first boot so I can revert to that state to run this script).
September 11, 2014, 12:06:55 AM
That's cool. I didn't know that script was available and rather than waiting for offline bundles I looked at ways to do it myself. With Synaptic offline I also like that I can add other packages or get updates to my offline box, pretty much the same way as I would on an online PC.
September 10, 2014, 10:47:13 PM
Sorry to steal your thunder here, Newar
I have a script I use to make offline bundles. Given how easy this script is to use, we could probably be putting out more bundles with each release (though we haven't because more released binaries & bundles complicates our release process). This script requires making a fresh VM of the same OS, but I need the VM for testing anyway, so it's actually saving me a step (it would be an extra step for others). Either way, it's not terribly difficult to install the distro with the default install options in a VM and run this script there (use the same ISO for the VM and the offline computer)
So the steps to use my script (for Debian-based OSes):
That's it. You can now install Armory on the offline computer and all dependencies will already be installed!
Under the hood what it's doing is extremely simple:
That just gets the download links and MD5 hashes of the entire depenency tree. The script then downloads those, and checks their MD5 sums. It only saves files that have a matching MD5 hashes.
When you run the script on the offline computer, it runs the same command to get the list of downloads and MD5 sums, but now it reads the .deb files from the directory (instead of wget'ing them) and checks that the hashes match. If so, it adds it to the list of files to install. When it's done, it simply runs:
You can use the Synaptic method mentioned by Newar, and you won't need a fresh VM of the same operating system. Or my scripts can be adapted do the same thing: generate the wget commands on the offline computer, execute on the online computer, take back to the offline computer to install. For me, I need the VM anyway so it's actually saving me time and I don't have to go through that pesky GUI
Here's the script:
Pretty darned short considering it handles both downloading and installing (depending on command line argument, argv[1]), and does all the hash checks for you. This is why I love python!
I have a script I use to make offline bundles. Given how easy this script is to use, we could probably be putting out more bundles with each release (though we haven't because more released binaries & bundles complicates our release process). This script requires making a fresh VM of the same OS, but I need the VM for testing anyway, so it's actually saving me a step (it would be an extra step for others). Either way, it's not terribly difficult to install the distro with the default install options in a VM and run this script there (use the same ISO for the VM and the offline computer)
So the steps to use my script (for Debian-based OSes):
- (1) Download ISO to be installed on offline computer
- (2) Install OS on VirtualBox virtual machine
- (3) Run the script (at the bottom of this post) using "python dl_offline_deps.py download". This will create a directory with all dependencies, and a copy of the script (itself)
- (4) Take directory to offline computer, run "sudo python dl_offline_deps.py install" within download directory
That's it. You can now install Armory on the offline computer and all dependencies will already be installed!
Under the hood what it's doing is extremely simple:
Code:
apt-get install --print-uris --yes
That just gets the download links and MD5 hashes of the entire depenency tree. The script then downloads those, and checks their MD5 sums. It only saves files that have a matching MD5 hashes.
When you run the script on the offline computer, it runs the same command to get the list of downloads and MD5 sums, but now it reads the .deb files from the directory (instead of wget'ing them) and checks that the hashes match. If so, it adds it to the list of files to install. When it's done, it simply runs:
Code:
sudo dpkg - i
You can use the Synaptic method mentioned by Newar, and you won't need a fresh VM of the same operating system. Or my scripts can be adapted do the same thing: generate the wget commands on the offline computer, execute on the online computer, take back to the offline computer to install. For me, I need the VM anyway so it's actually saving me time and I don't have to go through that pesky GUI
Here's the script:
Code:
################################################################################
#
# DOWNLOAD: python dl_offline_deps.py download
#
# DOWNLOADING MUST BE RUN ON A FRESH INSTALL OF THE TARGET OPERATING SYSTEM!
#
# This will download and check the md5sums of all Armory dependencies, and
# their dependency trees, in order to install Armory on a fresh install onto
# an offline system.
#
# It is recommended you make a virtual machine using VirtualBox of the exact
# same Linux distro, then run this script on the first boot. It will create
# a directory containing every .deb package in the dependency tree, which can
# be taken to the offline computer and installed using this script but with
# the "install" command instead (and sudo)
#
################################################################################
#
# INSTALL: sudo python dl_offline_deps.py install
#
# If run with admin privilege using the "install" command, it will get the
# raw .deb files from the current directory (instead of wget'ing), it will
# check the hashes, and then install them.
#
################################################################################
import os
import shutil
import platform
import hashlib
from subprocess import check_output
from sys import argv
if len(argv)<2 or not argv[1] in ['download','install']:
print 'Usage: '
print ' python %s download' % argv[0]
print ' or:'
print ' sudo python %s install' % argv[0]
exit(1)
# This is actually all packages needed for compiling, not just installing Armory
pkglist = 'git-core build-essential pyqt4-dev-tools swig libqtcore4 libqt4-dev python-qt4 python-dev python-twisted python-psutil dpkg-sig'
# Use the package manager to pretend to install it, collect URLs and hashes
uriList = check_output('apt-get install %s --print-uris --yes' % pkglist, shell=True)
# If we are downloading, make a directory to put them into
if argv[1]=='download':
opsys = platform.system()
distro = '-'.join(platform.linux_distribution())
osbits = 64 if platform.machine().lower()=='x86_64' else 32
dirname = 'offline_bundle_%s_%s_%dbit' % (opsys, distro, osbits)
if not os.path.exists(dirname):
os.mkdir(dirname)
shutil.copy(argv[0], dirname) # copy this script in there
# Download or read from file all .debs and check hashes
debList = []
for line in uriList.split('\n'):
if not line.startswith('\'http'):
continue
uri,deb,siz,md5 = line.split()
uri = uri.strip('\'')
md5 = md5.split(':')[-1]
if argv[1]=='download':
rawDeb = check_output('wget --quiet -O - %s' % uri, shell=True)
else:
rawDeb = open(deb,'rb').read()
if not hashlib.md5(rawDeb).hexdigest() == md5:
print "***ERROR: MD5 does not match! Not saving file"
continue
debList.append(deb)
if argv[1]=='download':
targfile = os.path.join(dirname, deb)
with open(targfile, 'wb') as f:
print ' Writing %s' % targfile
f.write(rawDeb)
if argv[1]=='install':
dlcmd = 'dpkg -i "%s"' % '" "'.join(debList)
print dlcmd
check_output(dlcmd, shell=True)
#
# DOWNLOAD: python dl_offline_deps.py download
#
# DOWNLOADING MUST BE RUN ON A FRESH INSTALL OF THE TARGET OPERATING SYSTEM!
#
# This will download and check the md5sums of all Armory dependencies, and
# their dependency trees, in order to install Armory on a fresh install onto
# an offline system.
#
# It is recommended you make a virtual machine using VirtualBox of the exact
# same Linux distro, then run this script on the first boot. It will create
# a directory containing every .deb package in the dependency tree, which can
# be taken to the offline computer and installed using this script but with
# the "install" command instead (and sudo)
#
################################################################################
#
# INSTALL: sudo python dl_offline_deps.py install
#
# If run with admin privilege using the "install" command, it will get the
# raw .deb files from the current directory (instead of wget'ing), it will
# check the hashes, and then install them.
#
################################################################################
import os
import shutil
import platform
import hashlib
from subprocess import check_output
from sys import argv
if len(argv)<2 or not argv[1] in ['download','install']:
print 'Usage: '
print ' python %s download' % argv[0]
print ' or:'
print ' sudo python %s install' % argv[0]
exit(1)
# This is actually all packages needed for compiling, not just installing Armory
pkglist = 'git-core build-essential pyqt4-dev-tools swig libqtcore4 libqt4-dev python-qt4 python-dev python-twisted python-psutil dpkg-sig'
# Use the package manager to pretend to install it, collect URLs and hashes
uriList = check_output('apt-get install %s --print-uris --yes' % pkglist, shell=True)
# If we are downloading, make a directory to put them into
if argv[1]=='download':
opsys = platform.system()
distro = '-'.join(platform.linux_distribution())
osbits = 64 if platform.machine().lower()=='x86_64' else 32
dirname = 'offline_bundle_%s_%s_%dbit' % (opsys, distro, osbits)
if not os.path.exists(dirname):
os.mkdir(dirname)
shutil.copy(argv[0], dirname) # copy this script in there
# Download or read from file all .debs and check hashes
debList = []
for line in uriList.split('\n'):
if not line.startswith('\'http'):
continue
uri,deb,siz,md5 = line.split()
uri = uri.strip('\'')
md5 = md5.split(':')[-1]
if argv[1]=='download':
rawDeb = check_output('wget --quiet -O - %s' % uri, shell=True)
else:
rawDeb = open(deb,'rb').read()
if not hashlib.md5(rawDeb).hexdigest() == md5:
print "***ERROR: MD5 does not match! Not saving file"
continue
debList.append(deb)
if argv[1]=='download':
targfile = os.path.join(dirname, deb)
with open(targfile, 'wb') as f:
print ' Writing %s' % targfile
f.write(rawDeb)
if argv[1]=='install':
dlcmd = 'dpkg -i "%s"' % '" "'.join(debList)
print dlcmd
check_output(dlcmd, shell=True)
Pretty darned short considering it handles both downloading and installing (depending on command line argument, argv[1]), and does all the hash checks for you. This is why I love python!
September 10, 2014, 12:54:46 PM
Just installed Armory on a Lubuntu 14.04 and thought I'd write it down. Hopefully this is of use to others too. The added benefit using Synpatic offfline is that you can easily add other packages from the official release without having to rely on any third party. It's a lot of text, but once you got the hang of it, it is actually not that hard to do. This is for the Trusty release, but can be done for other dists by changing the offline repo script.
Lubuntu 14.04 still comes with the Synaptic Package Manager. After installing Lubuntu on your offline PC take the USB stick back to an online PC and run the below script. It will create an offline repository on your USB stick as outlined here: https://help.ubuntu.com/community/AptGet/Offline/Repository It creates the necessary folder structure and gets all the info for 32- and 64-bit and also trusty-security, -updates and -backports. http://pastebin.com/mQU9Jy72
With this info on the USB take it back to the offline computer and add the USB repo to your sources list as outlined in the article. This can be done in Synaptic too, using
Code:
deb file:///media///us.archive.ubuntu.com/ubuntu trusty main restricted universe multiverse
After closing that dialog Synaptic will ask to reload the package information. You already should see some icons with arrows pointing up next to some packages, but close Synaptic for now, we'll come back to it.In the file manager (PCManFM) go to the USB stick where you have saved the verified armory_*.deb, choose "Tools" -> "Open current folder in terminal" or hit F4. In the terminal "sudo dpkg -i armory_*.deb", this will throw some errors. Don't worry.
Open Synaptic again -> "Edit" -> "Fix broken packages". On the bottom it will show you how many packages need installing. In my case there were 44. At this point you can also choose to install additional packages if you wish, for example I find QtQR handy to work with QR codes. You also can choose to "Mark All Upgrades" to keep your offline system updated. When you're done, select "File" -> "Generate package download script" and save that in an empty folder on the USB stick as
Take the stick to the offline machine. Before you can execute it, it needs a little change. The lines will look something like that:
Code:
file:///media///us.archive.ubuntu.com/ubuntu/pool/main/...
change that to Code:
http://us.archive.ubuntu.com/ubuntu/pool/main/...
Easily done in Leafpad, using the "Replace" command. Replace "file:///media/Go to the empty folder and again: "Tools" -> "Open current folder in terminal". Type
Code:
bash .sh
Wait for the download to finish. Take the USB stick back to the offline PC and in Synaptic select "File" -> "Add downloaded packages". Point it to the folder and it will install the packages and finish the Armory install as well. Armory (offline) is ready to use.
Jump to: