Classroom - 2011-07-09

Stefano Palazzo

4:10 PM

@TheEvilPhoenix Saturday the 9th you mean :-)

The Evil Phoenix

whatever :P

The Evil Phoenix

5:29 PM

The session on Packaging will begin in approximately 30 minutes.

in AskUbuntu Classroom Discussion - Packaging, 3 mins ago, by The Evil Phoenix

*** FILES YOU WILL NEED ***
http://kor.trekweb.org/packaging/class-files.zip <--- Files we will use for packaging demonstrations created and owned by myself for this class session.

Stefano Palazzo

5:52 PM

I will be moderating this session. If you have questions, please join the Classroom Discussion Room. Your questions will be relayed to the speaker.

The Evil Phoenix

BEGIN

Greetings, everyone! I am The Evil Phoenix, and I will be your instructor today.

This class session is called The Basics of Packaging on Ubuntu (Packaging: Part 1)

This session will cover the basics of creating .deb packages for Ubuntu installations.

Firstly, I would like to point out that during this session, we will be using the Command Line for most of the commands.

And before we continue, I must point out that during the session, there may be demonstrations of commands and actions.

The demonstrations will be using a file that is available via one of my servers: kor.trekweb.org/packaging/class-files.zip

This contains a basic file that will be used to create the demonstration package.

You will need to download and extract the files within the ZIP

Where you unzip the file to is irrelevant, but make it readily available.

Prerequisites for Packaging

In order to be able to create a package, it is recommended that users have a firm grasp of basic command line syntax as well as commands.

I'm going to assume that most of you have experience with the Terminal and Command-Line, however I will outline all commands as we go.

You must have a PGP key for yourself so that you may sign the .deb installer when you build it for deployment.

As this is a prerequisite, I will go briefly into how to create a PGP key via the GUI, using seahorse which ships standard with Ubuntu 11.04 (GNOME).

Stefano Palazzo

6:07 PM

@TheEvilPhoenix there's a question on that point:

James asked:

> How do you make a PGP key? Is there quick and dirty way to do that?

The Evil Phoenix

1) We will go through the easiest method of creating a PGP key via the GUI. It can be done via the command line, but the easiest method is the GUI.

First, you will need to run the Seahorse program. If you are using Unity, please open up the applications window, and type in "seahorse". An icon will show up that says "Passwords and Encryption keys". An easier method to open could be Alt + F2 (or in the terminal), then running seahorse

When you are in the GUI interface for seahorse, you will need to use the menu bar. Click "File" > "New".

A screen like this one will show up:

Select PGP Key, and hit continue.

The next screen will be similar to this:

Provide a name and an email address for the key, and hit "Create". A comment is not required, but may help you to differentiate between keys when you have several

The system will prompt you for a passphrase for the key.

It is strongly recommended you provide one.

After the key has been created, it will show up in the main window for Seahorse.

For this demonstration, you will not need to upload your PGP key to a key server.

However, if you were going to distribute your Debian packages, you would need to upload the PGP key to a key server, which I will very briefly explain here

Select the PGP key you created in the main seahorse window (the key should be under "personal keys").

Go to the menu bar, choose "Edit", and then "Preferences".

Under the Key Servers tab, it will look similar to this:

Make sure that under "Publish Keys" you select the Ubuntu keyserver entry.

close the preferences window, then, with your PGP key selected, go to the menu bar: "Remote" > "Sync and Publish Keys..."

click the "Sync" button.

Your PGP key will be uploaded to the Ubuntu keyservers, and will be available within a few minutes on the internet.

Now we will install all required software.

This command will run the installation of all required software for packaging:

sudo apt-get install build-essential devscripts ubuntu-dev-tools debhelper dh-make diff patch cdbs quilt gnupg fakeroot lintian pbuilder piuparts

A large number of packages will be installed and downloaded to your system for this.

Installation may take anywhere from 5 minutes to 30 minutes depending on various factors such as internet speed, system speeds, and other factors that contribute to a computer's speeds.

One of the packages needed within that command, however, is postfix. You can choose "Local configuration" when it asks you for what config to use.

if there are any unanswered questions up to this point, I will take them. (@StefanoPalazzo)

Rinzwind

6:26 PM

no questions yet

The Evil Phoenix

good.

After all the packages are installed, we will need to create the pbuilder environment. pbuilder is the "Personal Builder". It is used to build your packages after you've created the source package. This way, you can build the actual .deb in a chrooted environment.

Typically, you will be building the pbuilder build environment for the same distribution you are running, however you may also build environments for other distributions (Lucid, Maverick, etc).

This command will build the pbuilder environment for your distribution:

sudo pbuilder create --distribution $(lsb_release -cs) --othermirror "deb http://archive.ubuntu.com/ubuntu $(lsb_release -cs) main restricted universe multiverse"

The creation of the pbuilder environment will take time.

On this system that I am using, pbuilder creation of the base environment for Natty took approximately 10 minutes; on my netbook (running Maverick), it took 20 minutes.

During this time, we can continue to build the core of the package without actually compiling the files into a .deb.

While pbuilder builds in one terminal window, please open another terminal window.

In that window, please navigate to the location where you unzipped the class-files.zip file.

In this ZIP file, you will see only one folder: exec

This folder contains the executable .sh script that will be made into a package and subsequently installed into the system for use.

In my system, I unzipped the file on the Desktop, and named the folder packaging-demo-1.0

you may name it what you wish, but I recommend you name the folder the same as I name it for congruency between commands.

In the Terminal, you should navigate to the path to that folder: cd /path/to/packaging-demo-1.0

Within that folder, we will begin creating the Debian control files which steer the creation of the package.

The command we will now run is: dh_make

$ dh_make

Type of package: single binary, indep binary, multiple binary, library, kernel module, kernel patch?
 [s/i/m/l/k/n]

This is the output of the command when first run.

We will be choosing "single binary" as the option

 [s/i/m/l/k/n] s

Maintainer name : IGPF.us
Email-Address	: [email protected]
Date				: Sat, 09 Jul 2011 14:38:16 -0400
Package Name	 : packaging-demo
Version			: 1.0
License			: blank
Type of Package : Single
Hit <enter> to confirm:

The information here was gathered from your PGP key.

in this case, my folder that I unzipped was renamed to be packaging-demo-1.0, which means the package name would be packaging-demo

if you used the original directory name, it will be packaging-test

Stefano Palazzo

@TheEvilPhoenix there seems to be a small problem

The Evil Phoenix

Oh, whoops i believe i skipped a step, one moment.

Stefano Palazzo

6:41 PM

Amith, Rinziwind and David asked:

> question: why do we me have @unkown at the email adress?

The Evil Phoenix

yes, i'm finding the resolution now

(that's the step I skipped... whoops)

Whichever shell you're using (by default, its bash), you'll need to edit its configs.

Stefano Palazzo

also just so you're aware, people are reporting this error:

Could not find packaging-test_1.0.orig.tar.gz
Either specify an alternate file to use with -f,
or add --createorig to create one.

The Evil Phoenix

that's going to be covered

after we fix their @unknown issue

In this case, edit your `.bashrc`, and append this at the end:
export DEBFULLNAME="John Doe"
export DEBEMAIL="[email protected]"

and replace the text inside the quotes with the text you used in the PGP key.

afterwards, save that file, then run source .bashrc in your terminal

then run the dh_make command again.

As we did not create an .orig.tar.gz file, dh_make calls you out on it.

In which case, you may Ctrl + C the process, and instead run the command dh_make --createorig

Provide the same input we provided before

Single Binary

then hit "enter"

This will likely show up:

Currently there is no top level Makefile. This may require additional tuning.
Done. Please edit the files in the debian/ subdirectory now. You should also
check that the packaging-demo Makefiles install into $DESTDIR and not in / .

When packaging programs, you would usually create a Makefile which would dictate the destination of where things would be written.

we are going to use a different method for defining installation parameters for files.

via a debian control file called install

After dh_make has been run, run ls on the directory in terminal.

$ ls
total 16K
4.0K drwxr-xr-x 4 teward teward 4.0K 2011-07-09 14:46 ./
4.0K drwxr-xr-x 8 teward teward 4.0K 2011-07-09 14:46 ../
4.0K drwxr-xr-x 3 teward teward 4.0K 2011-07-09 14:46 debian/
4.0K drwxr-xr-x 2 teward teward 4.0K 2011-07-09 14:34 exec/

You can ignore the user fields, but as you can see, there is now a debian folder

Navigate into that folder.

You will notice many example files (*.ex files)

those were built in the event that you need to use them.

in this demo, we will not need them.

you may run the command rm *.ex *.EX to remove these files.

What is left should be similar to this:
$ ls
total 40K
4.0K drwxr-xr-x 3 teward teward 4.0K 2011-07-09 14:51 ./
4.0K drwxr-xr-x 4 teward teward 4.0K 2011-07-09 14:46 ../
4.0K -rw-r--r-- 1 teward teward  182 2011-07-09 14:46 changelog
4.0K -rw-r--r-- 1 teward teward    2 2011-07-09 14:46 compat
4.0K -rw-r--r-- 1 teward teward  533 2011-07-09 14:46 control
4.0K -rw-r--r-- 1 teward teward 1.5K 2011-07-09 14:46 copyright
   0 -rw-r--r-- 1 teward teward    0 2011-07-09 14:46 docs
4.0K -rw-r--r-- 1 teward teward  183 2011-07-09 14:46 README.Debian

I will give a brief explanation of these items, however these items are fully detailed in the Packaging Guide on the Ubuntu Wiki: wiki.ubuntu.com/PackagingGuide/Complete

Stefano Palazzo

a quick question: James asked

> Do we need to remove the .EX files?

The Evil Phoenix

2 mins ago, by The Evil Phoenix

in this demo, we will not need them.

2 mins ago, by The Evil Phoenix

you may run the command rm *.ex *.EX to remove these files.

to answer that question. You should remove the example files that you will not use prior to actually building the .deb

Stefano Palazzo

6:53 PM

thanks

The Evil Phoenix

The changelog file is exactly that: a log of changes between versions.

We will need to edit the changelog

Using nano, vi, or your favorite CLI text editor, open the changelog file.

This is the changelog generated on my system:

packaging-demo (1.0-1) unstable; urgency=low

  * Initial release (Closes: #nnnn)  <nnnn is the bug number of your ITP>

 -- IGPF.us <[email protected]>  Sat, 09 Jul 2011 14:46:41 -0400

Replace where it says unstable with your distribution version (in my case, natty)

if you are using Lucid Lynx (10.04), put lucid; for Maverick, maverick; for Oneric, oneric

You may edit the * Initial release line to say whatever, but for this demo, replace it with * Packaging Demonstration

keep the initial two spaces before the *

any questions thusfar?

Stefano Palazzo

yeah, just one

David asks:

> is it possible to specify for multiple versions (like maverick and natty)

The Evil Phoenix

Unfortunately, you cannot specify multiple versions in one changelog.

You would need to build the package multiple times, each time using a different distribution name

However, we will cover a method that allows you to use the PPA system of Launchpad to copy this one installer to other distributions. This will be covered in the second session, and not now.

After editing the changelog file, please save it.

The compat file we can ignore, but it is needed in order to identify compatibility levels with the various parts of building the .deb.

the control file is next, and we will need to edit it

Source: packaging-demo
Section: unknown
Priority: extra
Maintainer: IGPF.us <[email protected]>
Build-Depends: debhelper (>= 7.0.50~)
Standards-Version: 3.9.1
Homepage: <insert the upstream URL, if relevant>
#Vcs-Git: git://git.debian.org/collab-maint/packaging-demo.git
#Vcs-Browser: git.debian.org/?p=collab-maint/packaging-demo.git;a=summary

Package: packaging-demo
Architecture: any
Depends: ${shlibs:Depends}, ${misc:Depends}
Description: <insert up to 60 chars description>
 <insert long description, indented with spaces>

This is the control file that was generated within my system.

The commented lines for #Vcs-Git and #Vcs-Browser are unnecessary and may be removed.

This package is not connected to a home page, so that line is irrelevant today, however for a full project, you would put the URL to the project's home page (even if its a Launchpad project) in for that line.

You'll notice the Section: unknown line.

This determines what section the package is listed under in package managers

examples are net, utils, and others.

In this case, it doesnt really matter what we put (as long as its a valid section), and in this case we can put misc

which stands for miscellaneous packages

Priority stands for the importance of the package.

This excerpt from the Packaging Guide sums it up:

Priority: This sets the importance of the package to users. It should be one of the following:

Required - packages that are essential for the system to work properly. If they are removed it is highly likely that your system will break in an unrecoverable way.

Important - minimal set of packages for a usable system. Removing these packages will not produce an unrecoverable breakage of your system, but they are generally considered important tools without which any Linux installation would be incomplete. Note: This does not include things like Emacs or even the X Window System.

please expand to the full text of that line, there are more than just Required and Important.

We are going to leave our Priority as extra

because this is a specialized package that doesn't fall into any of the other established groups.

the Source and Package lines should remain the same. This denotes the name of the package and its source equivalent

The Architecture line determines whether this package is platform-dependent

This list explains the various possible inputs:

all - The source is not architecture-dependent. Programs that use Python or other interpreted languages would use this. The resulting binary package would end with _all.deb.

any - The source is architecture-dependent and should compile on all the supported architectures. There will be a .deb file for each architecture (_i386.deb for instance)

A subset of architectures (i386, amd64, ppc, etc.) can be listed to indicate that the source is architecture-dependent and does not work for all architectures supported by Ubuntu.

In our case, as this is using a .sh file, there is no dependency for architecture, so we will use all for the Architecture line.

Replace Description: <insert up to 60 chars description> with Description: Demo Package, and replace ` <insert long description, indented with spaces>` with ` Demo Package`, keeping the space in front of it.

The first description line is a short description of the package.

the long description line can be paragraphs long of information.

This is one of the most important lines:

Depends: ${shlibs:Depends}, ${misc:Depends}

This defines any dependencies for the package.

For example, if you are packaging a python program and you require Python of version 2.6 or newer, you'd likely want to define that: Depends: python (>= 2.6)

However, as this is a Bash script, we really only need to worry about bash being installed.

use this for the line: Depends: ${shlibs:Depends}, ${misc:Depends}, bash

the last of the premade files we need to worry about is copyright

This details the copyright information for the package.

Format: dep.debian.net/deps/dep5
Upstream-Name: packaging-demo
Source: <url://example.com>

Files: *
Copyright: <years> <put author's name and email here>
           <years> <likewise for another author>
License: <special license>
 <Put the license of the package here indented by 1 space>
 <This follows the format of Description: lines in control file>
 .
 <Including paragraphs>

# If you want to use GPL v2 or later for the /debian/* files use
# the following clauses, or change it to suit. Delete these two lines

This is my copyright file after running dh_make

Stefano Palazzo

7:14 PM

if I can interrupt quickly:

The Evil Phoenix

sure

Stefano Palazzo

@dv3500ea asks:

> what are '${shlibs:Depends}, ${misc:Depends}' - they don't look like package names?

The Evil Phoenix

shlibs:Depends calculates any dependencies for any dependent packages you specified in Depends:. In this case, any dependent files for bash would be installed at the same time.

misc:Depends are miscellaneous dependencies that may or may not be needed

In any case, you do not need to have them

however, the shlibs:Dependsarea is strongly recommended

there are manpages for the shlibs:Depends one: man dh_makeshlibs

Give me a few seconds to modify the Copyright file that was generated

The copyright file dictates the copyright rules.

Format: dep.debian.net/deps/dep5
Upstream-Name: packaging-demo
Source: <url://example.com>

Files: *
Copyright: 2011 <yourinfo>

License: GPL-2+
 This package is free software; you can redistribute it and/or modify
 it under the terms of the GNU General Public License as published by
 the Free Software Foundation; either version 2 of the License, or
 (at your option) any later version.
 .
 This package is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of

Where it says Source:, put any URL there

and where it says <yourinfo>, replace that with Your Name <email>

putting in your name and email

as matched on the PGP key

we are going to ignore the other files for now

and since i'm short on time, I will not answer any questions until the end of the session.

We must make one last file here: install

run the following command: touch install

then open up the new file

In here, we define were files go.

In this case, please put the following in: /exec/helloworld.sh /usr/bin

This will tell the installer to place the helloworld.sh file into /usr/bin

IN the install file, you will specify the location of where to place every file you included in the package.

with the exception of the Debian files

after you've done this, run cd ..

I'm going to skip the pbuilder demo, and instead point out the basic debuild system.

Run the following: debuild

This will build the .deb file

hang on a sec

My key seems to be broken, however what it should do

is it will build the debian installer (package).

it will ask for the keyphrase for the PGP key you used.

After which you can ls the parent directory of the folder you used for packaging and find all the created files and the .deb installer

You may test the installer by running /usr/bin/helloworld.sh

if it installed and works, you've done the package

I am now finished with the base tutorial of packaging.

Questions will now be taken.

for 5 minutes

After which, there will be a 5 minute recess before the second packaging session.

Stefano Palazzo

7:33 PM

in AskUbuntu Classroom Discussion - Packaging, 13 secs ago, by dv3500ea

QUESTION: is the file debian/install specific to using the dh_make method? When I've made packages before I've written out the debian/rules file with lots of cps!

The Evil Phoenix

The debian/install method is one of several methods you can use.

you may of course run the rules file with thousands of cp commands.

however, debian/install allows you to specify source dest without using cp a thousand times

any other questions? If not, we will begin the 5 minute recess before the second session.

The second session covers building the deb for upload to a PPA

Stefano Palazzo

in AskUbuntu Classroom Discussion - Packaging, 26 secs ago, by Rinzwind

building a deb for LP. Is that based of the DEB made in THIS session?

The Evil Phoenix

Yes, but it can be done for any .deb made with the debuild command.

i will cover specifics in the next session

and the differences between building a standalone .deb and a .deb for PPAs

Stefano Palazzo

All right, looks like we're done with this part. We'll be back at 19:43 UTC for Part II

The Evil Phoenix

Welcome to the second session on packaging!

This session will continue on the adventures of packaging: Ubuntu Packaging for Launchpad PPAs (Packaging: Part 2)

This session requires far less steps than the first session which covered the entire creation of a standalone .deb

However, all the packaging requirements for the first session still apply, as does the dh_make command.

The key difference between creating a standalone .deb and uploading to a PPA are that you do not upload a standalone Debian file to the PPA.

You will be creating a source package which will be uploaded to the PPA

You will require one additional package: dput

Please install it now.

After you have installed the package dput, you will need to create its config file for the user if you wish to use SFTP or SSH-tunneled uploading. For 9.10 and later, you can use a different method.

You will also need to create a PPA on Launchpad.

In order to do this, you will need to have imported your PGP key that you created earlier into Launchpad

Open up seahorse and open up the properties of the PGP key you created.

On the "Details" tab, you will see text titled "Fingerprint"

You will need all the text under that area when you import the key into launchpad.,

In Launchpad, open up your user information page (after you've logged in).

If you do not have a launchpad account, you will need to create one.

On your user info page, there is a section called "OpenPGP Keys". Click the yellow pencil icon next to that field.

You will need to import a PGP key, which is where the fingerprint you just found is needed.

Copy and paste the entire fingerprint into the "Import a PGP Key" box on the Launchpad page.

then click "import"

You will need to check the email address you have registered for Launchpad for further instructions on finishing the import

On the email you get, it will have sent a PGP encrypted message using your key.

You will need to decrypt that message.

Go to the received email, and copy the text between the lines -----BEGIN PGP MESSAGE----- and -----END PGP MESSAGE-----

including those lines

Copy those lines into a text file.

You will be using the gpg --decrypt function from the command line to decrypt this text

Open the terminal and navigate to wherever you stored the text file with the encrypted message.

type gpg --decrypt ./filename, replacing filename with the name of the file

The content will be decrypted, and you will need to follow the instructions in the decrypted message in order to finish importing your PGP key.

After importing the PGP key, you will need to create a personal PPA for yourself on Launchpad

However, you may need to sign the Ubuntu Code of Conduct. Instructions for how to do this are on Launchpad's site.

any questions?

Stefano Palazzo

7:57 PM

yes, just one again :)

in AskUbuntu Classroom Discussion - Packaging, 36 secs ago, by Rinzwind

question: passphrase is a random text or the one used when creating the key in seahorse?

The Evil Phoenix

Use the passphrase you used when you created the key

After creating the PPA on Launchpad and accepted all terms of them, and also after importing the PGP key, we can continue.

I will wait for you to finish the imports of the PGP keys

Before I continue, I will briefly touch on a configuration you must do for dput

and I strongly recommend this.

Navigate to your home directory.

If there is not a .dput.cf file (and it would be hidden), please create it.

[my-ppa]
fqdn = ppa.launchpad.net
method = ftp
incoming = ~<your_launchpad_id>/<ppa_name>/ubuntu/
login = anonymous
allow_unsigned_uploads = 0

This is an example, however this is how you would configure it.

You may leave the login as anonymous but its recommended you use your own launchpad ID in place of anonymous

This is an example of a dput block, which I have for my addgpg-apt project:

[addgpg-apt]
fqdn = ppa.launchpad.net
method = ftp
incoming = ~addgpg-apt-developers/ppa
login = trekcaptainusa-tw
allow_unsigned_uploads = 0

After you have done this, and after you created the PPA on Launchpad, we are ready to continue.

sec... phone

sorry about that.

We are now ready to build the source package

If you followed the instructions through the first session, the procedure is similar to building the standalone deb

with one exception

if you did not yet build the program into a .deb from the first session, please disregard this next step.

If you built the program from the first session into a .deb, do the following in the directory that held the folder you unzipped/downloaded: rm packaging-test_1.0-*

Navigate into the source directory for the Debian package (the folder that contains any programs or source, and also has debian in it)

Run this command: debuild -S

Several files will be generated in the directory that is the parent of the source directory.

Go to the parent directory.

If you created the .dput.cf file as I specified earlier, you will only need to use the text that was between the [ ] in this step.

Type the following: dput my-ppa packaging-test_1.0-1_source.changes

Then allow the program to run (NOTE: It may ask for your launchpad password so you can upload to the PPA)

This will then upload all required files to the PPA system.

Subsequently, the package will be queued for building on Launchpad.

You will receive an email at your launchpad registered email and it will tell you whether the upload was accepted or rejected.

if accepted, then it will queue it for building

if rejected, it will explain why.

That's the basics of Packaging for Launchpad. Not entirely different from building standalone .deb installers.

Any questions?

Stefano Palazzo

8:19 PM

@TheEvilPhoenix no questions!

The Evil Phoenix

Then I thank you all for your time, and I thank you for listening to me teach about packaging.

Have a wonderful day.

*** END SESSIONS ***

Last question that did pop up:

in AskUbuntu Classroom Discussion - Packaging, 1 min ago, by Rinzwind

are there common reasons for 'rejecting' an upload? Some examples(?)

Here are a few examples of reasons an upload would be rejected:

duplicate versions of the package have been uploaded to the same PPA, unsigned packages were uploaded, or a failure in specifying which distro you're building for

the last of which is a failure to specify a correct distribution in the debian/changelog file

and i'd like to point you to one last item on PPAs:

5

Q: PPA & Packaging: Having versions of packages for multiple distros

(quote from chat) Got a PPA on launchpad for source packages for an IRC bot project I'm associated with (since its all the intermediate packaging for the stuff between releases, it's classified as an "experimental" PPA). Question: I've already packaged the stuff for a lucid build. Any clue ho...

The method outlined here in that question applies to ALL versions of packages uploaded to a PPA

You may copy any already-uploaded package that was built for Natty or other distros into the other distro

provided that they are compatible

if a dependency for the program is present in Maverick and later, but not in Lucid or earlier, the system will not catch it, but when you try and install the package from the PPA in Lucid or earlier, it will fail to install.

So make sure any dependencies you require are available in each of the distros you're building the packages for (or copying the packages to in PPAs).

Are there any other questions?

Notice: Two Minutes Remaining in this session

*** SESSION IS NOW OFFICIALLY OVER ***

Transcript for

♦ Classroom