summaryrefslogtreecommitdiffhomepage
path: root/blog/entry/tag2upload.mdwn
blob: ff3907c1bc946cf5424c41a64bc42edffd38552b (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
[[!meta title="Upload to Debian with just 'git tag' and 'git push'"]]

At a sprint over the weekend, Ian Jackson and I designed and
implemented a system to make it possible for Debian Developers to
upload new versions of packages by simply pushing a specially
formatted git tag to salsa (Debian's GitLab instance).  That's right:
the only thing you will have to do to cause new source and binary
packages to flow out to the mirror network is sign and push a git tag.

It works like this:

1. DD signs and pushes a git tag containing some metadata.  The tag is
   placed on the commit you want to release (which is probably the
   commit where you ran `dch -r`).

2. This triggers a GitLab webhook, which passes the public clone URI
   of your salsa project and the name of the newly pushed tag to a
   cloud service called *tag2upload*.

3. tag2upload verifies the signature on the tag against the Debian
   keyring,[^1] produces a `.dsc` and `.changes`, signs these, and
   uploads the result to ftp-master.[^2]

   (tag2upload does not have, nor need, push access to anyone's repos
   on salsa.  It doesn't make commits to the maintainer's branch.)

4. ftp-master and the autobuilder network push out the source and
   binary packages in the usual way.

The first step of this should be as easy as possible, so we've
produced a new script, `git debpush`, which just wraps `git tag` and
`git push` to sign and push the specially formatted git tag.

We've fully implemented tag2upload, though it's not running in the
cloud yet.  However, you can try out this workflow today by running
tag2upload on your laptop, as if in response to a webhook.  We did
this ourselves for a few real uploads to sid during the sprint.

1. First get the tools installed.  tag2upload reuses code from `dgit`
   and `dgit-infrastructure`, and lives in `bin:dgit-infrastructure`.
   `git debpush` is in a completely independent binary package which
   does not make any use of dgit.[^3]

   `% apt-get install git-debpush dgit-infrastructure dgit debian-keyring`

   (you need version 9.1 of the first three of these packages, only in
   Debian unstable at the time of writing.  However, the `.deb`s will
   install on stretch & buster.  And I'll put them in buster-backports
   once that opens.)

2. Prepare a source-only upload of some package that you normally push
   to salsa.  When you are ready to upload this, just type `git
   debpush`.

   If the package is non-native, you will need to pass a *quilt
   option* to inform tag2upload what [git branch
   layout](https://wiki.debian.org/GitPackagingSurvey) you are using
   -- it has to know this in order to produce a `.dsc`.  See
   [git-debpush(1)](https://manpages.debian.org/git-debpush) for the
   supported quilt options.

   The quilt option you select gets stored in the newly created tag,
   so for your next upload you won't need it, and `git debpush` alone
   will be enough.

   See the git-debpush(1) manpage for more options, but we've tried
   hard to ensure most users won't need any.

3. Now you need to simulate salsa's sending of a webhook to the
   tag2upload service.  This is how you can do that:

   ``` {.bash}
   % mkdir -p ~/tmp/t2u
   % cd ~/tmp/t2u
   % DGIT_DRS_EMAIL_NOREPLY=myself@example.org dgit-repos-server \
       debian . /usr/share/keyrings/debian-keyring.gpg,a --tag2upload \
       https://salsa.debian.org/dgit-team/dgit-test-dummy.git debian/1.23
   ```
   ... substituting your own service admin e-mail address, salsa repo
   URI and new tag name.

   Check the file `~/tmp/t2u/overall.log` to see what happened, and
   perhaps take a quick look at
   [Debian's upload queue](ftp://ftp.upload.debian.org/pub/UploadQueue/).

A few other notes about trying this out:

- tag2upload will delete various files and directories in your working
  directory, so be sure to invoke it in an empty directory like
  `~/tmp/t2u`.

- You won't see any console output, and (in my own case, partially as
  a consequence of that) the command might feel a bit slow.  Neither
  of these will matter when tag2upload is running as a cloud service,
  of course.  If there is an error, you'll get an e-mail.

- Running the script like this on your laptop will use your default
  PGP key to sign the `.dsc` and `.changes`.  The real cloud service
  will have its own PGP key.

- The shell invocation given above is complicated, but once the cloud
  service is deployed, no human is going to ever need to type it!

  What's important to note is the two pieces of user input the command
  takes: your salsa repo URI, and the new tag name.  The GitLab web
  hook will provide the tag2upload service with (only) these two
  parameters.

For some more discussion of this new workflow, see the
[git-debpush(1)](https://manpages.debian.org/git-debpush) manpage.  We
hope you have fun trying it out.

[^1]: Unfortunately, DMs can't try tag2upload out on their laptops,
    though they will certainly be able to use the final cloud service
    version of tag2upload.

[^2]: Only source-only uploads are supported, but this is by design.

[^3]: Do not be fooled by the string 'dgit' appearing in the generated
    tags!  We are just reusing a tag metadata convention that dgit
    also uses.