Arch Linux: Resolve GPG key signing issues during pacman package upgrades

Tidbits Jan 6, 2021

I ran into an issue running through Arch Linux package upgrades on 2021-01-06, after neglecting to install package upgrades for a few months:

[miliarch@arch ~]$ sudo pacman -Syu
:: Synchronizing package databases...
 core                132.8 KiB  18.5 MiB/s 00:00 [##################################] 100%
 extra              1637.1 KiB  28.0 MiB/s 00:00 [##################################] 100%
 community             5.3 MiB  65.3 MiB/s 00:00 [##################################] 100%
 sublime-text       1827.0   B  0.00   B/s 00:00 [##################################] 100%
 sublime-text.sig    543.0   B  0.00   B/s 00:00 [##################################] 100%
:: Starting full system upgrade...
... -truncated- ...
(541/541) checking keys in keyring                             [##################################] 100%
(541/541) checking package integrity                           [##################################] 100%
error: cdrtools: signature from "Jerome Leclanche <jerome@leclan.ch>" is unknown trust
:: File /var/cache/pacman/pkg/cdrtools-3.02a09-4-x86_64.pkg.tar.zst is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n] y
error: failed to commit transaction (invalid or corrupted package)
Errors occurred, no packages were upgraded.

It turns out the developer's GPG key, which determines whether the author of the package is trusted, had expired:

[miliarch@arch ~]$ pacman-key --list-sigs Jerome
gpg: Note: trustdb not writable
pub   rsa4096 2013-10-16 [SC] [expired: 2021-01-01]
      169704C6FB490C6892C7F23C37E0AF1FDA48F373
uid           [ expired] Jerome Leclanche <jerome@leclan.ch>
sig          3348882F6AC6A4C2 2015-02-10  Pierre Schmitz (Arch Linux Master Key) <pierre@master-key.archlinux.org>
sig          BA1DFB64FFF979E7 2015-02-12  Allan McRae (Arch Linux Master Key) <allan@master-key.archlinux.org>
sig          A88E23E377514E00 2015-12-23  Florian Pritz (Arch Linux Master Key) <florian@master-key.archlinux.org>
sig 3        37E0AF1FDA48F373 2018-06-10  Jerome Leclanche <jerome@leclan.ch>
sig          D6D055F927843F1C 2019-09-29  Levente Polyak (Arch Linux Master Key) <anthraxx@master-key.archlinux.org>

After some searching, I came across this thread, which contained the solution. I did try using pacman-key --refresh-keys, as suggested by Scimmia, and while it did refresh many keys (see command output below), it didn't pull in an update of the problematic key.

[miliarch@arch ~]$ sudo pacman-key --refresh-keys
gpg: refreshing 121 keys from hkps://hkps.pool.sks-keyservers.net
gpg: keyserver refresh failed: Server indicated a failure
==> ERROR: A specified local key could not be updated from a keyserver.

Note: The ERROR can be disregarded as it's against my system's local key, which does not exist on the server side

After further review of the thread, and some more tinkering in the terminal, I discovered that the archlinux-keyring package required an update to pull in updated GPG keys. I'm very thankful that these folks took the time to both ask the question and drop in guidance on how to validate and fix the problem.

Now, the main crux of the issue was that while archlinux-keyring was part of the list of distribution packages to update, the one package associated with an expired key (cdrtools) "clogged up the works" and prevented the upgrade from completing. To work around that, I pulled this package separately with pacman -S archlinux-keyring:

[miliarch@arch ~]$ sudo pacman -S archlinux-keyring
resolving dependencies...
looking for conflicting packages...

Packages (1) archlinux-keyring-20201210-1

Total Installed Size:  1.30 MiB
Net Upgrade Size:      0.00 MiB

:: Proceed with installation? [Y/n] y
(1/1) checking keys in keyring                                                 [############################################] 100%
(1/1) checking package integrity                                               [############################################] 100%
(1/1) loading package files                                                    [############################################] 100%
(1/1) checking for file conflicts                                              [############################################] 100%
(1/1) checking available disk space                                            [############################################] 100%
:: Processing package changes...
(1/1) upgrading archlinux-keyring                                              [############################################] 100%
==> Appending keys from archlinux.gpg...
==> Locally signing trusted keys in keyring...
  -> Locally signing key D8AFDDA07A5B6EDFA7D8CCDAD6D055F927843F1C...
  -> Locally signing key DDB867B92AA789C165EEFA799B729B06A680C281...
  -> Locally signing key 91FFE0700E80619CEB73235CA88E23E377514E00...
  -> Locally signing key 0E8B644079F599DFC1DDC3973348882F6AC6A4C2...
  -> Locally signing key AB19265E5D7D20687D303246BA1DFB64FFF979E7...
==> Importing owner trust values...
==> Disabling revoked keys in keyring...
  -> Disabling key 8F76BEEA0289F9E1D3E229C05F946DED983D4366...
  -> Disabling key 63F395DE2D6398BBE458F281F2DBB4931985A992...
  -> Disabling key 50F33E2E5B0C3D900424ABE89BDCF497A4BBCC7F...
  -> Disabling key 27FFC4769E19F096D41D9265A04F9397CDFD6BB0...
  -> Disabling key 779CD2942629B7FA04AB8F172E89012331361F01...
  -> Disabling key 39F880E50E49A4D11341E8F939E4F17F295AFBF4...
  -> Disabling key 8840BD07FC24CB7CE394A07CCF7037A4F27FB7DA...
  -> Disabling key 5559BC1A32B8F76B3FCCD9555FA5E5544F010D48...
  -> Disabling key 0B20CA1931F5DA3A70D0F8D2EA6836E1AB441196...
  -> Disabling key 07DFD3A0BC213FA12EDC217559B3122E2FA915EC...
  -> Disabling key 4FCF887689C41B09506BE8D5F3E1D5C5D30DB0AD...
  -> Disabling key 5A2257D19FF7E1E0E415968CE62F853100F0D0F0...
  -> Disabling key D921CABED130A5690EF1896E81AF739EC0711BF1...
  -> Disabling key 7FA647CD89891DEDC060287BB9113D1ED21E1A55...
  -> Disabling key BC1FBE4D2826A0B51E47ED62E2539214C6C11350...
  -> Disabling key 4A8B17E20B88ACA61860009B5CED81B7C2E5C0D2...
  -> Disabling key 5696C003B0854206450C8E5BE613C09CB4440678...
  -> Disabling key 684148BB25B49E986A4944C55184252D824B18E8...
  -> Disabling key 8CF934E339CAD8ABF342E822E711306E3C4F88BC...
  -> Disabling key F5A361A3A13554B85E57DDDAAF7EF7873CFD4BB6...
  -> Disabling key 5E7585ADFF106BFFBBA319DC654B877A0864983E...
  -> Disabling key 65EEFE022108E2B708CBFCF7F9E712E59AF5F22A...
  -> Disabling key 40440DC037C05620984379A6761FAD69BA06C6A9...
  -> Disabling key 34C5D94FE7E7913E86DC427E7FB1A3800C84C0A5...
  -> Disabling key 1A60DC44245D06FEF90623D6EEEEE2EEEE2EEEEE...
  -> Disabling key 81D7F8241DB38BC759C80FCE3A726C6170E80477...
  -> Disabling key E7210A59715F6940CF9A4E36A001876699AD6E84...
  -> Disabling key 5357F3B111688D88C1D88119FCF2CB179205AC90...
  -> Disabling key 4D913AECD81726D9A6C74F0ADA6426DD215B37AD...
  -> Disabling key FB871F0131FEA4FB5A9192B4C8880A6406361833...
  -> Disabling key 66BD74A036D522F51DD70A3C7F2A16726521E06D...
  -> Disabling key 487EACC08557AD082088DABA1EB2638FF56C0C53...
  -> Disabling key B1F2C889CB2CCB2ADA36D963097D629E437520BD...
  -> Disabling key 9515D8A8EAB88E49BB65EDBCE6B456CAF15447D5...
  -> Disabling key 76B4192E902C0A52642C63C273B8ED52F1D357C1...
  -> Disabling key 40776A5221EF5AD468A4906D42A1DB15EC133BAD...
  -> Disabling key D4DE5ABDE2A7287644EAC7E36D1A9E70E19DAA50...
  -> Disabling key 44D4A033AC140143927397D47EFD567D4C7EA887...
==> Updating trust database...
gpg: next trustdb check due at 2021-08-02
:: Running post-transaction hooks...
(1/1) Arming ConditionNeedsUpdate...

After checking the public GPG key for this author again, it was clear that it had been updated with an expiration date in the future in my local keyring:

[miliarch@arch ~]$ pacman-key --list-sigs Jerome
gpg: Note: trustdb not writable
pub   rsa4096 2013-10-16 [SC] [expires: 2023-01-01]
      169704C6FB490C6892C7F23C37E0AF1FDA48F373
uid           [  full  ] Jerome Leclanche <jerome@leclan.ch>
sig          3348882F6AC6A4C2 2015-02-10  Pierre Schmitz (Arch Linux Master Key) <pierre@master-key.archlinux.org>
sig          BA1DFB64FFF979E7 2015-02-12  Allan McRae (Arch Linux Master Key) <allan@master-key.archlinux.org>
sig          A88E23E377514E00 2015-12-23  Florian Pritz (Arch Linux Master Key) <florian@master-key.archlinux.org>
sig 3        37E0AF1FDA48F373 2018-06-10  Jerome Leclanche <jerome@leclan.ch>
sig          D6D055F927843F1C 2019-09-29  Levente Polyak (Arch Linux Master Key) <anthraxx@master-key.archlinux.org>
sig 3        37E0AF1FDA48F373 2020-10-28  Jerome Leclanche <jerome@leclan.ch>
sub   rsa4096 2013-10-16 [E] [expires: 2023-01-01]
sig          37E0AF1FDA48F373 2018-06-10  Jerome Leclanche <jerome@leclan.ch>
sig          37E0AF1FDA48F373 2020-10-28  Jerome Leclanche <jerome@leclan.ch>

My next attempt to run a full distro upgrade finished without an issue.

The lesson to take from this is that it may be wise to run a standalone upgrade of the archlinux-keyring package prior to running a full distribution upgrade, especially if it's been a while since you've upgraded packages. Alternatively, it's certainly a valid strategy to wait until an error is presented to address a problem; it's good to keep this package in mind if you encounter a GPG key signing issue on a core package when working with pacman.

Finding documentation on archlinux-keyring wasn't trivial, but the Adding Developer Keys section of the pacman/Package signing page in the Arch Wiki touches on the purpose of the package, which is effectively a collection of Trusted User (TU) GPG keys.

Until next time

- miliarch