I didn't notice this because the script was exiting with a 0 exit code. I've pushed an update to parabolaweb-utils so that errors get noticed sooner (and the systemd service fails), but IDK what's causing the error.

Of the 67 (wow, that's a lot!) total repos, there are errors from reporead on 36 of them. Not all the same error:

(25) django.db.utils.DataError: integer out of range

    community/os/armv7h/community.files.tar.gz
    community/os/i686/community.files.tar.gz
    community/os/x86_64/community.files.tar.gz
    core/os/armv7h/core.files.tar.gz
    core/os/i686/core.files.tar.gz
    core/os/x86_64/core.files.tar.gz
    extra/os/armv7h/extra.files.tar.gz
    extra/os/i686/extra.files.tar.gz
    extra/os/x86_64/extra.files.tar.gz
    java/os/i686/java.files.tar.gz
    java/os/x86_64/java.files.tar.gz
    libre/os/armv7h/libre.files.tar.gz
    libre/os/i686/libre.files.tar.gz
    libre/os/x86_64/libre.files.tar.gz
    nonprism/os/i686/nonprism.files.tar.gz
    pcr/os/armv7h/pcr.files.tar.gz
    pcr/os/i686/pcr.files.tar.gz
    pcr/os/x86_64/pcr.files.tar.gz
    testing/os/armv7h/testing.files.tar.gz
    testing/os/i686/testing.files.tar.gz
    testing/os/x86_64/testing.files.tar.gz
    ~lukeshu/os/i686/~lukeshu.files.tar.gz
    ~lukeshu/os/x86_64/~lukeshu.files.tar.gz
    multilib-testing/os/x86_64/multilib-testing.files.tar.gz
    multilib/os/x86_64/multilib.files.tar.gz

(8) AttributeError: 'NoneType' object has no attribute 'encode'

    libre-testing/os/i686/libre-testing.files.tar.gz
    libre-testing/os/x86_64/libre-testing.files.tar.gz
    ~jorginho/os/i686/~jorginho.files.tar.gz
    ~jorginho/os/x86_64/~jorginho.files.tar.gz
    ~smv/os/i686/~smv.files.tar.gz
    ~smv/os/x86_64/~smv.files.tar.gz
    ~xihh/os/i686/~xihh.files.tar.gz
    ~xihh/os/x86_64/~xihh.files.tar.gz

(2) main.models.DoesNotExist: Repo matching query does not exist.

    ~fauno/os/i686/~fauno.files.tar.gz
    ~fauno/os/x86_64/~fauno.files.tar.gz

(1) tarfile.ReadError: file could not be opened successfully

    extra/os/x86_64/community.files.tar.gz

The tarfile.ReadError is probably transient because of db-import-pkg not updating the file atomically.

TBH, I'm not too worried about The AttributeError (except maybe on [libre-testing]). I'm pretty sure it's caused by old packages that don't have PGP signatures; which makes sense--it's mostly on personal repos of hackers I haven't seen around in quite a while.

Now, django.db.utils.DataError: integer out of range is the target of real concern. I have no idea about what's causing it.

parabolitsa=> select max(id) from package_files;                                                                                                
    max                                                                                                                                         
------------                                                                                                                                    
 2147483413                                                                                                                                     
(1 row) 
parabolitsa=> \d+ package_files                                                                                                                 
                                                        Table "public.package_files"                                                            
    Column    |          Type           |                         Modifiers                          | Storage  | Stats target | Description    
--------------+-------------------------+------------------------------------------------------------+----------+--------------+-------------   
 id           | integer                 | not null default nextval('package_files_id_seq'::regclass) | plain    |              |                
...

integer 4 bytes typical choice for integer -2147483648 to +2147483647

https://www.postgresql.org/docs/9.1/static/datatype-numeric.html

$ bc
2147483647 - 2147483413
234

parabolitsa=> select * from package_files_id_seq;
    sequence_name     | last_value | start_value | increment_by |      max_value      | min_value | cache_value | log_cnt | is_cycled | is_called                                                                                                                                              
----------------------+------------+-------------+--------------+---------------------+-----------+-------------+---------+-----------+----------
 package_files_id_seq | 2147504862 |           1 |            1 | 9223372036854775807 |         1 |           1 |      32 | f         | t
(1 row)

bc
2147483647 - 2147504862
-21215

currently running:

parabolitsa=> ALTER SEQUENCE package_files_id_seq RESTART;                                                                                      
ALTER SEQUENCE                                                                                                                                  
parabolitsa=> UPDATE package_files SET id = DEFAULT;                                                                                            

Both CPUs are pegged on iowait, iotop tells me postgresql is dominating, although XFS-related kernel tasks also pop up. That makes sense, just wait it out.

systemd-journald is appearing too, which doesn't make too much sense to me.

Followed that with VACUUM ANALYZE package_files, and things seem mostly OK now.

But I've temporarily disabled parabolaweb-reporead-rsync.timer anyway. It seems that for some reason, some packages are deleted/recreated each run. Include a few packages with tens of thousands of files (like rocksndiamonds-data) in that, and that would explain the churn that lead to us overflowing an int32.

So I'm looking in to why that's happening. I suspect it's related to the signature errors that oaken-source is seeing.

Oh, I should verify that SELECT last_value FROM package_files_id_seq is >= SELECT max(id) FROM package_files.

They are in fact ==, which is what is expected.

With 3 consecutive parabolaweb-reporead-rsync runs with the rsync bit disabled (so we're reporeading over static non-changing data), we get the same output for each, with slight variation in package order. Each run takes 10-20 minutes.

It looks like [core] is a good place to debug this--there's some churn on core, but it's just a handful (<10) packages, each with a small number of files.

I just want to point out that db-import-pkg hasn't been importing anything anyway since the introduction of this commit . the script halts as soon as it fails to find ${LOCAL_CONFIG}. is that behaviour part of the ongoing debugging efforts?

No, I don't think that's affecting this.

However... whuuuu? The [[ -f ]] guard is meant to handle that. I know the overall status is != 0 for that expression, but set -e doesn't trip for commands with && in them. Is trap ... ERR what's tripping?

Oh, but the non-0 $? is also taken to be the return code of the source command in db-import-pkg:9.

FYI, I've just pushed a fix for db-import-pkg crashing (dbscripts 20180401).

lukeshu wrote:

No, I don't think that's affecting this.

i understand that. i was rather asking if your investigations of #note-14 would be disrupted by my appending of || : to db-import-pkg:9 or something along those lines.

many thanks to both of you and everyone who remains active.

this seems to be working now at least for the upstream imports - is it completely fixed for all repos ?

Until the "churn" mentioned in #1666-10, #1666-13, #1666-14 is fixed: Even when we "fix" the PostgreSQL DB (as in #1666-8, #1666-10), the django.db.utils.DataError: integer out of range issue will keep eventually coming back.

It's worked it's way back up and it's having that issue again now.

I'm trying to do the sequence again:

ALTER SEQUENCE package_files_id_seq RESTART;
UPDATE package_files SET id = DEFAULT;
VACUUM ANALYZE package_files;

But because some of the low ids are still used, I'm having to do the UPDATE in batches.

The churn appears to be very bad. I've sudo systemctl stop parabolaweb-reporead-rsync.timer in the mean time.

It's being caused by arch=(any) packages that differ between arches of the same repo, because of db-import-pkg.

1. You're saying that shouldn't happen with current db-import-pkg? Well it is! Between ALARM and Arch.
2. But they're in different pools, surely there's some isolation going on? Well, the assumption in the Arch tools (dbscripts, and archweb) is that all arches of a given reponame share the same pool; eg [extra] and [community] don't need to share the same pool, but [community] (armv7h) and [community] (i686) do.
3. Even if we fix the pool selection bug in db-import-pkg (item 1), that doesn't fix the problem that some arch=(any) packages are present in a repo on one arch but not others, like archlinux32-keyring getting imported to [core] on i686, but [libre] on other arches. This is almost certainly exacerbated by Arch32 holding packages in testing repos for longer than us/Arch/ALARM.

A hack that I'm tempted to do is to have reporead override the package's architecture, and pretend that all packages have the architecture of the repo they're in.

Yeah, until we make some long-term changes to db-import, that's what we'll have to do.

1. Still, it's a short-term bug that db-import-pkg is allowing the same filename in multiple pools.
2. db-update should forbid releasing a file that exists in another pool, unless the new file is a bit-for-bit copy of the old file (as has happened with packages released to [nonsystemd] having the same filename as packages in the packages pool.

I've modified reporead and added fixtures so now any packages show up as "any_${tarch}".

I'm now waiting on it to complete Package.objects.filter(arch_id=1).delete() to remove the packages with any as the arch, as reporead refuses to do it because it things something is wrong.

Given that this bug involves multiple edge cases across multiple different pieces of infrastructure, I think it's fitting that the issue number is #1666.

Oh, I should have just updated the arch_id to be the any_x86_64 id, it would have been faster, and the subsequent reporead would have been faster too. Oh well.

I'm not sure what's wrong, but the any_ arches aren't working, I'm going to go straight to tarch.

lukeshu wrote:

Given that this bug involves multiple edge cases across multiple different pieces of infrastructure, I think it's fitting that the issue number is #1666.

This means it will reoccur at 2666.

Oh, wait, it was this bit in reporead.py:

    arch in Arch.objects.filter(agnostic=True):                                                                                             
        packages_arches[arch.name] = []

Well, it was already done: no more "any" arch in parabolaweb.

Remaining errors from reporead:

 (10) AttributeError: 'NoneType' object has no attribute 'encode'

          Traceback (most recent call last):
          …
            File "/srv/http/www.parabola.nu/web/devel/management/commands/reporead.py", line 232, in populate_pkg
              dbpkg.signature_bytes = b64decode(repopkg.pgpsig.encode('utf-8'))
          AttributeError: 'NoneType' object has no attribute 'encode'

      libre-testing/os/i686/libre-testing.files.tar.gz
      libre-testing/os/x86_64/libre-testing.files.tar.gz
      ~jorginho/os/i686/~jorginho.files.tar.gz
      ~jorginho/os/x86_64/~jorginho.files.tar.gz
      ~lukeshu/os/i686/~lukeshu.files.tar.gz
      ~lukeshu/os/x86_64/~lukeshu.files.tar.gz
      ~smv/os/i686/~smv.files.tar.gz
      ~smv/os/x86_64/~smv.files.tar.gz
      ~xihh/os/i686/~xihh.files.tar.gz
      ~xihh/os/x86_64/~xihh.files.tar.gz

  (6) main.models.DoesNotExist: Repo matching query does not exist.

      nonsystemd-multilib/os/x86_64/nonsystemd-multilib.files.tar.gz
      nonsystemd/os/armv7h/nonsystemd.files.tar.gz
      nonsystemd/os/i686/nonsystemd.files.tar.gz
      nonsystemd/os/x86_64/nonsystemd.files.tar.gz
      ~fauno/os/i686/~fauno.files.tar.gz
      ~fauno/os/x86_64/~fauno.files.tar.gz

A no-op run of reporead now takes about 2.5 minutes.

I'm going to go ahead and re-enable the timer.

The missing repos have been added to parabolaweb. That just leaves unsigned packages.

Unsigned packages:

Filename                                                                Packager                                  Build date
-------------------------------------------------------------------------------------------------------------------------------------
libre-testing/os/i686/mozilla-searchplugins-1.0-1-any.pkg.tar.xz        Unknown Packager                          2011-12-27 02:14:04
libre-testing/os/x86_64/iceweasel-libre-9.0.1.1-2-x86_64.pkg.tar.gz     Luke Shumaker <lukeshu@sbcglobal.net>     2012-01-04 01:08:56
libre-testing/os/x86_64/mozilla-searchplugins-1.0-1-any.pkg.tar.xz      Unknown Packager                          2011-12-27 02:14:04
~fauno/os/i686/argyllcms-1.3.3-1-i686.pkg.tar.xz                        Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-07-28 17:41:35
~fauno/os/i686/distcc-zeroconf-3.1-1-i686.pkg.tar.xz                    Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-06-21 15:35:05
~fauno/os/i686/emerillon-0.1.90-2-i686.pkg.tar.xz                       Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-09-24 04:54:46
~fauno/os/i686/ethos-0.2.2-6-i686.pkg.tar.xz                            Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-09-24 04:19:04
~fauno/os/i686/faenza-icon-theme-1.2-2-any.pkg.tar.xz                   Nicolás Reynolds <fauno@kiwwwi.com.ar>    2012-08-31 19:13:05
~fauno/os/i686/geoclue-0.12-2-i686.pkg.tar.xz                           Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-09-24 03:56:56
~fauno/os/i686/get_flash_videos-1.24-2-i686.pkg.tar.xz                  Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-07-20 22:22:18
~fauno/os/i686/gnome-schedule-2.1.5-1-any.pkg.tar.xz                    Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-10-04 00:38:46
~fauno/os/i686/haskell-base64-bytestring-0.1.0.3-1-i686.pkg.tar.xz      Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-09-26 01:56:13
~fauno/os/i686/haskell-citeproc-hs-0.3.3-1-i686.pkg.tar.xz              Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-09-26 18:47:44
~fauno/os/i686/haskell-digest-0.0.1.0-1-i686.pkg.tar.xz                 Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-09-26 01:04:09
~fauno/os/i686/haskell-dlist-0.5-18-i686.pkg.tar.xz                     Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-09-26 18:51:15
~fauno/os/i686/haskell-hs-bibutils-4.12-4-i686.pkg.tar.xz               Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-09-26 02:49:02
~fauno/os/i686/haskell-json-0.4.4-18-i686.pkg.tar.xz                    Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-09-26 17:00:09
~fauno/os/i686/haskell-pandoc-types-1.8.2-1-i686.pkg.tar.xz             Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-09-26 18:21:41
~fauno/os/i686/haskell-tagsoup-0.12.3-1-i686.pkg.tar.xz                 Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-09-26 18:58:19
~fauno/os/i686/haskell-texmath-0.5.0.1-19-i686.pkg.tar.xz               Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-09-26 02:08:32
~fauno/os/i686/haskell-xml-1.3.9-1-i686.pkg.tar.xz                      Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-09-26 02:03:15
~fauno/os/i686/haskell-zip-archive-0.1.1.7-19-i686.pkg.tar.xz           Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-09-26 01:09:18
~fauno/os/i686/libchamplain-0.12.0-1-i686.pkg.tar.xz                    Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-09-24 04:44:23
~fauno/os/i686/librest-0.7.10-1-i686.pkg.tar.xz                         Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-09-24 04:32:55
~fauno/os/i686/python-html2text-3.02-1-any.pkg.tar.xz                   Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-07-27 15:35:55
~fauno/os/i686/rss2email-2.71-2-any.pkg.tar.xz                          Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-07-27 15:42:43
~fauno/os/i686/ruby-enterprise-1.8.7-12-i686.pkg.tar.xz                 Nicolás Reynolds <fauno@kiwwwi.com.ar>    2012-02-11 14:51:58
~fauno/os/i686/ruby-minitar-0.5.2-1-any.pkg.tar.xz                      Nicolás Reynolds <fauno@kiwwwi.com.ar>    2012-01-10 16:26:34
~fauno/os/i686/subtle-hg-3171-1-i686.pkg.tar.xz                         Nicolás Reynolds <fauno@kiwwwi.com.ar>    2012-01-10 16:30:32
~fauno/os/i686/transmission-remote-cli-0.10.4-1-any.pkg.tar.xz          Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-11-11 20:53:01
~fauno/os/i686/transmission-remote-cli-git-20110623-1-any.pkg.tar.xz    fauno <fauno@kiwwwi.com.ar>               2011-06-23 05:18:00
~fauno/os/i686/vala-notmuch-20120112-1-i686.pkg.tar.xz                  Nicolás Reynolds <fauno@kiwwwi.com.ar>    2012-01-12 14:44:44
~fauno/os/i686/vim-conque-2.3-1-any.pkg.tar.xz                          Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-09-28 17:37:29
~fauno/os/x86_64/faenza-icon-theme-1.2-2-any.pkg.tar.xz                 Nicolás Reynolds <fauno@kiwwwi.com.ar>    2012-08-31 19:13:05
~fauno/os/x86_64/gnome-schedule-2.1.5-1-any.pkg.tar.xz                  Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-10-04 00:38:46
~fauno/os/x86_64/python-html2text-3.02-1-any.pkg.tar.xz                 Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-07-27 15:35:55
~fauno/os/x86_64/rss2email-2.71-2-any.pkg.tar.xz                        Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-07-27 15:42:43
~fauno/os/x86_64/ruby-minitar-0.5.2-1-any.pkg.tar.xz                    Nicolás Reynolds <fauno@kiwwwi.com.ar>    2012-01-10 16:26:34
~fauno/os/x86_64/transmission-remote-cli-0.10.4-1-any.pkg.tar.xz        Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-11-11 20:53:01
~fauno/os/x86_64/transmission-remote-cli-git-20110623-1-any.pkg.tar.xz  fauno <fauno@kiwwwi.com.ar>               2011-06-23 05:18:00
~fauno/os/x86_64/vim-conque-2.3-1-any.pkg.tar.xz                        Nicolás Reynolds <fauno@kiwwwi.com.ar>    2011-09-28 17:37:29
~jorginho/os/i686/firestarter-1.0.3-12-i686.pkg.tar.xz                  Jorge Lopez <jorginho@adinet.com.uy>      2012-04-16 16:58:23
~jorginho/os/x86_64/firestarter-1.0.3-12-x86_64.pkg.tar.xz              Jorge Lopez <jorginho@adinet.com.uy>      2012-04-16 22:49:46
~lukeshu/os/i686/conkeror-git-20120228-1-any.pkg.tar.xz                 Luke Shumaker <lukeshu@sbcglobal.net>     2012-02-28 07:21:02
~lukeshu/os/x86_64/conkeror-git-20120228-1-any.pkg.tar.xz               Luke Shumaker <lukeshu@sbcglobal.net>     2012-02-28 07:21:02
~smv/os/i686/eggdrop-1.8-1-i686.pkg.tar.xz                              Sorin-Mihai Vârgolici <smv@yobicore.org>  2011-07-28 16:47:44
~smv/os/i686/gitosis-git-20110728-1-any.pkg.tar.xz                      Sorin-Mihai Vârgolici <smv@yobicore.org>  2011-07-28 06:16:09
~smv/os/i686/lxc-0.7.4.1-1-i686.pkg.tar.xz                              Sorin-Mihai Vârgolici <smv@yobicore.org>  2011-07-25 09:30:18
~smv/os/i686/pan-0.135-1-i686.pkg.tar.xz                                Sorin-Mihai Vârgolici <smv@yobicore.org>  2011-07-25 10:02:13
~smv/os/i686/planner-0.14.5-1-i686.pkg.tar.xz                           Sorin-Mihai Vârgolici <smv@yobicore.org>  2011-07-25 10:24:44
~smv/os/x86_64/eggdrop-1.8-1-x86_64.pkg.tar.xz                          Sorin-Mihai Vârgolici <smv@yobicore.org>  2011-07-28 16:44:05
~smv/os/x86_64/gitosis-git-20110728-1-any.pkg.tar.xz                    Sorin-Mihai Vârgolici <smv@yobicore.org>  2011-07-28 06:16:09
~smv/os/x86_64/lxc-0.7.4.1-1-x86_64.pkg.tar.xz                          Sorin-Mihai Vârgolici <smv@yobicore.org>  2011-07-25 09:28:55
~smv/os/x86_64/pan-0.135-1-x86_64.pkg.tar.xz                            Sorin-Mihai Vârgolici <smv@yobicore.org>  2011-07-25 09:47:52
~smv/os/x86_64/planner-0.14.5-1-x86_64.pkg.tar.xz                       Sorin-Mihai Vârgolici <smv@yobicore.org>  2011-07-25 10:15:27
~xihh/os/i686/couchapp-git-20120413-1-any.pkg.tar.xz                    Joshua I. Haase H. <hahj87@gmail.com>     2012-04-13 09:05:35
~xihh/os/i686/python2-argh-0.15.1-1-any.pkg.tar.xz                      Joshua I. Haase H. <hahj87@gmail.com>     2012-04-13 08:53:26
~xihh/os/i686/python2-pathtools-20120413-1-any.pkg.tar.xz               Joshua I. Haase H. <hahj87@gmail.com>     2012-04-13 08:48:59
~xihh/os/i686/python2-restkit-4.0.0-1-any.pkg.tar.xz                    Joshua I. Haase H. <hahj87@gmail.com>     2012-04-13 08:57:41
~xihh/os/i686/python2-socketpool-0.2.3-1-any.pkg.tar.xz                 Joshua I. Haase H. <hahj87@gmail.com>     2012-04-13 08:47:45
~xihh/os/i686/python2-watchdog-0.6.0-1-any.pkg.tar.xz                   Joshua I. Haase H. <hahj87@gmail.com>     2012-04-13 08:54:54
~xihh/os/x86_64/couchapp-git-20120413-1-any.pkg.tar.xz                  Joshua I. Haase H. <hahj87@gmail.com>     2012-04-13 09:05:35
~xihh/os/x86_64/python2-argh-0.15.1-1-any.pkg.tar.xz                    Joshua I. Haase H. <hahj87@gmail.com>     2012-04-13 08:53:26
~xihh/os/x86_64/python2-pathtools-20120413-1-any.pkg.tar.xz             Joshua I. Haase H. <hahj87@gmail.com>     2012-04-13 08:48:59
~xihh/os/x86_64/python2-restkit-4.0.0-1-any.pkg.tar.xz                  Joshua I. Haase H. <hahj87@gmail.com>     2012-04-13 08:57:41
~xihh/os/x86_64/python2-socketpool-0.2.3-1-any.pkg.tar.xz               Joshua I. Haase H. <hahj87@gmail.com>     2012-04-13 08:47:45
~xihh/os/x86_64/python2-watchdog-0.6.0-1-any.pkg.tar.xz                 Joshua I. Haase H. <hahj87@gmail.com>     2012-04-13 08:54:54

All of those are from either 2011 or 2012.

[repo@winston ~]$ db-remove libre-testing any mozilla-searchplugins
==> Removing packages from [libre-testing]...
  -> mozilla-searchplugins (x86_64)
    (none)
  -> mozilla-searchplugins (i686)
    (none)
  -> mozilla-searchplugins (armv7h)
    (none)
==> WARNING: No packages remain, creating empty database.

Oh look, it's a phantom package.

lukeshu wrote:

That just leaves unsigned packages.

is that #1652 ?

there is also the issue of the "maintainer" filter #1897

Discrepancies between desc files in the .files and .db databases:

[lukeshu@beefcake repo-sanity]$ diff -r db files
Only in files/libre-testing/os/i686: mozilla-searchplugins-1.0-1
Only in files/libre-testing/os/x86_64: iceweasel-libre-9.0.1.1-2
Only in files/libre-testing/os/x86_64: mozilla-searchplugins-1.0-1
Only in files/pcr/os/x86_64: sfml-git-20120812-1
Only in files/~fauno/os/i686: argyllcms-1.3.3-1
Only in files/~fauno/os/i686: get_flash_videos-1.24-2
Only in files/~fauno/os/i686: python-html2text-3.02-1
Only in files/~fauno/os/i686: rss2email-2.71-2
Only in files/~fauno/os/x86_64: python-html2text-3.02-1
Only in files/~fauno/os/x86_64: rss2email-2.71-2
Only in files/~smv/os/i686: eggdrop-1.8-1
Only in files/~smv/os/i686: gitosis-git-20110728-1
Only in files/~smv/os/i686: lxc-0.7.4.1-1
Only in files/~smv/os/i686: pan-0.135-1
Only in files/~smv/os/i686: planner-0.14.5-1
Only in files/~smv/os/x86_64: eggdrop-1.8-1
Only in files/~smv/os/x86_64: gitosis-git-20110728-1
Only in files/~smv/os/x86_64: lxc-0.7.4.1-1
Only in files/~smv/os/x86_64: pan-0.135-1
Only in files/~smv/os/x86_64: planner-0.14.5-1

bill-auger wrote:

lukeshu wrote:

That just leaves unsigned packages.

is that #1652 ?

No, that's something else (mis-configured makepkg.conf?) The unsigned packages thing is packages that were uploaded without a PGP signature. Consider https://www.parabola.nu/packages/testing/i686/python-astroid/ it says "Signed By: Unknown (0xA952AC0F)" because Erich's PGP key isn't set up in parabolaweb, but it at least has the key fingerprint there.

I just ran:

ssh repo@repo.parabola.nu
cd /srv/repo
(set -euE -o pipefail; find main/~{fauno,jorginho,smv,xihh} -not -type d|while read -r src; do dst=http/${src#main/}; mkdir -p -- "${dst%/*}"; mv -Tv -- "$src" "$dst"; done)
find main/~{fauno,jorginho,smv,xihh} -depth -type d -exec rmdir -- {} \;

I manually deleted these packages from parabolaweb:

• ~xihh/i686/django-merengue
• ~xihh-x86_64/django-merengue

Then I deleted these repos from parabolaweb:

• ~fauno
• ~jorginho
• ~smv
• ~xihh

I've tricked repo-remove in to removing the packages from the repos with discrepancies by renaming the repo.files.tar.gz to repo.db.tar.gz in a temporary directory.

It was also complaining that [build-support] for i686 had only 45% of the packages in the web DB. So I've marked it as a staging repo in parabolaweb to skip that check.

I've now run:

db-remove '~lukeshu' x86_64 conkeror-git
db-remove '~lukeshu' i686 conkeror-git

When I'd manually edited the .files databases, they lost g+w permissions, so only I could push to [pcr] or [libre-testing].