Project

General

Profile

Bug #2141

libremakepkg failing to build package due to read-only startdir in recent librechroots

GNUtoo - 8 months ago - . Updated 12 days ago.

Status:
confirmed
Priority:
discussion
Assignee:
% Done:

0%


Description

I've tried building it inside an i686 chroot with:

$ sudo libremakepkg -n parabola-i686

and this gives:
  INSTALL sound/usb/snd-usb-audio.ko
  INSTALL sound/usb/snd-usbmidi-lib.ko
  INSTALL sound/usb/usx2y/snd-usb-us122l.ko
  INSTALL sound/usb/usx2y/snd-usb-usx2y.ko
  INSTALL sound/x86/snd-hdmi-lpe-audio.ko
  INSTALL virt/lib/irqbypass.ko
  DEPMOD  4.19.8-gnu-1
  -> Installing hooks...
/startdir/PKGBUILD: line 466: /startdir/linux.install.pkg: Read-only file system
==> ERROR: A failure occurred in package_linux-libre().
    Aborting...

related: https://lists.parabola.nu/pipermail/dev/2019-May/007207.html


Files

PKGBUILD (300 Bytes) PKGBUILD GNUtoo, 2019-04-16 06:17 PM
hello.install (77 Bytes) hello.install GNUtoo, 2019-04-16 06:20 PM
hello.sh (29 Bytes) hello.sh GNUtoo, 2019-04-16 06:20 PM
v20180804.diff (515 Bytes) v20180804.diff GNUtoo, 2019-05-16 02:55 PM

History

#1

Updated by GNUtoo 8 months ago

  • Subject changed from linux-libre not building with libremakepkg i686 to linux-libre-4.19.8_gnu-1-i686 not building with libremakepkg
#2

Updated by bill-auger 8 months ago

  • Assignee set to Megver83
#3

Updated by Megver83 8 months ago

  • Assignee deleted (Megver83)
  • Status changed from open to not-a-bug

two things:

1) this issue is not related with linux-libre specifically

2) man, don't be that obvious, your chroot is broken or corrupted. Try building linux-libre from another one and delete parabola-i686.

#4

Updated by bill-auger 8 months ago

Megver83 wrote:

Try building linux-libre from another one and delete parabola-i686.

yes, that is always a last resort option - i have needed to destroy the chroot a few times lately - and not only the working chroot, but the entire read-only replica like:

  # librechroot -n default delete
  # librechroot -n i686 delete
  # rn -rf /var/lib/archbuild/default
  # rn -rf /var/lib/archbuild/i686
#5

Updated by GNUtoo 7 months ago

  • Status changed from not-a-bug to confirmed

Before reporting the bug I made sure to delete the chroot and recreate it before.

I tried again to delete and recreate the chroot, and I still have the exact same issue with the package I'm interested in building (linux-libre-x86_64).

 |  /startdir/PKGBUILD: line 334: /startdir/linux.install.pkg: Read-only file system
 |  ==> ERROR: A failure occurred in package_linux-libre-x86_64().

and I do have space:

$ df -h --output=avail /var/lib/archbuild/
Avail
  27G

#6

Updated by GNUtoo 7 months ago

I've tried again with linux-libre and I've the same issue:

 |    DEPMOD  4.20.6-gnu-1
 |    -> Installing hooks...
 |  /startdir/PKGBUILD: line 466: /startdir/linux.install.pkg: Read-only file system
 |  ==> ERROR: A failure occurred in package_linux-libre().
 |      Aborting...
==> Copying log and package files out of the chroot...

I didn't rebuild the chroot since my last try yesterday with linux-libre_x86-64 though.

#7

Updated by GNUtoo 7 months ago

  • Subject changed from linux-libre-4.19.8_gnu-1-i686 not building with libremakepkg to linux-libre-4.19.8_gnu-1-i686 not building with libremakepkg with newly created librechroots

I tried again on winston.parabola.nu and I had the exact same issue, so this is probably related to the creation of a new chroot. It may work on older chroot, which I don't have lying around.

#8

Updated by oaken-source 7 months ago

this behavior is a more general problem, and can also be observed when building any package that relies on the pkgver() function to update the PKGBUILD. it appears that for some reason, the build processes in the chroot have lost write permissions to the files in /startdir.

this might be a recent change to our build toolchain, I distinctly remember this working before.

#9

Updated by GNUtoo 7 months ago

A workaround for that work at least for pkgver() is to build the package twice:
  • once with makepkg so it updates the PKGBUILD
  • once with libremakepkg after having updated the PKGBUILD
I wonder what would be the best way to deal with that?
  • Add a function to makepkg so it would update the PKGBUILD?
  • Enable the chroot to modify the PKGBUILD and copy it back to the directory in abslibre?
  • don't care about the changes made to PKGBUILD or other files in abslibre?
#10

Updated by GNUtoo 7 months ago

  • Subject changed from linux-libre-4.19.8_gnu-1-i686 not building with libremakepkg with newly created librechroots to libremakepkg failing to build package due to read-only startdir in recent librechroots
#11

Updated by GNUtoo 5 months ago

I've made a quick and dirty test case that I uploaded:

$ sudo libremakepkg 
==> Initializing the chroot...
==> Starting pre-build activities...
 |  ==> Downloading blacklist of proprietary software packages...done
 |    -> Inspecting package pkgname=hello.sh (0-1)
==> Downloading sources...
 |  ==> Making package: hello.sh 0-1 (mar. 16 avril 2019 20:15:44 CEST)
 |  ==> Retrieving sources...
 |    -> Found hello.sh
 |  ==> Validating source files with sha512sums...
 |      hello.sh ... Skipped
 |  ==> Entering fakeroot environment...
 |  ==> Creating source package...
 |    -> Adding PKGBUILD...
 |    -> Generating .SRCINFO file...
 |    -> Adding hello.sh...
 |    -> Compressing source package...
 |  ==> Leaving fakeroot environment.
 |  ==> Source package created: hello.sh (mar. 16 avril 2019 20:15:46 CEST)
==> Starting to build the package...
 |  ==> Cleaning chroot...
 |    -> Creating a full list of packages...
 |    -> No packages to remove
 |    -> No packages to add
 |  ==> Making package: hello.sh 0-1 (Tue 16 Apr 2019 08:15:49 PM CEST)
 |  ==> Checking runtime dependencies...
 |  ==> Checking buildtime dependencies...
 |  ==> Retrieving sources...
 |    -> Found hello.sh
 |  ==> WARNING: Skipping all source file integrity checks.
 |  ==> Extracting sources...
 |  ==> Entering fakeroot environment...
 |  ==> Starting package()...
 |  /startdir/PKGBUILD: line 13: /startdir/hello.install.pkg: Read-only file system
 |  ==> ERROR: A failure occurred in package().
 |      Aborting...
==> Copying log and package files out of the chroot...

and with makepkg:

$ makepkg
==> Making package: hello.sh 0-1 (mar. 16 avril 2019 20:17:30 CEST)
==> Checking runtime dependencies...
==> Checking buildtime dependencies...
==> Retrieving sources...
  -> Found hello.sh
==> Validating source files with sha512sums...
    hello.sh ... Skipped
==> Extracting sources...
==> Entering fakeroot environment...
==> Starting package()...
==> Tidying install...
  -> Removing libtool files...
  -> Purging unwanted files...
  -> Removing static library files...
  -> Stripping unneeded symbols from binaries and libraries...
  -> Compressing man and info pages...
==> Checking for packaging issues...
==> Creating package "hello.sh"...
  -> Generating .PKGINFO file...
  -> Generating .BUILDINFO file...
  -> Adding install file...
  -> Generating .MTREE file...
  -> Compressing package...
==> Leaving fakeroot environment.
==> Finished making: hello.sh 0-1 (mar. 16 avril 2019 20:17:33 CEST)

#12

Updated by GNUtoo 5 months ago

#13

Updated by GNUtoo 5 months ago

#14

Updated by eschwartz 5 months ago

bill-auger: No, that PKGBUILD doesn't work -- the install= must be global OR in the package() function. Furthermore, makepkg installs it relative to $startdir, not $srcdir -- so you can declare it during package to ./hello.install, but when makepkg reads it, it sees it as "$startdir/./hello.install", so you're still using the unmodified version.

And if you try using install="${srcdir}/hello.install", makepkg will error out because it is not permitted to use an install file that isn't in startdir.

There's really only one solution, and that is, use a read/write mounted startdir. This fixes both the linux packages which modify the install file, and packages which use pkgver() and modify the PKGBUILD.

#15

Updated by bill-auger 5 months ago

eli was referring to my suggestion of including the the install hook file in the sources() array then modifying it the prepare() stage - i deleted the suggestion at the same time eli was commenting, because it does not actually work

#16

Updated by GNUtoo 4 months ago

Here's a workaround:

$ cd dir/where/PKGBUILD/resides
$ git clone git://git.parabola.nu/packages/libretools.git
$ git -C libretools/ checkout 646ac0258c3295943778142468aadfe5b04ad6d1~1
$ git -C libretools/ apply v20180804.diff
$ ln -s libretools/src/chroot-tools/libremakepkg ./
$ sudo ./libremakepkg 
[...]
$ ls
$ ls 
hello.install      hello.sh                       hello.sh-0-1-i686.pkg.tar.xz  libremakepkg
hello.install.pkg  hello.sh-0-1-i686-package.log  hello.sh-0-1.src.tar.gz       PKGBUILD

Thanks a lot to elibrokeit on the #parabola IRC channel on Freenode for pointing the point where the statedir was mounted to me and bill-auger. This enabled me to very easily find the issue.

#17

Updated by bill-auger 4 months ago

  • Priority changed from bug to discussion
  • Assignee set to lukeshu
  • Description updated (diff)
#18

Updated by bill-auger 4 months ago

the log message for the commit that introduced this change indicates that it was necessary for some reason - i think it would be hasty to revert this until we know what the trade-off would be

"- We mount the temporary directory containing the extracted source package files read-only,
to be sure that makepkg doesn't modify the PKGBUILD. This is necessary because
--holdver only disables pkgver() if it's a VCS package."

right now, i think the only packages that are broken by this are the kernels - there is a single command that requires a writable /startdir, and it does not appear to have any value whatsoever - all that it does is write the 'pkgbase' into the .install script - 'pkgbase' will never change, so that could be hard-coded into the .install script anyways

#19

Updated by bill-auger 4 months ago

megver says he did not see this problem because his chroots use openrc/nonsystemd - apparently the read-only mount only relates to systemd-nspawn - dunno if that is a bug or a feature

#20

Updated by Megver83 4 months ago

bill-auger wrote:

megver says he did not see this problem because his chroots use openrc/nonsystemd - apparently the read-only mount only relates to systemd-nspawn - dunno if that is a bug or a feature

I'm using https://www.parabola.nu/packages/nonsystemd/x86_64/libretools/ to build packages. Yesterday I built linux-libre-hardened and -pae successfully without having to do any extra arrangements. So please, do not (yet) do things like this (or at least with kernels)

#21

Updated by bill-auger 4 months ago

i made that change mainly to make the point clear that it served no purpose - it was a code smell - the only effect of reverting that change now, would be to make the package not able to build with the standard tools - anything like that is an indication that something is not well thought out and should be discussed

maybe that is how arch does it, but the goal of staying close to arch, does not mean we need to do any useless things they do - the change that broke these packages was already a deviation from arch, and a more drastic one - this change is completely benign - even if we reverted the change that made the /startdir read-only as gnutoo suggested, there would still be no justification for reverting this change - it is more of a curiosity why that ugly hack was there in the first place

#22

Updated by eschwartz 4 months ago

It's not an ugly hack. It is a curious appendage designed to be useful in the sole case where some user wants to create a custom kernel.

Your "(NOTE: hard-coded into linux.install)" is quite missing the point of why any of that exists in the libre/linux-libre or upstream core/linux PKGBUILDs at all -- it is now no longer possible to simply change the pkgbase and generate a forked custom kernel, so you should either revert the change, or do one better and strip out the rest of the workarounds intended solely for custom kernel derivations.

#23

Updated by bill-auger 4 months ago

i wonder if thats why i could not publish the first kernels i built - i originally did change the pkgbase intentionally, so that i would not replace any existing kernels; but the packages were rejected with "this package was not built in a chroot"

maybe that was not related, im not sure - it was confusing and i was in a rush, and i tried several crazy hacks to get it out; but that was the original reason i pushed the changes i made to the PKGBUILDs

#24

Updated by bill-auger 4 months ago

i noticed that libremakepkg has options to control mounting ro or
rw - maybe that was intended to be the solution all along

-w <PATH[:INSIDE_PATH[:OPTIONS]]>     Bind mount a file or
directory, read/write
-r <PATH[:INSIDE_PATH[:OPTIONS]]>     Bind mount a file or
directory, read-only
#25

Updated by lukeshu 4 months ago

There is a real issue where pkgver() isn't getting evaluated before the sourceball is created, which then causes problems with the read-only $startdir. But: the PKGBUILD should be doing it's work in $srcdir, not $startdir.

However, I would tend to be of the opinion that "work" should happen in $srcdir. Would using an absolute path to the modified script not work?

  sed "$subst" "$startdir/$install" > "$srcdir/$install.pkg" 
  true && install=$srcdir/$install.pkg
#26

Updated by bill-auger 4 months ago

no i tried that - see eli's comment # 14 https://labs.parabola.nu/issues/2141#note-14

#27

Updated by bill-auger 12 days ago

oaken-source reverted the change that originally spurred this BR - there has not been any problems with that yet, although lukeshu indicated that problem that the change was intended to solve will turn up someday - maybe this issue is fixed? - only time will tell :)

Also available in: Atom PDF