How does mksource() work?

theova - over 2 years ago -

Hi there

I'm trying to update the PKGBUILD of texlive-latexextra.

To do this, I have several questions:
  1. What does mksource() in the following section do?
    mksource() {
       mkdir $pkgname-$pkgver
       pushd $pkgname-$pkgver
      # ... some code I understand...
  2. How can I modify the PKGBUILD to build the new version? I have already tried to just update pkgver and the checksum, but then I receive
    ==> Downloading sources...
     |  ==> Making package: texlive-latexextra 2019.50920-1.par1 (Fri 26 Jul 2019 05:09:59 PM CEST)
     |  ==> Retrieving sources...
     |    -> Downloading texlive-latexextra-libre-2019.50920-src.tar.xz...
     |    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
     |                                   Dload  Upload   Total   Spent    Left  Speed
     |    0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
     |  curl: (22) The requested URL returned error: 404 
     |  ==> Making source: texlive-latexextra 2019.50920-1.par1 (Fri 26 Jul 2019 05:10:00 PM CEST)
     |  ==> Checking source dependencies...
     |  ==> Retrieving sources...
     |    -> Found
     |  ==> Validating source files with sha256sums...
     | ... FAILED
     |  ==> ERROR: One or more files did not pass the validity check!
     |  ==> ERROR: Failure while downloading
     |      Aborting...
     |  ==> ERROR: Could not download sources.
  3. Where can I find the documentation to such questions? Parabola-/ArchWiki didn't help.

Replies (11)

RE: How does mksource() work? - bill-auger - over 2 years ago -

mksource() is a nonstandard function - it expects you to source the PKGBUILD and call mksource in a shell, if you need to prepare the custom source-ball; before running makepkg as usual

$ source ./PKGBUILD
$ mksource
$ makepkg -irs

the reason for it as i understand, is to remove any non-free code from the parabola source package - that is what the prepare() function os for though; so its not clear why mksource() is necessary

RE: How does mksource() work? - theova - over 2 years ago -

Right. Thanks.

that still gives me the same error

I guess I would need to upload the src to
Therefore I would like to work with the local folder as a source, i.e. source=(texlive-latexextra-2019.50920), but this doesn't work either.

How do I need to adapt $source$?

The current PKGBUILD I'm working on is attached.


RE: How does mksource() work? - bill-auger - over 2 years ago -

with these params:


when running mksource,

the mksource=() line gets a file from arch with the following name:

which evaluates to:

the bsdtar line then creates a tarball with the same name:

 bsdtar xfv ../$pkgname-$
which evaluates to:

when running makepkg, the source=() line: is looking for a tarball in the abslibre dir with a different name:

which evaluates to:

if that file is not found in the abslibre director, makepkg will try to download it
in this case, that file does not exist and is not available on the network either -thats the reason for this error:

-> Downloading texlive-latexextra-libre-2019.50920-src.tar.xz...
==> ERROR: Failure while downloading

when running `makepkg -g`, this error probably should not happen:

==> Retrieving sources...
  -> Found
==> Validating source files with sha256sums... ... FAILED

presumably, the parabola dev is expected to rename the libre tarball to 'texlive-latexextra-libre-2018.49282-src.tar.xz' and then upload it to the server manually - anyone else building this package could copy it or add a symlink with the expected name:

$ ln -s texlive-latexextra-{,libre-}2018.49282-src.tar.xz

RE: How does mksource() work? - freemor - over 2 years ago -

Unless there is a clear need for the mksource() stuff It might be better to
move the mksource() steps into prepare and same future people the same

I could understand the need to mksource() if for some bizarre reason the steps
needed to libreate the source could not exist in prepare() or in cases like
iceweasel where it created the supplimentry branding package. But if neither of
those are the case I'd say it is fine to move the steps to prepare and make it
a more standard PKGBUILD.

RE: How does mksource() work? - theova - over 2 years ago -

Wow, thanks again bill-auger for the comprehensive step-by-step guide.

Now, I was able to build the new version (I'll announce the PKGBUILD on the mailinglist) and removed the mksource() stuff.

So is mksource() a standard procedure? Then I may some hints to the wiki page Creating -libre packages .

RE: How does mksource() work? - bill-auger - over 2 years ago -

im still not very clear about mksource() - IIRC when discussing the 'ruby' package, we concluded that it was probably intended per the FSDG, to provide complete and pre-cleaned source-balls to the distro users; which would entail that the devs keep some of the libre-fications tools private

the FSDG does not require support for versions of 'foo' that were not included in any distro release however; but OTOH, this PKGBUILD contains instructions for downloading known non-free software

on the third hand (the philosophical hand), the 'gist' of the four freedoms is not about possessing or sharing non-free software; but that the user is able to do with it as they see fit - if the only thing that one wishes to do with that software is to merely posses it or share it, then anything re- distributable would satisfy the 'gist' of the four freedoms for that person

probably, the simplest way to resolve this class of issue, is to add a note to the PKGBULID with sed above every instance of 'mksource()', asking the reader to ignore it and ask a parabola dev to publish the intermediate source-ball, if they are building ahead of the current release; then to publish the missing source-ball for any enthusiastic user who asks for one - a better solution to this class of issue, would be for the PKGBUILD to explaining how to run mksource() and have the mksource() function create the source-ball with expected libre name in the sources() array - if that does not conflict with the FSDG - an even better solution to this class of issue, is to eliminate it by eradicating mksource() from abslibre somehow, even if that is simply moving the PKGBUILD to a private git branch

this incident is not a peculiar though - it is currently systematic AFAIK - i would also suggest elevating this to a proper bug report in the tooling category - im sure there is room for improvements in this area

RE: How does mksource() work? - freemor - over 2 years ago -

Since libremakepkg rolls up a source tarball which get uploaded with the
package during librerelease. Would it be possible just to make that step, the
rolling of the source tarball hold off until after prepare(). That way the
source tarball would be pre-cleaned.

I persoanlly feel it is an unecessary step to create a libre source tarball
since we provide all the steps and patches to clean it in the currently
existing source tarballs that are uploaded. Therefore it is easy to create the
cleaned package. I also see added benefit to doing it the way it is currently
rolled, because it lets people see what was required to free the package and
how it was done. The user/dev/whatever gets the whole thing from origional
source, the patches/seds/rm commands/etc required to clean it and a recipe to
have makepkg bake it. Tthe cleaned source is just a makepkg -o away.

RE: How does mksource() work? - bill-auger - over 2 years ago -

there was that patch to librechroot making the chroot readonly; but i think oaken-soucre reverted it - i opened a ticket on the [libre-tools] tracker for now - maybe move it to the [documentation] tracker if that is decided as the best approach

RE: How does mksource() work? - freemor - over 2 years ago -

I guess I just see the mksource() system (except where required for the
creation of supplemental bits etc) as a disingenuous kicking of the "unclean"
source one step down the road.

Because as Theova's problem points out. In order to make version next you still
have to pull the unclean source. By running the mksource() function. And the
URI of the "unclean" source is right there in the package build.

To me it seems both more honest, and more robust, to just say "here is the
mess, here is how we fixed it" rather then trying to obfuscate the mess with
mksource and "cleaned" tarballs.

RE: How does mksource() work? - theova - almost 2 years ago -

Months later, I am back to this issue...

How can I create a libre tarball for texlive-latexextra? I would like to use Parabola's PKGBUILD (without modification) and create the libre tarball (instead of using the one from Parabola).

I tried (in the directory of the PKGBUILD):

librefetch -C 
==> ERROR: $PKGEXT does not contain a valid package suffix (needs '.pkg.tar*', got 'texlive-latexextra-libre-2019.50920-src.tar.xz')

What is the correct way of using librefetch?

RE: How does mksource() work? - bill-auger - over 1 year ago -

theova -

you should not need to run librefetch directly - the instructions above should be sufficient for any package with a mksource()

$ source ./PKGBUILD ; mksource ; makepkg -irs

i wonder if this is the package that oaken-source was asking about yesterday - do you have 'texlive-latexextra' packaged successfully now?