Bug #46

Sourcing PKGBUILDs might change variables used

mtjm - almost 12 years ago - . Updated almost 7 years ago.

in progress
% Done:



E.g. toru used _repo while a PKGBUILD changed the value of this variable, making fullpkg ignore dependencies from the community repo. There should be another solution which wouldn't lead to difficult to debug problems.



Updated by fauno almost 12 years ago

Parsing PKGBUILDs is a mess (see AUR), wouldn't it be better to prefix toru's internal vars ("toru_repo") or something like that?


Updated by mtjm almost 12 years ago

This problem would be certainly fixed by writing and using a script which would source the PKGBUILD in a different process (properly, with e.g. CARCH defined) and output a script setting specified variables. Then we could implement e.g. caching for it in a single place, so it wouldn't be much slower.


Updated by xihh over 11 years ago

Maybe that's the reason why some scripts from «devtools» use subshels when sourcing PKGBUILDs.


Updated by lukeshu over 11 years ago

Devtools standards say to not use variables that begin with "_" as that acts as a prefix of variables that are safe to use in PKGBUILDs. And PKGBUILDs should not use variables that do not start with "_" except for the documented ones.


Updated by lukeshu about 11 years ago

Also, there are several approaches to this that are used by makepkg and devtools.

  • Source them in a subshell and print the results.
    VAR="$(. FILE; echo $VAR)"
  • Extract the line that sets it using grep, and `eval` it.
    eval $(grep '^\s*VAR=' FILE)

They both have the possibility to run side-effects. The first is more "correct" and rubust, but is more likely to run unintentional side effects in a poorly written PKGBUILD.


Updated by mtjm about 11 years ago

  • Source them in a subshell and print the results. [...]

It will work, not sure about spaces in variables; arrays need different handling. (I have written a script outputting it with nul-separated fields, seems robust although nonobvious to use in shell scripts.)

  • Extract the line that sets it using grep, and `eval` it. [...]

This certainly won't work for makedepends in many mips64el PKGBUILDs, with ifs and +=s.


Updated by lukeshu almost 11 years ago

mtjm: your script might be over complicated.

Spit out a null-delimited array:

printf '%s\0' "${array[@]}" 

Updated by lukeshu almost 11 years ago

pkgbuild-check-nonfree now (in git) sources the PKGBUILD in a subshell, so that should be safe. Should it set CARCH?


Updated by mtjm almost 11 years ago

Maybe source /etc/makepkg.conf? I'm not sure if other variables are used.


Updated by lukeshu over 10 years ago

Definitely source makepkg.conf (actually, <tt>load_files makepkg</tt>) if we need to set CARCH. But is setting CARCH important?

Also, as far as exporting arrays from a subshell:

eval "$(. FILE &>/dev/null; declare -ap VARNAME)" 

Other than the possibility of FILE having side-effects, I believe that the only caveat to that is that if it is done in a function, it will be local to that function.


Updated by mtjm over 10 years ago

But is setting CARCH important?

We remove some pkgname array entries depending on CARCH, so setting CARCH affects what toru finds. It is more important in scripts like fullpkg: we add and remove dependencies, changing build orders.


Updated by lukeshu over 10 years ago

I've added a "load_PKGBUILD [file]" function to lib/, and adjusted all the other programs to use it. It first unsets all PKGBUILD variables and functions, then sets CARCH, then loads the file.


Updated by lukeshu about 10 years ago

  • Status changed from open to in progress
  • % Done changed from 10 to 20

Updated by lukeshu almost 7 years ago

I'd kind of like to move everything to use makepkg --printsrcinfo. It would be a nice sanitization layer between the PKGBUILD and our runtime. There's a Python parser for the format. I suppose there's also a PHP parser that is part of AUR4. It's an easy format. It would also make it easier to move some of the tooling to a better language than Bash.

Also available in: Atom PDF