Project

General

Profile

Freedom Issue #3107

[opencv] Downloads and installs binary blobs

nona - over 1 year ago - . Updated about 2 months ago.

Status:
in progress
Priority:
freedom issue
Assignee:
-
% Done:

0%

Description

  • steps to reproduce:
    1. Download the PKGBUILD and libre.patch
    2. Run makepkg
  • expected result:
    binary-(blob)-free package and compilation
 parallel 'file {} | grep " data" | grep -v "\(image\|text\|JSON\|gzip\)"' ::: $(find . -type f)

Besides https://github.com/opencv/ade/archive/v0.1.1f.zip is downloaded during compilation.
  • proposed solution
    either rm -fr modules/{aruco,cnn_3dobj,dnns_easily_fooled,wechat_qrcode} or add -DOPENCV_FORCE_3DPARTY_BUILD=ON to configuration options (still testing, will report later)

    regarding the ADE file, add to sources and (sym)link to "${srcdir}"/"${pkgbase}"-"${pkgver}"/.cache/ade/b624b995ec9c439cbc2e9e6ee940d3a2-v0.1.1f.zip

Files

opencv-4.5.3-patches.tar.gz (2.08 KB) opencv-4.5.3-patches.tar.gz 4 patches to de-blob sources nona, 2021-10-07 05:18 PM
PKGBUILD (13.1 KB) PKGBUILD binary examples go in opencv-contrib samples dir nona, 2021-10-07 10:15 PM

History

#1

Updated by nona over 1 year ago

#2

Updated by nona over 1 year ago

  • File deleted (PKGBUILD)
#3

Updated by nona over 1 year ago

  • File PKGBUILD added

I am attaching a PKGBUILD which works in my computer. I still don't know how to use mksource, but should be simple enough to adapt the prepare function.

#4

Updated by nona over 1 year ago

  • File deleted (PKGBUILD)
#5

Updated by nona over 1 year ago

#6

Updated by Zuss about 1 year ago

  • Status changed from unconfirmed to in progress

I was updating OpenCV to 4.5.5 and noticed it trying to download external files as well.
It tried to download ADE, IPPICV, face, wechat_qrcode. I had opened a PR (#51) to update the package before noticing this issue.
The command you listed in the description returns a lot of files and I'm not quite sure what is meant to happen with them.

Though I assume we can merge the ADE patch and can leave out the rest? The PKGBUILD can't fetch the third party dependencies because network access is disabled within build() anyways.
I'm just looking to bring down the maintenance and testing required for newer releases.
We could possibly mention that Parabola's OpenCV package won't have xyz module support and leave the downloads fail, or add them into the sources much like how you did it with ADE. Though I'm not sure what's actually needed. Removing the folder from the module directory seems like the way to go about it if we were to strip them from the package.

Lapack is found but I don't really know why LAPACK_ is appended to those lines in the patch.

 |  -- LAPACK(Unknown): LAPACK_LIBRARIES: /usr/lib/liblapack.so;/usr/lib/libblas.so;/usr/lib/libcblas.so
 |  -- Looking for /usr/include/cblas.h
 |  -- Looking for /usr/include/cblas.h - found
 |  -- Looking for /usr/include/lapacke.h
 |  -- Looking for /usr/include/lapacke.h - found
 |  -- LAPACK(Unknown): Support is enabled.

#7

Updated by nona about 2 months ago

Regarding the LAPACK_ prefix: it seems to come from Arch (I'm not that smart ;) ):
https://github.com/opencv/opencv/issues/19846

Regarding the command (parallel 'file {} | grep " data" | grep -v "\(image\|text\|JSON\|gzip\)"' ::: $(find . -type f)?): It shows files which are some form of data (e.g. not images, text, JSON or gzip). It can be used to detect binaries (you get some jar files, for instance).

Regarding the build v.s. disabled network: I tried this like a year ago, and it may be that you are right, but today (without the patches):

- When I disable the network:
#+begin_example
-- wechat_qrcode: Download: detect.caffemodel
-- Try 1 failed
-- =======================================================================
Couldn't download files from the Internet.
Please check the Internet access on this host. =======================================================================
#+end_example

- When I enable the network:
#+begin_example
-- Registering hook 'INIT_MODULE_SOURCES_opencv_dnn': /pkg/opencv/src/opencv-4.5.5/modules/dnn/cmake/hooks/INIT_MODULE_SOURCES_opencv_dnn.cmake
-- opencv_dnn: filter out cuda4dnn source code
-- Excluding from source files list: <BUILD>/modules/dnn/layers/layers_common.rvv.cpp
-- wechat_qrcode: Download: detect.caffemodel
-- wechat_qrcode: Download: detect.prototxt
-- wechat_qrcode: Download: sr.caffemodel
-- wechat_qrcode: Download: sr.prototxt
#+end_example

Which shows that a connection is made in build. Thus, there may be something wrong with my configuration which enables the network in build (in the PKGBUILD with makepkg), but this actually happened in a different computer.

I think that I didn't want the .caffemodel and such because they are also binaries (although .jar, .png, etc. are also binaries). If I remember correctly, they are optional, but (also related to maintainability) filtering them out would be more work. I can't remember why it was relevant to download ADE and TBB, but not those other packages.

Sorry for the delay. This one went under my radar.

#8

Updated by bill-auger about 2 months ago

Which shows that a connection is made in build. Thus, there may be something wrong with my configuration which enables the network in build (in the PKGBUILD with makepkg), but this actually happened in a different computer.

`makepkg` does not disable the network - the requirement to
disable networking is referring to `libremakepkg`
all packages built for the parabola repos are built with
`libremakepkg`, never `makepkg`

Also available in: Atom PDF