View Issue Details

IDProjectCategoryView StatusLast Update
0000390Cinelerra-GG[All Projects] Bugpublic2020-04-05 21:49
ReporterMatN Assigned ToPhyllisSmith  
PrioritynormalSeverityblockReproducibilityalways
Status closedResolutionfixed 
PlatformX86_64OSMint XFCEOS Version19.3
Product Version 
Target VersionFixed in Version 
Summary0000390: Build from 20200318 git source fails.
DescriptionI just did a git pull, and now the build fails. The 20200228 release source builds fine.
After the pull, did a "sudo make clean" , then "sudo ./bld.sh" . I think there is an error related to the AV1 decoder.
See attached build log file around line 3780 .
TagsNo tags attached.

Activities

PhyllisSmith

PhyllisSmith

2020-04-05 21:49

manager   ~0003097

Some times strange things just happen for no reason.
@Andrew-R
Thanks for the bugzilla url reference to libvpx bug. We will wait until a newer release to upgrade since it was only a point upgrade from 1.8.1 to 1.8.2. We were only attempting to get all libraries upgraded at once.
MatN

MatN

2020-04-05 20:14

reporter   ~0003096

Since the 2020-03 release it builds fine for me. I don't think there were any relevant system updates apart from GCC, but I don't think that was the problem. I'm at a loss to explain why it did not build after dav1d was rolled back. Anyway, all fine now.
Andrew-R

Andrew-R

2020-03-22 11:07

reporter   ~0002929

something like this apparently works, at least build continues into plugins .....

c11_configure.ac_from_oxr_1.patch (1,858 bytes)
diff --git a/cinelerra-5.1/configure.ac b/cinelerra-5.1/configure.ac
index f61ef27..a9d280a 100644
--- a/cinelerra-5.1/configure.ac
+++ b/cinelerra-5.1/configure.ac
@@ -8,6 +8,45 @@ AC_LANG_CPLUSPLUS
 AC_LANG_C
 AC_PROG_CXX
 
+dnl Enable and choose c++ standard
+AC_ARG_ENABLE(cxxstd,
+              AC_HELP_STRING([--enable-cxxstd=14],
+                             [enable ISO c++ standard 11/14 [[default=auto]]]),
+              [cxxstd="${enableval}"], [cxxstd=14])
+
+if test "${cxxstd}" = 17 ; then
+    AX_CXX_COMPILE_STDCXX([17], [noext], [mandatory])
+	CXXFLAGS="$CXXFLAGS -std=c++17"
+else
+  if test "${cxxstd}" = 14 ; then
+      AX_CXX_COMPILE_STDCXX([14], [noext], [mandatory])
+	CXXFLAGS="$CXXFLAGS -std=c++14"
+  else
+    if test "${cxxstd}" = 11 ; then
+      AX_CXX_COMPILE_STDCXX([11], [noext], [mandatory])
+      CXXFLAGS="$CXXFLAGS -std=c++11"
+    else
+      if test "${cxxstd}" = 03 ; then
+        CXXFLAGS="$CXXFLAGS -std=c++03"
+      else
+        dnl automatically determine...
+        AX_CXX_COMPILE_STDCXX([11], [noext], [optional])
+        AX_CXX_COMPILE_STDCXX([14], [noext], [optional])
+        AX_CXX_COMPILE_STDCXX([17], [noext], [optional])
+        if test "$HAVE_CXX14" = 1 ; then
+	 CXXFLAGS="$CXXFLAGS -std=c++14"
+          cxxstd = 14
+        else
+          if test "$HAVE_CXX11" = 1 ; then
+	 CXXFLAGS="$CXXFLAGS -std=c++11"
+            cxxstd = 11
+          fi
+        fi
+      fi
+    fi
+  fi
+fi
+
 CFG_CFLAGS+=" -fno-omit-frame-pointer -fno-math-errno -fno-signed-zeros"
 CFG_CFLAGS+=" -pthread -Wall"
 # misguided pedantic warnings
@@ -1111,6 +1150,7 @@ echo "endif"
 echo "endif"
 
 echo "CFLAGS := \$(CFLAGS_)"
+echo "CFLAGS += \$(CXXFLAGS_)"
 echo "CFLAGS += '-DCIN=\"\$(WANT_CIN)\"'"
 echo "CFLAGS += '-DCINLIB_DIR=\"$WANT_CINLIB_DIR\"'"
 echo "CFLAGS += '-DCINDAT_DIR=\"$WANT_CINDAT_DIR\"'"
Andrew-R

Andrew-R

2020-03-22 08:46

reporter   ~0002928

oh, this still doesn't work yet, but may be because in cinelerra/Makefile I see


CFLAGS := \
-I/usr/local/include \
-I$(GUICAST) \
-I$(LIBZMPEG3) \
$(static_incs) \
$(CFLAGS)

[...]
$(shell echo $(CFLAGS) > $(OBJDIR)/c_flags)

and $(OBJDIR)/c_flags used for building essentially C++ files, and one of them (fileexr) references c++ library, compiled with different set of CXXFLAGS ....

even if updated configure patch (attached, with hack, currently compiling) set CXXFLAGS correctly, or at least in same way as OpenEXR?

c11_configure.ac_from_oxr.patch (1,618 bytes)
diff --git a/cinelerra-5.1/configure.ac b/cinelerra-5.1/configure.ac
index f61ef27..ae9aefd 100644
--- a/cinelerra-5.1/configure.ac
+++ b/cinelerra-5.1/configure.ac
@@ -8,6 +8,46 @@ AC_LANG_CPLUSPLUS
 AC_LANG_C
 AC_PROG_CXX
 
+dnl Enable and choose c++ standard
+AC_ARG_ENABLE(cxxstd,
+              AC_HELP_STRING([--enable-cxxstd=14],
+                             [enable ISO c++ standard 11/14 [[default=auto]]]),
+              [cxxstd="${enableval}"], [cxxstd=14])
+
+if test "${cxxstd}" = 17 ; then
+    AX_CXX_COMPILE_STDCXX([17], [noext], [mandatory])
+	CXXFLAGS="$CXXFLAGS -std=c++17"
+else
+  if test "${cxxstd}" = 14 ; then
+      AX_CXX_COMPILE_STDCXX([14], [noext], [mandatory])
+	CXXFLAGS="$CXXFLAGS -std=c++14"
+  else
+    if test "${cxxstd}" = 11 ; then
+      AX_CXX_COMPILE_STDCXX([11], [noext], [mandatory])
+      CXXFLAGS="$CXXFLAGS -std=c++11"
+    else
+      if test "${cxxstd}" = 03 ; then
+        CXXFLAGS="$CXXFLAGS -std=c++03"
+      else
+        dnl automatically determine...
+        AX_CXX_COMPILE_STDCXX([11], [noext], [optional])
+        AX_CXX_COMPILE_STDCXX([14], [noext], [optional])
+        AX_CXX_COMPILE_STDCXX([17], [noext], [optional])
+        if test "$HAVE_CXX14" = 1 ; then
+	 CXXFLAGS="$CXXFLAGS -std=c++14"
+          cxxstd = 14
+        else
+          if test "$HAVE_CXX11" = 1 ; then
+	 CXXFLAGS="$CXXFLAGS -std=c++11"
+            cxxstd = 11
+          fi
+        fi
+      fi
+    fi
+  fi
+fi
+
+CFG_CFLAGS+= $CXXFLAGS
 CFG_CFLAGS+=" -fno-omit-frame-pointer -fno-math-errno -fno-signed-zeros"
 CFG_CFLAGS+=" -pthread -Wall"
 # misguided pedantic warnings
Andrew-R

Andrew-R

2020-03-22 07:23

reporter   ~0002927

from pkg description - just updated my autoconf-archive into 2019 version (from late 2015)

PACKAGE DESCRIPTION:
# autoconf-archive (GNU Autoconf Macro Archive)
#
# The Autoconf Macro Archive aims to provide a collection of reusable
# Autoconf macros as free software. The archive currently features more
# than 500 macros which perform portability tests ranging from compiler
# support for weird language extensions to automatic generation of
# sophisticated Automake rules.
#
# http://www.gnu.org/software/autoconf-archive/

still, apparently some Ubuntu failed to provide this archive .. then ppl complained about autotools being broken, and switched to cmake :} joke ......
https://github.com/SFTtech/nyan/issues/6 ...not sure if I should smile or not. probably not.

Also, libvpx was giving some ..fun to Mozilla, too!
https://bugzilla.mozilla.org/show_bug.cgi?id=1525393
Andrew-R

Andrew-R

2020-03-22 06:58

reporter   ~0002926

Ah, there is some code in OpenEXR's configure.ac:

dnl Enable and choose c++ standard
AC_ARG_ENABLE(cxxstd,
              AC_HELP_STRING([--enable-cxxstd=14],
                             [enable ISO c++ standard 11/14 [[default=auto]]]),
              [cxxstd="${enableval}"], [cxxstd=14])

if test "${cxxstd}" = 17 ; then
    AX_CXX_COMPILE_STDCXX([17], [noext], [mandatory])
<------>CXXFLAGS="$CXXFLAGS -std=c++17"
else
  if test "${cxxstd}" = 14 ; then
      AX_CXX_COMPILE_STDCXX([14], [noext], [mandatory])
  <---->CXXFLAGS="$CXXFLAGS -std=c++14"
  else
    if test "${cxxstd}" = 11 ; then
      AX_CXX_COMPILE_STDCXX([11], [noext], [mandatory])
      CXXFLAGS="$CXXFLAGS -std=c++11"
    else
      if test "${cxxstd}" = 03 ; then
        CXXFLAGS="$CXXFLAGS -std=c++03"
      else
        dnl automatically determine...
        AX_CXX_COMPILE_STDCXX([11], [noext], [optional])
        AX_CXX_COMPILE_STDCXX([14], [noext], [optional])
        AX_CXX_COMPILE_STDCXX([17], [noext], [optional])
        if test "$HAVE_CXX14" = 1 ; then
  <----> CXXFLAGS="$CXXFLAGS -std=c++14"
          cxxstd = 14
        else
          if test "$HAVE_CXX11" = 1 ; then
  <----> CXXFLAGS="$CXXFLAGS -std=c++11"
            cxxstd = 11
          fi
        fi
      fi
    fi
  fi
fi

:}

still, it detects c++14 on my gcc 5.5.0 (and this ver. of gcc basically forced on me due to all those kernel vulnerabilities from 2018) and thus build lib in c++11 mode at very least ?

I tried to add some elementary c++ level checking in Cin's configure.ac, see patch, it works with m4 1.4.17 (from 2013), currently upgraded to latest m4 1.4.18 (for some reason gnu site lists latest as m4-latest.tar.bz2 2013-09-22 03:08 1.4M while in reality latest is [ ] m4-1.4.18.tar.bz2 2016-12-31 17:39 1.4M - see at https://ftp.gnu.org/gnu/m4/ )

c11_configure.ac.patch (394 bytes)
diff --git a/cinelerra-5.1/configure.ac b/cinelerra-5.1/configure.ac
index f61ef27..c30bab1 100644
--- a/cinelerra-5.1/configure.ac
+++ b/cinelerra-5.1/configure.ac
@@ -7,6 +7,8 @@ AM_PROG_AS
 AC_LANG_CPLUSPLUS
 AC_LANG_C
 AC_PROG_CXX
+AX_CXX_COMPILE_STDCXX_11
+AX_CXX_COMPILE_STDCXX_14
 
 CFG_CFLAGS+=" -fno-omit-frame-pointer -fno-math-errno -fno-signed-zeros"
 CFG_CFLAGS+=" -pthread -Wall"
Andrew-R

Andrew-R

2020-03-21 21:47

reporter   ~0002925

Updated version, still it must be tweaked, for example lame also has dependency on nasm (I currently uninstalled nasm completely)

nasm_thirdparty-2.patch (3,941 bytes)
diff --git a/cinelerra-5.1/configure.ac b/cinelerra-5.1/configure.ac
index f61ef27..00552c6 100644
--- a/cinelerra-5.1/configure.ac
+++ b/cinelerra-5.1/configure.ac
@@ -328,6 +328,12 @@ PKG_3RD([mjpegtools],[yes],
     mpeg2enc/.libs/libmpeg2encpp.a ],
   [ .  lavtools utils ])
 
+PKG_3RD([nasm],[yes],
+ [nasm-2.14.02],
+ [ ],
+ [ . ])
+
+
 PKG_3RD([openexr],[auto],
   [openexr-2.4.1],
   [ usr/local/lib/libHalf.a \
@@ -458,23 +464,23 @@ ARCH=[`uname -m`]
 I86=[`expr "x$ARCH" : 'xi[346]86.*'`]
 X86=[`expr "x$ARCH" : 'x..._64*'`]
 
-if test "x$I86$X86" != "x00" ; then
+#if test "x$I86$X86" != "x00" ; then
   # Checks for ix86 programs.
-  REQUIRE_PROG(NASM, [nasm])
+#  REQUIRE_PROG(NASM, [nasm])
   # libx264 nasm fix
-  AC_MSG_CHECKING([nasm x264 compatible])
-  echo "vmovdqa32 [[eax]]{k1}{z}, zmm0" > conftest.asm
-  nasm conftest.asm -o conftest.o > /dev/null 2>&1
-  if test $? != 0 ; then
-   AC_MSG_RESULT([no])
-   AC_MSG_WARN([libx264 built without assembly code])
-   X264_CFG_PARAMS="$X264_CFG_PARAMS --disable-asm"
-  else
-   AC_MSG_RESULT([yes])
-  fi
-  rm -f conftest.asm conftest.o
-  REQUIRE_PROG(YASM, [yasm])
-fi
+#  AC_MSG_CHECKING([nasm x264 compatible])
+#  echo "vmovdqa32 [[eax]]{k1}{z}, zmm0" > conftest.asm
+#  nasm conftest.asm -o conftest.o > /dev/null 2>&1
+#  if test $? != 0 ; then
+#   AC_MSG_RESULT([no])
+#   AC_MSG_WARN([libx264 built without assembly code])
+#   X264_CFG_PARAMS="$X264_CFG_PARAMS --disable-asm"
+#  else
+#   AC_MSG_RESULT([yes])
+#  fi
+#  rm -f conftest.asm conftest.o
+#  REQUIRE_PROG(YASM, [yasm])
+#fi
 
 ## end arch dep tests
 
@@ -911,6 +917,7 @@ PKG_PROVIDE([mjpegtools])
 PKG_PROVIDE([libaom])
 PKG_PROVIDE([dav1d])
 PKG_PROVIDE([libwebp])
+PKG_PROVIDE([nasm])
 PKG_PROVIDE([openExr], [$WANT_OPENEXR])
 PKG_PROVIDE([openexr], [$WANT_OPENEXR])
 PKG_PROVIDE([openjpeg])
@@ -1219,6 +1226,7 @@ echo ""
 echo "export thirdparty_libraries libraries"
 echo "export CFLAGS_ CXXFLAGS_ LDFLAGS_"
 echo "unexport CFLAGS CXXFLAGS LDFLAGS"
+echo "export PATH=`realpath $(echo $PWD)`/thirdparty/nasm-2.14.02:$PATH"
 
 if test "x$HAVE_tiff" = "xyes"; then
   if test "x$HAVE_jbig" != "xyes"; then
diff --git a/cinelerra-5.1/thirdparty/Makefile b/cinelerra-5.1/thirdparty/Makefile
index 0c595ee..29c84c4 100644
--- a/cinelerra-5.1/thirdparty/Makefile
+++ b/cinelerra-5.1/thirdparty/Makefile
@@ -234,6 +234,7 @@ libtheora.ldflags?="$(call ld_path,libvorbis,lib/.libs) $(call ld_path,libogg,sr
 libtheora.cfg_params?= --disable-examples --disable-spec --enable-shared=no
 libuuid.cfg_params?=--enable-shared=no
 libvorbis.cfg_params?= --disable-oggtest --enable-shared=no
+nasm.cfg_params?= --disable-doc
 openjpeg.cfg_params?= -DBUILD_SHARED_LIBS:BOOL=OFF
 openjpeg.cfg_vars?=$(call cmake_config,.)
 openjpeg.mak_params?= ; cd $(call bld_path,openjpeg,src/lib/openjp2); ln -sf . openjpeg-2.1
@@ -314,6 +315,7 @@ else
 rules=$(eval $(1))
 endif
 
+$(call rules,$(call std-build,nasm))
 $(call rules,$(call std-build,a52dec,djbfft))
 $(call rules,$(call std-build,djbfft))
 $(call rules,$(call std-build,audiofile))
@@ -329,7 +331,7 @@ $(call rules,$(call std-build,ilmBase, openexr))
 $(call rules,$(call std-build,ladspa))
 $(call rules,$(call std-build,lame))
 $(call rules,$(call std-build,libaom))
-$(call rules,$(call std-build,dav1d))
+$(call rules,$(call std-build,dav1d,nasm))
 $(call rules,$(call std-build,libwebp))
 $(call rules,$(call std-build,libavc1394,libraw1394))
 $(call rules,$(call std-build,libdv))
@@ -350,8 +352,8 @@ $(call rules,$(call std-build,opus))
 $(call rules,$(call std-build,speech_tools))
 $(call rules,$(call std-build,tiff, libwebp))
 $(call rules,$(call std-build,twolame))
-$(call rules,$(call std-build,x264))
-$(call rules,$(call std-build,x265))
+$(call rules,$(call std-build,x264,nasm))
+$(call rules,$(call std-build,x265,nasm))
 $(call rules,$(call std-build,libvpx))
 $(call rules,$(call std-build,lv2))
 $(call rules,$(call std-build,serd))
nasm_thirdparty-2.patch (3,941 bytes)
Andrew-R

Andrew-R

2020-03-21 19:28

reporter   ~0002924

something like this (completely untested)

nasm_thirdparty.patch (3,719 bytes)
diff --git a/cinelerra-5.1/configure.ac b/cinelerra-5.1/configure.ac
index f61ef27..00552c6 100644
--- a/cinelerra-5.1/configure.ac
+++ b/cinelerra-5.1/configure.ac
@@ -328,6 +328,12 @@ PKG_3RD([mjpegtools],[yes],
     mpeg2enc/.libs/libmpeg2encpp.a ],
   [ .  lavtools utils ])
 
+PKG_3RD([nasm],[yes],
+ [nasm-2.14.02],
+ [ ],
+ [ . ])
+
+
 PKG_3RD([openexr],[auto],
   [openexr-2.4.1],
   [ usr/local/lib/libHalf.a \
@@ -458,23 +464,23 @@ ARCH=[`uname -m`]
 I86=[`expr "x$ARCH" : 'xi[346]86.*'`]
 X86=[`expr "x$ARCH" : 'x..._64*'`]
 
-if test "x$I86$X86" != "x00" ; then
+#if test "x$I86$X86" != "x00" ; then
   # Checks for ix86 programs.
-  REQUIRE_PROG(NASM, [nasm])
+#  REQUIRE_PROG(NASM, [nasm])
   # libx264 nasm fix
-  AC_MSG_CHECKING([nasm x264 compatible])
-  echo "vmovdqa32 [[eax]]{k1}{z}, zmm0" > conftest.asm
-  nasm conftest.asm -o conftest.o > /dev/null 2>&1
-  if test $? != 0 ; then
-   AC_MSG_RESULT([no])
-   AC_MSG_WARN([libx264 built without assembly code])
-   X264_CFG_PARAMS="$X264_CFG_PARAMS --disable-asm"
-  else
-   AC_MSG_RESULT([yes])
-  fi
-  rm -f conftest.asm conftest.o
-  REQUIRE_PROG(YASM, [yasm])
-fi
+#  AC_MSG_CHECKING([nasm x264 compatible])
+#  echo "vmovdqa32 [[eax]]{k1}{z}, zmm0" > conftest.asm
+#  nasm conftest.asm -o conftest.o > /dev/null 2>&1
+#  if test $? != 0 ; then
+#   AC_MSG_RESULT([no])
+#   AC_MSG_WARN([libx264 built without assembly code])
+#   X264_CFG_PARAMS="$X264_CFG_PARAMS --disable-asm"
+#  else
+#   AC_MSG_RESULT([yes])
+#  fi
+#  rm -f conftest.asm conftest.o
+#  REQUIRE_PROG(YASM, [yasm])
+#fi
 
 ## end arch dep tests
 
@@ -911,6 +917,7 @@ PKG_PROVIDE([mjpegtools])
 PKG_PROVIDE([libaom])
 PKG_PROVIDE([dav1d])
 PKG_PROVIDE([libwebp])
+PKG_PROVIDE([nasm])
 PKG_PROVIDE([openExr], [$WANT_OPENEXR])
 PKG_PROVIDE([openexr], [$WANT_OPENEXR])
 PKG_PROVIDE([openjpeg])
@@ -1219,6 +1226,7 @@ echo ""
 echo "export thirdparty_libraries libraries"
 echo "export CFLAGS_ CXXFLAGS_ LDFLAGS_"
 echo "unexport CFLAGS CXXFLAGS LDFLAGS"
+echo "export PATH=`realpath $(echo $PWD)`/thirdparty/nasm-2.14.02:$PATH"
 
 if test "x$HAVE_tiff" = "xyes"; then
   if test "x$HAVE_jbig" != "xyes"; then
diff --git a/cinelerra-5.1/thirdparty/Makefile b/cinelerra-5.1/thirdparty/Makefile
index 0c595ee..d5ae729 100644
--- a/cinelerra-5.1/thirdparty/Makefile
+++ b/cinelerra-5.1/thirdparty/Makefile
@@ -234,6 +234,7 @@ libtheora.ldflags?="$(call ld_path,libvorbis,lib/.libs) $(call ld_path,libogg,sr
 libtheora.cfg_params?= --disable-examples --disable-spec --enable-shared=no
 libuuid.cfg_params?=--enable-shared=no
 libvorbis.cfg_params?= --disable-oggtest --enable-shared=no
+nasm.cfg_params?= --disable-doc
 openjpeg.cfg_params?= -DBUILD_SHARED_LIBS:BOOL=OFF
 openjpeg.cfg_vars?=$(call cmake_config,.)
 openjpeg.mak_params?= ; cd $(call bld_path,openjpeg,src/lib/openjp2); ln -sf . openjpeg-2.1
@@ -329,7 +330,7 @@ $(call rules,$(call std-build,ilmBase, openexr))
 $(call rules,$(call std-build,ladspa))
 $(call rules,$(call std-build,lame))
 $(call rules,$(call std-build,libaom))
-$(call rules,$(call std-build,dav1d))
+$(call rules,$(call std-build,dav1d,nasm))
 $(call rules,$(call std-build,libwebp))
 $(call rules,$(call std-build,libavc1394,libraw1394))
 $(call rules,$(call std-build,libdv))
@@ -350,8 +351,8 @@ $(call rules,$(call std-build,opus))
 $(call rules,$(call std-build,speech_tools))
 $(call rules,$(call std-build,tiff, libwebp))
 $(call rules,$(call std-build,twolame))
-$(call rules,$(call std-build,x264))
-$(call rules,$(call std-build,x265))
+$(call rules,$(call std-build,x264,nasm))
+$(call rules,$(call std-build,x265,nasm))
 $(call rules,$(call std-build,libvpx))
 $(call rules,$(call std-build,lv2))
 $(call rules,$(call std-build,serd))
nasm_thirdparty.patch (3,719 bytes)
Andrew-R

Andrew-R

2020-03-21 18:57

reporter   ~0002923

ok, libvpx version + 0.0.1 doesn't sounds very exiting :}

but missing asm sounds as potential slowdown on all machines without nasm > 2.13 (because x264/x265/ffmpeg all use it ?)

I tried to add this to configure.ac, in addition to other lines:
  echo "export PATH=`realpath $(echo $PWD)`/thirdparty/nasm-2.14.02:$PATH"

now I have line in global_config

[a lot of line skipped]

export thirdparty_libraries libraries
export CFLAGS_ CXXFLAGS_ LDFLAGS_
unexport CFLAGS CXXFLAGS LDFLAGS
export PATH=/dev/shm/tmp/cinelerra-goodguy-20200321/cinelerra-5.1/thirdparty/nasm-2.14.02:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/bin:/usr/bin:/bin:/usr/X11R7/bin:/usr/games:/usr/lib/java/bin:/bin:

so, will it work or not .....

sorry, currently my thirdparty hacks a bit of mess, because they try to deal with few different issues at one go ..:/
PhyllisSmith

PhyllisSmith

2020-03-21 18:28

manager   ~0002922

I completely forgot to mention that last month when upgrading the libraries that are listed int the file cinelerra_path/thirdparty/downloads.txt, libvpx currently at version1.8.1 had a problem at 1.8.2 so it was not upgraded.

No analysis was done on that at all - it did compile on Fedora but when I loaded a test file, it did not correctly load so we just dropped it.
Andrew-R

Andrew-R

2020-03-21 17:34

reporter   ~0002921

I tried to add nasm to thirdparty buildsystem,

in thirdparty/Makefile

diff --git a/cinelerra-5.1/thirdparty/Makefile b/cinelerra-5.1/thirdparty/Makefile
index 0c595ee..f2b89ac 100644
--- a/cinelerra-5.1/thirdparty/Makefile
+++ b/cinelerra-5.1/thirdparty/Makefile
@@ -234,6 +234,7 @@ libtheora.ldflags?="$(call ld_path,libvorbis,lib/.libs) $(call ld_path,libogg,sr
 libtheora.cfg_params?= --disable-examples --disable-spec --enable-shared=no
 libuuid.cfg_params?=--enable-shared=no
 libvorbis.cfg_params?= --disable-oggtest --enable-shared=no
+nasm.cfg_params?= --disable-doc
 openjpeg.cfg_params?= -DBUILD_SHARED_LIBS:BOOL=OFF
 openjpeg.cfg_vars?=$(call cmake_config,.)
 openjpeg.mak_params?= ; cd $(call bld_path,openjpeg,src/lib/openjp2); ln -sf . openjpeg-2.1
@@ -248,13 +249,14 @@ tiff.cfg_params+= --enable-shared=no --disable-zstd $(call if_pkg,libwebp,\
  --with-webp-lib-dir=$(call pkg_libs,libwebp))
 twolame.cfg_params?=--enable-shared=no
 x264.cfg_params?= --enable-static --enable-pic
-x265.cfg_vars?=$(call cmake_config,source)
+#x265.cfg_vars?=$(call cmake_config,source)
 x265.cfg_params?= -DENABLE_SHARED=no
 libvpx.cfg_params?= --enable-pic

 DS:=$$$$$$$$
 pkg_cfg=$(call bld_path,$(1),usr/local/lib/pkgconfig):
 LV2_PKGCFG:=$(subst : ,:,$(foreach p,lv2 serd sord sratom lilv suil,$(call pkg_cfg,$(p))))
+LV2_PKGCFG:=+ $(PKG_CONFIG_PATH)

 # lv2
 define waf-setup
@@ -313,13 +315,13 @@ rules=$(info $(1))
 else
 rules=$(eval $(1))
 endif
-
+$(call rules,$(call std-build,nasm))
 $(call rules,$(call std-build,a52dec,djbfft))
 $(call rules,$(call std-build,djbfft))
 $(call rules,$(call std-build,audiofile))
 $(call rules,$(call std-build,encore))
 $(call rules,$(call std-build,esound,audiofile))


in configure.ac

diff --git a/cinelerra-5.1/configure.ac b/cinelerra-5.1/configure.ac
index f61ef27..edc569a 100644
--- a/cinelerra-5.1/configure.ac
+++ b/cinelerra-5.1/configure.ac
@@ -363,6 +363,11 @@ PKG_3RD([twolame],[auto],
   [ libtwolame/.libs/libtwolame.a ],
   [ libtwolame ])

+PKG_3RD([nasm],[yes],
+ [nasm-2.14.02],
+ [ ],
+ [ . ])
+
 PKG_3RD([x264],[auto],
   [x264-snapshot-20191217-2245-stable],
   [ libx264.a ],
@@ -373,7 +378,7 @@ PKG_3RD([x265],[auto],
   [ libx265.a ],
   [ . source ])

-PKG_3RD([libvpx],[auto],
+PKG_3RD([libvpx],[yes],
   [libvpx-1.8.1],
   [ libvpx.a ],
   [ . ])
@@ -908,6 +913,7 @@ PKG_PROVIDE([libtheora])
 PKG_PROVIDE([libuuid])
 PKG_PROVIDE([libvorbis])
 PKG_PROVIDE([mjpegtools])
+PKG_PROVIDE([nasm])
 PKG_PROVIDE([libaom])
 PKG_PROVIDE([dav1d])
 PKG_PROVIDE([libwebp])

sorry this libvpx thing sneaked in again ....

and nasm-2.14.02.tar.xz in thirdparty/src
but how to add newly-build nasm executable to PATH ?
Andrew-R

Andrew-R

2020-03-21 16:41

reporter   ~0002920

Sorry about failing builds. May be at some point in the future at least some libraries will re-add this backward compatibility. In the meantime re-using already existing download mechanism (used for ffmpeg/OpenCV) will be helpful in my case .... (but this will require some conditionals and two or more set of patches).
PhyllisSmith

PhyllisSmith

2020-03-21 16:00

manager   ~0002919

So today I am doing a production test build. Looks like the latest version of openexr is broken, and dav1d is also so do not make the cut. It is important to realize that upgrading is a double edge sword. It does bring fixes and adds vitality to the contemporary state of the library series. Frequently, the thirdparty build exceeds the vendor offerings, Particularly with high use visibility packages like ffmpeg. But it also bring risks. When a build fails, it may involve restarting the entire series, after the mod freeze, and with almost no time to test anything. This is not a great idea when you are looking a bug in a console log, and the rest of the builds are still chugging along. What do you do? Almost without exception, an emergency fix is added which removes the cause of the problem, usually by dropping a mod. This consequence is almost always an untested result.

What to do, what to do... When a package "does not play well with others", I usually freeze it. OpenCV has been frozen ever since they dropped sift and surf in a sort of bait and switch source bases. dav1d is also in peril since they make no effort to provide usable source on the systems available in the community. The failing instruction seems to be only used a s "shift". There are no other ways to do this which work for the mortal man? say like divide, or what about a regular shift? Almost all code that has historical relevance has some sort of conditioning to provide historical context. I am sort of expecting that dav1d is also about to be a "bait and switch", probably with the encoder. This makes it much less likely to be carried, and is against the "open source" policy that they use as bait.
Andrew-R

Andrew-R

2020-03-20 16:16

reporter   ~0002915

> IMHO it would be very bad to make it dependent on some advanced tools (like the latest and greatest version of nasm) that are not supported on the platforms.

well, this is true for GCC/libstdc++ itself, because they have huge, system-wide impact. But nasm itself small enough for updating.

> Just being able to build it one one site and just distributes the binaries is not what we should aim for I think, quite the contrary.

I agree, just I usually find some possible solution AFTER I speak out some idea ....
see https://www.mail-archive.com/[email protected]/msg01170.html
(" AVX512 asm instructions - idea for configure.ac") and
https://www.mail-archive.com/[email protected]/msg01172.html
("stdc++11 detecion via autotools") - damn, typo! Should be detection!

>> a .vdi size of between 10 and 15G per distro ..

mmm ... sorry, I should buy external hdd again :}
LANG=C df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 5.8G 1.1M 5.8G 1% /dev
/dev/sda2 9.8G 9.4G 0 100% /
/dev/sda3 269G 269G 70M 100% /home
/dev/loop0 177M 177M 0 100% /opt/libreoffice-5
none 5.8G 5.8G 25M 100% /dev/shm

you see, not a whole lot of space ... but, this is silly excuse. Will fix it later ... hopefully this month?

> If you find a fix for a bug or new feature, build CinGG with that on at least one CinGG supported other platform, to make sure you catch build problems.

Yeah, and from experience I better to try on OLD distro, because new ones usually have more eyes on it ....
MatN

MatN

2020-03-20 15:11

reporter   ~0002914

@Andrew-R,
I applaud you enthusiasm to improve CinGG. However, I feel there are some important things to keep in mind.
- CinGG is very stable, and it should be a goal not to lower it. It is better to have less features in a stable program than nice bells-and-whistles in one that crashes. CinGG has a reputation to lose!
- CinGG is available for many platforms, and can be built on many. IMHO it would be very bad to make it dependent on some advanced tools (like the latest and greatest version of nasm) that are not supported on the platforms. Just being able to build it one one site and just distributes the binaries is not what we should aim for I think, quite the contrary.
- I keep a set of VirtualBox VMs with different Linux distros just for testing. I don't know how much space you have, but using dynamic disks and not installing more than is needed, then after installing CinGG build tools and compiling it, I end up having a .vdi size of between 10 and 15G per distro (with Mint XFCE so far the smallest). Maybe you can set one up for a CinGG supported customer friendly 64 bit distribution, like Ubuntu, or Mint? I normally use Mint XFCE. And I do not upgrade my VMs with levels of software beyond what the distro has, otherwise it isn't a good representation of what is in the field, I only use the distro's package manager.
- If you find a fix for a bug or new feature, build CinGG with that on at least one CinGG supported other platform, to make sure you catch build problems.
Andrew-R

Andrew-R

2020-03-19 21:43

reporter   ~0002913

I think configure (in CinGG) option is "--enable-dav1d", so for disabling it you need "--disable-dav1d". You can go dirty way and install locally-built nasm in /usr/local/bin (or even in your home dir, just make sure PATH variable is updated). I think because main speed of libdav1d comes from assembler language fragments - their reliance on new nasm understandable ....
Andrew-R

Andrew-R

2020-03-19 19:53

reporter   ~0002912

I see .. sorry, it was my idea to upgrade dav1d.. On Slackware it was easy to upgrade nasm. It should be relatively safe to upgrade nasm alone on buildfarm systems (unlike, say, gcc/libstdc++ itself ..). May eb I really should try and do some 'admin' work there? (only thing I remotely good at is building and rebuilding stuff ..until it, hopefully, works). But I don't have GPG key or some known good reputation.... and anyway, updating build systems probably best to try AFTER monthly rebuild finished successfully.
PhyllisSmith

PhyllisSmith

2020-03-19 17:29

manager   ~0002911

Looks like you found the CDEF_FILTER_4x4 problem. We will back out 0.6 in the next 20 minutes in GIT.

GG writes below:
The first complaint from nasm seems to be:
src/x86/cdef.asm:1902: error: parser: instruction expected
src/x86/cdef.asm:1894: ... from macro `CDEF_FILTER_4x4_PRI' defined here
which is:
    vgf2p8affineqb m9, m2, [r3+r6*8] {1to8}, 0 ; abs(diff) >> shift
You have to admit, anybody who embeds the word "affine" in an instruction
is probably already in trouble... yep, it seems to be an actual instruction, not
a macro.
The dav1d guys seem to have no regard for whether or not it may work in
the field. I sent them a makefile that prevents the problems it already has
with meson, and they rejected it. Apparently working on your machine is
not a goal for them. It also seems that gitlab now require credentials to just
access the current source. Seems like the whole idea of opensource is not
important and is being factored out of this code base. I think this product
just went static, and will not be upgraded. It has no effective encoder, and
is only available on 4 of 18 platforms. Sorry to hear that.
MatN

MatN

2020-03-19 16:24

reporter   ~0002910

I think that the assembler complains about instruction "vpcmpub", the first one in macro CDEF_FILTER_4x4_PRI . That instruction is I think only available for AVX512 (Intel). Is there not a configuration option to not used AVX512?
PhyllisSmith

PhyllisSmith

2020-03-19 16:00

manager   ~0002909

Attached is the NASM levels from our build computer using xen.

NASM_levels.txt (2,351 bytes)
== /os/arch02 NASM version 2.13.03 compiled on Apr 1 2018
== /os/arch03 NASM version 2.14.02 compiled on Jan 22 2019
== /os/avlnx NASM version 2.12.01
== /os/cent01 NASM version 2.10.07 compiled on Jun 9 2014
== /os/cent03 NASM version 2.10.07 compiled on Jun 9 2014
== /os/cent04 NASM version 2.13.03 compiled on Jul 14 2018
== /os/cent05 NASM version 2.13.03 compiled on May 14 2019
== /os/deb01 NASM version 2.11.05 compiled on Sep 9 2014
== /os/deb02 NASM version 2.13.03 compiled on Jul 14 2018
== /os/deb02i NASM version 2.11.05 compiled on Sep 9 2014
== /os/deb04 NASM version 2.13.03 compiled on Jul 14 2018
== /os/deb05 NASM version 2.13.03 compiled on Apr 15 2018
== /os/deb06 NASM version 2.14
== /os/fc23 NASM version 2.11.08 compiled on Jun 17 2015
== /os/fc24 NASM version 2.12 compiled on Mar 3 2016
== /os/fc25 NASM version 2.13.01 compiled on Jul 26 2017
== /os/fc25n NASM version 2.13.01 compiled on Jul 26 2017
== /os/fc26 NASM version 2.13.02 compiled on Jan 2 2018
== /os/fc27 NASM version 2.13.02 compiled on Jan 2 2018
== /os/fc28 NASM version 2.13.03 compiled on Feb 8 2018
== /os/fc29 NASM version 2.13.03 compiled on Jul 14 2018
== /os/fc30 NASM version 2.13.03 compiled on Feb 2 2019
== /os/fc31 NASM version 2.14.02 compiled on Jul 25 2019
== /os/gentoo NASM version 2.14.02 compiled on Feb 1 2020
== /os/leap01 NASM version 2.10.09 compiled on Jul 26 2013
== /os/leap02 NASM version 2.10.09 compiled on Aug 4 2017
== /os/leap03 NASM version 2.13.03 compiled on Jul 14 2018
== /os/leap04 NASM version 2.13.02 compiled on Jan 24 2018
== /os/leap05 NASM version 2.13.02 compiled on Jan 24 2018
== /os/leap-nv NASM version 2.13.02 compiled on Jan 24 2018
== /os/mint02 NASM version 2.11.08
== /os/mint03 NASM version 2.13.03 compiled on Jul 14 2018
== /os/mint04 NASM version 2.13.02
== /os/mint05 NASM version 2.13.02
== /os/slk32 NASM version 2.13.01 compiled on Jun 23 2017
== /os/slk64 NASM version 2.13.01 compiled on Jun 23 2017
== /os/suse NASM version 2.10.09 compiled on Jul 26 2013
== /os/tweed NASM version 2.14.02 compiled on Nov 30 2019
== /os/ub14 NASM version 2.13.03 compiled on Jul 14 2018
== /os/ub1601 NASM version 2.13.03 compiled on Jul 14 2018
== /os/ub1602 NASM version 2.12.01
== /os/ub17 NASM version 2.12.02
== /os/ub18 NASM version 2.13.02
== /os/ubuntu NASM version 2.13.03 compiled on Apr 15 2018
NASM_levels.txt (2,351 bytes)
PhyllisSmith

PhyllisSmith

2020-03-19 15:51

manager   ~0002908

The 0.6 version of dav1d requires NASM 2.14. We are backing down to 0.5 and will stay at that version. It is ridiculous that the dav1d developers do not care if it works on Mint 19.3 which is the current Mint. The only distros that we build that are at 2.14 are the latest versions Arch, Debian, Gentoo, Tumbleweed, and Fedora.
The rest are at 2.12 and 2.13 including the most widely used Ubuntu.
PhyllisSmith

PhyllisSmith

2020-03-19 14:52

manager   ~0002907

Good to get this early warning. The problem appears to be the dav1d upgrade from 0.5 to 0.6 checked into GIT on March 12 with a macro. GG is looking at it now. As a temporary workaround I think here is a parameter in the build to script to build without dav1d (but I did not look it up).

It is getting more and more difficult to keep some of these thirdparty libraries up to date as the developers switch from tried and true standard tools to newer, less standard tools. For example OpenEXR is the worse, OpenCV is not really "Open" and dav1d WITHOUT providing any new capabilities. We are going to re-evaluate and "if it is not broken and provides nothing new, we are not going to change it so we can concentrate on Cinelerra code instead. However, in the case of dav1d, the speed up gains in version 0.6 would be good to have.
MatN

MatN

2020-03-19 12:45

reporter  

log (272,771 bytes)

Issue History

Date Modified Username Field Change
2020-03-19 12:45 MatN New Issue
2020-03-19 12:45 MatN File Added: log
2020-03-19 14:52 PhyllisSmith Assigned To => PhyllisSmith
2020-03-19 14:52 PhyllisSmith Status new => confirmed
2020-03-19 14:52 PhyllisSmith Note Added: 0002907
2020-03-19 15:51 PhyllisSmith Note Added: 0002908
2020-03-19 16:00 PhyllisSmith File Added: NASM_levels.txt
2020-03-19 16:00 PhyllisSmith Note Added: 0002909
2020-03-19 16:24 MatN Note Added: 0002910
2020-03-19 17:29 PhyllisSmith Note Added: 0002911
2020-03-19 19:53 Andrew-R Note Added: 0002912
2020-03-19 21:43 Andrew-R Note Added: 0002913
2020-03-20 15:11 MatN Note Added: 0002914
2020-03-20 16:16 Andrew-R Note Added: 0002915
2020-03-21 16:00 PhyllisSmith Note Added: 0002919
2020-03-21 16:41 Andrew-R Note Added: 0002920
2020-03-21 17:34 Andrew-R Note Added: 0002921
2020-03-21 18:28 PhyllisSmith Note Added: 0002922
2020-03-21 18:57 Andrew-R Note Added: 0002923
2020-03-21 19:28 Andrew-R File Added: nasm_thirdparty.patch
2020-03-21 19:28 Andrew-R Note Added: 0002924
2020-03-21 21:47 Andrew-R File Added: nasm_thirdparty-2.patch
2020-03-21 21:47 Andrew-R Note Added: 0002925
2020-03-22 06:58 Andrew-R File Added: c11_configure.ac.patch
2020-03-22 06:58 Andrew-R Note Added: 0002926
2020-03-22 07:23 Andrew-R Note Added: 0002927
2020-03-22 08:46 Andrew-R File Added: c11_configure.ac_from_oxr.patch
2020-03-22 08:46 Andrew-R Note Added: 0002928
2020-03-22 11:07 Andrew-R File Added: c11_configure.ac_from_oxr_1.patch
2020-03-22 11:07 Andrew-R Note Added: 0002929
2020-04-05 20:14 MatN Note Added: 0003096
2020-04-05 21:49 PhyllisSmith Status confirmed => closed
2020-04-05 21:49 PhyllisSmith Resolution open => fixed
2020-04-05 21:49 PhyllisSmith Note Added: 0003097