View Issue Details

IDProjectCategoryView StatusLast Update
0000562Cinelerra-GG[All Projects] Bugpublic2021-04-13 21:07
ReporterMatN Assigned ToPhyllisSmith  
PrioritynormalSeveritymajorReproducibilityalways
Status acknowledgedResolutionopen 
PlatformX86_64OSMint XFCEOS Version20.1
Product Version 
Target VersionFixed in Version 
Summary0000562: AppImage 201-02, during render, wrong file is accessed
DescriptionI started the AppImage from a terminal. I was rendering DNxHR_HQR, and something went wrong, but it kept on going anyway. In the end there was no output file. Fair enough. The terminal showed multiples times the same error.
However, after reporting the end, it kept on trying to go on with another file, one that came right after it the directory. I think this is very bad, trying to access the wrong file.

That part of the directory looked like:

LG_Dolby_Trailer_4K_Demo.ts (input file, 300MB)
LG_trailer_4K_dnxhr_hqx.mov (intended output file, size 0)
Marjan_65jaar_720p.mp4 (next file in the folder)


FFMPEG::open_decoder: some stream times estimated: /home/mat/Videos/LG_Dolby_Trailer_4K_Demo.ts
Failed HW device create.
dev:vaapi
  err: Input/output error
HW device init failed, using SW decode.
file:/home/mat/Videos/LG_Dolby_Trailer_4K_Demo.ts
  err: Operation not permitted
Render::render_single: Session finished.
** rendered 1939 frames in 431.779 secs, 4.491 fps
Decoder dnxhd does not support device type vaapi.
HW device init failed, using SW decode.
file:/home/mat/Marjan_65jaar_720p.mov
  err: Operation not permitted

TagsNo tags attached.

Activities

PhyllisSmith

PhyllisSmith

2021-04-13 21:07

manager   ~0004707

@Andrea_Paz
Interesting - I did not know that about ffmpeg usage from the command line being very CPU happy..
In my CInGG experiments just now, I am finding big differences in CPU usage when I render the same file using different formats. The older formats may not be incorporating any multi-threading in their library code. Results:

MP4 - uses as much as 1500 cpu's out of 1600.
MOV - uses as much as 1400 cpu's out of 1600.
DVD - uses as much as 750 cpu's out of 1600 -- a much older format.
OGG - uses as much as 165 cpu's out of 1600 -- a really old theora format.
Andrea_Paz

Andrea_Paz

2021-04-13 20:52

manager   ~0004706

Last edited: 2021-04-13 20:53

View 2 revisions

Even I can't use all the cores with CinGG. For encoding I have to use the Render Farm to get the CPU used at 100%. See:
https://www.mail-archive.com/[email protected]/msg01565.html
https://www.mail-archive.com/[email protected]/msg00854.html

(Ffmpeg from command line uses all cores!)

PhyllisSmith

PhyllisSmith

2021-04-13 19:59

manager   ~0004705

@MatN
About "is there something in ffmpeg that limits it?" for CPU usage, I would have to just guess "yes" something is limiting it at times but not necessarily ffmpeg; it could be Cinelerra or the just about anything else that might be involved such as the particular render decode format being used which in this case is dnxhr_hqx. Cinelerra has locks all over the place and this would definitely limit the CPU usage. If I figure anything specific out about this, I will report back.
MatN

MatN

2021-04-13 18:59

reporter   ~0004704

@Phyllis,
I tested again on a Mint 20.1 laptop, with on CinGG on it except the AppImage 20210331. I found I also have the LG Comparison file, so used that. Because the laptop only has 8G RAM, I created a 32G swapfile; the advantage is also you can see how much memory is used by seeing how much of the swapfile is used (regular "free -h" entries in another terminal).
With "vaapi" (and this AMD 4500U machine does have vaapi hardware), it did 5.8 fps, end result was a close to 20G file (!) with bitrate almost 700 Mb/s. Max swapfile use was 0.9 G.
With "none", it did 5.9 fps, same size end result, but max swapfile was 2.0G . This is the same which I noted before, that with vaapi it uses less memory.
Maybe I really did run out of disk space, the desktop I tried it on before had about 20G free. But "df" did not report it full. I'll try again on that machine.

One thing I noticed: The CPU use was far from 100% (this CPU has 6 cores, no SMT). It constantly hovered around 50%, sometimes for a few seconds it was 100%. Both with and without vaapi. because certainly without vaapi all de- and encoding must be done by software, why is the CPU not constant 100%? I have seen it so on other encodes. Is there something in ffmpeg that limits it?
PhyllisSmith

PhyllisSmith

2021-04-12 17:12

manager   ~0004703

Last edited: 2021-04-12 17:15

View 2 revisions

@MatN
Instead of using the file you used: LG_Dolby_Trailer_4K_Demo.ts (input file, 300MB), I used "LG Dolby Compariuson 4K Demo.ts" of 308MB, because this is what I had on my computer (actual size was: 308 162 644).

I ran with vaapi enabled and none with the same output file generated of size: 6 590 411 856. This file is HUGE at 20 times larger. I think you may have really run out of disk space because not only does the rendered output file take space, the $HOME/.bcast5 file uses some space also AND with vaapi enabled, potentially space is used temporarily for processing that is unseen.

In my scenario, when I ran out of disk space, I still had a size of 3 198 955 520 for the output file (about half of what the final file would have been) instead of 0 size that you had for LG_trailer_4K_dnxhr_hqx.mov. I think the difference is due to the way the O/S Fedora operates versus Mint as far as reporting disk space. My Mint O/S, as well as all except the one laptop I used in this experiment, use Nvidia boards/vdpau and are AMD based so I have nothing to test this on Mint. (Just the HP laptop has Intel and an Intel graphics board for vaapi).

Also, as far as the initial report of "something went wrong, but it kept going anyway...no output file...trying to access the wrong file", is it possible that you were out of disk space that time also? When that happens, things can really go badly.

BTW: the error message of "HW device init failed, using SW decode" means that you are really not taking advantage of the graphics board as it can not handle the graphics directives being given to it. However, it may be attempting to use it in some cases anyway, which I think may be using temporary disk space. There is some slight evidence in the case I ran with Vaapi and with None in that with Vaapi, render statistics showed 3.278 fps versus 3.319 fps for None. I also had the "using SW decode" error message.

Summary: I do not think we can definitively attribute these problems to CinGG, but I could be wrong as always!

PhyllisSmith

PhyllisSmith

2021-04-12 15:21

manager   ~0004702

Last edited: 2021-04-12 15:32

View 2 revisions

@MatN
(JUST EDITED NOW)
Not done looking at this yet; I wanted to give the update that I also get the "No space left on device" BUT unlike you, I really was out of disk space. I am using Fedora and vaapi.

MatN

MatN

2021-04-10 06:42

reporter   ~0004701

@Phyllis,
Indeed, I will try again, but I was so surprised by this error that I need to see where this comes from. I cannot imagine it was CinGG, or something really weird is going on. I don't see CinGG keeping track of used/available disk space. I did not restart CinGG between the two renders, that's next to try (if it can be reproduced). System was Mint XFCE 20.1 with the AppImage from the CinGG site.
Almost any Linux will have vaapi (or it can be installed). It is the default video accelerator API for Intel and AMD hardware, the old vdpau from Nvidia is much less capable. Install program "vainfo" to check.
PhyllisSmith

PhyllisSmith

2021-04-09 22:56

manager   ~0004700

@MatN
What does "To be repeated." mean? Does it mean it is reproducible or that you are going to try it again? I will see if I can find a computer that does vaapi -- I know one of them does and now with AppImage it will be simple enough to try.
MatN

MatN

2021-04-09 19:41

reporter   ~0004699

Tried to re-create this problem using the 2021-03 AppImage.
- New project, format set to RGBA float, 6 audio channels, 2160p23.97.
- Load file LG demo .ts, append to existing tracks. Play a short while, fine.
- Set in "performance" settings hwdevice to "none"
- Render to file format ffmepg .mov, video compression dnxhr_hqx.mov, cin-pix-format yuv422p10le.
- Render went OK at 4.7 fps, result was a 6.6G file. Max memory use about 7.6GB.
- Deleted the output file file, set hwdevice to vaapi, started same render. This failed after the output file reached 3.1G an "err: No space left on device" . Command "df" shows almost 7 GB still free.
Weird. To be repeated.

Issue History

Date Modified Username Field Change
2021-03-11 10:02 MatN New Issue
2021-04-09 19:41 MatN Note Added: 0004699
2021-04-09 22:56 PhyllisSmith Note Added: 0004700
2021-04-10 06:42 MatN Note Added: 0004701
2021-04-12 15:21 PhyllisSmith Note Added: 0004702
2021-04-12 15:32 PhyllisSmith Note Edited: 0004702 View Revisions
2021-04-12 17:12 PhyllisSmith Note Added: 0004703
2021-04-12 17:12 PhyllisSmith Assigned To => PhyllisSmith
2021-04-12 17:12 PhyllisSmith Status new => acknowledged
2021-04-12 17:15 PhyllisSmith Note Edited: 0004703 View Revisions
2021-04-13 18:59 MatN Note Added: 0004704
2021-04-13 19:59 PhyllisSmith Note Added: 0004705
2021-04-13 20:52 Andrea_Paz Note Added: 0004706
2021-04-13 20:53 Andrea_Paz Note Edited: 0004706 View Revisions
2021-04-13 21:07 PhyllisSmith Note Added: 0004707