View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0000497Cinelerra-GG[All Projects] Bugpublic2020-08-31 00:592020-10-07 21:15
ReporterAndrew-R Assigned ToPhyllisSmith  
PrioritynormalSeverityminorReproducibilitysometimes
Status closedResolutionfixed 
Product Version2020-07 
Target VersionFixed in Version2020-09 
Summary0000497: Background rendering sometimes segfault?
DescriptionUsing Cin version commit 6ad20126d5f82618e5dd4dd2d14b0682a5529d17 I tried background render feature, it mostly works, but sometimes segfaults (probably by reading not-yet written file?)
Steps To Reproduce

1) Set background rendering to some heavy compressor, like tiff or png.
2) set background render directory to tmpfs (/dev/shm/brender , for example)
3) Load heavy video (lie 1080p/60 fps h264)
4) Select region on timeline
5) Turn on Background rendering from menu.
6) Watch as red bar started to appear.
7) Hit play
8) Watch how playback cursor run ahead of already-rendered portion of timeline

9) sometimes it will crash like this:

cin
Cinelerra Infinity - built: Aug 31 2020 01:29:58
git://git.cinelerra-gg.org/goodguy/cinelerra.git
(c) 2006-2019 Heroine Virtual Ltd. by Adam Williams
2007-2020 mods for Cinelerra-GG by W.P.Morrow aka goodguy
Cinelerra is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. There is absolutely no warranty for Cinelerra.

RenderFarmClient::main_loop: client started
FFMPEG::open_decoder: some stream times estimated: /home/guest/New_hdd/Swimming with orca in New Zealand-JQ3mDXF3bcE.mkv
FFMPEG::open_decoder: some stream times estimated: /home/guest/New_hdd/Swimming with orca in New Zealand-JQ3mDXF3bcE.mkv
FFMPEG::open_decoder: some stream times estimated: /home/guest/New_hdd/Swimming with orca in New Zealand-JQ3mDXF3bcE.mkv
FFMPEG::open_decoder: some stream times estimated: /home/guest/New_hdd/Swimming with orca in New Zealand-JQ3mDXF3bcE.mkv
FFMPEG::open_decoder: some stream times estimated: /home/guest/New_hdd/Swimming with orca in New Zealand-JQ3mDXF3bcE.mkv
FFMPEG::open_decoder: some stream times estimated: /home/guest/New_hdd/Swimming with orca in New Zealand-JQ3mDXF3bcE.mkv
FFMPEG::open_decoder: some stream times estimated: /home/guest/New_hdd/Swimming with orca in New Zealand-JQ3mDXF3bcE.mkv
FFMPEG::open_decoder: some stream times estimated: /home/guest/New_hdd/Swimming with orca in New Zealand-JQ3mDXF3bcE.mkv
BRenderThread::start 1 map=0 equivalent=0 brender_start=4532 result=4532 end=5778
RenderFarmClient::main_loop: Session started from /tmp/cinelerra.d73cd1ef-7464-4c0d-a041-b94ffca8a0e5
FFMPEG::open_decoder: some stream times estimated: /home/guest/New_hdd/Swimming with orca in New Zealand-JQ3mDXF3bcE.mkv
FFMPEG::open_decoder: some stream times estimated: /home/guest/New_hdd/Swimming with orca in New Zealand-JQ3mDXF3bcE.mkv
FileTIFF: Cannot read TIFF header.
** segv at 0xf7583d7f in pid 8190, tid 8346
writing debug data to /tmp/cinelerra_8190.dmp
lock_items: 23
lock_frees: 5
AudioALSA::write_buffer err -32(Broken pipe) at sample 174435
** dump complete
Ошибка сегментирования
TagsNo tags attached.

Activities

PhyllisSmith

PhyllisSmith

2020-09-30 23:56

manager   ~0004093

The print statement was removed earlier. Will close in a day or 2.
Andrew-R

Andrew-R

2020-09-05 08:28

reporter   ~0003994

Seems to work with tiff and exr :}
There is some sort of debug print left, but it doesn't hurt much, may be leave it there until next release (I'll try to stress-test this feature more)?
----
BRender::set_video_map(5600, 2)
BRender::set_video_map(5601, 2)
BRender::set_video_map(5602, 2)
-----
PhyllisSmith

PhyllisSmith

2020-09-05 02:41

manager   ~0003992

@Andrew-R
A fix for reported problem has been checked into GIT.
"move brender set_video_map update to write_frame_done"
More later.
Andrew-R

Andrew-R

2020-09-04 19:23

reporter   ~0003988

patch (this one was already send some time ago to mail List, but just in case you missed it ..)

If you apply patch and run Cin with background render set o OpenEXR - it surely will crash too on playback of region you already rendering slowly ... . But I tried with standard choices like tiff and png just to be sure about bug being confined to OpenEXR loading ..it and it turned to be generic?

OpenEXR as brender is not very fast, but my point was to use 32 bit per channel floating point formats, so all those (possible) errors accumulating from float-> 8 bit int rgb conversion if you work in RGBA-float will not show up ....

FORMAT_brender_exr_added.diff (452 bytes) 
diff --git a/cinelerra-5.1/cinelerra/formatpopup.C b/cinelerra-5.1/cinelerra/formatpopup.C
index addd4a2..dc1e951 100644
--- a/cinelerra-5.1/cinelerra/formatpopup.C
+++ b/cinelerra-5.1/cinelerra/formatpopup.C
@@ -92,7 +92,10 @@ void FormatPopup::create_objects()
 	if(!use_brender)
 		post_item(FILE_TIFF);
 	post_item(FILE_TIFF_LIST);
-
+#ifdef HAVE_OPENEXR
+	if(use_brender)
+	post_item(FILE_EXR_LIST);
+#endif
 	update(&format_items, 0, 0, 1);
 }
Andrew-R

Andrew-R

2020-09-04 19:16

reporter   ~0003987

Also, while complete not based on this commit

https://github.com/olive-editor/olive/commit/27effb9473e66d3d40866df740b148bcb6e02d4a
" use OIIO DWAA for cache "

I was also experimenting with 'EXR for background render' and this was exactly how I found this bug :}
Andrew-R

Andrew-R

2020-09-04 19:11

reporter   ~0003986

Also, I was looking in Olive (video editor using qt5) git, and found those two lines:
https://github.com/olive-editor/olive/commits/master?after=e42ba3f9d4042eb062e0575ac3af8b187f1e064c+1189&branch=master
"made auto-deleting cache on close an option"
"added disk preferences pane and the ability to set the disk cache loc… "

I think this functionality close to Cin's background render, so solution like 'remove brender files on close' actually might be working idea (but still, I'm not sure if such solution will work well in renderfarm setup, I only tried one instance of Cin running locally, while I think I saw few 'cin -b /tmp/long-string-of-letters-and-numbers' processes anyway)
Andrew-R

Andrew-R

2020-09-01 19:54

reporter   ~0003959

> Here is the problem with CinGG deleting them -- When do you do this? When you quit out?

I think? Or what happen when I set background render to one type of files over same region, and then after it worked, decided to switch filetype? It will be confused, I think ... May be (yet another!) preference, like deleting background render files on quit/disable event ?

Also, I use /dev/shm for speed (because my hdd where my real /tmp is resided is not speed daemon), and other users may want to put those files on fast NVME/ssd :}
PhyllisSmith

PhyllisSmith

2020-09-01 19:18

manager   ~0003957

"Also, even if I un-toggle background rendering and red-brown bar disappear from timeline - files created during its run still around even after I quit CinGG ... Isn't this a bug, too?"

Suggestion, which should be added to the manual, is to plan on putting the files in /tmp (like is the default) so that on most systems, when you reboot they will automatically be deleted.

Here is the problem with CinGG deleting them -- When do you do this? When you quit out? So if you are being called to dinner and you want to hurry, then instead of quitting right away, Cinelerra is fooling around deleting these files and your food gets cold. Do you have a better suggestion?
Andrew-R

Andrew-R

2020-08-31 02:23

reporter   ~0003953

Also, even if I un-toggle background rendering and red-brown bar disappear from timeline - files created during its run still around even after I quit CinGG ... Isn't this a bug, too? (those files can be big, and will eat disk space ...for now I manually delete them after each test run)
Andrew-R

Andrew-R

2020-08-31 02:19

reporter   ~0003952

I watched directory where CinGG was creating brender files (pressing and holding ctrl-r in Midnight commander) - sometimes 0-sized files appeared there, and then after some time replaced by properly-sized files. may be check for this case (0-sized file) in readers? Ah, no, files not created in instant either (currently I recompile CinGG, so machine under load, and I can watch this process of file creation in slow motion).

Another idea was to add .lock file in writers, remove it when file is surely written, and watch for this .lock file in readers ....
PhyllisSmith

PhyllisSmith

2020-08-31 01:45

manager   ~0003951

Read the dump and GG was able to reproduce using Big Buck Bunny. He says "yes" that is a bug. He is looking at it now but do not know how long that will take. You analysis of "probably by reading not-yet written file?" sounds like a good possibility.
Andrew-R

Andrew-R

2020-08-31 00:59

reporter  

cinelerra_8190.dmp (147,185 bytes)

Issue History

Date Modified Username Field Change
2020-08-31 00:59 Andrew-R New Issue
2020-08-31 00:59 Andrew-R File Added: cinelerra_8190.dmp
2020-08-31 01:45 PhyllisSmith Assigned To => PhyllisSmith
2020-08-31 01:45 PhyllisSmith Status new => confirmed
2020-08-31 01:45 PhyllisSmith Note Added: 0003951
2020-08-31 02:19 Andrew-R Note Added: 0003952
2020-08-31 02:23 Andrew-R Note Added: 0003953
2020-09-01 19:18 PhyllisSmith Note Added: 0003957
2020-09-01 19:54 Andrew-R Note Added: 0003959
2020-09-04 19:11 Andrew-R Note Added: 0003986
2020-09-04 19:16 Andrew-R Note Added: 0003987
2020-09-04 19:23 Andrew-R File Added: FORMAT_brender_exr_added.diff
2020-09-04 19:23 Andrew-R Note Added: 0003988
2020-09-05 02:41 PhyllisSmith Note Added: 0003992
2020-09-05 08:28 Andrew-R Note Added: 0003994
2020-09-30 23:56 PhyllisSmith Status confirmed => resolved
2020-09-30 23:56 PhyllisSmith Resolution open => fixed
2020-09-30 23:56 PhyllisSmith Fixed in Version => 2020-09
2020-09-30 23:56 PhyllisSmith Note Added: 0004093
2020-10-07 21:15 PhyllisSmith Status resolved => closed