[Scott9mm]

Symptom: Atomic Parsley never completes after KMTTG transcodes (MPG to MP4) some programs. It seems almost all of these were recorded over the air and from a station that has good signal sometimes and marginal sometimes. That transmitter is 80 miles away and is about 55 degrees off the antenna axis so I'm surprised I get it at all but I like the Star Trek stuff on H&I network.

More: When this happens only a *-temp.mp4 file is produced and in many cases it is corrupted (badly pixelated, picture halts while sound continues) although the mpg file is more playable. In most cases I would consider the resulting mp4 file un-watchable.

Theory: I think the *-temp,mp4 is too corrupted for Atomic Parsley to process so Parsley just hangs ... forever.

Current procedure: When I see Atomic Parsley is stuck I quick-look at the *-temp.mp4 file. If it looks tolerable I delete -temp from the filename. Otherwise I delete the offending program from the TiVo and hope for a better recording later. These are all reruns anyway.

Question: What does Atomic Parsley actually do? I looked briefly but only saw vague answers. If nothing else, it seems useful as a quality check on the mp4 file.

 

[Scott9mm]

Follow-on:

This "atomic parsley hang" problem continues but my previous theory is wrong. Atomic Parsley hangs with both over-the-air recordings (which may be noisy) and also on cable recordings that are clean.

The common denominator seems to be old shows (i.e., reruns) like Wings and Star Trek sequels, and then, only for a minority of those shows.

If anyone wants to look into this, here are kmttg log entries. First, for a successful atomic run:

2020_10_13_14:43:11 >> Running AtomicParsley on \\DISKSTATION216\HD Video\NCIS Los Angeles\NCIS Los Angeles - s10e04 - Hit List.mp4 ...
2020_10_13_14:43:11 C:\Users\Scott\Desktop\KMTTG\AtomicParsley\AtomicParsley.exe "\\DISKSTATION216\HD Video\NCIS Los Angeles\NCIS Los Angeles - s10e04 - Hit List.mp4" --overWrite --stik "TV Show" --title "Hit List" --year 2018-10-21T00:00:00Z --description "A cartel hit list includes pictures of the NCIS team, as well as Mosley and her sons names. Also, NCIS Deputy Director Louis Ochoa arrives with Special Prosecutor John Rogers to interview Mosley about the off-the-books mission in Mexico. " --longdesc "A cartel hit list includes pictures of the NCIS team, as well as Mosley and her sons names. Also, NCIS Deputy Director Louis Ochoa arrives with Special Prosecutor John Rogers to interview Mosley about the off-the-books mission in Mexico. " --contentRating TV-14 --hdvideo true --TVShowName "NCIS: Los Angeles" --artist "NCIS: Los Angeles" --albumArtist "NCIS: Los Angeles" --TVEpisode 1004 --TVEpisodeNum 04 --tracknum 1004 --TVSeasonNum 10 --album "NCIS: Los Angeles, Season 10" --TVNetwork WRPXDT -d 1/1
....
2020_10_13_14:43:57 NOTE: AtomicParsley job completed: 0:00:46
2020_10_13_14:43:57 ---DONE--- job=atomic output=\\DISKSTATION216\HD Video\NCIS Los Angeles\NCIS Los Angeles - s10e04 - Hit List.mp4

And here are the log entries for a show that hung:

2020_10_13_14:22:49 >> Running AtomicParsley on \\DISKSTATION216\HD Video\Wings\Wings - s07e16 - Love at First Flight.mp4 ...
2020_10_13_14:22:49 C:\Users\Scott\Desktop\KMTTG\AtomicParsley\AtomicParsley.exe "\\DISKSTATION216\HD Video\Wings\Wings - s07e16 - Love at First Flight.mp4" --overWrite --stik "TV Show" --title "Love at First Flight" --year 1996-02-13T00:00:00Z --description "On Valentines Day, Cupid shoots Brian and his sole passenger, whos flying in for her wedding; and Roy makes a date with Antonios ex. Melissa: Ally Walker. David: Matthew Porretta. Brian: Steven Weber. Antonio: Tony Shalhoub. " --longdesc "On Valentines Day, Cupid shoots Brian and his sole passenger, whos flying in for her wedding; and Roy makes a date with Antonios ex. Melissa: Ally Walker. David: Matthew Porretta. Brian: Steven Weber. Antonio: Tony Shalhoub. " --contentRating TV-PG --TVShowName Wings --artist Wings --albumArtist Wings --TVEpisode 716 --TVEpisodeNum 16 --tracknum 716 --TVSeasonNum 7 --album "Wings, Season 7" --TVNetwork WNCNDT2 -d 1/1

At this point, there us still a yyyy-temp-xxx.mp4 file in the destination directory and there are no more log entries until I kill atomic parsley manually:

2020_10_13_16:39:44 NOTE: Killing 'atomic' job: C:\Users\Scott\Desktop\KMTTG\AtomicParsley\AtomicParsley.exe "\\DISKSTATION216\HD Video\Wings\Wings - s07e16 - Love at First Flight.mp4" --overWrite --stik "TV Show" --title "Love at First Flight" --year 1996-02-13T00:00:00Z --description "On Valentines Day, Cupid shoots Brian and his sole passenger, whos flying in for her wedding; and Roy makes a date with Antonios ex. Melissa: Ally Walker. David: Matthew Porretta. Brian: Steven Weber. Antonio: Tony Shalhoub. " --longdesc "On Valentines Day, Cupid shoots Brian and his sole passenger, whos flying in for her wedding; and Roy makes a date with Antonios ex. Melissa: Ally Walker. David: Matthew Porretta. Brian: Steven Weber. Antonio: Tony Shalhoub. " --contentRating TV-PG --TVShowName Wings --artist Wings --albumArtist Wings --TVEpisode 716 --TVEpisodeNum 16 --tracknum 716 --TVSeasonNum 7 --album "Wings, Season 7" --TVNetwork WNCNDT2 -d 1/1

This is just a minor annoyance but it would be nice if it didn't happen. I have set kmttg to four simultaneous encodes so kmttg stops processing if I reach four hung instances of atomic parsley. So far, I just manually kill the hung processes and rename the temp files. I suspect there is something about the xxx.mp4.txt file that atomic parsley doesn't like.

Maybe I should just disable Atomic Parsley? If so, what would be lost?

Thanks for any insights.

 

[reneg]

I allow two active jobs and I see a hang issue with Atomic Parsley rarely. When I double click on the hung job, for me, it's usually stuck near the end. I'm not sure if it's an Atomic Parsely problem or kmttg job problem. After file renaming, if I run Atomic Parsley on it's own, it works.

I you disable Atomic Parley, it will prohibit embedding the encoded file with metadata.

 

[ej42137]

There is a newer version of AtomicParsley here; it might be worth a try.

AtomicParsley inserts metadata into the video file. Video players and servers can access that information and display things like title, year, actors, etc.

 

[dlfl]

And AP can read embedded metadata from supported video files, e.g., .mp4, .m4v. The two programs linked in my signature use AP for both reading and embedding.

 

[Scott9mm]

There is a newer version of AtomicParsley here; it might be worth a try.

AtomicParsley inserts metadata into the video file. Video players and servers can access that information and display things like title, year, actors, etc.

Thanks for the tip but the new version didn't seem to help with my problem (parsley runs forever). Sometimes if I manually reran the download -> encode process Parsley terminated normally. Totally unpredictable but seems to happen only with old reruns and only some shows.

 

[dlfl]

Thanks for the tip but the new version didn't seem to help with my problem (parsley runs forever). Sometimes if I manually reran the download -> encode process Parsley terminated normally. Totally unpredictable but seems to happen only with old reruns and only some shows.

You might consider submitting an issue on the GitHub linked for the new version. You have to sign up for a free account and then you would probably need to find a way to provide a sample video file that illustrates the problem. Don't know how responsive the author would be.

 

[Scott9mm]

Thanks for this suggestion. I'll need to find a good test case. Sometimes it is not repeatable.
Just now, AtomicParsley failed to complete on an episode of All in the Family but ran correctly (to completion) when I let it run again (by deleting associated auto.history line). I think I'll need more specifics before asking for help.

 

[Scott9mm]

I'm the OP so I'd like to close this out with my solution. I have not addressed this on GitHub but didn't see anything promising there.

AtomicParsley completes normally in most cases. When it fails the line in the kmttg status window shows AtomicParsley still running (after many minutes or even hours). Looking at the logs, the AP command line appears correct and AP renames the mp4 file (xxxx-temp-xxxx.mp4); that temp file seems to play fine but I don't know if it has the metadata. However, the next time AP runs to completion it finds and deletes any xxx-temp-xxx.mp4 files. Bummer.

If I catch it before the temp files get deleted I can remove the "-temp-xxxx" from the filename and run AP manually (sometimes needed more than once).

I think I need a better solution.

 

[Scott9mm]

Wow, I just understood "double click" on a kmttg status line from post 3 and tried it!!

Here's what it shows when AP fails to complete:

AtomicParsley warning: desc was trimmed to 255 characters (33 characters over)

 Started writing to temp file.
 Progress: >  0% -------------------------------------------------------|
 ...
 Progress: =>  1% ------------------------------------------------------|
 Progress: ======> 12% -------------------------------------------------|
 Progress: ==================> 32% -------------------------------------|
 Progress: =============================> 53% --------------------------|
 Progress: ========================================> 73% ---------------|
 Progress: ====================================================> 94% ---|

 Finished writing to temp file.
Unable to write to a directory lacking write permission

So the problem is access permission. Note that this access failure occurs immediately after writing the successful temp file. These files are on a NAS, maybe that's the problem? There were two other "hung" instances of AP at this time and it required two (or three?) manual attempts to get AP to complete normally.

I need to experiment. Maybe it will work if I avoid the NAS?

It looks like AP has become an orphan with no real updates in years. Any new clues?

 

[reneg]

I need to experiment. Maybe it will work if I avoid the NAAS?

It looks like AP has become an orphan with no real updates in years. Any new clues?

I'm running it on a PC with all the kmttg activity on a SSD, so I don't think it's your NAS. I don't have any other suggestions.

 

[Scott9mm]

Currently, I'm writing all the encode (.MP4) files to my C: drive which is an SSD. Too soon to be sure but AP has completed normally every time so far with this setup. I am beginning to think that AP is trying to rename the "temp" file before the NAS finishes "unprotecting" the folder from the immediately-preceding write. It acts like AP just needs a pause or handshake between those two (write-rename) steps.

I put a query on GitHub but don't expect an answer.

For now, I'll just move files from my C: drive to the NAS manually.

 

[ej42137]

Finished writing to temp file.
Unable to write to a directory lacking write permission

I need to experiment. Maybe it will work if I avoid the NAS?

You have been told it is a permission error. Moving it off the NAS would fix that, but you could also investigate and correct the permission problem.

 

[reneg]

You have been told it is a permission error. Moving it off the NAS would fix that, but you could also investigate and correct the permission problem.

I doubt it. I suspect the OP theory is more likely based on what i've observed on my system.

 

[Scott9mm]

You have been told it is a permission error. Moving it off the NAS would fix that, but you could also investigate and correct the permission problem.

I changed the encoding destination from my NAS to my C: drive (which is a SSD). Since then, EVERTY Atomic Parsley job has run fine, and I've encoded hundreds of videos since that change.
Recall that most AP runs were OK even sending the encode file to the NAS but there were occasional failures (that stacked up). It appears that AP tries to rename the temp file name so soon after writing it and the file is still locked in some cases.
This problem started suddenly so I think it is related to some NAS OS change, but that's a guess. For now, I'll continue to encode to the temp directory on my C: drive and move files to the NAS later.
For some reason, new SRT files were left in the kmttg temp directory instead of overwriting old SRT files on the NAS. So I had to manually move them anyway. So also moving MP4 and TXT files is no big deal.

 

[kdmorse]

I changed the encoding destination from my NAS to my C: drive (which is a SSD). Since then, EVRTY Atomic Parsley job has run fine, and I've encoded hundreds of videos since that change.
Recall that most AP runs were OK even sending the encode file to the NAS but there were occasional failures (that stacked up). It appears that AP tries to rename the temp file name so soon after writing it that the file is still locked in some cases.
This problem started suddenly so I think it is related to some NAS OS change, but that's a guess. For now, I'll continue to encode to the temp directory on my C: drive and move files to the NAS later.
For some reason, new SRT files were left in the kmttg temp directory instead of overwriting old SRT files on the NAS. So I had to manually move them anyway. Moving MP4 and TXT files is no big deal.

Do you have any ability to rebuild AP? I looked at the source, it doesn't look that difficult to build (But I also didn't try it). And I looked at the code in question, and you are correct that it goes straight from fwrite() to fclose() to rename(), then when it fails it emits the exact message. It wouldn't be too hard to add short retry loop to that stage...

 

[Scott9mm]

Do you have any ability to rebuild AP? I looked at the source, it doesn't look that difficult to build (But I also didn't try it). And I looked at the code in question, and you are correct that it goes straight from fwrite() to fclose() to rename(), then when it fails it emits the exact message. It wouldn't be too hard to add short retry loop to that stage...

Sorry I'm just now seeing your reply. I have a workable "fix" for now but this is a good idea for the future. I too have never compiled AP (or anything else) in last 30+ years so there would be a lot of learning before I succeed. But this is a great idea for the "to do someday" list. Thanks for checking it out for me.