diff --git a/.assets/radarr-quality-profile.png b/.assets/radarr-quality-profile.png new file mode 100644 index 0000000..753448e Binary files /dev/null and b/.assets/radarr-quality-profile.png differ diff --git a/.assets/sonarr-language-profile.png b/.assets/sonarr-language-profile.png new file mode 100644 index 0000000..27a38e8 Binary files /dev/null and b/.assets/sonarr-language-profile.png differ diff --git a/.assets/striptracks-v3-custom-script.png b/.assets/striptracks-v3-custom-script.png index d9e8564..4190e73 100644 Binary files a/.assets/striptracks-v3-custom-script.png and b/.assets/striptracks-v3-custom-script.png differ diff --git a/README.md b/README.md index c31bded..3e962e0 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,9 @@ # About -A [Docker Mod](https://github.com/linuxserver/docker-mods) for the LinuxServer.io Radarr/Sonarr Docker container that adds a script to automatically strip out unwanted audio and subtitle streams, keeping only the desired languages. +A [Docker Mod](https://github.com/linuxserver/docker-mods) for the LinuxServer.io Radarr/Sonarr v3 Docker container that adds a script to automatically strip out unwanted audio and subtitle streams, keeping only the desired languages. -**One unified script works in both Radarr and Sonarr. Use this mod in either container!** +**Beginning with version 2.0 of this mod, it only supports v3 or later of Radarr/Sonarr. For legacy Radarr/Sonarr v2 please use mod release 1.3 or earlier** + +**This unified script works in both Radarr and Sonarr. Use this mod in either container!** >**NOTE:** This mod supports Linux OSes only. Container info: @@ -10,8 +12,6 @@ Container info: Production Container info: ![Docker Image Size](https://img.shields.io/docker/image-size/linuxserver/mods/radarr-striptracks "Container Size") # Installation ->**NOTE:** See the [Sonarr/Radarr v2 Builds](./README.md#sonarrradarr-v2-builds) section below for important differences to these instructions for v2 builds. - 1. Pull your selected container ([linuxserver/radarr](https://hub.docker.com/r/linuxserver/radarr "LinuxServer.io's Radarr container") or [linuxserver/sonarr](https://hub.docker.com/r/linuxserver/sonarr "LinuxServer.io's Sonarr container")) from Docker Hub: `docker pull linuxserver/radarr:latest` OR `docker pull linuxserver/sonarr:latest` @@ -20,7 +20,7 @@ Production Container info: ![Docker Image Size](https://img.shields.io/docker/im **[linuxserver/radarr](https://hub.docker.com/r/linuxserver/radarr "Radarr Docker container")** **[linuxserver/sonarr](https://hub.docker.com/r/linuxserver/sonarr "Sonarr Docker container")** 1. Add the **DOCKER_MODS** environment variable to the `docker run` command, as follows: - + - Dev/test release: `-e DOCKER_MODS=thecaptain989/radarr-striptracks:latest` - Stable release: `-e DOCKER_MODS=linuxserver/mods:radarr-striptracks` *Example Docker CLI Configuration* @@ -44,17 +44,14 @@ Production Container info: ![Docker Image Size](https://img.shields.io/docker/im 2. Start the container. -3. After the above configuration is complete, to use mkvmerge, configure a custom script from Radarr's or Sonarr's *Settings* > *Connect* screen and type the following in the **Path** field: - `/usr/local/bin/striptracks-eng.sh` +3. Configure a custom script from Radarr's or Sonarr's *Settings* > *Connect* screen and type the following in the **Path** field: + `/usr/local/bin/striptracks.sh` *Example* ![striptracks v3](.assets/striptracks-v3-custom-script.png "Radarr/Sonarr custom script settings") - This is a wrapper script that calls striptracks.sh with the following arguments, which keep English audio and subtitles only! - `:eng:und :eng` - - *For any other combinations of audio and subtitles you **must** either use one of the [included wrapper scripts](./README.md#included-wrapper-scripts) or create a custom script with the codes for the languages you want to keep. See the [Syntax](./README.md#syntax) section below.* - *Do not put `striptracks.sh` in the **Path** field!* + The script will detect the language defined in the video profile for the movie or TV show and only keep the audio and subtitles selected. + Alternatively, a wrapper script may be used to more granularly define which tracks to keep. See [wrapper scripts](./README.md#wrapper-scripts) for more details. ## Usage The source video can be any mkvtoolnix supported video format. The output is an MKV file with the same name. @@ -65,14 +62,27 @@ If you've configured the Radarr/Sonarr **Recycle Bin** path correctly, the origi ![danger] **NOTE:** If you have *not* configured the Recycle Bin, the original video file will be deleted/overwritten and permanently lost. ### Syntax ->**NOTE:** The **Arguments** field for Custom Scripts was removed in Radarr and Sonarr v3 due to security concerns. To support options with these versions and later, -a wrapper script must be manually created that will call *striptracks.sh* with the required arguments. +Beginning with version 2.0 of this mod, the script may be called with no arguments. In this configuration it will detect the language(s) defined in the profile (Quality Profile for Radarr, Language Profile for Sonarr) configured on the particular movie or TV show. + +#### Automatic Language Detection +Both audio and subtitles that match the selected language(s) are kept. + +*Radarr Quality Profile Example* +![radarr profile](.assets/radarr-quality-profile.png "Radarr Quality Profile settings") + +*Sonarr Language Profile Example* +![sonarr profile](.assets/sonarr-language-profile.png "Sonarr Language Profile settings") + +>**Note:** The intent of the Radarr language selection 'Original' is not well documented. For the purposes of this script, it has the same function as 'Any' and will preserve all languages in the video file. + +#### Manual Override +The script still supports command line arguments that can override what is detected. More granular control can be therefore be exerted or extended using tagging and defining multiple Connect scripts (this is outside the scope of this documentation). The script accepts two command line arguments and one option: -`[-d] ` +`[-d] [ []]` -The `` and `` arguments are colon (:) prepended language codes in [ISO639-2](https://en.wikipedia.org/wiki/List_of_ISO_639-2_codes "List of ISO 639-2 codes") format. +The `` and `` are optional arguments that are colon (:) prepended language codes in [ISO639-2](https://en.wikipedia.org/wiki/List_of_ISO_639-2_codes "List of ISO 639-2 codes") format. For example: * :eng @@ -82,9 +92,7 @@ For example: ...etc. Multiple codes may be concatenated, such as `:eng:spa` for both English and Spanish. -**These codes are mandatory.** There are no defaults. -The wrapper script noted above uses `:eng:und :eng`, which will keep English and Undetermined audio and English subtitles. >**NOTE:** The script is smart enough to not remove the last audio track. This way you don't have to specify every possible language if you are importing a foreign film, for example. @@ -98,11 +106,15 @@ The `-d` option enables debug logging. # Spanish subtitles ``` -### Included Wrapper Scripts +### Wrapper Scripts +To supply arguments to the script, one of the included wrapper scripts may be used or a custom wrapper script must be created. + +#### Included Wrapper Scripts For your convenience, several wrapper scripts are included in the `/usr/local/bin/` directory. -You may use any of these scripts in place of the `striptracks-eng.sh` mentioned in the [Installation](./README.md#installation) section above. +You may use any of these scripts in place of `striptracks.sh` mentioned in the [Installation](./README.md#installation) section above. ``` +striptracks-debug.sh # Use detected languages, but enable debug logging striptracks-dut.sh # Keep Dutch audio and subtitles striptracks-eng.sh # Keep English and Undetermined audio and English subtitles striptracks-eng-debug.sh # Keep English and Undetermined audio and English subtitles, and enable debug logging @@ -114,7 +126,7 @@ striptracks-ger.sh # Keep German audio and subtitles striptracks-spa.sh # Keep Spanish audio and subtitles ``` -### Example Wrapper Script +#### Example Wrapper Script To configure the last entry from the [Examples](./README.md#examples) section above, create and save a file called `striptracks-custom.sh` to `/config` containing the following text: ```shell #!/bin/bash @@ -126,7 +138,7 @@ Make it executable: chmod +x /config/striptracks-custom.sh ``` -Then put `/config/striptracks-custom.sh` in the **Path** field in place of `/usr/local/bin/striptracks-eng.sh` mentioned in the [Installation](./README.md#installation) section above. +Then put `/config/striptracks-custom.sh` in the **Path** field in place of `/usr/local/bin/striptracks.sh` mentioned in the [Installation](./README.md#installation) section above. >**Note:** If you followed the Linuxserver.io recommendations when configuring your container, the `/config` directory will be mapped to an external storage location. It is therefore recommended to place custom scripts in the `/config` directory so they will survive container updates, but they may be placed anywhere that is accessible by Radarr or Sonarr. @@ -147,31 +159,6 @@ Log rotation is performed with 5 log files of 512KB each being kept. ___ -## Sonarr/Radarr v2 Builds ->![warning] **Warning: Legacy Releases** ->The Sonarr/Radarr v2 Builds of Radarr and Sonarr are no longer being developed and are considered legacy. However, this mod works with all versions of the container. - -Important differences for Sonarr/Radarr v2 Builds -### Legacy Installation -Substitute the following step for step #3 noted in the [Installation](./README.md#installation) section above. -3. After all of the above configuration is complete, to use mkvmerge: - 1. Configure a custom script from the Radarr/Sonnar *Settings* > *Connect* screen and type the following in the **Path** field: - `/usr/local/bin/striptracks.sh` - - 2. Add the codes for the audio and subtitle languages you want to keep as **Arguments** (details in the [Syntax](./README.md#syntax) section above): - Suggested Example - `:eng:und :eng` - - *Example* - ![striptracks v2](.assets/striptracks-v2-custom-script.png "Radarr/Sonarr custom script settings") - -### Legacy Triggers -The only events/notification triggers that have been tested are **On Download** and **On Upgrade** - -### Legacy Logs -The log can be inspected or downloaded from Radarr/Sonarr under *System* > *Log Files* - -___ # Credits This would not be possible without the following: diff --git a/SECURITY.md b/SECURITY.md index b8a7d50..aa90eea 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -6,8 +6,7 @@ Only the latest major and minor version are supported. | Version | Supported | | ------- | ------------------ | -| 1.3.x | :heavy_check_mark: | -| < 1.3 | :x: | +| 2.x | :heavy_check_mark: | ## Reporting a Vulnerability diff --git a/root/usr/local/bin/striptracks-debug.sh b/root/usr/local/bin/striptracks-debug.sh new file mode 100644 index 0000000..747569a --- /dev/null +++ b/root/usr/local/bin/striptracks-debug.sh @@ -0,0 +1,3 @@ +#!/bin/bash + +. /usr/local/bin/striptracks.sh -d diff --git a/root/usr/local/bin/striptracks-dut.sh b/root/usr/local/bin/striptracks-dut.sh index e2c19af..a55829d 100644 --- a/root/usr/local/bin/striptracks-dut.sh +++ b/root/usr/local/bin/striptracks-dut.sh @@ -1,3 +1,3 @@ #!/bin/bash -. /usr/local/bin/striptracks.sh :dut :dut +. /usr/local/bin/striptracks.sh :nld:dut :nld:dut diff --git a/root/usr/local/bin/striptracks.sh b/root/usr/local/bin/striptracks.sh index 373d003..3ca2828 100644 --- a/root/usr/local/bin/striptracks.sh +++ b/root/usr/local/bin/striptracks.sh @@ -10,6 +10,8 @@ # # Put a colon `:` in front of every language code. Expects ISO639-2 codes +# NOTE: This has been updated to work with v3 API only. Far too many complications trying to keep multiple version compatible. + # Dependencies: # mkvmerge # awk @@ -28,6 +30,7 @@ # 5 - specified video file not found # 6 - unable to rename video to temp video # 7 - unknown environment +# 8 - unsupported Radarr/Sonarr version (v2) # 10 - remuxing completed, but no output file found # 20 - general error @@ -39,38 +42,40 @@ export striptracks_log=/config/logs/striptracks.txt export striptracks_maxlogsize=512000 export striptracks_maxlog=4 export striptracks_debug=0 +export striptracks_langcodes= export striptracks_type=$(printenv | sed -n 's/_eventtype *=.*$//p') if [[ "${striptracks_type,,}" = "radarr" ]]; then export striptracks_video="$radarr_moviefile_path" - export striptracks_api_endpoint="movie" + export striptracks_video_api="movie" + export striptracks_video_id="${radarr_movie_id}" + export striptracks_videofile_api="moviefile" + export striptracks_videofile_id="${radarr_moviefile_id}" + export striptracks_rescan_id="${radarr_movie_id}" export striptracks_json_quality_root=".movieFile" export striptracks_video_type="movie" + export striptracks_profile_type="quality" export striptracks_title="$radarr_movie_title ($radarr_movie_year)" elif [[ "${striptracks_type,,}" = "sonarr" ]]; then export striptracks_video="$sonarr_episodefile_path" - export striptracks_api_endpoint="episodefile" - export striptracks_json_quality_root="" + export striptracks_video_api="episode" + export striptracks_video_id="${sonarr_episodefile_episodeids}" + export striptracks_videofile_api="episodefile" + export striptracks_videofile_id="${sonarr_episodefile_id}" + export striptracks_rescan_id="${sonarr_series_id}" + export striptracks_json_quality_root=".episodeFile" export striptracks_video_type="series" + export striptracks_profile_type="language" export striptracks_title="$sonarr_series_title $(numfmt --format "%02f" ${sonarr_episodefile_seasonnumber:-0})x$(numfmt --format "%02f" ${sonarr_episodefile_episodenumbers:-0}) - $sonarr_episodefile_episodetitles" else echo "Unknown environment: ${striptracks_type}" exit 7 fi -export striptracks_api="Rescan${striptracks_video_type^}" +export striptracks_rescan_api="Rescan${striptracks_video_type^}" export striptracks_json_key="${striptracks_video_type}Id" -export striptracks_api_endpoint_idname="${striptracks_type,,}_${striptracks_api_endpoint}_id" -export striptracks_api_endpoint_id="${!striptracks_api_endpoint_idname}" -export striptracks_video_idname="${striptracks_type,,}_${striptracks_video_type}_id" -export striptracks_video_id="${!striptracks_video_idname}" export striptracks_eventtype="${striptracks_type,,}_eventtype" export striptracks_tempvideo="${striptracks_video}.tmp" export striptracks_newvideo="${striptracks_video%.*}.mkv" -export striptracks_db="/config/${striptracks_type,,}.db" -if [ ! -f "$striptracks_db" ]; then - striptracks_db=/config/nzbdrone.db -fi -export striptracks_recyclebin=$(sqlite3 $striptracks_db 'SELECT Value FROM Config WHERE Key="recyclebin"') -RET=$?; [ "$RET" != 0 ] && >&2 echo "WARNING[$RET]: Unable to read recyclebin information from database \"$striptracks_db\"" +striptracks_isocodemap='{"languages":[{"language":{"id":-1,"name":"Any","iso639-2":["ara","bul","zho","chi","ces","cze","dan","nld","dut","eng","fin","fra","fre","deu","ger","ell","gre","heb","hin","hun","isl","ice","ita","jpn","kor","lit","nor","pol","por","ron","rom","rus","spa","swe","tha","tur","vie","und"]}},{"language":{"id":-2,"name":"Original","iso639-2":["ara","bul","zho","chi","ces","cze","dan","nld","dut","eng","fin","fra","fre","deu","ger","ell","gre","heb","hin","hun","isl","ice","ita","jpn","kor","lit","nor","pol","por","ron","rom","rus","spa","swe","tha","tur","vie","und"]}},{"language":{"id":27,"name":"Hindi","iso639-2":["hin"]}},{"language":{"id":26,"name":"Arabic","iso639-2":["ara"]}},{"language":{"id":0,"name":"Unknown","iso639-2":["und"]}},{"language":{"id":13,"name":"Vietnamese","iso639-2":["vie"]}},{"language":{"id":17,"name":"Turkish","iso639-2":["tur"]}},{"language":{"id":14,"name":"Swedish","iso639-2":["swe"]}},{"language":{"id":3,"name":"Spanish","iso639-2":["spa"]}},{"language":{"id":11,"name":"Russian","iso639-2":["rus"]}},{"language":{"id":18,"name":"Portuguese","iso639-2":["por"]}},{"language":{"id":12,"name":"Polish","iso639-2":["pol"]}},{"language":{"id":15,"name":"Norwegian","iso639-2":["nor"]}},{"language":{"id":24,"name":"Lithuanian","iso639-2":["lit"]}},{"language":{"id":21,"name":"Korean","iso639-2":["kor"]}},{"language":{"id":8,"name":"Japanese","iso639-2":["jpn"]}},{"language":{"id":5,"name":"Italian","iso639-2":["ita"]}},{"language":{"id":9,"name":"Icelandic","iso639-2":["isl","ice"]}},{"language":{"id":22,"name":"Hungarian","iso639-2":["hun"]}},{"language":{"id":23,"name":"Hebrew","iso639-2":["heb"]}},{"language":{"id":20,"name":"Greek","iso639-2":["ell","gre"]}},{"language":{"id":4,"name":"German","iso639-2":["deu","ger"]}},{"language":{"id":2,"name":"French","iso639-2":["fra","fre"]}},{"language":{"id":19,"name":"Flemish","iso639-2":["nld","dut"]}},{"language":{"id":16,"name":"Finnish","iso639-2":["fin"]}},{"language":{"id":1,"name":"English","iso639-2":["eng"]}},{"language":{"id":7,"name":"Dutch","iso639-2":["nld","dut"]}},{"language":{"id":6,"name":"Danish","iso639-2":["dan"]}},{"language":{"id":25,"name":"Czech","iso639-2":["ces","cze"]}},{"language":{"id":10,"name":"Chinese","iso639-2":["zho","chi"]}}]}' ### Functions function usage { @@ -81,7 +86,7 @@ Video remuxing script designed for use with Radarr and Sonarr Source: https://github.com/TheCaptain989/radarr-striptracks Usage: - $0 [-d] + $0 [-d] [ []] Options and Arguments: -d enable debug logging @@ -106,8 +111,8 @@ function log {( while read do echo $(date +"%Y-%-m-%-d %H:%M:%S.%1N")\|"[$striptracks_pid]$REPLY" >>"$striptracks_log" - local FILESIZE=$(stat -c %s "$striptracks_log") - if [ $FILESIZE -gt $striptracks_maxlogsize ] + local striptracks_filesize=$(stat -c %s "$striptracks_log") + if [ $striptracks_filesize -gt $striptracks_maxlogsize ] then for i in $(seq $((striptracks_maxlog-1)) -1 0); do [ -f "${striptracks_log::-4}.$i.txt" ] && mv "${striptracks_log::-4}."{$i,$((i+1))}".txt" @@ -120,68 +125,98 @@ function log {( # Inspired by https://stackoverflow.com/questions/893585/how-to-parse-xml-in-bash function read_xml { local IFS=\> - read -d \< ENTITY CONTENT + read -d \< striptracks_xml_entity striptracks_xml_content } # Get video information function get_video_info { - [ $striptracks_debug -eq 1 ] && echo "Debug|Getting video information for $striptracks_api_endpoint '$striptracks_api_endpoint_id'. Calling ${striptracks_type^} API using GET and URL 'http://$striptracks_bindaddress:$striptracks_port$striptracks_urlbase/api/$striptracks_api_endpoint/$striptracks_api_endpoint_id?apikey=(removed)'" | log - RESULT=$(curl -s -H "Content-Type: application/json" \ - -X GET http://$striptracks_bindaddress:$striptracks_port$striptracks_urlbase/api/$striptracks_api_endpoint/$striptracks_api_endpoint_id?apikey=$striptracks_apikey) - [ $striptracks_debug -eq 1 ] && echo "API returned: $RESULT" | awk '{print "Debug|"$0}' | log - if [ "$(echo $RESULT | jq -crM .path)" != "null" ]; then - local RET=0 + [ $striptracks_debug -eq 1 ] && echo "Debug|Getting video information for $striptracks_video_api '$striptracks_video_id'. Calling ${striptracks_type^} API using GET and URL '$striptracks_api_url/v3/$striptracks_video_api/$striptracks_video_id'" | log + striptracks_result=$(curl -s -H "X-Api-Key: $striptracks_apikey" \ + -X GET "$striptracks_api_url/v3/$striptracks_video_api/$striptracks_video_id") + [ $striptracks_debug -eq 1 ] && echo "API returned: $striptracks_result" | awk '{print "Debug|"$0}' | log + if [ "$(echo $striptracks_result | jq -crM .hasFile)" = "true" ]; then + local striptracks_return=0 else - local RET=1 + local striptracks_return=1 fi - return $RET + return $striptracks_return } -# Initiate API Rescan request -function rescan { - MSG="Info|Calling ${striptracks_type^} API to rescan ${striptracks_video_type}, try #$i" - echo "$MSG" | log - [ $striptracks_debug -eq 1 ] && echo "Debug|Forcing rescan of $striptracks_json_key '$striptracks_video_id', try #$i. Calling ${striptracks_type^} API '$striptracks_api' using POST and URL 'http://$striptracks_bindaddress:$striptracks_port$striptracks_urlbase/api/command?apikey=(removed)'" | log - RESULT=$(curl -s -d "{name: '$striptracks_api', $striptracks_json_key: $striptracks_video_id}" -H "Content-Type: application/json" \ - -X POST http://$striptracks_bindaddress:$striptracks_port$striptracks_urlbase/api/command?apikey=$striptracks_apikey) - [ $striptracks_debug -eq 1 ] && echo "API returned: $RESULT" | awk '{print "Debug|"$0}' | log - JOBID="$(echo $RESULT | jq -crM .id)" - if [ "$JOBID" != "null" ]; then - local RET=0 +# Get video file information +function get_videofile_info { + [ $striptracks_debug -eq 1 ] && echo "Debug|Getting video file information for $striptracks_videofile_api id '$striptracks_videofile_id'. Calling ${striptracks_type^} API using GET and URL '$striptracks_api_url/v3/$striptracks_videofile_api/$striptracks_videofile_id'" | log + striptracks_result=$(curl -s -H "X-Api-Key: $striptracks_apikey" \ + -X GET "$striptracks_api_url/v3/$striptracks_videofile_api/$striptracks_videofile_id") + [ $striptracks_debug -eq 1 ] && echo "API returned: $striptracks_result" | awk '{print "Debug|"$0}' | log + if [ "$(echo $striptracks_result | jq -crM .path)" != "null" ]; then + local striptracks_return=0 else - local RET=1 + local striptracks_return=1 fi - return $RET + return $striptracks_return +} +# Initiate Rescan request +function rescan { + striptracks_message="Info|Calling ${striptracks_type^} API to rescan ${striptracks_video_type}, try #$i" + echo "$striptracks_message" | log + [ $striptracks_debug -eq 1 ] && echo "Debug|Forcing rescan of $striptracks_json_key '$striptracks_rescan_id', try #$i. Calling ${striptracks_type^} API '$striptracks_rescan_api' using POST and URL '$striptracks_api_url/v3/command'" | log + striptracks_result=$(curl -s -H "X-Api-Key: $striptracks_apikey" -H "Content-Type: application/json" \ + -d "{\"name\": \"$striptracks_rescan_api\", \"$striptracks_json_key\": $striptracks_rescan_id}" \ + -X POST "$striptracks_api_url/v3/command") + [ $striptracks_debug -eq 1 ] && echo "API returned: $striptracks_result" | awk '{print "Debug|"$0}' | log + striptracks_jobid="$(echo $striptracks_result | jq -crM .id)" + if [ "$striptracks_jobid" != "null" ]; then + local striptracks_return=0 + else + local striptracks_return=1 + fi + return $striptracks_return } # Check result of rescan job function check_rescan { local i=0 for ((i=1; i <= 15; i++)); do - [ $striptracks_debug -eq 1 ] && echo "Debug|Checking job $JOBID completion, try #$i. Calling ${striptracks_type^} API using GET and URL 'http://$striptracks_bindaddress:$striptracks_port$striptracks_urlbase/api/command/$JOBID?apikey=(removed)'" | log - RESULT=$(curl -s -H "Content-Type: application/json" \ - -X GET http://$striptracks_bindaddress:$striptracks_port$striptracks_urlbase/api/command/$JOBID?apikey=$striptracks_apikey) - [ $striptracks_debug -eq 1 ] && echo "API returned: $RESULT" | awk '{print "Debug|"$0}' | log - if [ "$(echo $RESULT | jq -crM .status)" = "completed" ]; then - local RET=0 + [ $striptracks_debug -eq 1 ] && echo "Debug|Checking job $striptracks_jobid completion, try #$i. Calling ${striptracks_type^} API using GET and URL '$striptracks_api_url/v3/command/$striptracks_jobid'" | log + striptracks_result=$(curl -s -H "X-Api-Key: $striptracks_apikey" \ + -X GET "$striptracks_api_url/v3/command/$striptracks_jobid") + [ $striptracks_debug -eq 1 ] && echo "API returned: $striptracks_result" | awk '{print "Debug|"$0}' | log + if [ "$(echo $striptracks_result | jq -crM .status)" = "completed" ]; then + local striptracks_return=0 break else - if [ "$(echo $RESULT | jq -crM .status)" = "failed" ]; then - local RET=2 + if [ "$(echo $striptracks_result | jq -crM .status)" = "failed" ]; then + local striptracks_return=2 break else - local RET=1 + # It may have timed out, so let's wait a second + local striptracks_return=1 + [ $striptracks_debug -eq 1 ] && echo "Debug|Job not done. Waiting 1 second." | log sleep 1 fi fi done - return $RET + return $striptracks_return +} +# Get language/quality profiles +function get_profiles { + [ $striptracks_debug -eq 1 ] && echo "Debug|Getting list of $striptracks_profile_type profiles. Calling ${striptracks_type^} API using GET and URL '$striptracks_api_url/v3/${striptracks_profile_type}Profile'" | log + striptracks_result=$(curl -s -H "X-Api-Key: $striptracks_apikey" \ + -X GET "$striptracks_api_url/v3/${striptracks_profile_type}Profile") + # This returns A LOT of data, and it is normally not needed + # [ $striptracks_debug -eq 1 ] && echo "API returned: $striptracks_result" | awk '{print "Debug|"$0}' | log + if [ "$(echo $striptracks_result | jq -crM '.message?')" != "NotFound" ]; then + local striptracks_return=0 + else + local striptracks_return=1 + fi + return $striptracks_return } # Process options while getopts ":d" opt; do case ${opt} in d ) # For debug purposes only - MSG="Debug|Enabling debug logging." - echo "$MSG" | log - >&2 echo "$MSG" + striptracks_message="Debug|Enabling debug logging." + echo "$striptracks_message" | log + >&2 echo "$striptracks_message" striptracks_debug=1 printenv | sort | sed 's/^/Debug|/' | log ;; @@ -189,31 +224,57 @@ while getopts ":d" opt; do done shift $((OPTIND -1)) -# Check for required command line options -if [ -z "$1" ]; then - MSG="Error|No audio languages specified!" - echo "$MSG" | log - >&2 echo "$MSG" - usage - exit 2 -fi - -if [ -z "$2" ]; then - MSG="Error|No subtitles languages specified!" - echo "$MSG" | log - >&2 echo "$MSG" - usage - exit 3 -fi - # Check for required binaries if [ ! -f "/usr/bin/mkvmerge" ]; then - MSG="Error|/usr/bin/mkvmerge is required by this script" - echo "$MSG" | log - >&2 echo "$MSG" + striptracks_message="Error|/usr/bin/mkvmerge is required by this script" + echo "$striptracks_message" | log + >&2 echo "$striptracks_message" exit 4 fi +# Check for config file +if [ -f "$striptracks_arr_config" ]; then + # Read *arr config.xml + [ $striptracks_debug -eq 1 ] && echo "Debug|Reading from ${striptracks_type^} config file '$striptracks_arr_config'" | log + while read_xml; do + [[ $striptracks_xml_entity = "Port" ]] && striptracks_port=$striptracks_xml_content + [[ $striptracks_xml_entity = "UrlBase" ]] && striptracks_urlbase=$striptracks_xml_content + [[ $striptracks_xml_entity = "BindAddress" ]] && striptracks_bindaddress=$striptracks_xml_content + [[ $striptracks_xml_entity = "ApiKey" ]] && striptracks_apikey=$striptracks_xml_content + done < $striptracks_arr_config + + [[ $striptracks_bindaddress = "*" ]] && striptracks_bindaddress=localhost + + # Build URL to Radarr/Sonarr API + striptracks_api_url="http://$striptracks_bindaddress:$striptracks_port$striptracks_urlbase/api" + + # Check Radarr/Sonarr version + [ $striptracks_debug -eq 1 ] && echo "Debug|Getting ${striptracks_type^} version. Calling ${striptracks_type^} API using GET and URL '$striptracks_api_url/system/status'" | log + striptracks_arr_version=$(curl -s -H "X-Api-Key: $striptracks_apikey" \ + -X GET "$striptracks_api_url/system/status" | jq -crM .version) + [ $striptracks_debug -eq 1 ] && echo "Debug|Detected ${striptracks_type^} version $striptracks_arr_version" | log + + # Requires API v3 + if [ "${striptracks_arr_version/.*/}" = "2" ]; then + # Radarr/Sonarr version 2 + striptracks_message="Error|This script does not support ${striptracks_type^} version ${striptracks_arr_version}. Please upgrade." + echo "$striptracks_message" | log + >&2 echo "$striptracks_message" + exit 8 + fi + + # Get RecycleBin + [ $striptracks_debug -eq 1 ] && echo "Debug|Getting ${striptracks_type^} RecycleBin. Calling ${striptracks_type^} API using GET and URL '$striptracks_api_url/v3/config/mediamanagement'" | log + striptracks_recyclebin=$(curl -s -H "X-Api-Key: $striptracks_apikey" \ + -X GET "$striptracks_api_url/v3/config/mediamanagement" | jq -crM .recycleBin) + [ $striptracks_debug -eq 1 ] && echo "Debug|Detected ${striptracks_type^} RecycleBin '$striptracks_recyclebin'" | log +else + # No config file means we can't call the API. Best effort at this point. + striptracks_message="Warn|Unable to locate ${striptracks_type^} config file: '$striptracks_arr_config'" + echo "$striptracks_message" | log + >&2 echo "$striptracks_message" +fi + # Handle Test event if [[ "${!striptracks_eventtype}" = "Test" ]]; then echo "Info|${striptracks_type^} event: ${!striptracks_eventtype}" | log @@ -223,48 +284,121 @@ fi # Check if called from within Radarr/Sonarr if [ -z "$striptracks_video" ]; then - MSG="Error|No video file specified! Not called from Radarr/Sonarr?" - echo "$MSG" | log - >&2 echo "$MSG" + striptracks_message="Error|No video file specified! Not called from Radarr/Sonarr?" + echo "$striptracks_message" | log + >&2 echo "$striptracks_message" usage exit 1 fi # Check if source video exists if [ ! -f "$striptracks_video" ]; then - MSG="Error|Input file not found: \"$striptracks_video\"" - echo "$MSG" | log - >&2 echo "$MSG" + striptracks_message="Error|Input file not found: \"$striptracks_video\"" + echo "$striptracks_message" | log + >&2 echo "$striptracks_message" exit 5 fi +#### Detect languages configured in Radarr/Sonarr +# Check for URL +if [ -n "$striptracks_api_url" ]; then + # Get quality/language profile info + if get_profiles; then + striptracks_profiles="$striptracks_result" + # Get video profile + if get_video_info; then + # Per environment logic + if [[ "${striptracks_type,,}" = "radarr" ]]; then + striptracks_profileid="$(echo $striptracks_result | jq -crM .qualityProfileId)" + striptracks_languages=$(echo $striptracks_profiles | jq -crM ".[] | select(.id == $striptracks_profileid) | .language.id") + elif [[ "${striptracks_type,,}" = "sonarr" ]]; then + striptracks_profileid="$(echo $striptracks_result | jq -crM .series.languageProfileId)" + striptracks_languages=$(echo $striptracks_profiles | jq -crM ".[] | select(.id == $striptracks_profileid) | .languages | .[] | select(.allowed).language.id") + else + # Should never fire due to previous check, but just in case + striptracks_message "Error|Unknown environment: ${striptracks_type}" + echo "$striptracks_message" | log + >&2 echo "$striptracks_message" + exit 7 + fi + striptracks_profilename=$(echo $striptracks_profiles | jq -crM ".[] | select(.id == $striptracks_profileid).name") + [ $striptracks_debug -eq 1 ] && echo "Debug|Detected $striptracks_profile_type profile '$striptracks_profilename' id '$striptracks_profileid'" | log + [ $striptracks_debug -eq 1 ] && echo "Debug|Detected language ids of '$(echo ${striptracks_languages})'" | log + # Lookup ISO codes + for i in $striptracks_languages; do + striptracks_langcodes+=$(echo $striptracks_isocodemap | jq -jcrM ".languages | .[] | select(.language.id == $i) | .language | \":\(.\"iso639-2\"[])\"") + done + [ $striptracks_debug -eq 1 ] && echo "Debug|Mapped language codes '$(echo ${striptracks_languages})' to ISO639-2 code string '$striptracks_langcodes'" | log + else + # 'hasFile' is False in returned JSON. + striptracks_message="Warn|The '$striptracks_video_api' API with id $striptracks_video_id returned a false hasFile." + echo "$striptracks_message" | log + >&2 echo "$striptracks_message" + fi + else + # Get Profiles API failed + striptracks_message="Warn|Unable to retrieve $striptracks_profile_type profiles from ${striptracks_type^} API" + echo "$striptracks_message" | log + >&2 echo "$striptracks_message" + fi +else + # No URL means we can't call the API + striptracks_message="Warn|Unable to determine ${striptracks_type^} API URL." + echo "$striptracks_message" | log + >&2 echo "$striptracks_message" +fi + +# Check for command line options; will override the detected languages +if [ -z "$1" -a -z "$striptracks_langcodes" ]; then + striptracks_message="Error|No audio languages specified!" + echo "$striptracks_message" | log + >&2 echo "$striptracks_message" + usage + exit 2 +elif [ -z "$1" ]; then + striptracks_audiokeep="$striptracks_langcodes" +else + striptracks_audiokeep="$1" +fi +if [ -z "$2" -a -z "$striptracks_langcodes" ]; then + striptracks_message="Error|No subtitles languages specified!" + echo "$striptracks_message" | log + >&2 echo "$striptracks_message" + usage + exit 3 +elif [ -z "$2" ]; then + striptracks_subskeep="$striptracks_langcodes" +else + striptracks_subskeep="$2" +fi + #### BEGIN MAIN -FILESIZE=$(numfmt --to iec --format "%.3f" $(stat -c %s "$striptracks_video")) -MSG="Info|${striptracks_type^} event: ${!striptracks_eventtype}, Video: $striptracks_video, Size: $FILESIZE, AudioKeep: $1, SubsKeep: $2" -echo "$MSG" | log +striptracks_filesize=$(numfmt --to iec --format "%.3f" $(stat -c %s "$striptracks_video")) +striptracks_message="Info|${striptracks_type^} event: ${!striptracks_eventtype}, Video: $striptracks_video, Size: $striptracks_filesize, AudioKeep: $striptracks_audiokeep, SubsKeep: $striptracks_subskeep" +echo "$striptracks_message" | log # Rename the original video file to a temporary name [ $striptracks_debug -eq 1 ] && echo "Debug|Renaming: \"$striptracks_video\" to \"$striptracks_tempvideo\"" | log mv -f "$striptracks_video" "$striptracks_tempvideo" | log -RET=$?; [ "$RET" != 0 ] && { - MSG="ERROR[$RET]: Unable to rename video: \"$striptracks_video\" to temp video: \"$striptracks_tempvideo\". Halting." - echo "$MSG" | log - >&2 echo "$MSG" +striptracks_return=$?; [ "$striptracks_return" != 0 ] && { + striptracks_message="ERROR[$striptracks_return]: Unable to rename video: \"$striptracks_video\" to temp video: \"$striptracks_tempvideo\". Halting." + echo "$striptracks_message" | log + >&2 echo "$striptracks_message" exit 6 } # Read in the output of mkvmerge info extraction [ $striptracks_debug -eq 1 ] && echo "Debug|Executing: /usr/bin/mkvmerge -J \"$striptracks_tempvideo\"" | log -JSON=$(/usr/bin/mkvmerge -J "$striptracks_tempvideo") -RET=$?; [ "$RET" != 0 ] && { - MSG="ERROR[$RET]: Error executing mkvmerge." - echo "$MSG" | log - >&2 echo "$MSG" +striptracks_json=$(/usr/bin/mkvmerge -J "$striptracks_tempvideo") +striptracks_return=$?; [ "$striptracks_return" != 0 ] && { + striptracks_message="ERROR[$striptracks_return]: Error executing mkvmerge." + echo "$striptracks_message" | log + >&2 echo "$striptracks_message" } # This and the modified AWK script are a hack, and I know it. JQ is crazy hard to learn, BTW. # Mimic the mkvmerge --identify-verbose option that has been deprecated -JSON_PROCESSED=$(echo $JSON | jq -jcrM ' +striptracks_json_processed=$(echo $striptracks_json | jq -jcrM ' ( if (.chapters | .[] | .num_entries) then "Chapters: \(.chapters | .[] | .num_entries) entries\n" else @@ -274,19 +408,19 @@ JSON_PROCESSED=$(echo $JSON | jq -jcrM ' ( .tracks | .[] | ( "Track ID \(.id): \(.type) (\(.codec)) [", - ( [.properties | to_entries |.[] | "\(.key):\(.value | tostring | gsub(" "; "\\s"))"] | join(" ")), + ( [.properties | to_entries | .[] | "\(.key):\(.value | tostring | gsub(" "; "\\s"))"] | join(" ")), "]\n" ) ) ') -[ $striptracks_debug -eq 1 ] && echo "$JSON_PROCESSED" | awk '{print "Debug|"$0}' | log +[ $striptracks_debug -eq 1 ] && echo "$striptracks_json_processed" | awk '{print "Debug|"$0}' | log -echo "$JSON_PROCESSED" | awk -v Debug=$striptracks_debug \ +echo "$striptracks_json_processed" | awk -v Debug=$striptracks_debug \ -v OrgVideo="$striptracks_video" \ -v TempVideo="$striptracks_tempvideo" \ -v MKVVideo="$striptracks_newvideo" \ -v Title="$striptracks_title" \ --v AudioKeep="$1" \ --v SubsKeep="$2" ' +-v AudioKeep="$striptracks_audiokeep" \ +-v SubsKeep="$striptracks_subskeep" ' BEGIN { MKVMerge="/usr/bin/mkvmerge" FS="[\t\n: ]" @@ -382,114 +516,119 @@ END { if [ -s "$striptracks_newvideo" ]; then # Use Recycle Bin if configured if [ "$striptracks_recyclebin" ]; then - [ $striptracks_debug -eq 1 ] && echo "Debug|Moving: \"$striptracks_tempvideo\" to \"${striptracks_recyclebin%/}/$(basename "$striptracks_video")"\" | log + [ $striptracks_debug -eq 1 ] && echo "Debug|Recycling: \"$striptracks_tempvideo\" to \"${striptracks_recyclebin%/}/$(basename "$striptracks_video")"\" | log mv "$striptracks_tempvideo" "${striptracks_recyclebin%/}/$(basename "$striptracks_video")" | log else [ $striptracks_debug -eq 1 ] && echo "Debug|Deleting: \"$striptracks_tempvideo\"" | log rm "$striptracks_tempvideo" | log fi else - MSG="Error|Unable to locate or invalid remuxed file: \"$striptracks_newvideo\". Undoing rename." - echo "$MSG" | log - >&2 echo "$MSG" + striptracks_message="Error|Unable to locate or invalid remuxed file: \"$striptracks_newvideo\". Undoing rename." + echo "$striptracks_message" | log + >&2 echo "$striptracks_message" [ $striptracks_debug -eq 1 ] && echo "Debug|Renaming: \"$striptracks_tempvideo\" to \"$striptracks_video\"" | log mv -f "$striptracks_tempvideo" "$striptracks_video" | log exit 10 fi -FILESIZE=$(numfmt --to iec --format "%.3f" $(stat -c %s "$striptracks_newvideo")) -MSG="Info|New size: $FILESIZE" -echo "$MSG" | log +striptracks_filesize=$(numfmt --to iec --format "%.3f" $(stat -c %s "$striptracks_newvideo")) +striptracks_message="Info|New size: $striptracks_filesize" +echo "$striptracks_message" | log -# Call Radarr/Sonarr API to RescanMovie/RescanSeries -if [ -f "$striptracks_arr_config" ]; then - # Read *arr config.xml - while read_xml; do - [[ $ENTITY = "Port" ]] && striptracks_port=$CONTENT - [[ $ENTITY = "UrlBase" ]] && striptracks_urlbase=$CONTENT - [[ $ENTITY = "BindAddress" ]] && striptracks_bindaddress=$CONTENT - [[ $ENTITY = "ApiKey" ]] && striptracks_apikey=$CONTENT - done < $striptracks_arr_config - - [[ $striptracks_bindaddress = "*" ]] && striptracks_bindaddress=localhost - - # Check for video ID - if [ "$striptracks_video_id" ]; then - # Call API - if [ "${striptracks_type,,}" = "radarr" ] && get_video_info; then +#### Call Radarr/Sonarr API to RescanMovie/RescanSeries +# Check for URL +if [ -n "$striptracks_api_url" ]; then + # Check for video IDs + if [ "$striptracks_video_id" -a "$striptracks_videofile_id" ]; then + # Get video file info + if get_videofile_info; then # Save original quality - ORGQUALITY=$(echo $RESULT | jq -crM ${striptracks_json_quality_root}.quality) - fi - # Loop a maximum of twice - for ((i=1; $i <= 2; i++)); do - # Scan the disk for the new movie file - if rescan; then - # Check that the Rescan completed - if check_rescan; then - # This whole section doesn't work under Sonarr because the episodefile_id changes after the RescanSeries if the filename changes - # Should look into just using a PUT to change everything at once instead of a Rescan. - if [ "${striptracks_type,,}" = "radarr" ]; then + striptracks_original_quality=$(echo $striptracks_result | jq -crM .quality) + [ $striptracks_debug -eq 1 ] && echo "Debug|Detected quality '$(echo $striptracks_original_quality | jq -crM .quality.name)'." | log + # Loop a maximum of twice + # Radarr needs to Rescan twice when the file extension changes + # (.avi -> .mkv for example) + for ((i=1; $i <= 2; i++)); do + # Scan the disk for the new movie file + if rescan; then + # Give it a beat + sleep 1 + # Check that the Rescan completed + if check_rescan; then + # Get new video file id if get_video_info; then - # Check that the file didn't get lost in the Rescan. - # Radarr sometimes needs to Rescan twice when the file extension changes - # (.avi -> .mkv for example) - if [ "$(echo $RESULT | jq -crM .hasFile)" = "true" ]; then + # Get new video file ID + striptracks_videofile_id=$(echo $striptracks_result | jq -crM ${striptracks_json_quality_root}.id) + [ $striptracks_debug -eq 1 ] && echo "Debug|Set new video file id '$striptracks_videofile_id'." | log + # Get new video file info + if get_videofile_info; then + # Check that the file didn't get lost in the Rescan. # If we lost the quality information, put it back - # NOTE: This "works" with Radarr in that the change shows up in the GUI, but only until the page changes. - # It doesn't seem to write the info permanently. Maybe an API bug? - if [ "$(echo $RESULT | jq -crM ${striptracks_json_quality_root}.quality.quality.name)" = "Unknown" ]; then - [ $striptracks_debug -eq 1 ] && echo "Debug|Updating quality to '$(echo $ORGQUALITY | jq -crM .quality.name)'. Calling ${striptracks_type^} API using PUT and URL 'http://$striptracks_bindaddress:$striptracks_port$striptracks_urlbase/api/$striptracks_api_endpoint/$striptracks_video_id?apikey=(removed)'" | log - RESULT=$(curl -s -d "$(echo $RESULT | jq -crM "${striptracks_json_quality_root}.quality=$ORGQUALITY")" -H "Content-Type: application/json" \ - -X PUT http://$striptracks_bindaddress:$striptracks_port$striptracks_urlbase/api/$striptracks_api_endpoint/$striptracks_video_id?apikey=$striptracks_apikey) - [ $striptracks_debug -eq 1 ] && echo "API returned: $RESULT" | awk '{print "Debug|"$0}' | log - if [ "$(echo $RESULT | jq -crM ${striptracks_json_quality_root}.quality.quality.name)" = "Unknown" ]; then - MSG="Warn|Unable to update ${striptracks_type^} $striptracks_api_endpoint '$striptracks_title' to quality '$(echo $ORGQUALITY | jq -crM .quality.name)'" - echo "$MSG" | log - >&2 echo "$MSG" + if [ "$(echo $striptracks_result | jq -crM .quality.quality.name)" != "$(echo $striptracks_original_quality | jq -crM .quality.name)" ]; then + [ $striptracks_debug -eq 1 ] && echo "Debug|Updating from quality '$(echo $striptracks_result | jq -crM .quality.quality.name)' to '$(echo $striptracks_original_quality | jq -crM .quality.name)'. Calling ${striptracks_type^} API using PUT and URL '$striptracks_api_url/v3/$striptracks_videofile_api/editor'" | log + striptracks_result=$(curl -s -H "X-Api-Key: $striptracks_apikey" -H "Content-Type: application/json" \ + -d "{\"${striptracks_videofile_api}Ids\":[${striptracks_videofile_id}],\"quality\":$striptracks_original_quality}" \ + -X PUT "$striptracks_api_url/v3/$striptracks_videofile_api/editor") + [ $striptracks_debug -eq 1 ] && echo "API returned: $striptracks_result" | awk '{print "Debug|"$0}' | log + # Check that the returned result shows the update + if [ "$(echo $striptracks_result | jq -crM .[].quality.quality.name)" = "$(echo $striptracks_original_quality | jq -crM .quality.name)" ]; then + # Updated successfully + [ $striptracks_debug -eq 1 ] && echo "Debug|Successfully updated quality to '$(echo $striptracks_result | jq -crM .[].quality.quality.name)'." | log + break + else + striptracks_message="Warn|Unable to update ${striptracks_type^} $striptracks_video_api '$striptracks_title' to quality '$(echo $striptracks_original_quality | jq -crM .quality.name)'" + echo "$striptracks_message" | log + >&2 echo "$striptracks_message" fi + else + # The quality is already correct + [ $striptracks_debug -eq 1 ] && echo "Debug|Quality of '$(echo $striptracks_original_quality | jq -crM .quality.name)' remained unchanged." | log + break fi - # The video record is [now] good - break else - # Loop again because there was no file - continue + # No '.path' in returned JSON + striptracks_message="Warn|The '$striptracks_videofile_api' API with ${striptracks_video_api}File id $striptracks_videofile_id returned no path." + echo "$striptracks_message" | log + >&2 echo "$striptracks_message" fi else - # No 'path' in returned JSON. - MSG="Warn|The '$striptracks_api' API with $striptracks_api_endpoint $striptracks_api_endpoint_id returned no path." - echo "$MSG" | log - >&2 echo "$MSG" + # 'hasFile' is False in returned JSON. + striptracks_message="Warn|The '$striptracks_video_api' API with id $striptracks_video_id returned a false hasFile (Normal with Radarr on try #1)." + echo "$striptracks_message" | log + >&2 echo "$striptracks_message" fi else - # Didn't do anything because we're in Sonarr - break + # Timeout or failure + striptracks_message="Warn|${striptracks_type^} job ID $striptracks_jobid timed out or failed." + echo "$striptracks_message" | log + >&2 echo "$striptracks_message" fi else - # Timeout or failure - MSG="Warn|${striptracks_type^} job ID $JOBID timed out or failed." - echo "$MSG" | log - >&2 echo "$MSG" + # Error from API + striptracks_message="Error|The '$striptracks_rescan_api' API with $striptracks_json_key $striptracks_video_id failed." + echo "$striptracks_message" | log + >&2 echo "$striptracks_message" fi - else - # Error from API - MSG="Error|The '$striptracks_api' API with $striptracks_json_key $striptracks_video_id failed." - echo "$MSG" | log - >&2 echo "$MSG" - fi - done + done + else + # No '.path' in returned JSON + striptracks_message="Warn|The '$striptracks_videofile_api' API with ${striptracks_video_api}File id $striptracks_videofile_id returned no path." + echo "$striptracks_message" | log + >&2 echo "$striptracks_message" + fi else # No video ID means we can't call the API - MSG="Warn|Missing environment variable: $striptracks_video_idname" - echo "$MSG" | log - >&2 echo "$MSG" + striptracks_message="Warn|Missing or empty environment variable: striptracks_video_id='$striptracks_video_id' or striptracks_videofile_id='$striptracks_videofile_id'" + echo "$striptracks_message" | log + >&2 echo "$striptracks_message" fi else - # No config file means we can't call the API - MSG="Warn|Unable to locate ${striptracks_type^} config file: '$striptracks_arr_config'" - echo "$MSG" | log - >&2 echo "$MSG" + # No URL means we can't call the API + striptracks_message="Warn|Unable to determine ${striptracks_type^} API URL." + echo "$striptracks_message" | log + >&2 echo "$striptracks_message" fi # Cool bash feature -MSG="Info|Completed in $(($SECONDS/60))m $(($SECONDS%60))s" -echo "$MSG" | log +striptracks_message="Info|Completed in $(($SECONDS/60))m $(($SECONDS%60))s" +echo "$striptracks_message" | log