[iOS & tvOS] Unused Localization Cleanup

[JPKribs]

Summary

Removes Localizations with no usages in code. Alphabetizes localizations and cleans up comments. This is a must more intrusive part of #518. This may change our webplates but Swiftfin itself will be unchanged.

Some localizations may be worth retaining for future usage. An example of this is WIP is currently unused but if we ever need something to be tagged as WIP in the future it will be good to keep this localized.

I have validated that both tvOS and iOS still appropriately build with these removed. The next step in this process, I have a remove with all "Strings" that exist in Swiftin. Most of them are fine like " / " but I'll submit another PR with the unlocalized strings localized.

The following localizations will be removed by this PR. Let me know if any of these should be retained and I can place them back in:

/// Who's watching?
"WhosWatching" = "Who's watching?";

/// Active Devices
"activeDevices" = "Active Devices";

/// Administration
"administration" = "Administration";

/// All Genres
"allGenres" = "All Genres";

/// Allowed tags
"allowedTags" = "Allowed tags";

/// Only show media to this user with at least one of the specified tags.
"allowedTagsDescription" = "Only show media to this user with at least one of the specified tags.";

/// Apply
"apply" = "Apply";

/// Audio & Captions
"audioAndCaptions" = "Audio & Captions";

/// Audio Track
"audioTrack" = "Audio Track";

/// Blocked tags
"blockedTags" = "Blocked tags";

/// Hide media with at least one of the specified tags.
"blockedTagsDescription" = "Hide media with at least one of the specified tags.";

/// Cancelled
"canceled" = "Cancelled";

/// CAST
"cast" = "CAST";

/// Change Server
"changeServer" = "Change Server";

/// Changes not saved
"changesNotSaved" = "Changes not saved";

/// Cinematic Views
"cinematicViews" = "Cinematic Views";

/// Closed Captions
"closedCaptions" = "Closed Captions";

/// Coming soon
"comingSoon" = "Coming soon";

/// Confirm Close
"confirmClose" = "Confirm Close";

/// Connect Manually
"connectManually" = "Connect Manually";

/// Connect to Jellyfin
"connectToJellyfin" = "Connect to Jellyfin";

/// Connect to a Jellyfin server
"connectToJellyfinServer" = "Connect to a Jellyfin server";

/// Continue Watching
"continueWatching" = "Continue Watching";

/// Current Position
"currentPosition" = "Current Position";

/// Dictates back to the Jellyfin Server what this device hardware is capable of playing.
"customDeviceProfileDescription" = "Dictates back to the Jellyfin Server what this device hardware is capable of playing.";

/// Default Scheme
"defaultScheme" = "Default Scheme";

/// Discard Changes
"discardChanges" = "Discard Changes";

/// Discovered Servers
"discoveredServers" = "Discovered Servers";

/// Edit Jump Lengths
"editJumpLengths" = "Edit Jump Lengths";

/// Empty Next Up
"emptyNextUp" = "Empty Next Up";

/// Existing Server
"existingServer" = "Existing Server";

/// Existing User
"existingUser" = "Existing User";

/// Feature access
"featureAccess" = "Feature access";

/// File
"file" = "File";

/// File Path
"filePath" = "File Path";

/// Filter Results
"filterResults" = "Filter Results";

/// Find Missing Metadata
"findMissingMetadata" = "Find Missing Metadata";

/// %@fps
"fpsWithString" = "%@fps";

/// Haptic Feedback
"hapticFeedback" = "Haptic Feedback";

/// Information
"information" = "Information";

/// You do not have permission to delete this item.
"itemDeletionPermissionFailure" = "You do not have permission to delete this item.";

/// %1$@ / %2$@
"itemOverItem" = "%1$@ / %2$@";

/// Jump Gestures Enabled
"jumpGesturesEnabled" = "Jump Gestures Enabled";

/// %s seconds
"jumpLengthSeconds" = "%s seconds";

/// Loading
"loading" = "Loading";

/// Login
"login" = "Login";

/// Login to %@
"loginToWithString" = "Login to %@";

/// This setting may result in media failing to start playback.
"mayResultInPlaybackFailure" = "This setting may result in media failing to start playback.";

/// More Like This
"moreLikeThis" = "More Like This";

/// %d users
"multipleUsers" = "%d users";

/// Networking
"networking" = "Networking";

/// No Cast devices found..
"noCastdevicesfound" = "No Cast devices found..";

/// No Codec
"noCodec" = "No Codec";

/// N/A
"notAvailableSlash" = "N/A";

/// 1 user
"oneUser" = "1 user";

/// Online
"online" = "Online";

/// Operating System
"operatingSystem" = "Operating System";

/// Other User
"otherUser" = "Other User";

/// Overlay
"overlay" = "Overlay";

/// Overlay Type
"overlayType" = "Overlay Type";

/// Page %1$@ of %2$@
"pageOfWithNumbers" = "Page %1$@ of %2$@";

/// Play Next
"playNext" = "Play Next";

/// Playback settings
"playbackSettings" = "Playback settings";

/// Player Gestures Lock Gesture Enabled
"playerGesturesLockGestureEnabled" = "Player Gestures Lock Gesture Enabled";

/// Present
"present" = "Present";

/// Invalid Quick Connect code
"quickConnectInvalidError" = "Invalid Quick Connect code";

/// Note: Quick Connect not enabled
"quickConnectNotEnabled" = "Note: Quick Connect not enabled";

/// Rated
"rated" = "Rated";

/// Refresh
"refresh" = "Refresh";

/// Released
"released" = "Released";

/// Reload
"reload" = "Reload";

/// Remaining Time
"remainingTime" = "Remaining Time";

/// Remove
"remove" = "Remove";

/// Remove All Users
"removeAllUsers" = "Remove All Users";

/// Remove From Resume
"removeFromResume" = "Remove From Resume";

/// Report an Issue
"reportIssue" = "Report an Issue";

/// Request a Feature
"requestFeature" = "Request a Feature";

/// Reset App Settings
"resetAppSettings" = "Reset App Settings";

/// Resume 5 Second Offset
"resume5SecondOffset" = "Resume 5 Second Offset";

/// Scan All Libraries
"scanAllLibraries" = "Scan All Libraries";

/// Scheduled Tasks
"scheduledTasks" = "Scheduled Tasks";

/// Search…
"searchDots" = "Search…";

/// Searching…
"searchingDots" = "Searching…";

/// Seasons
"seasons" = "Seasons";

/// Seek Slide Gesture Enabled
"seekSlideGestureEnabled" = "Seek Slide Gesture Enabled";

/// Select Cast Destination
"selectCastDestination" = "Select Cast Destination";

/// Server Details
"serverDetails" = "Server Details";

/// Server Information
"serverInformation" = "Server Information";

/// A new trigger was created for '%1$@'.
"serverTriggerCreated" = "A new trigger was created for '%1$@'.";

/// The selected trigger was deleted from '%1$@'.
"serverTriggerDeleted" = "The selected trigger was deleted from '%1$@'.";

/// Show Cast & Crew
"showCastAndCrew" = "Show Cast & Crew";

/// Show Chapters Info In Bottom Overlay
"showChaptersInfoInBottomOverlay" = "Show Chapters Info In Bottom Overlay";

/// Flatten Library Items
"showFlattenView" = "Flatten Library Items";

/// Sign in to get started
"signInGetStarted" = "Sign in to get started";

/// Signed in as %@
"signedInAsWithString" = "Signed in as %@";

/// Sort by
"sortBy" = "Sort by";

/// STUDIO
"studio" = "STUDIO";

/// Suggestions
"suggestions" = "Suggestions";

/// System Control Gestures Enabled
"systemControlGesturesEnabled" = "System Control Gestures Enabled";

/// Time Limit (%@)
"timeLimitWithUnit" = "Time Limit (%@)";

/// Too Many Redirects
"tooManyRedirects" = "Too Many Redirects";

/// Try again
"tryAgain" = "Try again";

/// Unable to connect to server
"unableToConnectServer" = "Unable to connect to server";

/// User %s is already signed in
"userAlreadySignedIn" = "User %s is already signed in";

/// Your Favorites
"yourFavorites" = "Your Favorites";

[LePips]

The next step in this process, I have a remove with all "Strings" that exist in Swiftin.

This should not be the end goal. Strings in logging statements and assertions shouldn't be localized as these are largely developer facing and are used for debugging. Only strings that face the user through official use/design should be localized.

[JPKribs]

I'm going to wait for some of the outstanding PRs then re-run this script but this time only remove unused localization. I jumped the gun a little adding new localizations in. It'll be easier to re-run the script that to try and dissect out my "String" replacements. This time, I'll only do the reorganized strings + drop unused strings.

[JPKribs]

@LePips Re-ran this on current Main. All this does is remove unused Strings from localization. Existing strings are alphabetized and the comment about them is the content of the string in English.

The only freeform item I added to this is MediaStream.swift when I first did localization I localized those but I now understand those to be part of the URL building and have reverted them back to plaintext. I can revert that as needed or put it back as needed. Having trouble determining what is a string and what is supposed to be plaintext.

I've updated the list of removed localizations on the main section of this post.

Below are 2 files output from the script.

1. unlocalized.log are all items that HAVE an L10n localization but are not localized. I went through these and these are all supposed to be plaintext.
2. strings.log are all "Strings" in Swiftfin. I don't really know what to do with those for now but eventually I (or some brave volunteer) can go through and localize any strings in there that don't have existing localizations.

For now, I'm happy with the 2 steps we've already taken. The first being localize strings that have existing localizations. The second because remove unused localizations.

strings.log
unlocalized.log

[JPKribs]

If we ever needs this again, here is the Python Script I used. It's not the best thing in the world but it works. You point it at the project folder, the en.lproj/translation.swift file, and you can provide a file containing file paths that you want it to ignore.

I'm just providing this for documentation. If you need me to re-run this to clean up localizations, feel free to ping me!

import os
import re
from collections import defaultdict

def extract_localizations(file_path):
    """Extract keys and values from the Localizable.strings file."""
    localization_pattern = re.compile(r'^"(\w+)"\s*=\s*"(.*?)";')
    localizations = {}

    with open(file_path, "r", encoding="utf-8") as file:
        for line in file:
            match = localization_pattern.match(line.strip())
            if match:
                key, value = match.groups()
                localizations[key] = value

    return localizations

def load_ignore_file(ignore_file_path):
    """Load files to ignore from the ignore file."""
    ignored_files = set()
    if os.path.isfile(ignore_file_path):
        with open(ignore_file_path, "r", encoding="utf-8") as file:
            for line in file:
                ignored_files.add(line.strip())
    return ignored_files

def find_localization_usage(directory, keys, ignored_files):
    """Search the project for instances of L10n.<key>, skipping ignored files."""
    results = defaultdict(int)
    localization_pattern = re.compile(r"L10n\.(\w+)")

    for root, _, files in os.walk(directory):
        for file in files:
            if file.endswith(".swift"):
                file_path = os.path.join(root, file)
                if file_path in ignored_files:
                    continue  # Skip ignored files

                with open(file_path, "r", encoding="utf-8") as swift_file:
                    for line in swift_file:
                        matches = localization_pattern.findall(line)
                        for match in matches:
                            if match in keys:
                                results[match] += 1

    return results

def find_direct_string_usage(directory, localizations, ignored_files):
    """Search the project for direct string usage instead of localization."""
    string_usage = defaultdict(list)

    for key, value in localizations.items():
        pattern = re.compile(rf'"{re.escape(value)}"')  # Match the value in quotes

        for root, _, files in os.walk(directory):
            for file in files:
                if file.endswith(".swift"):
                    file_path = os.path.join(root, file)
                    if file_path in ignored_files:
                        continue  # Skip ignored files

                    with open(file_path, "r", encoding="utf-8") as swift_file:
                        for line_number, line in enumerate(swift_file, start=1):
                            if pattern.search(line):
                                string_usage[key].append(f"{file_path} (Line {line_number})")

    return string_usage

def find_unlocalized_strings(directory, localized_values, ignored_files):
    """Find all strings in the Swift project that are not localized."""
    unlocalized_strings = defaultdict(list)
    string_pattern = re.compile(r'"([^"]+)"')  # Matches anything in quotes
    system_prefix_pattern = re.compile(r'system(?:name|image):', re.IGNORECASE)  # Matches "SystemName:" or "SystemImage:"

    for root, _, files in os.walk(directory):
        for file in files:
            if file.endswith(".swift"):
                file_path = os.path.join(root, file)
                if file_path in ignored_files:
                    continue  # Skip ignored files

                with open(file_path, "r", encoding="utf-8") as swift_file:
                    for line_number, line in enumerate(swift_file, start=1):
                        matches = string_pattern.findall(line)
                        for match in matches:
                            if (
                                match not in localized_values  # Skip localized strings
                                and not system_prefix_pattern.search(line)  # Skip system-prefixed strings
                            ):
                                unlocalized_strings[match].append(f"{file_path} (Line {line_number})")

    return unlocalized_strings

def write_localization_usage(file_path, localizations, usage_counts):
    """Write localization usage details to a new .strings file, excluding 0 usage and no total usages."""
    with open(file_path, "w", encoding="utf-8") as file:
        for key in sorted(localizations.keys()):  # Alphabetize by key
            value = localizations[key]
            # Write only localizations with usages greater than 0
            if usage_counts.get(key, 0) > 0:
                file.write(f'/// {value}\n')
                file.write(f'"{key}" = "{value}";\n\n')

def write_removed_strings(file_path, localizations, usage_counts):
    """Write localizations with 0 usages to removed.strings."""
    with open(file_path, "w", encoding="utf-8") as file:
        for key in sorted(localizations.keys()):  # Alphabetize by key
            value = localizations[key]
            # Write localizations with 0 usages
            if usage_counts.get(key, 0) == 0:
                file.write(f'/// {value}\n')
                file.write(f'"{key}" = "{value}";\n\n')

def write_unlocalized_log(file_path, direct_usages, localizations):
    """Write unlocalized string details to a log file."""
    with open(file_path, "w", encoding="utf-8") as file:
        for key, locations in sorted(direct_usages.items()):
            file.write(f"{localizations[key]} (use L10n.{key} instead):\n")
            for location in locations:
                file.write(f"  - {location}\n")
            file.write("\n")  # Add an extra blank line between entries

def write_strings_log(file_path, unlocalized_strings):
    """Write unlocalized strings to strings.log."""
    with open(file_path, "w", encoding="utf-8") as file:
        for string, locations in sorted(unlocalized_strings.items()):
            file.write(f'"{string}" found at:\n')
            for location in locations:
                file.write(f"  - {location}\n")
            file.write("\n")  # Add an extra blank line between entries

if __name__ == "__main__":
    # Input paths
    project_directory = input("Enter the path to your Swift project: ").strip()
    localizable_file = input("Enter the path to your Localizable.strings file: ").strip()
    ignore_file_path = input("Enter the path to your ignore file: ").strip()

    if not os.path.isdir(project_directory):
        print("Invalid project directory. Please provide a valid path.")
    elif not os.path.isfile(localizable_file):
        print("Invalid Localizable.strings file path. Please provide a valid path.")
    elif not os.path.isfile(ignore_file_path):
        print("Invalid ignore file path. Please provide a valid path.")
    else:
        # Load ignored files
        ignored_files = load_ignore_file(ignore_file_path)

        # Extract keys and values from Localizable.strings
        localizations = extract_localizations(localizable_file)
        localized_values = set(localizations.values())  # Extract just the values for comparison

        # Search for L10n.<key> usage in the project
        usage_counts = find_localization_usage(project_directory, localizations.keys(), ignored_files)

        # Search for direct string usage
        direct_usages = find_direct_string_usage(project_directory, localizations, ignored_files)

        # Find unlocalized strings
        unlocalized_strings = find_unlocalized_strings(project_directory, localized_values, ignored_files)

        # Output files
        script_folder = os.path.dirname(os.path.abspath(__file__))
        localization_output_file = os.path.join(script_folder, "localization_usage.strings")
        removed_strings_file = os.path.join(script_folder, "removed.strings")
        unlocalized_log_file = os.path.join(script_folder, "unlocalized.log")
        strings_log_file = os.path.join(script_folder, "strings.log")

        # Write results to files
        write_localization_usage(localization_output_file, localizations, usage_counts)
        write_removed_strings(removed_strings_file, localizations, usage_counts)
        write_unlocalized_log(unlocalized_log_file, direct_usages, localizations)
        write_strings_log(strings_log_file, unlocalized_strings)

        print(f"Localization usage written to {localization_output_file}")
        print(f"Removed localizations written to {removed_strings_file}")
        print(f"Unlocalized strings log written to {unlocalized_log_file}")
        print(f"Strings log written to {strings_log_file}")

[LePips]

I do like the cleanup and removing all of these unused strings. My only concern is that in the future we may want to reintroduce strings, like Refresh, but I think starting fresh and working from the ground up now is good.

Regarding the scripts, I think something written in Swift that sorts the main .strings file prior to swiftgen would be cool to have.

I think we'll be fine when it comes to the Weblate but we'll see and can revert if necessary.

[JPKribs]

Regarding the scripts, I think something written in Swift that sorts the main .strings file prior to swiftgen would be cool to have.

That's a great idea! I moved this over the Swift and added it to the Build Phase before SwiftGen. I wasn't sure how to handle Scripts so I made a folder for it. I put the PR out here:

#1372