Switch to the Playlist API endpoint (#107)

* clean up unwanted code

* switch to pathlib to create required directories

* add placeholder validate_spotify_url

* fix bug in creating path

* switch to new Playlist Endpoint

* use client auth flow for Spotify Auth
remove scope which is not used
update README & GETTING_STARTED

* fix wrong download directory

* update secret refs

* reduce verbosity of logs

* some changes to make deepsource happy

* exit with error if item type is not valid

more changes to make deepsource happy

* docstring fixes

* docstring fixes, more deepsource happy fixes

* fix correct path to where the files are saved

.github/workflows/pythonpackage.yml

@@ -30,6 +30,8 @@ jobs:
     - name: Test with pytest
       env:
         YOUTUBE_DEV_KEY:  ${{ secrets.YOUTUBE_DEV_KEY }}
+        SPOTIPY_CLIENT_ID:  ${{ secrets.SPOTIPY_CLIENT_ID }}
+        SPOTIPY_CLIENT_SECRET:  ${{ secrets.SPOTIPY_CLIENT_SECRET }}
       run: |
         pip install pytest pytest-cov
         pytest --cov=spotify_dl tests/

GETTING_STARTED.md

@@ -7,27 +7,23 @@ Pre-requisite: You need Python 3.6+
         pip3 install spotify_dl
 
 2. Login to [Spotify developer console](https://developer.spotify.com/my-applications/#!/applications) and click on "Create an App". Fill in details for name and description
-    - Click on "Edit Settings"
-    - Under "Redirect URIs", enter `https://spotifydl.sathyabh.at/`
-    - Click on "Save".
 
-3. Make a note of Client ID and Client Secret. These values need to be then set `SPOTIPY_CLIENT_ID`, `SPOTIPY_CLIENT_SECRET` environment variables respectively. Also `SPOTIPY_REDIRECT_URI` environment variable to `https://spotifydl.sathyabh.at/`
+3. Make a note of Client ID and Client Secret. These values need to be then set `SPOTIPY_CLIENT_ID`, `SPOTIPY_CLIENT_SECRET` environment variables respectively.
 
     You can set environment variables in Linux like so:
 
             export SPOTIPY_CLIENT_ID=your-spotify-client-id
             export SPOTIPY_CLIENT_SECRET=your-spotify-client-secret
-            export SPOTIPY_REDIRECT_URI=https://spotifydl.sathyabh.at/
 
     Windows users, check for [this question](http://superuser.com/a/284351/4377) for details on how you can set environment variables. If you don't wish to use my URL for the redirect, you are free to use any valid URL. Just ensurethe redirect URL set as the environment variable matches with what you have entered in the developer console & in the environment variable above.
 
-4. Create your YouTube API key & fetch the keys from [Google Developer Console](https://console.developers.google.com/apis/api/youtube/overview). Set the key as `YOUTUBE_DEV_KEY` environment variable as mentioned above. Note that as of version 5 you do not have to set this - it will fallback to scraping the YouTube page.
+4. Create your YouTube API key & fetch the keys from [Google Developer Console](https://console.developers.google.com/apis/api/youtube/overview). Set the key as `YOUTUBE_DEV_KEY` environment variable as mentioned above. Note that as of **version 5 you do not have to set this** - it will fallback to scraping the YouTube page.
 
         export YOUTUBE_DEV_KEY=youtube-dev-key
 
 5. Run the script using `spotify_dl`. spotify_dl accepts different parameters, for more details run `spotify_dl -h`. 
 
-   For most users `spotify_dl -l spotify_playlist_link -o download_directory` should do where
+   For most users `spotify_dl -l spotify_playlist_link -o download_directory -s yes` should do where
 
    - `spotify_playlist_link` is a link to Spotify's playlist. You can get it from the 3-dot menu. 
 
@@ -37,13 +33,7 @@ Pre-requisite: You need Python 3.6+
 
    - `download_directory` is the location where the songs must be downloaded to. If you give a `.` then it will download to the current directory.
 
-   Alternatively, `spotify_dl -p playlist_id -u user_name -o download_directory` will also work
-
-   - `playlist_id` is the id of the playlist where songs need to be downloaded. If this is skipped then it will download songs ftom your "My Music" collection
-   - `user_name` is the user name who created the playlist. 
-   - `download_directory` is the location where the songs must be downloaded to. 
-6. A first time run will require authentication; you will need to click on the URL prompted to authenticate. Once logged in, paste the URL back in.
-7. To retrieve download songs as MP3, you will need to install ffmpeg. If you prefer to skip MP3 conversion, pass `-m` or `--skip_mp3` as a parameter when running the script
+6. To retrieve download songs as MP3, you will need to install ffmpeg. If you prefer to skip MP3 conversion, pass `-m` or `--skip_mp3` as a parameter when running the script
   - Linux users can get them by installing libav-tools by using apt-get (`sudo apt-get install -y libav-tools`) or a package manager which comes with your distro
   - Windows users can download FFMPEG pre-built binaries from [here](http://ffmpeg.zeranoe.com/builds/). Extract the file using [7-zip](http://7-zip.org/) to a foldrer and [add the folder to your PATH environment variable](http://www.wikihow.com/Install-FFmpeg-on-Windows) 
 

README.md

@@ -1,5 +1,5 @@
 ## spotify_dl
-Downloads songs from any Spotify playlist or from your "My Music" collection.
+Downloads songs from any Spotify playlist, album or track.
 
 [![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com)
 [![PyPI download month](https://img.shields.io/pypi/dm/spotify_dl.svg)](https://pypi.python.org/pypi/spotify_dl/)

spotify_dl/cache.py

@@ -5,27 +5,27 @@
 
 def check_if_in_cache(search_term):
     """
-    Checks if the specified search term is in the local database cache
+    Checks if the specified search term is in the local database cache.
     and returns the video id if it exists.
     :param search_term: String to be searched for in the cache
     :return A tuple with Boolean and video id if it exists
     """
     try:
         song = Song.get(search_term=search_term)
-        log.info(f"Found id {song.video_id} for {search_term} in cache")
+        log.debug(f"Found id {song.video_id} for {search_term} in cache")
         return True, song.video_id
     except DoesNotExist:
-        log.info(f"Couldn't find id for {search_term} in cache")
+        log.debug(f"Couldn't find id for {search_term} in cache")
         return False, None
 
 
 def save_to_cache(search_term, video_id):
     """
-    Saves the search term and video id to the database cache so it can be looked up later
+    Saves the search term and video id to the database cache so it can be looked up later.
     :param search_term: Search term to be saved to in the cache
     :param video_id: Video id to be saved to in the cache
     :return Video id saved in the cache
     """
     song_info, saved = Song.get_or_create(search_term=search_term, video_id=video_id)
-    log.info(f"Saved: {saved} video id {song_info.video_id} in cache")
+    log.debug(f"Saved: {saved} video id {song_info.video_id} in cache")
     return song_info.video_id

spotify_dl/constants.py

@@ -6,4 +6,3 @@
 YOUTUBE_VIDEO_URL = 'https://www.youtube.com/watch?v='
 VERSION = '6.0.0'
 SAVE_PATH = '~/.spotifydl'
-SCOPE = "user-library-read"

spotify_dl/spotify.py

@@ -1,58 +1,65 @@
-from __future__ import unicode_literals
-import re
-
 import youtube_dl
-
 from spotify_dl.scaffold import *
 
-def fetch_tracks(sp, playlist, user_id):
-    """Fetches tracks from Spotify user's saved
-        tracks or from playlist(if playlist parameter is passed
-        and saves song name and artist name to songs list
+
+def fetch_tracks(sp, item_type, url):
+    """
+    Fetches tracks from the provided URL.
+    :param sp: Spotify client
+    :param type: Type of item being requested for: album/playlist/track
+    :param url: URL of the item
+    :return Dictionary of song and artist
     """
-    log.debug('Fetching saved tracks')
-    offset = 0
     songs_dict = {}
-    if user_id is None:
-        current_user_id = sp.current_user()['id']
-    else:
-        current_user_id = user_id
-    while True:
-        if playlist is None:
-            results = sp.current_user_saved_tracks(limit=50, offset=offset)
-        else:
-            results = sp.user_playlist_tracks(current_user_id, playlist, None,
-                                              limit=50, offset=offset)
-
-        log.debug(f'Got result json keys {results.keys()}', )
-        for item in results['tracks']['items']:
-            track = item['track']
-
-            if track is not None:
-                track_name = str(track['name'])
-                track_artist = str(track['artists'][0]['name'])
-                log.debug('Appending %s to'
-                        'songs list', (track['name'] + ' - ' + track['artists'][0]['name']))
+    offset = 0
+
+    if item_type == 'playlist':
+        items = sp.playlist_items(playlist_id=url, fields='items.track.name,items.track.artists(name),items.track.album(name),total,next,offset', additional_types=['track'])
+        while True:
+            for item in items['items']:
+                track_name = item['track']['name']
+                track_artist = " ".join([artist['name'] for artist in item['track']['artists']])
+                songs_dict.update({track_name: track_artist})
+                offset += 1
+
+            if items.get('next') is None:
+                log.info('All pages fetched, time to leave. Added %s songs in total', offset)
+                break
+
+    elif item_type == 'album':
+        items = sp.album_tracks(album_id=url)
+        while True:
+            for item in items['items']:
+                track_name = item['name']
+                track_artist = " ".join([artist['name'] for artist in item['artists']])
                 songs_dict.update({track_name: track_artist})
-            else:
-                log.warning("Track/artist name for %s not found, skipping", track)
+                offset += 1
 
-            offset += 1
+            if items.get('next') is None:
+                log.info('All pages fetched, time to leave. Added %s songs in total', offset)
+                break
 
-        if results.get('next') is None:
-            log.info('All pages fetched, time to leave.'
-                     ' Added %s songs in total', offset)
-            break
+    elif item_type == 'track':
+        items = sp.track(track_id=url)
+        track_name = items['name']
+        track_artist = " ".join([artist['name'] for artist in items['artists']])
+        songs_dict.update({track_name: track_artist})
     return songs_dict
 
 
-def download_songs(info, download_directory, format_string, skip_mp3):
+def download_songs(songs_dict, download_directory, format_string, skip_mp3):
     """
-    Downloads songs from the YouTube URL passed to either
-       current directory or download_directory, is it is passed
+    Downloads songs from the YouTube URL passed to either current directory or download_directory, is it is passed.
+    :param songs_dict: Dictionary of songs and associated artist
+    :param download_directory: Location where to save
+    :param format_string: format string for the file conversion
+    :param skip_mp3: Whether to skip conversion to MP3
     """
-    for number, item in enumerate(info):
+    download_directory = f"{download_directory}\\"
+    log.debug(f"Downloading to {download_directory}")
+    for number, item in enumerate(songs_dict):
         log.debug('Songs to download: %s', item)
+
         url_, track_, artist_ = item
         download_archive = download_directory + 'downloaded_songs.txt'
         outtmpl = download_directory + '%(title)s.%(ext)s'
@@ -82,29 +89,48 @@ def download_songs(info, download_directory, format_string, skip_mp3):
                 continue
 
 
-def extract_user_and_playlist_from_uri(uri, sp):
-    playlist_re = re.compile("(spotify)(:user:[\w,.]+)?(:playlist:[\w]+)")
-    user_id = sp.current_user()['id']
-    for playlist_uri in ["".join(x) for x in playlist_re.findall(uri)]:
-        segments = playlist_uri.split(":")
-        if len(segments) >= 4:
-            user_id = segments[2]
-            playlist_id = segments[4]
-            log.info('List ID: ' + str(playlist_id))
-        else:
-            playlist_id = segments[2]
-            log.info('List ID: ' + str(playlist_id))
-    log.info('List owner: ' + str(user_id))
-    return user_id, playlist_id
+def parse_spotify_url(url):
+    """
+    Parse the provided Spotify playlist URL and determine if it is a playlist, track or album.
+    :param url: URL to be parsed
+    :param download_directory: Location where to save
+    :param format_string: format string for the file conversion
+    :return tuple indicating the type and id of the item
+    """
+    parsed_url = url.replace("https://open.spotify.com/", "")
+    item_type = parsed_url.split("/")[0]
+    item_id = parsed_url.split("/")[1]
+    return item_type, item_id
 
 
-def playlist_name(uri, sp):
-    user_id, playlist_id = extract_user_and_playlist_from_uri(uri, sp)
-    return get_playlist_name_from_id(playlist_id, user_id, sp)
+def get_item_name(sp, item_type, item_id):
+    """
+    Fetch the name of the item.
+    :param sp: Spotify Client
+    :param item_type: Type of the item
+    :param item_id: id of the item
+    :return String indicating the name of the item
+    """
+    if item_type == 'playlist':
+        name = sp.playlist(playlist_id=item_id, fields='name').get('name')
+    elif item_type == 'album':
+        name = sp.album(album_id=item_id).get('name')
+    elif item_type == 'track':
+        name = sp.track(track_id=item_id).get('name')
+    return name
 
 
-def get_playlist_name_from_id(playlist_id, user_id, sp):
-    playlist = sp.user_playlist(user_id, playlist_id,
-                                fields="tracks, next, name")
-    name = playlist['name']
-    return name
+def validate_spotify_url(url):
+    """
+    Validate the URL and determine if the item type is supported.
+    :return Boolean indicating whether or not item is supported
+    """
+    item_type, item_id = parse_spotify_url(url)
+    log.debug(f"Got item type {item_type} and item_id {item_id}")
+    if item_type not in ['album', 'track', 'playlist']:
+        log.error("Only albums/tracks/playlists are supported")
+        return False
+    if item_id is None:
+        log.error("Couldn't get a valid id")
+        return False
+    return True