Skip to content
This repository has been archived by the owner on Oct 19, 2024. It is now read-only.

Commit

Permalink
Making many improvements on the user friendliness of the colorization…
Browse files Browse the repository at this point in the history
… notebooks
  • Loading branch information
jantic committed Apr 19, 2019
1 parent bd05e2f commit ebb4eba
Show file tree
Hide file tree
Showing 7 changed files with 340 additions and 71 deletions.
180 changes: 180 additions & 0 deletions ImageColorizer.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"os.environ['CUDA_VISIBLE_DEVICES']='1' "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from __future__ import print_function\n",
"from ipywidgets import interact, interactive, fixed, interact_manual\n",
"import ipywidgets as widgets"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from fasterai.visualize import *\n",
"plt.style.use('dark_background')\n",
"torch.backends.cudnn.benchmark=True"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"NOTE: Set artistic to False if you're having trouble getting a good render. Chances are it will work with the Stable model. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"colorizer = get_image_colorizer(artistic=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Instructions\n",
"\n",
"### source_url\n",
"Type in a url to a direct link of an image. Usually that means they'll end in .png, .jpg, etc. NOTE: If you want to use your own image, you can set source_url to None and just upload the image to /test_images/ in Jupyter. Just make sure that the source_path parameter matches the file you uploaded.\n",
"\n",
"### source_path\n",
"Name this whatever sensible image path (plus extension of jpg/png/ext) you want! Sensible means the path exists and the file exists if source_url=None.\n",
"\n",
"### render_factor\n",
"The default value of 35 has been carefully chosen and should work -ok- for most scenarios (but probably won't be the -best-). This determines resolution at which the color portion of the image is rendered. Lower resolution will render faster, and colors also tend to look more vibrant. Older and lower quality images in particular will generally benefit by lowering the render factor. Higher render factors are often better for higher quality images, but the colors may get slightly washed out. \n",
"\n",
"### result_path\n",
"Ditto- don't change.\n",
"\n",
"### How to Download a Copy\n",
"Simply shift+right click on the displayed image and click \"Save Image As...\"!\n",
"\n",
"\n",
"## Pro Tips\n",
"1. You can evaluate how well the image is rendered at each render_factor by using the code at the bottom (that cell under \"See how well render_factor values perform on a frame here\"). \n",
"2. Keep in mind again that you can go up top and set artistic to False for the colorizer to use the 'Stable' model instead. This will often tend to do better on portraits, and natural landscapes. \n",
"\n",
"\n",
"## Troubleshooting\n",
"If you get a 'CUDA out of memory' error, you probably have the render_factor too high. The max is 45 on 11GB video cards."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Colorize!!"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#NOTE: Max is 45 with 11GB video cards. 35 is a good default\n",
"render_factor=35\n",
"#NOTE: Make source_url None to just read from file at ./video/source/[file_name] directly without modification\n",
"source_url='https://i.imgur.com/NajrX6Z.jpg'\n",
"source_path = 'test_images/image.png'\n",
"result_path = None\n",
"\n",
"if source_url is not None:\n",
" result_path = colorizer.plot_transformed_image_from_url(url=source_url, path=source_path, render_factor=render_factor)\n",
"else:\n",
" result_path = colorizer.plot_transformed_image(path=source_path, render_factor=render_factor)\n",
"\n",
"show_image_in_notebook(result_path)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## See how well render_factor values perform on the image here"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"for i in range(5,46):\n",
" colorizer.plot_transformed_image(source_path, render_factor=i, display_render_factor=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.0"
},
"toc": {
"colors": {
"hover_highlight": "#DAA520",
"navigate_num": "#000000",
"navigate_text": "#333333",
"running_highlight": "#FF0000",
"selected_highlight": "#FFD700",
"sidebar_border": "#EEEEEE",
"wrapper_background": "#FFFFFF"
},
"moveMenuLeft": true,
"nav_menu": {
"height": "67px",
"width": "252px"
},
"navigate_menu": true,
"number_sections": true,
"sideBar": true,
"threshold": 4,
"toc_cell": false,
"toc_section_display": "block",
"toc_window_display": false,
"widenNotebook": false
}
},
"nbformat": 4,
"nbformat_minor": 2
}
File renamed without changes.
59 changes: 41 additions & 18 deletions ImageColorizerColab.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -158,31 +158,37 @@
"colorizer = get_image_colorizer(artistic=True)"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "sUQrbSYipiJn"
},
"source": [
"#◢ Colorize!!"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Instructions\n",
"# Instructions\n",
"\n",
"#### source_url\n",
"Any direct link to an image should do (will end with extension .jpg, .png, etc). NOTE: If you want to use your own image, upload it first to a site like Imgur. \n",
"### source_url\n",
"Type in a url to a direct link of an image. Usually that means they'll end in .png, .jpg, etc. NOTE: If you want to use your own image, upload it first to a site like Imgur. \n",
"\n",
"\n",
"#### render_factor\n",
"The default value of 35 has been carefully chosen and should work -ok- for most scenarios (but probably won't be the -best-). This determines resolution at which the photo is rendered. Lower resolution will render faster, and colors also tend to look more vibrant. Older and lower quality photos in particular will generally benefit by lowering the render factor. Higher render factors are often better for higher quality photos.\n",
"### render_factor\n",
"The default value of 35 has been carefully chosen and should work -ok- for most scenarios (but probably won't be the -best-). This determines resolution at which the color portion of the image is rendered. Lower resolution will render faster, and colors also tend to look more vibrant. Older and lower quality images in particular will generally benefit by lowering the render factor. Higher render factors are often better for higher quality images, but the colors may get slightly washed out. \n",
"\n",
"#### How to Download a Copy\n",
"Simply right click on the displayed image and click \"Save image as...\"!"
"Simply right click on the displayed image and click \"Save image as...\"!\n",
"\n",
"## Pro Tips\n",
"1. You can evaluate how well the image is rendered at each render_factor by using the code at the bottom (that cell under \"See how well render_factor values perform on a frame here\"). \n",
"2. Keep in mind again that you can go up top and set artistic to False for the colorizer to use the 'Stable' model instead. This will often tend to do better on portraits, and natural landscapes. \n",
"\n",
"## Troubleshooting\n",
"If you get a 'CUDA out of memory' error, you probably have the render_factor too high. The max is 45 on 11GB video cards."
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "sUQrbSYipiJn"
},
"source": [
"#◢ Colorize!!"
]
},
{
Expand All @@ -192,7 +198,7 @@
"outputs": [],
"source": [
"source_url = '' #@param {type:\"string\"}\n",
"render_factor = 35 #@param {type: \"slider\", min: 7, max: 46}\n",
"render_factor = 35 #@param {type: \"slider\", min: 7, max: 45}\n",
"\n",
"if source_url is not None and source_url !='':\n",
" image_path = colorizer.plot_transformed_image_from_url(url=source_url, render_factor=render_factor)\n",
Expand All @@ -201,6 +207,23 @@
" print('Provide an image url and try again.')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## See how well render_factor values perform on the image here"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"for i in range(5,46):\n",
" colorizer.plot_transformed_image(source_path, render_factor=i, display_render_factor=True)"
]
},
{
"cell_type": "markdown",
"metadata": {
Expand Down
File renamed without changes.
93 changes: 68 additions & 25 deletions VideoColorizer.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -27,33 +27,58 @@
"metadata": {},
"outputs": [],
"source": [
"#Adjust render_factor (int) if image doesn't look quite right (max 45 on 11GB GPU). \n",
"#Lower render factors (as low as 12-15) tend to work well for old and low quality videos.\n",
"#High render factors (25-45) tend to work well for higher quality and more recent videos\n",
"colorizer = get_video_colorizer()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Instructions\n",
"\n",
"#Not satisfied with color saturation? Lower the render factor. \n",
"#Unacceptable object flicker? Increase the render factor.\n",
"### source_url\n",
"Type in a url hosting a video from YouTube, Imgur, Twitter, Reddit, Vimeo, etc. Many sources work! GIFs also work. Full list here: https://ytdl-org.github.io/youtube-dl/supportedsites.html NOTE: If you want to use your own video, you can set source_url to None and just upload the file to video/source/ in Jupyter. Just make sure that the file_name parameter matches the file you uploaded.\n",
"\n",
"#It literally just is a number multiplied by 16 to get the square render resolution. \n",
"#Note that this doesn't affect the resolution of the final output- the output is the same resolution as the input.\n",
"#Example: render_factor=21 => color is rendered at 16x21 = 336x336 px. \n",
"render_factor=21\n",
"#Specify media_url. Many sources will work (YouTube, Imgur, Twitter, Reddit, etc). \n",
"#Complete list here: https://rg3.github.io/youtube-dl/supportedsites.html. \n",
"#NOTE: Make source_url None to just read from file at ./video/source/[file_name] directly without modification\n",
"#source_url= 'https://twitter.com/silentmoviegifs/status/1112256563182489600'\n",
"#source_url='https://archive.org/details/impact'\n",
"source_url='https://twitter.com/silentmoviegifs/status/1116751583386034176'\n",
"file_name = 'DogBath.mp4'"
"\n",
"### file_name\n",
"Name this whatever sensible file name you want (minus extension)! It should actually exist in video/source if source_url=None\n",
"\n",
"\n",
"### render_factor\n",
"The default value of 21 has been carefully chosen and should work -ok- for most scenarios (but probably won't be the -best-). This determines resolution at which the color portion of the video is rendered. Lower resolution will render faster, and colors also tend to look more vibrant. Older and lower quality film in particular will generally benefit by lowering the render factor. Higher render factors are often better for higher quality videos and inconsistencies (flashy render) will generally be reduced, but the colors may get slightly washed out. \n",
"\n",
"\n",
"### file_name_ext\n",
"There's no reason to changes this.\n",
"\n",
"\n",
"### result_path\n",
"Ditto- don't change.\n",
"\n",
"\n",
"### How to Download a Copy\n",
"Simply shift+right click on the displayed video and click \"Save video as...\"!\n",
"\n",
"\n",
"## Pro Tips\n",
"1. If a video takes a long time to render and you're wondering how well the frames will actually be colorized, you can preview how well the frames will be rendered at each render_factor by using the code at the bottom. Just stop the video rendering by hitting the stop button on the cell, then run that bottom cell under \"See how well render_factor values perform on a frame here\". It's not perfect and you may still need to experiment a bit especially when it comes to figuring out how to reduce frame inconsistency. But it'll go a long way in narrowing down what actually works.\n",
"\n",
"\n",
"## Troubleshooting\n",
"The video player may wind up not showing up, in which case- make sure to wait for the Jupyter cell to complete processing first (the play button will stop spinning). Then follow these alternative download instructions\n",
"\n",
"1. In the menu to the left, click Home icon.\n",
"2. By default, rendered video will be in /video/result/\n",
"\n",
"If a video you downloaded doesn't play, it's probably because the cell didn't complete processing and the video is in a half-finished state.\n",
"If you get a 'CUDA out of memory' error, you probably have the render_factor too high. The max is 44 on 11GB video cards."
]
},
{
"cell_type": "code",
"execution_count": null,
"cell_type": "markdown",
"metadata": {},
"outputs": [],
"source": [
"colorizer = get_video_colorizer()"
"## Colorize!!"
]
},
{
Expand All @@ -62,20 +87,38 @@
"metadata": {},
"outputs": [],
"source": [
"#NOTE: Max is 44 with 11GB video cards. 21 is a good default\n",
"render_factor=21\n",
"#NOTE: Make source_url None to just read from file at ./video/source/[file_name] directly without modification\n",
"source_url='https://twitter.com/silentmoviegifs/status/1116751583386034176'\n",
"file_name = 'DogShy1926'\n",
"file_name_ext = file_name + '.mp4'\n",
"result_path = None\n",
"\n",
"if source_url is not None:\n",
" video_path = colorizer.colorize_from_url(source_url, file_name, render_factor=render_factor)\n",
" show_video_in_notebook(video_path)\n",
" result_path = colorizer.colorize_from_url(source_url, file_name_ext, render_factor=render_factor)\n",
"else:\n",
" video_path = colorizer.colorize_from_file_name(file_name)\n",
" show_video_in_notebook(video_path)"
" result_path = colorizer.colorize_from_file_name(file_name_ext)\n",
"\n",
"show_video_in_notebook(result_path)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## See how well render_factor values perform on a frame here"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
"source": [
"for i in range(5,44):\n",
" colorizer.vis.plot_transformed_image('video/bwframes/' + file_name + '/00001.jpg', render_factor=i, display_render_factor=True)"
]
}
],
"metadata": {
Expand Down
Loading

0 comments on commit ebb4eba

Please sign in to comment.