gif: Nostr Build GIF Keyboard #2387

ericholguin · 2024-08-23T23:34:16Z

This PR adds a gif keyboard, kind of, to the posting view. Leverages the nostr build API to get latest gifs.

Changelog-Added: Nostr Build GIF keyboard

danieldaquino · 2024-08-27T23:11:12Z

Thanks @ericholguin! I am still reviewing and testing the new GIF keyboard, but just to follow up on that issue with GIFs not always loading on the poster's device, I made an experiment and I believe the issue is around the image cache.

Here is the experiment I performed:

Device: iPhone 15 simulator
iOS: 17.5
Damus: 2a4f4117ddd2be812dcc566124315726a8a00d2a
Procedure:

Post a GIF using the new GIF keyboard
Check if posted GIF is animated
Restart app
Check if posted GIF is animated
Go to appearance settings, clear cache, and restart app right away (without going back to the feed)
Check if posted GIF is animated

Results:

GIF not animated on step 2
GIF not animated on step 4
GIF is animated on step 6

danieldaquino

Added some general comments

danieldaquino · 2024-08-27T23:19:52Z

damus/Views/GIFs/NostrBuildGIFGrid.swift

+    @State var results:[NostrBuildGif] = []
+    @State var cursor: Int = 0
+    @State var errorAlert: Bool = false
+    @SceneStorage("NostrBuildGIFGrid.show_nsfw_alert") var show_nsfw_alert : Bool = true


Do we need to persist show_nsfw_alert?

Yes this way users are not presented with the alert again.

damus/Models/GIFs/NostrBuildGIF.swift

damus/Views/GIFs/NostrBuildGIFGrid.swift

danieldaquino · 2024-08-27T23:52:45Z

damus/Views/GIFs/NostrBuildGIFGrid.swift

+            TopBar
+            ScrollView {
+                LazyVGrid(columns: columns, spacing: 5) {
+                    ForEach($results) { gifResult in


Personal opinion/optional comment: Instead of having results as an array that starts out empty and later gets filled up, and a separate errorAlert state, I would create a single local enum to track the state, with cases like "loading", "failure", "loaded(results: [NostrBuildGif])".

This helps differentiate the UI when there are no results vs when we are still loading results. It also simplifies state management, as an error on the NostrBuild request is interconnected with the GIF results

That makes sense, I might need some help with that.

I can help with that if you need anything, just let me know what you would need help with!

damus/Views/PostView.swift

danieldaquino · 2024-08-28T01:58:28Z

@ericholguin did another experiment — this time to try to find where the image caching is going wrong.

Here is the second experiment I performed:

Device: iPhone 15 simulator
iOS: 17.5
Damus: 2a4f4117ddd2be812dcc566124315726a8a00d2a
Procedure:

Open Damus on two simulator devices
Open the GIF keyboard on both devices (page 1)
Choose a GIF that appeared on both devices.
Post that GIF from one of the devices, and just watch the feed from the second device (Second account must follow the first)
Check if GIF is animated from the second device

Results: GIF is not animated on the second device

This experiment points in the direction that the caching issue is likely getting triggered from the KFAnimatedImage on the GIF keyboard view

danieldaquino · 2024-08-28T02:14:18Z

@ericholguin I believe I found a way to fix it!

diff --git a/damus/Views/GIFs/NostrBuildGIFGrid.swift b/damus/Views/GIFs/NostrBuildGIFGrid.swift
index b8319903..29604724 100644
--- a/damus/Views/GIFs/NostrBuildGIFGrid.swift
+++ b/damus/Views/GIFs/NostrBuildGIFGrid.swift
@@ -82,6 +82,7 @@ struct NostrBuildGIFGrid: View {
                             if let url = URL(string: gifResult.url.wrappedValue) {
                                 ZStack {
                                     KFAnimatedImage(url)
+                                        .imageContext(.note, disable_animation: damus_state.settings.disable_animation)
                                         .cancelOnDisappear(true)
                                         .configure { view in
                                             view.framePreloadCount = 3

This seems to do the trick for me. My test:

Device: iPhone 15 simulator
iOS: 17.5
Damus: 2a4f4117ddd2be812dcc566124315726a8a00d2a (the tip of this PR) with the patch above applied
Procedure:

Clear image cache (important) and restart app right away
Post using GIF keyboard
Check if GIF appears animated on the home feed on the device that posted it
Results: GIF is animated!

danieldaquino · 2024-08-28T02:22:21Z

Furthermore, if we wanted to fix GIF animation in the PostView itself (while the user is composing) (which does not seem very important anyways), here is a rough patch that seems to do the trick:

diff --git a/damus/Views/GIFs/NostrBuildGIFGrid.swift b/damus/Views/GIFs/NostrBuildGIFGrid.swift
index 29604724..cae33271 100644
--- a/damus/Views/GIFs/NostrBuildGIFGrid.swift
+++ b/damus/Views/GIFs/NostrBuildGIFGrid.swift
@@ -87,6 +87,7 @@ struct NostrBuildGIFGrid: View {
                                         .configure { view in
                                             view.framePreloadCount = 3
                                         }
+//                                        .imageModifier(ImageHandler(handler: $image))
                                         .clipShape(RoundedRectangle(cornerRadius: 12.0))
                                         .frame(width: 120, height: 120)
                                         .aspectRatio(contentMode: .fill)
diff --git a/damus/Views/PostView.swift b/damus/Views/PostView.swift
index 3b64b9ea..b8daae9a 100644
--- a/damus/Views/PostView.swift
+++ b/damus/Views/PostView.swift
@@ -7,6 +7,7 @@
 
 import SwiftUI
 import AVFoundation
+import Kingfisher
 
 enum NostrPostResult {
     case post(NostrPost)
@@ -335,7 +336,7 @@ struct PostView: View {
                 }
                 let blurhash = await blurhash
                 let meta = blurhash.map { bh in calculate_image_metadata(url: url, img: img, blurhash: bh) }
-                let uploadedMedia = UploadedMedia(localURL: media.localURL, uploadedURL: url, representingImage: img, metadata: meta)
+                let uploadedMedia = UploadedMedia(localURL: media.localURL, uploadedURL: url, metadata: meta)
                 uploadedMedias.append(uploadedMedia)
                 
             case .failed(let error):
@@ -468,12 +469,13 @@ struct PostView: View {
                         return
                     }
                     self.preUploadedMedia = PreUploadedMedia.processed_image(url)
-                    if let media = generateMediaUpload(preUploadedMedia) {
-                        let img = getImage(media: media)
+                    let fakeUploadedMedia = PreUploadedMedia.processed_image(URL(string: "https://image.nostr.build/51ccc5f8fc4ab5a70b2094afc8a3c5d0332ae53a90a8337a663062efecc4e000.gif")!)
+                    if let media = generateMediaUpload(preUploadedMedia), let fakeMedia = generateMediaUpload(fakeUploadedMedia) {
+                        let img = getImage(media: fakeMedia)
                         Task {
                             async let blurhash = calculate_blurhash(img: img)
                             let meta = await blurhash.map { bh in calculate_image_metadata(url: url, img: img, blurhash: bh) }
-                            let uploadedMedia = UploadedMedia(localURL: media.localURL, uploadedURL: url, representingImage: img, metadata: meta)
+                            let uploadedMedia = UploadedMedia(localURL: media.localURL, uploadedURL: url, metadata: meta)
                             uploadedMedias.append(uploadedMedia)
                         }
                     }
@@ -543,21 +545,18 @@ struct PVImageCarouselView: View {
     var body: some View {
         ScrollView(.horizontal, showsIndicators: false) {
             HStack {
-                ForEach(media.map({$0.representingImage}), id: \.self) { image in
+                ForEach($media, id: \.self) { media_item in
                     ZStack(alignment: .topTrailing) {
-                        Image(uiImage: image)
-                            .resizable()
+                        KFAnimatedImage(media_item.uploadedURL.wrappedValue)
                             .aspectRatio(contentMode: .fill)
                             .frame(width: media.count == 1 ? deviceWidth*0.8 : 250, height: media.count == 1 ? 400 : 250)
                             .cornerRadius(10)
                             .padding()
                             .contextMenu {
-                                if let uploadedURL = media.first(where: { $0.representingImage == image })?.uploadedURL {
-                                    Button(action: {
-                                        UIPasteboard.general.string = uploadedURL.absoluteString
-                                    }) {
-                                        Label(NSLocalizedString("Copy URL", comment: "Label for button in context menu to copy URL of the selected uploaded media asset."), image: "copy")
-                                    }
+                                Button(action: {
+                                    UIPasteboard.general.string = media_item.uploadedURL.wrappedValue.absoluteString
+                                }) {
+                                    Label(NSLocalizedString("Copy URL", comment: "Label for button in context menu to copy URL of the selected uploaded media asset."), image: "copy")
                                 }
                             }
                         Image("close-circle")
@@ -565,7 +564,7 @@ struct PVImageCarouselView: View {
                             .padding(20)
                             .shadow(radius: 5)
                             .onTapGesture {
-                                if let index = media.map({$0.representingImage}).firstIndex(of: image) {
+                                if let index = media.map({$0.uploadedURL}).firstIndex(of: media_item.uploadedURL.wrappedValue) {
                                     media.remove(at: index)
                                 }
                             }
@@ -611,11 +610,15 @@ fileprivate func getImage(media: MediaUpload) -> UIImage {
     return uiimage
 }
 
-struct UploadedMedia: Equatable {
+struct UploadedMedia: Equatable, Hashable {
     let localURL: URL
-    let uploadedURL: URL
-    let representingImage: UIImage
+    var uploadedURL: URL
     let metadata: ImageMetadata?
+    
+    func hash(into hasher: inout Hasher) {
+        hasher.combine(localURL)
+        hasher.combine(uploadedURL)
+    }
 }

However, I am not sure why we needed representingImage on UploadedMedia in the first place, and whether getting rid of it will cause other issues. @jb55 do you happen to know why we would need representingImage on UploadedMedia? Is fixing GIF animations on PostView an important thing in your view (or is it something that can be done separately)?

danieldaquino · 2024-08-28T02:27:54Z

@ericholguin btw the new GIF keyboard functionality is very cool 🔥🔥!

I hope my comments are helpful. Please let me know if you have any questions!

jb55 · 2024-08-29T09:00:53Z

this is fine but I would like to reiterate the same issue I had with adding nostr.build in the first place. once it goes away half of our app will be broken (image uploads, gif keyboard)

of course we can comb over this now but the proper solution is using protocols not platforms:

https://github.com/nostr-protocol/nips/blob/master/96.md (media uploads)
https://github.com/nostr-protocol/nips/blob/master/94.md (sourcing gifs)

This PR adds a gif keyboard, kind of, to the posting view. Leverages the nostr build API to get latest gifs. Changelog-Added: Nostr Build GIF keyboard Signed-off-by: ericholguin <[email protected]>

danieldaquino

LGTM!

danieldaquino · 2024-09-18T22:28:13Z

LGTM! Handing over to @jb55 if he has any concerns about these changes

alltheseas · 2024-09-23T16:51:50Z

if this is near/ready, would be a fun testflight addition in the near term

jb55

sure lets try it

danieldaquino

Adding this review change request as a reminder for myself that we still need to implement search and NIP-94 support before merging this PR (from our standup discussions)

Please feel free to correct me if I am wrong. Thank you!

alltheseas · 2024-10-04T17:51:47Z

@fishcakeday yall adding search this quarter 👀

jb55 · 2024-10-08T07:40:36Z

On Fri, Oct 04, 2024 at 10:46:18AM -0700, Daniel D’Aquino wrote: @danieldaquino requested changes on this pull request. Adding this review change request as a reminder for myself that we still need to implement search and NIP-94 support before merging this PR (from our standup discussions) Please feel free to correct me if I am wrong. Thank you!

if we're doing a centralized integration I don't see why we don't just use tenor or something

ericholguin requested a review from danieldaquino August 23, 2024 23:34

danieldaquino reviewed Aug 28, 2024

View reviewed changes

gif: Nostr Build GIF Keyboard

5caec85

This PR adds a gif keyboard, kind of, to the posting view. Leverages the nostr build API to get latest gifs. Changelog-Added: Nostr Build GIF keyboard Signed-off-by: ericholguin <[email protected]>

ericholguin force-pushed the the-infamous-gif-keyboard branch from 2a4f411 to 5caec85 Compare September 12, 2024 03:47

danieldaquino approved these changes Sep 18, 2024

View reviewed changes

danieldaquino requested a review from jb55 September 18, 2024 22:27

alltheseas added this to the 1.11 - lists milestone Sep 23, 2024

jb55 approved these changes Sep 24, 2024

View reviewed changes

danieldaquino requested changes Oct 4, 2024

View reviewed changes

alltheseas mentioned this pull request Oct 14, 2024

Verifiable media [Feature Request] #994

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gif: Nostr Build GIF Keyboard #2387

gif: Nostr Build GIF Keyboard #2387

ericholguin commented Aug 23, 2024

danieldaquino commented Aug 27, 2024

danieldaquino left a comment

danieldaquino Aug 27, 2024

ericholguin Sep 12, 2024

danieldaquino Aug 27, 2024

ericholguin Sep 12, 2024

danieldaquino Sep 17, 2024

danieldaquino commented Aug 28, 2024

danieldaquino commented Aug 28, 2024

danieldaquino commented Aug 28, 2024

danieldaquino commented Aug 28, 2024

jb55 commented Aug 29, 2024

danieldaquino left a comment

danieldaquino commented Sep 18, 2024

alltheseas commented Sep 23, 2024

jb55 left a comment

danieldaquino left a comment

alltheseas commented Oct 4, 2024

jb55 commented Oct 8, 2024 via email

gif: Nostr Build GIF Keyboard #2387

Are you sure you want to change the base?

gif: Nostr Build GIF Keyboard #2387

Conversation

ericholguin commented Aug 23, 2024

danieldaquino commented Aug 27, 2024

danieldaquino left a comment

Choose a reason for hiding this comment

danieldaquino Aug 27, 2024

Choose a reason for hiding this comment

ericholguin Sep 12, 2024

Choose a reason for hiding this comment

danieldaquino Aug 27, 2024

Choose a reason for hiding this comment

ericholguin Sep 12, 2024

Choose a reason for hiding this comment

danieldaquino Sep 17, 2024

Choose a reason for hiding this comment

danieldaquino commented Aug 28, 2024

danieldaquino commented Aug 28, 2024

danieldaquino commented Aug 28, 2024

danieldaquino commented Aug 28, 2024

jb55 commented Aug 29, 2024

danieldaquino left a comment

Choose a reason for hiding this comment

danieldaquino commented Sep 18, 2024

alltheseas commented Sep 23, 2024

jb55 left a comment

Choose a reason for hiding this comment

danieldaquino left a comment

Choose a reason for hiding this comment

alltheseas commented Oct 4, 2024

jb55 commented Oct 8, 2024 via email