Only partial import & no notes?

[timbruls]

My highlights are only partially imported (about 50 books with highlights) and notes are not visible at all? Still very happy that I now have a direct way and interested in where the plugin goes from here!

[bitwiseops]

Hello!

Unfortunately, notes were not in my mind while I was developing this plugin. I could add this feature request in the TODOs.

About highlights, can you provide more info?

For example

• I can't see any highlights from a book or I can only see 70% of highlights from a single book
• Books from which I couldn't extract highlights are in PDF format

Thanks

[timbruls]

Yes please add it to TODO if possible,

I can see all the highlights from individual books, but not all the books with highlights. None are in PDF, it is all Epub for me (or in some cases the kobo proprietary epub format

[bitwiseops]

So, something goes wrong with some book "types" ...

At this point, you could check by yourself why some books are not considered during the extraction ( the whole process revolves around this query )

I can help you from here if you need support in doing this.

( I didn't want to inspect your sqlite DB beacuse I don't want you to share it publicly... )

[timbruls]

I will play around with the query. If i Need help I will ask. Thanks again for the plugin,

[timbruls]

You are welcome to my DB if it helps you, Is there a non public way to get it to you?

I see nothing out of the ordinary in the query, but I am not well versed in SQL, it has been a long time since I did anything with that.
Running the query in DB explorer for SQLite gets 70 results (each result is one highlight in a book). When manually filtering in the DB I get 543 highlights

I see no error or anything to explain why it caps at 70

[bitwiseops]

Unfortunately there is no way to send direct messages to Github users, the only way to communicate is through projects.

There is something wrong with my query because I built it looking only on my specific database...
My goal was to extract highlights along with some metadata ( for example chapter title ) that are not easy to link. What I mean is that I couldn't find a better pattern than the one I used to build the query.
Connection between an highlight and a book is made by joining Bookmark and content tables. If there is no match between key columns i used to make this join, some books may be filtered out.

Other tools, such as this one, might help you to extract all highlights, but without metadata I was mentioning before. Can you please check if using this tool you could extract all your highlights?

[timbruls]

I am not sure the issue is in the query per se, but then again there should not be a difference in the DB structure between two kobo's to account for the difference in output.

[bitwiseops]

Yes I agree with you, DB model should be the same among devices.

Okay, the only way left to understand what's happening is to inspect data... Without sharing the whole sqlite file, could be ok if you provide only

• content table
• Bookmark table

in csv fromat.

Before doing this, check if no sensible data goes in your outputs. Remember that you're doing this at your own risk, I don't take any responsibility.

[caalberts]

SELECT T2.ContentID as ID, BookTitle, Title as ChapterTitle, Text as Highlight, T2.DateCreated, T2.DateModified, VolumeIndex, StartContainerPath, EndContainerPath, SubChapters
FROM (SELECT *, group_concat(Title, '-') as SubChapters FROM content GROUP BY ChapterIDBookmarked ORDER BY VolumeIndex) as T1 INNER JOIN Bookmark as T2 ON T1.ChapterIDBookmarked = T2.ContentID
GROUP BY T2.BookmarkID 
ORDER BY ChapterIDBookmarked, ChapterProgress

@bitwiseops I think the IDs used to join the 2 tables should be content.ContentId = Bookmark.VolumeId.

SELECT T2.VolumeID as ID, BookTitle, Title as ChapterTitle, Text as Highlight, T2.DateCreated, T2.DateModified, VolumeIndex, StartContainerPath, EndContainerPath, SubChapters
FROM (SELECT *, group_concat(Title, '-') as SubChapters FROM content GROUP BY ContentId ORDER BY VolumeIndex) as T1 INNER JOIN Bookmark as T2 ON T1.ContentId = T2.VolumeID
GROUP BY T2.BookmarkID 
ORDER BY T1.ContentId, ChapterProgress

With this query, I was able to retrieve more highlights.

Retrieving the note is a matter of adding Annotation into SELECT.

[OGKevin]

General update: this is on my list to try and implement in my fork :)

context: obsidianmd/obsidian-releases#1110

[OGKevin]

The partial import issue has been fixed in 1.2.0!

[OGKevin]

Notes export is now included in 1.3.0.

@bitwiseops if you could please close this one as well 🙏🏾