Add exporting to Denote (#161)

* ekg-denote export changes (#144) Adds export from ekg to denote. Authored-by: Jay Rajput * fix #146 (#148) Co-authored-by: Jay Rajput * Note new denote export in docs * Add denote as an extra dependency * denote-export: linting and test failures (#155) * documentation string updates for linting errors. * documentation string updates for linting errors * update function names to start with ekg-denote-test * updates for denote changes --------- Co-authored-by: Jay Rajput <[email protected]> * Allow emoji tags (#156) This should fix #149. * Set version number to 0.6.1 (#157) * Bump version to 0.7.0 * Rethink version number, let's call the new release 0.6.1 It isn't quite a new release, even though it has some minor functionality / UI changes (how tag actions happen). * Enable the CI for the develop branch (#158) --------- Co-authored-by: Jay Rajput <[email protected]> Co-authored-by: Jay Rajput <[email protected]>
ahyatt · Jul 28, 2024 · c31a6e7 · c31a6e7
1 parent aa481fa
commit c31a6e7
Show file tree

Hide file tree

Showing 5 changed files with 523 additions and 0 deletions.
diff --git a/Eldev b/Eldev
@@ -3,3 +3,4 @@
 (eldev-use-package-archive 'gnu-elpa)
 (eldev-use-plugin 'maintainer)
 (eldev-add-extra-dependencies 'test '(:package markdown-mode :archive melpa))
+(eldev-add-extra-dependencies 'test 'denote)
diff --git a/doc/ekg.org b/doc/ekg.org
@@ -78,6 +78,7 @@ triples library is already installed.
 - Run tag hooks when a valid tag is typed, instead of relying on completion, which often doesn't work.
 - Fix issue in =markdown-mode= where the beginning of the tag values was incorrectly read-only.
 - Allow emojis as tags.
+- Add export to [[https://github.com/protesilaos/denote][denote]] (by [[https://github.com/jayrajput][Jay Rajput]])
 ** Version 0.6.0
 - Add tag and similar notes to the context when having LLMs add to or replace
   notes.  ~ekg-llm-default-instructions~ replaces the previous variable,
@@ -1110,6 +1111,60 @@ To start using this, you need to require the module and turn on =ekg-auto-save-m
 #+end_src
 
 In the capture mode, this. is equivalent to saving periodically (to drafts).  In edit mode, it will save the latest version while editing.
+** Denote
+:PROPERTIES:
+:CUSTOM_ID: denote
+:END:
+ekg can export notes as denotes. Denote is a note taking and file
+naming tool. Primary reason for export is taking backup of your notes
+in a git backed repository. Import is in road map (PR is welcome). To
+export to denote, =ekg-denote= module is required.
+
+#+begin_src emacs-lisp
+  (use-package ekg-denote
+    :init
+    (setq ekg-denote-export-add-front-matter nil)
+    (setq ekg-denote-export-backup-on-conflict t)
+    (setq ekg-denote-export-title-max-len 50)
+    (setq ekg-denote-export-combined-keywords-len 150))
+#+end_src
+
+To export, call =ekg-denote-export= which will export any modified
+note after the last export as a denote in the =denote-directory=
+defined by =denote= package. If it is your first time exporting, all
+the notes will be exported to the =denote-directory=.
+
+=ekg-denote= keeps record of the last export time in the ekg db and
+use that to find the notes changed since last export. This way the
+exports are much faster. It is suggested to export your notes every
+day.
+
+User can optionally enable adding of denote front-matter to exported
+denotes by setting =ekg-denote-export-add-front-matter= to
+=t=. Denote front matter is added using =denote-add-front-matter=
+function defined by =denote= package which open note in an emacs
+buffer and requires manual execution of =save-buffer= by the user.
+
+Ekg and denote have differences, due to which following customization
+are made available:
+
+- =ekg-denote-export-title-max-len= to trim the title during export. Default
+  is 50 characters.
+- =ekg-denote-export-combined-keywords-len= to trim the combined length of
+  keywords. Default is 150 characters.
+- =ekg-denote-export-backup-on-conflict= to backup the denote if both
+  ekg and denote are found to be updated after last export. Default is
+  =t=.
+
+It is user responsibility to backup the denotes before and after
+export to protect against accidental deletes. This can be easily done
+by keeping denotes in a git repository and making sure to check-in any
+changes before and after export.
+
+If ekg and denote are both found to be updated after the last export
+which should ideally not happen, denote is updated with ekg. A backup
+is taken based on =ekg-denote-export-backup-on-conflict= setting.
+
 * Design
 ** The triple database
 The ekg package uses the triples package to interface with a sqlite database.

diff --git a/doc/ekg.texi b/doc/ekg.texi
@@ -107,6 +107,7 @@ Extras
 * Logseq::
 * LLM::
 * Auto save::
+* Denote::
 
 Logseq
 
@@ -227,6 +228,8 @@ Run tag hooks when a valid tag is typed, instead of relying on completion, which
 Fix issue in @samp{markdown-mode} where the beginning of the tag values was incorrectly read-only.
 @item
 Allow emojis as tags.
+@item
+Add export to @uref{https://github.com/protesilaos/denote, denote} (by @uref{https://github.com/jayrajput, Jay Rajput})
 @end itemize
 
 @node Version 060
@@ -1174,6 +1177,7 @@ package.
 * Logseq::
 * LLM::
 * Auto save::
+* Denote::
 @end menu
 
 @node Embeddings
@@ -1533,6 +1537,64 @@ To start using this, you need to require the module and turn on @samp{ekg-auto-s
 
 In the capture mode, this. is equivalent to saving periodically (to drafts).  In edit mode, it will save the latest version while editing.
 
+@node Denote
+@section Denote
+
+ekg can export notes as denotes. Denote is a note taking and file
+naming tool. Primary reason for export is taking backup of your notes
+in a git backed repository. Import is in road map (PR is welcome). To
+export to denote, @samp{ekg-denote} module is required.
+
+@lisp
+(use-package ekg-denote
+  :init
+  (setq ekg-denote-export-add-front-matter nil)
+  (setq ekg-denote-export-backup-on-conflict t)
+  (setq ekg-denote-export-title-max-len 50)
+  (setq ekg-denote-export-combined-keywords-len 150))
+@end lisp
+
+To export, call @samp{ekg-denote-export} which will export any modified
+note after the last export as a denote in the @samp{denote-directory}
+defined by @samp{denote} package. If it is your first time exporting, all
+the notes will be exported to the @samp{denote-directory}.
+
+@samp{ekg-denote} keeps record of the last export time in the ekg db and
+use that to find the notes changed since last export. This way the
+exports are much faster. It is suggested to export your notes every
+day.
+
+User can optionally enable adding of denote front-matter to exported
+denotes by setting @samp{ekg-denote-export-add-front-matter} to
+@samp{t}. Denote front matter is added using @samp{denote-add-front-matter}
+function defined by @samp{denote} package which open note in an emacs
+buffer and requires manual execution of @samp{save-buffer} by the user.
+
+Ekg and denote have differences, due to which following customization
+are made available:
+
+@itemize
+@item
+@samp{ekg-denote-export-title-max-len} to trim the title during export. Default
+is 50 characters.
+@item
+@samp{ekg-denote-export-combined-keywords-len} to trim the combined length of
+keywords. Default is 150 characters.
+@item
+@samp{ekg-denote-export-backup-on-conflict} to backup the denote if both
+ekg and denote are found to be updated after last export. Default is
+@samp{t}.
+@end itemize
+
+It is user responsibility to backup the denotes before and after
+export to protect against accidental deletes. This can be easily done
+by keeping denotes in a git repository and making sure to check-in any
+changes before and after export.
+
+If ekg and denote are both found to be updated after the last export
+which should ideally not happen, denote is updated with ekg. A backup
+is taken based on @samp{ekg-denote-export-backup-on-conflict} setting.
+
 @node Design
 @chapter Design
 

diff --git a/ekg-denote-test.el b/ekg-denote-test.el
@@ -0,0 +1,187 @@
+;;; ekg-denote-test.el --- Tests for ekg-denote  -*- lexical-binding: t; -*-
+
+;; Copyright (c) 2024  Jay Rajput <[email protected]>
+
+;; This program is free software; you can redistribute it and/or
+;; modify it under the terms of the GNU General Public License as
+;; published by the Free Software Foundation; either version 2 of the
+;; License, or (at your option) any later version.
+;;
+;; This program is distributed in the hope that it will be useful, but
+;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+;; General Public License for more details.
+;;
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; These tests should be run and pass before every commit.
+
+;;; Code:
+
+(require 'ert)
+(require 'ekg-test-utils)
+(require 'ekg-denote)
+(require 'cl-lib)
+
+(ert-deftest ekg-denote-test-sublist-keywords ()
+  "Combined length of the sublist keywords is as per the allowed combined length argument."
+  (should (equal '("kw1") (ekg-denote-sublist-keywords '("kw1" "kw2" "kw 3") 4)))
+  (should (equal '("kw1" "kw2") (ekg-denote-sublist-keywords '("kw1" "kw2" "kw 3") 8)))
+  (should (equal '("kw1" "kw2" "kw 3") (ekg-denote-sublist-keywords '("kw1" "kw2" "kw 3") 13))))
+
+(ert-deftest ekg-denote-test-notes-having-duplicate-creation-time ()
+  "Notes with duplicate creation time should error."
+  (let* ((time (time-convert (current-time) 'integer))
+	 (note1 (make-ekg-note :id "ID1" :creation-time time))
+	 (note2 (make-ekg-note :id "ID2" :creation-time time))
+	 (notes (list note1 note2)))
+    (should-error (ekg-denote-assert-notes-have-unique-creation-time notes))))
+
+(ert-deftest ekg-denote-test-notes-having-unique-creation-tiem ()
+  "Notes with unique creation time should not error."
+  (let* ((time (time-convert (current-time) 'integer))
+	 (note1 (make-ekg-note :id "ID1" :creation-time time))
+	 (note2 (make-ekg-note :id "ID2" :creation-time (1+ time)))
+	 (notes (list note1 note2)))
+    (should-not (ekg-denote-assert-notes-have-unique-creation-time notes))))
+
+(ert-deftest ekg-denote-test-notes-missing-creation-time ()
+  "Notes missing creation time should error."
+  (let* ((time (time-convert (current-time) 'integer))
+	 (note1 (make-ekg-note :id "ID1" :creation-time time))
+	 (note2 (make-ekg-note :id "ID2")))
+    (should-error (ekg-denote-assert-notes-have-creation-time notes))))
+
+(ert-deftest ekg-denote-test-notes-not-missing-creation-time ()
+  "Notes having creation time should not error."
+  (let* ((time (time-convert (current-time) 'integer))
+	 (note1 (make-ekg-note :id "ID1" :creation-time time))
+	 (note2 (make-ekg-note :id "ID2" :creation-time (1+ time)))
+	 (notes (list note1 note2)))
+    (should-not (ekg-denote-assert-notes-have-creation-time notes))))
+
+(ert-deftest ekg-denote-test-create ()
+  "Verify creation of `ekg-denote' against given ekg note."
+
+  ; date tag is removed.
+  ; text defaults to ""
+  ; ext defaults to ekg-default-capture-mode
+  ; title defaults to ""
+  ; path uses denote-id and tags
+  ; ekg-id is copied
+  (let* ((time (time-convert (current-time) 'integer))
+	 (denote-directory "/tmp")
+	 (denote-id (format-time-string denote-id-format time))
+	 (ekg-default-capture-mode 'org-mode)
+	 (note (make-ekg-note :id "ID1"
+			      :creation-time time
+			      :tags '("date/20230101" "portfolio" "tag2")))
+	 (denote (make-ekg-denote :id denote-id
+				  :note-id "ID1"
+				  :title ""
+				  :text ""
+				  :path (format "/tmp/%s__portfolio_tag2.org" denote-id )
+				  :kws '("portfolio" "tag2"))))
+    (should (equal denote (ekg-denote-create note))))
+
+  ;; scalar title values shall work
+  (let* ((time (time-convert (current-time) 'integer))
+	 (denote-directory "/tmp")
+	 (denote-id (format-time-string denote-id-format time))
+	 (ekg-default-capture-mode 'org-mode)
+	 (note (make-ekg-note :id "ID1"
+			      :creation-time time
+			      :text "Text"
+			      :properties `(:titled/title "MyTitle")
+			      :tags '("date/20230101" "portfolio")))
+	 (denote (make-ekg-denote :id denote-id
+				  :note-id "ID1"
+				  :title "mytitle"
+				  :text "Text"
+				  :path (format "/tmp/%s--mytitle__portfolio.org" denote-id )
+				  :kws '("portfolio"))))
+    (should (equal denote (ekg-denote-create note))))
+
+  ;; text is copied as it is
+  ;; title is truncated, copied and sluggified
+  (let* ((time (time-convert (current-time) 'integer))
+	 (denote-directory "/tmp")
+	 (denote-id (format-time-string denote-id-format time))
+	 (ekg-default-capture-mode 'org-mode)
+	 (ekg-denote-export-title-max-len 10)
+	 (note (make-ekg-note :id "ID1"
+			      :creation-time time
+			      :text "Text"
+			      :properties `(:titled/title ,(list "Title 123456789"))
+			      :tags '("date/20230101" "portfolio")))
+	 (denote (make-ekg-denote :id denote-id
+				  :note-id "ID1"
+				  :title "title-1234"
+				  :text "Text"
+				  :path (format "/tmp/%s--title-1234__portfolio.org" denote-id )
+				  :kws '("portfolio"))))
+    (should (equal denote (ekg-denote-create note)))))
+
+
+(defun ekg-test--matching-denote (regexp)
+  "Get denote file name containing REGEXP.
+Enforces single match."
+  (let ((files (seq-filter (lambda (x) (string-match-p regexp x))
+			   (denote-directory-files nil nil :text-only))))
+    (when files (car files))))
+
+(defun ekg-test--denote-text (file)
+  "Get text from denote FILE."
+  (with-temp-buffer
+    (insert-file-contents file)
+    (buffer-string)))
+
+(defun ekg-test--creation-time-from-denote-file (file)
+  "Return creation time for the denote FILE."
+  (time-convert
+   (encode-time
+    (parse-time-string
+     (denote-retrieve-filename-identifier file)))
+   'integer))
+
+(defun ekg-test--denote-ekg (file)
+  "Return ekg note for given denote FILE."
+  (let* ((creation-time (ekg-test--creation-time-from-denote-file file))
+	 (triples (triples-db-select-pred-op ekg-db :time-tracked/creation-time '= creation-time)))
+    (should (length= triples 1))
+    (ekg-get-note-with-id (car (car triples)))))
+
+(ekg-deftest ekg-denote-test-export ()
+  "Verify export."
+  (let ((denote-directory (make-temp-file "denote" t)))
+    ;; ekg note creations are exported
+    (ekg-save-note (ekg-note-create :text "text1" :mode 'org-mode :tags '("portfolio")))
+    (sleep-for 1)
+    (ekg-denote-export)
+    (setq denote-file (ekg-test--matching-denote "__portfolio"))
+    (should (equal "text1" (ekg-test--denote-text denote-file)))
+    ;; ekg note updates are exported
+    (setq note (ekg-test--denote-ekg denote-file))
+    (setf (ekg-note-text note) "text2")
+    (ekg-save-note note)
+    (sleep-for 1)
+    (ekg-denote-export)
+    (setq denote-file (ekg-test--matching-denote "__portfolio"))
+    (should (equal "text2" (ekg-test--denote-text denote-file)))
+    ;; ekg note tag update cause denote file rename
+    (setq note (ekg-test--denote-ekg denote-file))
+    (setf (ekg-note-tags note) '("updatedtag"))
+    (ekg-save-note note)
+    (sleep-for 1)
+    (ekg-denote-export)
+    (should (ekg-test--matching-denote "__updatedtag"))
+    (should-not (ekg-test--matching-denote "__portfolio"))))
+
+(ekg-deftest ekg-denote-test-last-export ()
+  "Verify last export time updates."
+  (should (= 0 (ekg-denote-get-last-export)))
+  (ekg-denote-set-last-export 123)
+  (should (= 123 (ekg-denote-get-last-export))))