Binding data before a predicate #88
Comments
|
Hello, I am not the author of this package, but it looks like this issue is about extending Alternatively, you might be able to improve the performance using memoization. |
|
@akirak Thanks! I hadn't read about memoize before. The following changes made it run much faster: |
|
Hi Erik, Aha, welcome to the Regarding the idea of binding data around a query: that's an idea I've thought about, although I can't seem to find any notes I've made about it. Generally, I think it would be helpful for data that is specific to each buffer searched in a query, like per-buffer to-do keywords. Caching data like that would avoid having to retrieve it every time the predicate is called. For your specific use case, I think it's not necessary. I looked at your code and I think you can easily optimize it with a few changes. Here are some thoughts:
For example, instead of defining this predicate: (org-ql--defpred org-brain ()
"Return non-nil if entry could be a part of `org-brain'.
Entry should have an ID and `org-brain-entry-at-point-excludedp' should return nil."
(and (org-entry-get (point) "ID")
(not (org-brain-entry-at-point-excludedp))))Which you then use in (defun org-brain-headline-entries ()
"Get all org-brain headline entries."
(org-ql-select (org-brain-files) '(org-brain)
:action #'org-brain--headline-entry-at-point))Use existing predicates in a query, like: (defun org-brain-headline-entries ()
"Get all org-brain headline entries."
(org-ql-select (org-brain-files)
'(and (property "ID")
(not (org-brain-entry-at-point-excludedp)))
:action #'org-brain--headline-entry-at-point))That should be faster, because the (and (regexp "^:ID: org-brain-")
(property "ID") ; Ensure the match is actually a property in a drawer
(not (org-brain-entry-at-point-excludedp)))Also, when possible, prefer to do as much work as possible in the
Do you understand what I mean? Let me know if I should explain more or more clearly. :) BTW, regarding |
|
Also, @akirak: Yes, ideally |
|
@alphapapa Thank you for the comment. I will keep it in mind. Also, I liked memoization because memoize.el is so easy to use and convenient, but I didn't know of that discussion. Thank you for the info. |
|
Ah, here's the note I was thinking of: Line 212 in d28d54c |
|
I've been sitting for some hours now trying different ways to make What I realized though was that it didn't matter much what I put into the query or action, it took about the same time. The most stripped down version I've tried is this: This takes about 20 seconds the first time it is run. It scans 25 files and finds 224 matches. Is that the expected speed? I have only tried it on my Windows config. During the first run I get lots of messages in the mini-buffer: |
|
No, 25 files and 224 matches should be collected much faster than that. That fontification error appears to come from It's hard to say what is causing the fontification errors. Once in a while I see those errors in my own config when doing random Org-related things. You might have some Org files with invalid syntax, so you might try using |
|
I deactivated a package named This gets me down to 0.13 seconds! I've had similar problems during the development of The downside here is ofcourse that the mode-hooks aren't run. If I enable them I get about 1.9 seconds on the first call. |
|
Yeah, it sounds like your config has Org doing a lot of initialization in the mode hooks. Even some built-in Org modules can be slow to initialize in large files, like So I'd recommend testing by writing a script to do something like this:
Also, I recommend using this macro for your benchmarking: https://github.com/alphapapa/emacs-package-dev-handbook#bench-multi-lexical It lets you compare forms, and it uses lexical-binding, which can make a big difference (and you always want your "production" code to use lexical-binding anyway). |
|
There's also a lot of stuff in |
|
The 3 caches are initialized here: Line 85 in fb830f8 defvar forms (make sure to do all 3), or the equivalent.
|
Happy new year! I've experimented with adding
org-qltoorg-brain, and it has been a huge performance boost. See commit here (if you're interested): Kungsgeten/org-brain@3160386I had an idea to provide
org-qlpredicates for some oforg-brainspecific things, in case users want to query theirorg-braincontent. Here's an example:This works, but is slow since the
let-clause has to be run for every headline in the query. In reality thelet-clause is only needed to be run once, before the search starts, and then the results could be used in each try.It would be nice to have a way to bind data before the search starts, and have that data be available in the
defpred. Perhaps something like this:It could be though that this won't be needed in any other use case, and thus doesn't fit into
org-ql. Another posibility could be for theBUFFERS-OR-FILESargument inorg-ql-selectto also allow a list of markers to headlines.The text was updated successfully, but these errors were encountered: