Timestamp ranges not matched the same way org-agenda does

[yantar92]

I have a headline like the following:

***** NEXT [#A] Manuel Macías [Email] (2021) "Continuous Integration and TeX with Org-Mode", by Rohit Goswani (TUG 2021) :BOOKMARK:misc:email:
SCHEDULED: <2021-08-07 Sat 16:30-17:30>

I expect this headline to be matched by (scheduled 0) sexp, but it is not.

[alphapapa]

I may be mistaken, but I don't think it's valid to use a range timestamp as an entry's SCHEDULED timestamp.

[yantar92]

I may be mistaken, but I don't think it's valid to use a range timestamp as an entry's SCHEDULED timestamp.

It is indeed discouraged in the manual. However, org-agenda does show scheduled entries with scheduled timestamp being a time interval. Also, `ts` predicate will not match timestamps with interval (see `org-ql-regexp-part-ts-time').

[alphapapa]

I may be mistaken, but I don't think it's valid to use a range timestamp as an entry's SCHEDULED timestamp.

Well, I guess we'll have to do it too, then. :)

It is indeed discouraged in the manual. However, org-agenda does show scheduled entries with scheduled timestamp being a time interval. Also, ts predicate will not match timestamps with interval (see `org-ql-regexp-part-ts-time').

Hm, I thought I had that covered, but maybe I overlooked it. Thanks.

[alphapapa]

Retargeting this for 0.7. 0.6 has been delayed for too long.

[stfl]

It's also relevant for ts-active timestamps. In particular this shows up for me when syncing my calendar with org-gcal which will create active ranged timestamps.

[enko]

Anything hindering this pr to be merged?

[alphapapa]

@enko Well, yes: since this has been a bug, the fix should be accompanied by explicit tests, i.e. there should be a new test added for timestamp ranges, rather than just changing one of the test data entries, as the PR currently does. Maybe even multiple tests for active/inactive timestamps, in planning/deadline/scheduled lines, and in headings/content.

It also needs a changelog entry--which I typically take care of myself when merging, but if a contributor provides it in the patch, it saves me the time and makes it easier for me to merge (assuming it's done correctly; even when I ask someone to provide one, and they do, it usually contains typographical errors that I have to fix myself, which defeats the purpose).

Other than that, the limitation is time, which I don't have as much of as I used to.

In the meantime, you're free to use the PR's code and report back as to how well it works. Any bugs you could discover before merging would be helpful.

[alphapapa]

As well, fixing this properly probably requires more work, because a ranged timestamp essentially represents two timestamps, one at each end of the range, and searching for timestamps within specific ranges must account for both of them.

[dmitrym0]

Do you think it's possible to split this is up into two tasks:

1. getting ranged timestamp to match scheduled and planning
2. searching for a timestamp in a range.

I came here look for 1, looks like I'm not the only one. Would be happy to contribute tests for it as well and improve the original request.

[magogware]

I tried out your most recent commit @dmitrym0 and had no luck with either the scheduled predicate or the ts-active predicate.

[dmitrym0]

@magogware

Here's a part of my test data:

* TODO This heading should be there because it contains scheduling info with time
SCHEDULED: <2026-01-23 Sun 14:00-15:00>
:PROPERTIES:
:ID:       FAKE_ID1
:END:

and the associated org-ql query that returns it:

'(or
  (ts-active)
  (clocked)
  (and
   (todo "TODO" "DONE")
   (or
    (scheduled) (planning))))

[magogware]

Ack, sorry. A (Doom) Emacs restart ensured I was actually using the package built from your commit, and now everything works. Cheers! It all works a charm.

[dmitrym0]

Can't take credit for it; so thanks to @yantar92. Of course caveat emptor since this hasn't been properly tested.

[alphapapa]

@yantar92 FYI, it appears that something in the Buttercup library has changed, and now the test suite goes into an infinite loop and never terminates. Obviously, this presents a bit of a roadblock to developing this and other fixes and features. And my time and enthusiasm for debugging a third-party library that used to work fine is limited.

I've half a mind to convert all the tests to use ERT, but I like Buttercup's nice-looking, grouped and colorized output. Maybe what's needed is some kind of lightweight wrapper around ERT to use ERT as a backend while providing nicer output and test grouping.

Anyway, if anyone would like to dig into Buttercup's recent changes and figure out why it's broken, that would certainly be appreciated. Otherwise, it seems that I will have to find time to do some serious yak shaving (or skinning, even).

[ivanperez-keera]

I've now added this change to my installation of org-ql (merging it individually as I have org-ql installed with elpa).

I have a file with more than 15 years worth of history. It won't be representative of all possible cases, but it's also not a trivial test.

It seems to be working well for me so far. I'll report back if I notice any issues.

[alphapapa]

I've now added this change to my installation of org-ql (merging it individually as I have org-ql installed with elpa).

I have a file with more than 15 years worth of history. It won't be representative of all possible cases, but it's also not a trivial test.

It seems to be working well for me so far. I'll report back if I notice any issues.

Thank you, I would appreciate that. My time for these projects has been limited lately, and ones that require careful review and testing like this are the hardest to find time for.

[ParetoOptimalDev]

I've now added this change to my installation of org-ql (merging it individually as I have org-ql installed with elpa).

I have a file with more than 15 years worth of history. It won't be representative of all possible cases, but it's also not a trivial test.

It seems to be working well for me so far. I'll report back if I notice any issues.

@ivanperez-keera If you can push your changes up, I can also test with 10+ years of history.

[ivanperez-keera]

I'm not sure how to do that (I understand git, it's just that I'm using the version provided by ELPA so I don't know how that compares to your version).

It's only two lines that need to be adjusted. See: https://github.com/alphapapa/org-ql/pull/237/files#diff-34013f28e9b645ff9d9a6dab1c4f0ca707ae51e0361da84126e8440810452773R162-R163

It's quite easy to just add the change manually.

[ParetoOptimalDev]

It's only two lines that need to be adjusted. See: https://github.com/alphapapa/org-ql/pull/237/files#diff-34013f28e9b645ff9d9a6dab1c4f0ca707ae51e0361da84126e8440810452773R162-R163

This does match (todo) which wasn't before, but it doesn't seem to work with (scheduled :to today) or (ts-active :on today).

For example with that change this data:

*** TODO 15 minutes, standup
   SCHEDULED: <2024-06-24 Mon 09:00-09:15 ++7d>

Doesn't match (or (scheduled :to today) (ts-active :on today)).

[alphapapa]

@ParetoOptimalDev It works for me. It's likely that you did not load the changed library file correctly. i.e. defvar forms do not change the values of already-defined variables unless you individually set them with C-M-x, and doing that to one does not do that to others that depend on its value.

[alphapapa]

v0.8.7 will allow such ranges to be matched simply, but matching depending on the specific inner time ranges will not work yet. That will require some refactoring, and so it will be done in a future version (maybe v0.9, maybe not).