Train was missing in tstp

[PaulSzymanski]

While sitting in a delayed train that would miss the connecting train I was looking for an alternative route with TSTP.
Fortunately, I was also checking DB Navigator as it was giving me a connection that was not displayed in TSTP.
The train that was missing was the RE1 (4342) which was scheduled to leave at 19:19 from Koblenz and arriving 20:40 in Trier.
Interestingly the RE1 one hour later (20:19) was not missing in TSTP.

The navigator and TSTP refer to the Train as RE1. The train itself displays RE11. I asked the train attendee, she said that the train is actually called RE1 up to Trier and is called RE11 afterwards.

Can we figure out what went wrong here?

[traines-source]

I have looked into this, the issue is that apparently HAFAS for some trains gives out different trip IDs for the same train on the departure and arrival boards. E.g. the train you were talking about today toggles between
2|#VN#1#ST#1721675456#PI#0#ZI#275247#TA#0#DA#240724#1S#8000206#1T#1919#LS#8000134#LT#2040#PU#80#RT#1#CA#RE#ZE#4342#ZB#RE 4342#PC#3#FR#8000206#FT#1919#TO#8000134#TT#2040#

on arrival and

2|#VN#1#ST#1721675456#PI#0#ZI#1682887#TA#0#DA#240724#1S#8000206#1T#1919#LS#8200100#LT#2129#PU#80#RT#1#CA#RE#ZE#4342#ZB#RE 4342#PC#3#FR#8000206#FT#1919#TO#8200100#TT#2129#

on departure. This is not the case for all trains, I suspect it is somehow related to the splitting/renaming or the fact that it goes till Luxembourg.
TSTP is using these trip IDs to merge the trip from the individual departure/arrival boards, so that fails...
I have now experimentally switched to hafas-client's fahrtNr plus line.ID instead. This is only unique across one day as far as I can tell, so this may on the contrary lead to too many connections displayed by TSTP... Maybe HAFAS' trainStartDate could help here, but that would first need to be exposed upstream in hafas-client.
So let's try it like this for the time being, it's already deployed and Koblenz-Trier works. Thanks for reporting!
(Btw due to that renaming in Trier you would still need to add Trier as an interchange in order for the connection to be displayed e.g. from Koblenz to Luxembourg. Time for better automatic interchange finding...)

[iw0nderhow]

The fahrtNr+line.ID combo, as implemented, seems to break light rail and most buses in Düsseldorf - trips there currently have a fahrtNr of 0. This causes TSTP to confuse trips for each other, even trains of the same line going the other way. Try planning e.g. Heerdter Sandberg to Düsseldorf Hbf (Hafas comparison).

[traines-source]

I have changed it now to only take the fahrtNr and line.ID if they're longer than 3 characters, and fall back to tripID otherwise. This fixes Düsseldorf, but I fear that in other places, it will still go wrong (at the very least when querying across two days). But currently I don't see any more stable solution... I noticed some weird time travel issues in the past couple of days too but I couldn't yet reproduce them and wasn't sure whether it was due to the ID matching. So thanks for reporting – and if you notice any other places where it goes wrong, don't hesitate to tell me!

[iw0nderhow]

New issue: for some replacement buses, db-rest will return fahrtNr of 999999 and line.id of bus, causing the same confusion:

Example responses

{
  "departures": [
    {
      "tripId": "2|#VN#1#ST#1727296536#PI#0#ZI#2098482#TA#0#DA#270924#1S#8000211#1T#1736#LS#8000205#LT#1919#PU#80#RT#2#CA#BSV#ZE#SEV#ZB#Bus     #PC#3#FR#8000211#FT#1736#TO#8000205#TT#1919#",
      "stop": {
        "type": "station",
        "id": "8000211",
        "name": "Krefeld Hbf",
        "location": {
          "type": "location",
          "id": "8000211",
          "latitude": 51.32552,
          "longitude": 6.570054
        },
        "products": {
          "nationalExpress": true,
          "national": false,
          "regionalExpress": false,
          "regional": true,
          "suburban": false,
          "bus": true,
          "ferry": false,
          "subway": true,
          "tram": true,
          "taxi": false
        }
      },
      "when": "2024-09-27T17:36:00+02:00",
      "plannedWhen": "2024-09-27T17:36:00+02:00",
      "delay": 0,
      "platform": null,
      "plannedPlatform": null,
      "prognosisType": "prognosed",
      "direction": "Kleve",
      "provenance": null,
      "line": {
        "type": "line",
        "id": "bus",
        "fahrtNr": "999999",
        "name": "Bus",
        "public": true,
        "adminCode": "TDRR__",
        "productName": "Bus",
        "mode": "train",
        "product": "regional",
        "operator": {
          "type": "operator",
          "id": "rheinruhrbahn-transdev",
          "name": "RheinRuhrBahn (Transdev)"
        }
      },
      "remarks": [
        {
          "type": "status",
          "code": "alternative-trip",
          "text": "Sonderfahrt"
        }
      ],
      "origin": null,
      "destination": {
        "type": "station",
        "id": "8000205",
        "name": "Kleve",
        "location": {
          "type": "location",
          "id": "8000205",
          "latitude": 51.789687,
          "longitude": 6.146266
        },
        "products": {
          "nationalExpress": false,
          "national": false,
          "regionalExpress": false,
          "regional": true,
          "suburban": false,
          "bus": true,
          "ferry": false,
          "subway": false,
          "tram": false,
          "taxi": true
        }
      }
    }
  ],
  "realtimeDataUpdatedAt": 1727449905
}

{
  "departures": [
    
    {
      "tripId": "2|#VN#1#ST#1727296536#PI#0#ZI#2095475#TA#0#DA#270924#1S#8000238#1T#1746#LS#8001372#LT#1858#PU#80#RT#2#CA#BSV#ZE#SEV#ZB#Bus     #PC#3#FR#8000238#FT#1746#TO#8001372#TT#1858#",
      "stop": {
        "type": "station",
        "id": "8000238",
        "name": "Lüneburg",
        "location": {
          "type": "location",
          "id": "8000238",
          "latitude": 53.249742,
          "longitude": 10.419873
        },
        "products": {
          "nationalExpress": true,
          "national": true,
          "regionalExpress": true,
          "regional": true,
          "suburban": false,
          "bus": true,
          "ferry": false,
          "subway": false,
          "tram": false,
          "taxi": true
        }
      },
      "when": "2024-09-27T17:46:00+02:00",
      "plannedWhen": "2024-09-27T17:46:00+02:00",
      "delay": null,
      "platform": null,
      "plannedPlatform": null,
      "prognosisType": null,
      "direction": "Dannenberg Ost",
      "provenance": null,
      "line": {
        "type": "line",
        "id": "bus",
        "fahrtNr": "999999",
        "name": "Bus",
        "public": true,
        "adminCode": "X2____",
        "productName": "Bus",
        "mode": "train",
        "product": "regional",
        "operator": {
          "type": "operator",
          "id": "erixx",
          "name": "erixx"
        }
      },
      "remarks": [
        {
          "type": "status",
          "code": "alternative-trip",
          "text": "Sonderfahrt"
        }
      ],
      "origin": null,
      "destination": {
        "type": "station",
        "id": "8001372",
        "name": "Dannenberg Ost",
        "location": {
          "type": "location",
          "id": "8001372",
          "latitude": 53.105025,
          "longitude": 11.118317
        },
        "products": {
          "nationalExpress": false,
          "national": false,
          "regionalExpress": false,
          "regional": true,
          "suburban": false,
          "bus": true,
          "ferry": false,
          "subway": false,
          "tram": false,
          "taxi": false
        }
      }
    }
  ],
  "realtimeDataUpdatedAt": 1727449844
}

[traines-source]

Sorry for the wait but now the entire tripId is used unless all of the following conditions are satisfied:

• line.id >= 4 chars, including at least one digit
• fahrtNr >= 3 chars
• productName NOT being Bus (because buses don't split, do they)
• the tripId containing a date at the expected place, that will be used as start date

The tripId actually appears to contain all sorts of interesting information:

2|#VN#1#ST#1727296536#PI#0#ZI#2095475#TA#0#DA#270924#1S#8000238#1T#1746#LS#8001372#LT#1858#PU#80#RT#2#CA#BSV#ZE#SEV#ZB#Bus     #PC#3#FR#8000238#FT#1746#TO#8001372#TT#1858#
           |                  |               |         |          |       |          |                  |             |             |
           |                  |               |         |          |       |          |                  |             |             from, from time, to, to time same as 1. and last stop?
           |                  |               |         |          |       |          |                  |             trip name              
           |                  |               |         |          |       |          |                  sort of product name
           |                  |               |         |          |       |          last stop time
           |                  |               |         |          |       last stop
           |                  |               |         |          1. stop time
           |                  |               |         1. stop
           |                  |               start date
           |                  sort of vehicle id?
           creation/import timestamp?

But unfortunately the first and last stop differ for splitting trains and can't be used as a straightforward, reliable addition to the identifier for that reason. So I still don't see a less brittle way for the time being...