Can multiple Observables from bind be handled? I can't get it working.

[oehm-smith]

I was playing around with a responsive UI and split the list of results from fromFetch() in to individual Observables, hoping (initially expecting!) to render each one in React. However the stateObservable only returned the last Observable (this made sense after I looked at the documentation). So I subscribed to the state and expected to be able to handle it that way.

The subscription did indeed produce the expected results and I'd see each item one at a time. However there was no way I could get this in to React as a list. All attempts produced nothing or just one or two results.

I expect the problem is to do with the React life cycle interaction however I'm not sure exactly why.

Do you think there should be a solution for this? If this is something someone would like to look at I'll create a minimal viable example. Below is my actual code.

Use fromFetch to retrieve data:

private observations = (
  path: string,
  query?: QueryParams,
  params: RequestParams = {},
):Observable<EOWObservationsResponse[]> => {
  const queryString = query && '?' + this.toQueryString(query);
  const fullUrl = `${this.baseUrl}${path}${queryString}`

  return fromFetch(fullUrl).pipe(
    switchMap(response => {
      if (response.ok) {
        const r = response.json();
        return r;
      } else {
        return of({ error: true, message: `Error ${response.status}` });
      }
    }),
    catchError(err => {
      // Network or other error, handle appropriately
      console.error(err);
      return of({ error: true, message: err.message });
    })
  )
}

The caller of this massages the data (note that observationsWorld() fixes up params and calls observations() ):

  static getEOWDataLayerFeatures(bBoxString: string = ""): Observable<LayerDefinition[]|null> {
    // Retrieve data since start of data collection
    const now = Date.now();
    const msToMonths = 1000*60*60*24*30;
    const monthsSinceStart = Math.floor((now - EOWDataCollectionStarted)  / msToMonths);

    const bboxObj = bBoxString.length ? { bbox: bBoxString } : {};

    return eowApi
      .observationsWorld({ period: monthsSinceStart, limit: LIMIT, ...bboxObj })
      .pipe(
        mergeMap((m: EOWObservationsResponse[]) =>
          of(
            m.map((e: EOWObservationsResponse) => ({
              type: "Point",
              coordinates: [
                e.location?.lng,
                e.location?.lat,
              ],
              properties: e, //(e as EOWObservationsResponse),
            } as ParseData)),
          ),
        ),
        mergeMap((t: ParseData[]) => of(parse(t))),
        mergeMap((t: FeatureCollection) =>
          of({
            type: LayerType.GEOJSON,
            data: t,
            url: "",
            options: {
              createLayer: true,
              useAsWaterBodySource: false,
              layerDisplayName: "EOW",
              visible: true,
            },
          }),
        ),
        toArray(),
      )
  }

This is passed through more rxjs pipes to extract just the data I want. It includes this part to create an Observable for each item in the array returned from fromFetch:

          concatMap((a) => {
            const features = a && a.length ? a[0].data?.features : null;
            return features ? from(features) : null;
          }),

And finally ends with a bind:

export const [measurementFromEOWData, MeasurementFromEOWData$] = bind<WrappedSingle<MeasurementsSubset>>(
  globalOrLocalEOWData$.pipe(
    map((wsfeature: WrappedSingle<Feature>) => {
      return { data: buildMeasurementSubset(wsfeature.data), key: wsfeature.key};
    }),
    tap((wsMeasure: WrappedSingle<MeasurementsSubset>) => Logger.debug(`measurement from pipe: ${wsMeasure.data.date_photo}, ${wsMeasure.data.key}`)),
  ),
);

And the React component subscribes to it:

  useEffect(() => {
    MeasurementFromEOWData$.subscribe((m) => {
      Logger.debug(`measurement from sub - ${JSON.stringify(m)}`);
      setMeasurmentsList([...measurmentsList, {date: m.data.date_photo}]);
      keys.push(m.data);
    });
  }, []);

The Log message shows every observable as being separate. But I can't get the data in to React.

• The MeasurementsList is empty
• and the simple keys array is being populated but empty by the time the rendering is performed.

DEBUG: measurement from sub - {"data":{"fu_processed":8,"date_photo":"2019-05-13 04:44:31.000","latLngs":{"lat":-35.2901107119725,"lng":149.1261494906145},"key":"659cd94bdd79038e45780f9c6909d7073a6bcaeab0aea9b767e63087d4ff240f"},"key":"659cd94bdd79038e45780f9c6909d7073a6bcaeab0aea9b767e63087d4ff240f"} -

DEBUG: measurement from sub - {"data":{"fu_processed":11,"date_photo":"2019-05-13 04:43:51.000","latLngs":{"lat":-35.29031430840053,"lng":149.1262304597992},"key":"0a4c0ea78df123c9a5706fdc3e7659898d0d277c18e34215da9566d9fa02a15f"},"key":"0a4c0ea78df123c9a5706fdc3e7659898d0d277c18e34215da9566d9fa02a15f"} -

DEBUG: measurement from sub - {"data":{"fu_processed":10,"date_photo":"2019-01-22 04:07:29.000","latLngs":{"lat":-35.29002408500322,"lng":149.1262198147821},"key":"ccbcfbe00b74d082dbd49adc35619823f1f102791f48cb62c2531ebb801cff12"},"key":"ccbcfbe00b74d082dbd49adc35619823f1f102791f48cb62c2531ebb801cff12"} -

DEBUG: measurement from sub - {"data":{"fu_processed":11,"date_photo":"2019-01-22 04:06:37.000","latLngs":{"lat":-35.28989437505164,"lng":149.1263265164095},"key":"5c012881f754701500959b395625805f2451ffa9c6eb65494b995b72dd7eaeee"},"key":"5c012881f754701500959b395625805f2451ffa9c6eb65494b995b72dd7eaeee"} -

Any ideas on how I could get this working?

---

Solved I think - but wonder if better way?

I left this for a couple hours and then did some more research. I changed to using useRef instead of useState and it seems to retain the values and can be used in the React render.

Is there a solution that is better suited for React-RXJS? Including - is there a way to get a StateObservable (that only returns the latest Observable) to return each one? So that can be used instead of subscribing to the state.

[voliva]

There are few things here, which makes it harder to understand how does the data get transformed. It sounds like what you really need at the end is an Observable<Array<something>>. In observables, usually past emitted values are already gone, the only relevant one is the most recent. If you need a history of values, it's better to represent this as an Observable of an Array, this way the latest value will always have every value emitted.

Sorry for the nit-pick, but in many places you do mergeMap(args => of(x)). This makes it hard to read, it's more idiomatic to use map(args => x). This makes it easier to understand on first sight that for a given value, you get only one other value out.

That chain makes the following transform: EOWObservationsResponse[] => ParseData[] => FeatureCollection => LayerDefinition.

The toArray method that comes next is confusing. If I understand correctly, eowApi.observationsWorld is originating from fromFetch(fullUrl), which means that only 1 response will be emitted. This value will get parsed into an array EOWObservationsResponse[] and down to a LayerDefintion, which I guess that contains all the info from that request-response. Doing a toArray on this observable would be the same as doing map(x => [x]) (wrapping in an array of just 1 element).

Then, probably this is part of the culprit:

          concatMap((a) => {
            const features = a && a.length ? a[0].data?.features : null;
            return features ? from(features) : null;
          }),

My guess is that features is actually the array you want to receive. If you do concatMap((..) => .. from(features)), you will be getting an observable that emits each feature one-by-one, and every one of them just gets replaced with the new one. Of course if you subscribe to it you will see how you get all of them, but remember in terms of React-RxJS only the latest value is relevant.

You could use toArray() here to turn it back into an array, but it would be more straightforward to just use map instead:

          map((a) => {
            const features = a && a.length ? a[0].data?.features : null;
            return features ?? null;
          }),

And the other part of the culprit is the infamous stale closure problem in react's useEffect:

  useEffect(() => {
    MeasurementFromEOWData$.subscribe((m) => {
      Logger.debug(`measurement from sub - ${JSON.stringify(m)}`);
      setMeasurmentsList([...measurmentsList, {date: m.data.date_photo}]);
      keys.push(m.data);
    });
  }, []);

This function is closing over measurmentsList when the component got rendered for the first time, which was an empty array. Then every time you get a value from that, you are essentially just changing the state as setMeasurmentsList([...[], { date: m.date.date_photo })

This is why it works with useRef. A more React-idiomatic solution would be to use the function overload:

setMeasurementList(oldList => [...oldList, { date: m.data.date_photo }]);

This way it always has a reference to the value before

Conclusion

Let me know if I missunderstood anything. All in all, I don't really recommend using React's useState/useRef/useEffect to deal with observables when React-RxJS can manage that.

Try to rebuild your observable as an Observable<Array<the value you want>>, and then when you use the hook returned by bind you will actually get your array of values ready to be used by your component.

[oehm-smith]

Hi @voliva , thanks for your well thought out reply.

The point of what I was trying to achieve was avoiding an Observable<Array<the value you want>> (given that fromFetch returns Observable<data that includes Array<Feature>> anyway) and instead effectively I wanted to receive Array<Observable<Feature>> with each Observable being emitted separately so I could process it separately and async. I see that is foolish and doesn't solve the responsive-ui problem.

Instead I am rewriting the fromFetch api calls to use pagination and emit eachObservable<data that includes Array<Feature>> from each "page". And then process each one of these asynchronously and separately.

The toArray() was because I was finding it easier to get the data back as a (single item) array. I was expecting to clean this up after solving all the other details.

Thanks for the tips and reminder of setState (setMeasurementsList) taking an arg.