Status of push and other external services

[jf87]

Hi Shankari,

I am wondering about the current status/requirement of push notifications through firebase.
Is it still used for silent push on iOS?

Otherwise, am I correct that it is only needed to send surveys to single users or a group of users?

From my understanding, the firebase plugin on the phone will generate some device token the first time the app is used. Then this token is sent to firebase server and to our server.
If we want to send a notification, we can make the mapping between uuid/email on our server and ask firebase to send a notification to specific device tokens.
Is this correct?

It seems that this results also in some collection of analytics data on firebase servers.
See screenshot below (somehow only for iOS users for me, this might be a bug or some mis-configuration on my side).

According to firebase doc, this can be turned off: https://firebase.google.com/docs/cloud-messaging/android/client#prevent-auto-init
Would this break anything?

Generally, I need to understand other external services we use and what data is shared with them in order to have the correct data protection documentation. Besides firebase, I guess there is the openstreetmap server for mapping. I have not enabled habitica. Can you think of anything else?

Thanks :-)

[shankari]

@jf87

tl;dr

• yes, fixes to the manifest work, but it should really be put into the plugin.xml for the data collection plugin
• there are, and have been, similar statistics available through the store dashboards (app store/play store). I believe individual users can opt out of providing metrics to the store dashboard on iOS but not on android. As far as I can tell, the statistics are not personally identifiable.
• we make calls to nominatim to reverse geocode the start and end places, and to the overpass.de service to look up public transit stops near the start and end of sections. There are no further external integrations other than habitica on the main branches. The greentrip branch integrates with yelp but that is not merged to master.
• back in 2015 when I first started testing this, the visit detection API was kind of flaky, but multiple studies since then have run successfully without the silent push notification. So if you bump up the minimum iOS version supported, you can skip the silent push notification step.
  • since you are building your own app, if you also don't need visible push notifications, you can remove the push plugin and the associated code completely

Details:

ah yes. before the giant upgrade earlier this year (#325) we used GCM which did not do any automatic analysis tracking on android. However, google was deprecating GCM as of April 10, so I also decided to migrate GCM -> FCM at the same time (#325 (comment)). And that time I did notice in the logs that FCM was sending information automatically, and I even disabled it using the change to AndroidManifest.xml but I did not make the corresponding changes to the plugin.xml and it got overwritten as I made other plugin changes.

I used the instructions here https://firebase.google.com/docs/analytics/configure-data-collection

Also, the way that the token generation works is different on android and iOS. On android, since FCM is the native notification mechanism, it generates a token natively, which we store on the server. On iOS, however, since APNS is the native notification mechanism, we get an APNS token from the server. We send it to the e-mission server, and then map it to an FCM token by calling an FCM API
https://github.com/e-mission/e-mission-server/blob/master/emission/net/ext_service/push/notify_interface_impl/firebase.py#L97

This allows us to switch to a different provider for APNS or even use APNS directly in the future. No way of getting around FCM for android, as far as I can tell, though.

[shankari]

@jf87 the silent push notification essentially works as a coarse timer since we do not have the ability to schedule periodic tasks on iOS. As part of the coarse timer on android, I sync to the server, but also check whether location services are enabled and whether the app has appropriate permissions. The coarse timer on iOS currently only syncs data but I would like to add similar checks to avoid confusion and "the app is not tracking anything" concerns.

so it seems to me that we should continue to have the silent push notifications enabled in the long term.

What do you think? we don't really need the coarse timer if we don't want to perform periodic housekeeping tasks. But as you know, most robust, long-running systems do need to perform periodic housekeeping tasks. and ongoing background data collection is essentially a robust, long-running system.

[jf87]

Thank you very much for the long reply. 👍
Can you point me to the code which handles the periodic silent notifications for iOS? I could not find it yesterday.

I think in general, push notifications are useful for survey/crowdsourcing related tasks in this context.
Plus, as you mention the useful "hack" for iOS.
So I would also continue to have it enabled. The context of my question was related to data-protection/gdpr etc. We need to state somewhere which external services are used because we share potentially personal data with them (IP address, app usage, GPS coordinates etc). So it's more about not missing anything. Personally, I don't like to share too much with Google and others, but I guess that is how you pay for a free service ;-)

[shankari]

Can you point me to the code which handles the periodic silent notifications for iOS? I could not find it yesterday.

https://github.com/e-mission/e-mission-data-collection/blob/master/src/ios/BEMDataCollection.m#L231

which calls

https://github.com/e-mission/e-mission-data-collection/blob/master/src/ios/BEMAppDelegate.m#L142

Note that we actually register for notifications in the javascript code using the push plugin
https://github.com/e-mission/e-mission-phone/blob/master/www/js/splash/pushnotify.js

So the flow is:

• register from javascript (so that we can use the push plugin and get bugfixes, etc)
• in javascript, detect whether this is a silent push or a visible push
• if a silent push, call native code plugin to handle it

[shankari]

Personally, I don't like to share too much with Google and others

Neither do I, which is why I have been very careful about external integrations (e.g. not using Google's or MapQuests' APIs for map matching). But there aren't really a lot of alternatives to Google and Apple for the push notifications since they are integrated into the OS.

I will make the changes to the plugin.xml to turn off firebase analytics by default. Unless you get to it first ?!

[jf87]

I can do, but I am not sure what is the right way. It seems that we are using https://github.com/phonegap/phonegap-plugin-push

Is it possible to pass this setting though config.xml using a CLI variable?
Otherwise, am I right that we need to either have a fork of this push plugin or point to our own plugin.xml?

[shankari]

I was going to do something easy like add the entries to the e-mission-data-collection plugin.xml but you are right that the better solution is to add it to the push-plugin. The only problem is that I am not currently on the latest version of the push plugin because it requires <engine name="cordova-android" version=">=7.1.0"/>, which requires me to update the packaging for all the plugins. The version that we use is the latest one that works with <engine name="cordova-android" version=">=6.3.0"/>

I think we could submit a PR to that repository and state that it fixes the open issue around advertising ID + analytics collection
phonegap/phonegap-plugin-push#2571

But for now, we could:

• add it to the data collection's plugin.xml, OR
• bring our fork of the push plugin which I had created for some similar reason earlier up to date and use it instead.

[shankari]

our fork is at https://github.com/e-mission/phonegap-plugin-push

Is it possible to pass this setting though config.xml using a CLI variable?

Yup! You can look at the camera plugin for an example.
https://github.com/apache/cordova-plugin-camera#ios-quirks