From 5db7b75f78fc4101cc07347d42e66c2d50892f48 Mon Sep 17 00:00:00 2001 From: Michael Huebler Date: Sun, 3 Apr 2022 10:17:30 +0200 Subject: [PATCH] Changed "CCTG Mode (root)" to "CCTG Import", user now needs to select a file. This file can be copied e.g. by doing `adb shell su -c cp /data/data/de.corona.tracing/databases/exposure.db /storage/emulated/0/Download/`. This file can also be copied from another device, if desired. --> Closes #161 and #107 and several feature requests received by e-mail. This will also ensure that the original CCTG database is not touched and can therefore not be corrupted. --- README.md | 18 +- .../coronawarncompanion/MainActivity.java | 22 +- .../microgreadout/CctgDbOnDisk.java | 201 +++++++----------- .../src/main/res/values-de/strings.xml | 6 +- .../src/main/res/values/strings.xml | 6 +- 5 files changed, 113 insertions(+), 140 deletions(-) diff --git a/README.md b/README.md index 2dfd969..9e20d14 100644 --- a/README.md +++ b/README.md @@ -31,7 +31,7 @@ The `release` build variant will probably build out-of-the box only on macOS bec After a lot of user feedback and many iterations, the app meanwhile has several modes. The best user experience is available if the device is rooted and the app gets root permissions: With root permissions, the app can directly access the database _from a framework_ where the recorded Rolling Proximity IDs (RPIs) are stored. -However, since many users cannot or do not wish to root their device, two other options (RaMBLE and CCTG) are also available, where _another app_ can export their recorded RPIs, so that this app can read them without root rights, and work with them. +However, since many users cannot or do not wish to root their device, other options (RaMBLE and CCTG) are also available, where _another app_ can export their recorded RPIs, so that this app can read them without root rights, and work with them. The currently available modes are: @@ -40,25 +40,27 @@ The currently available modes are: ### Normal mode (root): This app reads RPIs from the Google Exposure Notifications service / framework. Works together with the official [Corona-Warn-App](https://www.coronawarn.app/de/). -### CCTG mode (root): -This app reads RPIs from the [Corona Contact Tracing Germany app](https://bubu1.eu/cctg/) (which brings its own microG framework that stores the RPIs in its own location). Works if you have installed CCTG, and give root access to this app. -***NOTE: Better use the CCTG export (as explained below) instead*** - ### microG mode (root): This app reads RPIs from the [microG ENF framework](https://microg.org). Works if you have installed microG and an app that activates microG ENF RPI recording, and give root access to this app. ## Modes that work without root: -### RaMBLE mode: +### RaMBLE Import: You need to record with the [RaMBLE app](https://apkpure.com/de/ramble-bluetooth-le-mapper/com.contextis.android.BLEScanner) (to record, tap the "▶" button), and then on the next day, export the database (tap the "⋮" button in the top right corner, select "Export Database"). Then set this app to RaMBLE Mode (tap the "⋮" menu button in the top right corner, select "RaMBLE Mode"). *You then need to select the newest "RaMBLE_..." file from the "Downloads" directory*, where RaMBLE has stored its exported database. Note that the next time you want to use this app, you need to follow the same steps again (export from RaMBLE, select the file in this app). -### CCTG export (not a special mode within this app): -This requires the installation of the [Corona Contact Tracing Germany app](https://bubu1.eu/cctg/). On the next day, export the tracing information from CCTG and share with this app as described in the [CCTG FAQ](https://codeberg.org/corona-contact-tracing-germany/cwa-android#how-do-i-access-the-microg-exposure-notitifaction-settings-in-the-standalone-app). If you tap "export" on the microG ENF screen, and select to share with this app (tap on the Friendly Dog icon), this app is called and automatically reads the data, and also automatically selects "microG mode" in the process. +### CCTG Export (not a special mode within this app): +This requires the installation of the [Corona Contact Tracing Germany app](https://bubu1.eu/cctg/). On the next day, export the tracing information from CCTG and share with this app as described in the [CCTG FAQ](https://codeberg.org/corona-contact-tracing-germany/cwa-android#how-do-i-access-the-microg-exposure-notitifaction-settings-in-the-standalone-app). If you tap "export" on the microG ENF screen, and select to *share* with this app (tap on the Friendly Dog icon), this app is called and automatically reads the data, and also automatically selects "microG mode" in the process. Note that the next time you want to use this app, you need to follow the same steps again (export and share from CCTG / microG ENF). +### CCTG Import: +*NOTE: It's easier to use the CCTG Export (as explained above) instead* +This app can read RPIs from a file created by the [Corona Contact Tracing Germany app](https://bubu1.eu/cctg/) (which brings its own microG framework that stores the RPIs in its own location: /data/data/de.corona.tracing/databases/exposure.db). Works if you have installed CCTG, can copy the database file to a location where you can select it from within this app - e.g. by doing `adb shell su -c cp /data/data/de.corona.tracing/databases/exposure.db /storage/emulated/0/Download/`. +One thing you can also do using this mode: Use a **copy of a database file** that was copied from a slow device, where analysis takes very long, on another more capable device, where analysis runs faster. + + --- (BTW, if you wonder why this explanation is not part of the Play Store text: This is because of the Play Store requirements for COVID-19 apps; this is not a category 1 or 2 app and must therefore not use COVID-19 related terms in its Play Store listing.) diff --git a/corona-warn-companion/src/main/java/org/tosl/coronawarncompanion/MainActivity.java b/corona-warn-companion/src/main/java/org/tosl/coronawarncompanion/MainActivity.java index a0de6ad..3e4ef1c 100644 --- a/corona-warn-companion/src/main/java/org/tosl/coronawarncompanion/MainActivity.java +++ b/corona-warn-companion/src/main/java/org/tosl/coronawarncompanion/MainActivity.java @@ -115,6 +115,7 @@ public class MainActivity extends AppCompatActivity { public static final String EXTRA_MESSAGE_DAY = "org.tosl.coronawarncompanion.DAY_MESSAGE"; public static final String EXTRA_MESSAGE_COUNT = "org.tosl.coronawarncompanion.COUNT_MESSAGE"; public static final int INTENT_PICK_RAMBLE_FILE = 1; + public static final int INTENT_PICK_CCTG_FILE = 2; static boolean mainActivityShouldBeRecreated = false; private static CWCApplication.AppModeOptions desiredAppMode; private RpiList rpiList = null; @@ -441,9 +442,10 @@ protected void onCreate(Bundle savedInstanceState) { rpiList = microGDbOnDisk.getRpisFromContactDB(this, databaseFile); continueWhenRpisAreAvailable(); } else if (CWCApplication.appMode == CCTG_MODE) { - CctgDbOnDisk cctgDbOnDisk = new CctgDbOnDisk(this); - rpiList = cctgDbOnDisk.getRpisFromContactDB(this); - continueWhenRpisAreAvailable(); + intent = new Intent(Intent.ACTION_OPEN_DOCUMENT); + intent.addCategory(Intent.CATEGORY_OPENABLE); + intent.setType("application/octet-stream"); // this is the MIME type of a CCTG SQLITE database + startActivityForResult(intent, INTENT_PICK_CCTG_FILE); } else { throw new IllegalStateException(); } @@ -549,6 +551,7 @@ public void onActivityResult(int requestCode, int resultCode, super.onActivityResult(requestCode, resultCode, resultData); if (requestCode == INTENT_PICK_RAMBLE_FILE && resultCode == Activity.RESULT_OK) { + // Import the RaMBLE file: // The result data contains a URI for the document or directory that // the user selected. Uri uri; @@ -561,6 +564,19 @@ public void onActivityResult(int requestCode, int resultCode, getDaysFromMillis(System.currentTimeMillis()) - 14); continueWhenRpisAreAvailable(); } + } else if (requestCode == INTENT_PICK_CCTG_FILE + && resultCode == Activity.RESULT_OK) { + // Import the CCTG file: + // The result data contains a URI for the document or directory that + // the user selected. + Uri uri; + if (resultData != null) { + uri = resultData.getData(); + // Perform operations on the document using its URI. + CctgDbOnDisk cctgDbOnDisk = new CctgDbOnDisk(this, uri); + rpiList = cctgDbOnDisk.getRpisFromContactDB(); + continueWhenRpisAreAvailable(); + } } } diff --git a/corona-warn-companion/src/main/java/org/tosl/coronawarncompanion/microgreadout/CctgDbOnDisk.java b/corona-warn-companion/src/main/java/org/tosl/coronawarncompanion/microgreadout/CctgDbOnDisk.java index 9368bab..3d8621f 100644 --- a/corona-warn-companion/src/main/java/org/tosl/coronawarncompanion/microgreadout/CctgDbOnDisk.java +++ b/corona-warn-companion/src/main/java/org/tosl/coronawarncompanion/microgreadout/CctgDbOnDisk.java @@ -1,10 +1,9 @@ package org.tosl.coronawarncompanion.microgreadout; -import android.annotation.SuppressLint; -import android.app.Activity; import android.content.Context; import android.database.Cursor; import android.database.sqlite.SQLiteDatabase; +import android.net.Uri; import android.util.Log; import com.google.protobuf.ByteString; @@ -15,154 +14,110 @@ import java.io.File; import java.io.FileOutputStream; import java.io.IOException; +import java.io.InputStream; -import static org.tosl.coronawarncompanion.gmsreadout.ContactDbOnDisk.deleteDir; -import static org.tosl.coronawarncompanion.gmsreadout.Sudo.sudo; -import static org.tosl.coronawarncompanion.tools.Utils.byteArrayToHexString; import static org.tosl.coronawarncompanion.tools.Utils.getDaysFromMillis; +// Reads a CCTG / microg database file that the user should have copied +// from /data/data/de.corona.tracing/databases/exposure.db +// to a temporary location of their choice. +// User can e.g. do +// adb shell su -c cp /data/data/de.corona.tracing/databases/exposure.db /storage/emulated/0/Download/ public class CctgDbOnDisk { private static final String TAG = "CctgDbOnDisk"; - - @SuppressLint("SdCardPath") - private static final String gmsPathStr = "/data/data/de.corona.tracing/databases/"; - private static final String dbName = "exposure.db"; - private static final String dbNameModifier = "_"; - private static final String dbNameModified = dbName+dbNameModifier; - private static String cachePathStr = ""; - private static File cacheDir = null; - private final Context context; + private final Uri uri; - public CctgDbOnDisk(Context context) { + public CctgDbOnDisk(Context context, Uri uri) { this.context = context; + this.uri = uri; + Log.d(TAG, "Selected CCTG file: " + uri.toString()); } - public void copyFromGMS() { - // Copy the CCTG microG GMS database to local app cache - Log.d(TAG, "Trying to copy CCTG database"); - cacheDir = context.getExternalCacheDir(); - if (cacheDir == null) { - cacheDir = context.getCacheDir(); - } - assert cacheDir != null; - cachePathStr = cacheDir.getPath(); + public RpiList getRpisFromContactDB() { + RpiList rpiList = null; + InputStream inputStream; try { - File testFile = File.createTempFile("test_file", null, cacheDir); - FileOutputStream stream = new FileOutputStream(testFile); - try { - stream.write("test".getBytes()); - // get owner of testFile - String owner = sudo("ls -l "+testFile+"|head -n 1|cut -d \" \" -f 3").replace("\n", ""); - Log.d(TAG, "Cache file owner: "+owner); - // get group of testFile - String group = sudo("ls -l "+testFile+"|head -n 1|cut -d \" \" -f 4").replace("\n", ""); - Log.d(TAG, "Cache file group: "+group); - // get context of testFile - String context = sudo("ls -Z "+testFile+"|head -n 1|cut -d \" \" -f 1").replace("\n", ""); - Log.d(TAG, "Cache file context: "+context); - - // First rename the LevelDB directory, then copy it, then rename to the original name - String result = sudo( - "rm "+cachePathStr+"/"+dbNameModified, - "mv "+gmsPathStr+"/"+dbName+" "+gmsPathStr+"/"+dbNameModified, - "cp "+gmsPathStr+"/"+dbNameModified+" "+cachePathStr+"/", - "mv "+gmsPathStr+"/"+dbNameModified+" "+gmsPathStr+"/"+dbName, - "ls -lZ "+cachePathStr+"/"+dbNameModified - ); - Log.d(TAG, "Result from trying to copy LevelDB: "+result); - if (result.length() < 10) { - Log.e(TAG, "ERROR: Super User rights not granted!"); - } - - // set owner and context (required for Android 11), and group as well - result = sudo( - "chown -R "+owner+" "+cachePathStr+"/"+dbNameModified, - "chgrp -R "+group+" "+cachePathStr+"/"+dbNameModified, - "chcon -R "+context+" "+cachePathStr+"/"+dbNameModified, - "ls -lZ "+cachePathStr+"/"+dbNameModified - ); - Log.d(TAG, "Result from trying to set owner, group and context: "+result); - } finally { - stream.close(); - } - } catch (IOException e) { - e.printStackTrace(); - } - } + inputStream = this.context.getContentResolver().openInputStream(uri); + File file = File.createTempFile("cctg", "sqlite"); - public RpiList getRpisFromContactDB(Activity activity) { - RpiList rpiList = null; - - copyFromGMS(); + FileOutputStream outputStream = new FileOutputStream(file); + byte[] buff = new byte[1024]; + int read; + while ((read = inputStream.read(buff, 0, buff.length)) > 0) + outputStream.write(buff, 0, read); + inputStream.close(); + outputStream.close(); - try (SQLiteDatabase microGDb = SQLiteDatabase.openDatabase(cachePathStr + "/" + dbNameModified, - null, SQLiteDatabase.OPEN_READONLY)) { + try (SQLiteDatabase microGDb = SQLiteDatabase.openDatabase(file.getPath(), + null, SQLiteDatabase.OPEN_READONLY)) { - if (microGDb != null) { - Log.d(TAG, "Opened CCTG Database: " + gmsPathStr + "/" + dbNameModified); + if (microGDb != null) { + Log.d(TAG, "Opened temporary copy of the CCTG Database: " + file.getPath()); - Cursor cursor = microGDb.rawQuery("SELECT rpi, aem, timestamp, rssi, duration "+ - "FROM advertisements", null); + Cursor cursor = microGDb.rawQuery("SELECT rpi, aem, timestamp, rssi, duration "+ + "FROM advertisements", null); - rpiList = new RpiList(); + rpiList = new RpiList(); - while (cursor.moveToNext()) { - // parse entry from table "advertisements" - byte[] rpiBytes = cursor.getBlob(0); - if (rpiBytes == null) { - Log.w(TAG, "Warning: Found rpiBytes == null"); - } else { - byte[] aemBytes = cursor.getBlob(1); - if (aemBytes == null) { - Log.w(TAG, "Warning: Found aemBytes == null"); + while (cursor.moveToNext()) { + // parse entry from table "advertisements" + byte[] rpiBytes = cursor.getBlob(0); + if (rpiBytes == null) { + Log.w(TAG, "Warning: Found rpiBytes == null"); } else { - long timestampMs = cursor.getLong(2); - int rssi = (int) cursor.getLong(3); - int duration = cursor.getInt(4); - - //Log.d(TAG, "Scan read: " + byteArrayToHexString(rpiBytes) + " " + byteArrayToHexString(aemBytes) + - // " RSSI: " + rssi + ", Timestamp: " + timestampMs + ", Duration: " + duration); - - // limit RSSI, which could be a very large number, because of this bug: https://github.com/microg/android_packages_apps_GmsCore/issues/1230 - if (rssi < -200L) rssi = -200; - if (rssi > +200L) rssi = +200; - - // add scanRecord to contactRecords - ContactRecordsProtos.ContactRecords.Builder contactRecordsBuilder = - ContactRecordsProtos.ContactRecords.newBuilder(); - @SuppressWarnings("deprecation") ContactRecordsProtos.ScanRecord scanRecord = ContactRecordsProtos.ScanRecord.newBuilder() - .setTimestamp((int)(timestampMs/1000L)) - .setRssi(rssi) - .setAem(ByteString.copyFrom(aemBytes)) - .build(); - contactRecordsBuilder.addRecord(scanRecord); - //noinspection deprecation - scanRecord = ContactRecordsProtos.ScanRecord.newBuilder() - .setTimestamp((int)((timestampMs+duration)/1000L)) - .setRssi(rssi) - .setAem(ByteString.copyFrom(aemBytes)) - .build(); - contactRecordsBuilder.addRecord(scanRecord); - - // store entry (incl. contactRecords) in rpiList - int daysSinceEpochUTC = getDaysFromMillis(timestampMs); - rpiList.addEntry(daysSinceEpochUTC, rpiBytes, contactRecordsBuilder.build()); - if (getDaysFromMillis(timestampMs+duration) != daysSinceEpochUTC) { // extremely unlikely - rpiList.addEntry(daysSinceEpochUTC + 1, rpiBytes, contactRecordsBuilder.build()); + byte[] aemBytes = cursor.getBlob(1); + if (aemBytes == null) { + Log.w(TAG, "Warning: Found aemBytes == null"); + } else { + long timestampMs = cursor.getLong(2); + int rssi = (int) cursor.getLong(3); + int duration = cursor.getInt(4); + + //Log.d(TAG, "Scan read: " + byteArrayToHexString(rpiBytes) + " " + byteArrayToHexString(aemBytes) + + // " RSSI: " + rssi + ", Timestamp: " + timestampMs + ", Duration: " + duration); + + // limit RSSI, which could be a very large number, because of this bug: https://github.com/microg/android_packages_apps_GmsCore/issues/1230 + if (rssi < -200L) rssi = -200; + if (rssi > +200L) rssi = +200; + + // add scanRecord to contactRecords + ContactRecordsProtos.ContactRecords.Builder contactRecordsBuilder = + ContactRecordsProtos.ContactRecords.newBuilder(); + @SuppressWarnings("deprecation") ContactRecordsProtos.ScanRecord scanRecord = ContactRecordsProtos.ScanRecord.newBuilder() + .setTimestamp((int)(timestampMs/1000L)) + .setRssi(rssi) + .setAem(ByteString.copyFrom(aemBytes)) + .build(); + contactRecordsBuilder.addRecord(scanRecord); + //noinspection deprecation + scanRecord = ContactRecordsProtos.ScanRecord.newBuilder() + .setTimestamp((int)((timestampMs+duration)/1000L)) + .setRssi(rssi) + .setAem(ByteString.copyFrom(aemBytes)) + .build(); + contactRecordsBuilder.addRecord(scanRecord); + + // store entry (incl. contactRecords) in rpiList + int daysSinceEpochUTC = getDaysFromMillis(timestampMs); + rpiList.addEntry(daysSinceEpochUTC, rpiBytes, contactRecordsBuilder.build()); + if (getDaysFromMillis(timestampMs+duration) != daysSinceEpochUTC) { // extremely unlikely + rpiList.addEntry(daysSinceEpochUTC + 1, rpiBytes, contactRecordsBuilder.build()); + } } } } + cursor.close(); + microGDb.close(); } - cursor.close(); - microGDb.close(); + } catch (Exception e) { + e.printStackTrace(); } - } catch (Exception e) { + } catch (IOException e) { e.printStackTrace(); } - deleteDir(cacheDir); return rpiList; } } diff --git a/corona-warn-companion/src/main/res/values-de/strings.xml b/corona-warn-companion/src/main/res/values-de/strings.xml index ae55b08..beccf2b 100644 --- a/corona-warn-companion/src/main/res/values-de/strings.xml +++ b/corona-warn-companion/src/main/res/values-de/strings.xml @@ -46,9 +46,9 @@ Alle Begegnungen zeigen/verbergen Karte zeigen/verbergen Normaler Modus (root) - RaMBLE Modus + RaMBLE Import microG Modus (root) - CCTG Modus (root) + CCTG Import Demo Modus Open Source Software Lizenzen Weitere Open Source Software Lizenzen @@ -59,7 +59,7 @@ "DEMO " "RaMBLE-Warn-Companion" "microG-Warn-Companion" - "CCTG-Warn-Companion" + "CCTG-Import-Warn-Companion" @string/app_name DEMO Beispiel-Details diff --git a/corona-warn-companion/src/main/res/values/strings.xml b/corona-warn-companion/src/main/res/values/strings.xml index 1199fcc..07947ff 100644 --- a/corona-warn-companion/src/main/res/values/strings.xml +++ b/corona-warn-companion/src/main/res/values/strings.xml @@ -46,9 +46,9 @@ Show/Hide All Scans Show/Hide Location Normal Mode (root) - RaMBLE Mode + RaMBLE Import microG Mode (root) - CCTG Mode (root) + CCTG Import Demo Mode Open Source Software Licenses Further Open Source Software Licenses @@ -59,7 +59,7 @@ "DEMO " "RaMBLE-Warn-Companion" "microG-Warn-Companion" - "CCTG-Warn-Companion" + "CCTG-Import-Warn-Companion" @string/app_name DEMO sample details