Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(api): Lazy Loading and Custom Selection Set #2592

Merged
merged 112 commits into from
Oct 10, 2023
Merged
Show file tree
Hide file tree
Changes from 111 commits
Commits
Show all changes
112 commits
Select commit Hold shift + click to select a range
476f5b8
add lazy loading for api
eeatonaws Jun 1, 2023
8460163
Allow passing ModelIdentifier for querying based on CPK
tylerjroach Jun 5, 2023
017ac0a
Share RequestFactory work between API and DataStore
tylerjroach Jun 6, 2023
43edd8d
Mark helper to Amplify Internal
tylerjroach Jun 6, 2023
1a6e010
checkstyle
tylerjroach Jun 6, 2023
a5b63ad
Add CPK ModelQuery test
tylerjroach Jun 7, 2023
4f8b9c5
Code cleanup
tylerjroach Jun 8, 2023
c71680f
fix test
tylerjroach Jun 8, 2023
d4c8540
Merge branch 'main' into tjroach/cpk-api
tylerjroach Jun 8, 2023
a3666d2
GSON serializer should decide how to insert. Inserting as string is i…
tylerjroach Jun 9, 2023
f189db6
Merge remote-tracking branch 'origin/tjroach/cpk-api' into tjroach/cp…
tylerjroach Jun 9, 2023
bc26c47
Temporal Date/Times need to be Serializable to support usage as sort key
tylerjroach Jun 9, 2023
5a440e4
Add CPK API CRUD request tests
tylerjroach Jun 9, 2023
b9d8b51
lint
tylerjroach Jun 9, 2023
6717c54
CPK testing
tylerjroach Jun 12, 2023
36edbac
allow setting null association
tylerjroach Jun 12, 2023
121e3ea
Update how associations are handled in create and update mutations.
tylerjroach Jun 13, 2023
0e08245
Add test for updating nullable relation
tylerjroach Jun 13, 2023
50962d3
Test file rename and lint
tylerjroach Jun 13, 2023
23a6ee3
lint
tylerjroach Jun 13, 2023
ad538f4
Merge branch 'main' into tjroach/cpk-api
tylerjroach Jun 13, 2023
59afe46
retrieve items from lazy list
eeatonaws Jun 13, 2023
b706c42
allow query by id to figure out proper name and type
tylerjroach Jun 13, 2023
665fbab
Merge remote-tracking branch 'origin/tjroach/cpk-api' into tjroach/cp…
tylerjroach Jun 13, 2023
1201715
remove unnecessary changes
eeatonaws Jun 13, 2023
8b52b40
Fix getPrimaryKey SQLiteColumn to properly return CPK with no sort key
tylerjroach Jun 13, 2023
822ad20
Merge branch 'main' into api-lazy-loading
eeatonaws Jun 13, 2023
2098bae
Merge branch 'tjroach/cpk-api' into api-lazy-loading
eeatonaws Jun 13, 2023
e28b570
save lazy model id in create mutation for belongs to relationship
eeatonaws Jun 14, 2023
8767d42
include associated lazy model id when updating model
eeatonaws Jun 14, 2023
a960810
Merge branch 'main' into api-lazy-loading
eeatonaws Jul 6, 2023
ffebf91
remove formatting changes
eeatonaws Jul 6, 2023
d1a43cd
remove unneeded code and update lazy model apis
eeatonaws Jul 6, 2023
cf01846
Begin custom selection set implementation
tylerjroach Jul 13, 2023
adb0fd4
Begin custom selection set implementation
tylerjroach Jul 13, 2023
68620de
ModelPath updates
tylerjroach Jul 13, 2023
18bb144
Custom Selection Set
tylerjroach Jul 13, 2023
77f3b18
allow set mutation
tylerjroach Jul 14, 2023
995c250
Fix custom selection set
tylerjroach Jul 21, 2023
8431cfb
css work
tylerjroach Jul 25, 2023
2257d43
Merge branch 'main' into tjroach/custom-selection-set
tylerjroach Jul 26, 2023
e7b1c08
cleanup custom selection set
tylerjroach Jul 27, 2023
4d81891
Adding includes to additional query/mutations
tylerjroach Jul 27, 2023
f4ca874
Rename .java to .kt
tylerjroach Jul 27, 2023
5899222
Add includes to ModelSubscription
tylerjroach Jul 27, 2023
866791b
Rename .java to .kt
tylerjroach Jul 27, 2023
749fedd
Add includes to ModelMutation
tylerjroach Jul 27, 2023
deb7432
lint
tylerjroach Jul 27, 2023
7fd0622
Fix tests
tylerjroach Jul 27, 2023
d1274cb
Move API request options to be set for selection set
tylerjroach Jul 28, 2023
fe24f28
remove unused file
tylerjroach Jul 28, 2023
b57fccb
Better Java support
tylerjroach Aug 2, 2023
1cf07b1
Fix a few selection set merge issues.
tylerjroach Aug 2, 2023
e375efa
remove kotlin core dep on aws-api
tylerjroach Aug 3, 2023
5f6bdb2
Create LazyList during response deserialization
tylerjroach Aug 7, 2023
e1b5422
Move lazy list to use PaginatedResult
tylerjroach Aug 21, 2023
3012354
Pass apiName through to gson response parser to support custom apiNam…
tylerjroach Aug 22, 2023
28248b3
lint
tylerjroach Aug 22, 2023
9591ab3
Fix tests
tylerjroach Aug 22, 2023
e589cfc
Merge branch 'main' into tjroach/custom-selection-set
tylerjroach Aug 23, 2023
944068f
Rename .java to .kt
tylerjroach Aug 23, 2023
0d83f26
cleanup
tylerjroach Aug 23, 2023
ed39093
cleanup
tylerjroach Aug 23, 2023
c972f82
naming changes
tylerjroach Aug 28, 2023
5c19a55
lazy list loading updates
tylerjroach Sep 1, 2023
3eecd22
hide internal apis
tylerjroach Sep 1, 2023
55690e3
Merge branch 'main' into tjroach/custom-selection-set
tylerjroach Sep 1, 2023
b7d2356
model updates
tylerjroach Sep 1, 2023
181bf11
change includeAssociations to includeRelationships
tylerjroach Sep 12, 2023
cae2667
Remove unused targetNames from HasMany interface
tylerjroach Sep 12, 2023
d0883d1
includeRelationships fix
tylerjroach Sep 12, 2023
daa6da5
includeRelationships fix
tylerjroach Sep 14, 2023
ae172c4
single execution of lazy model reference
tylerjroach Sep 14, 2023
b0d5cde
Merge branch 'main' into tjroach/custom-selection-set
tylerjroach Sep 19, 2023
d26a044
Fail DataStore initialization if lazy model type discovered
tylerjroach Sep 19, 2023
e212550
lint
tylerjroach Sep 19, 2023
0cb4150
fix tests
tylerjroach Sep 21, 2023
2329d7a
Code cleanup to ensure public contracts are correct
tylerjroach Sep 21, 2023
a16001d
checkstyle fixes
tylerjroach Sep 21, 2023
1f4db83
test additions
tylerjroach Sep 22, 2023
f282d1f
Add copyright notices
tylerjroach Sep 22, 2023
1fd4e1f
lint
tylerjroach Sep 22, 2023
9702183
tests
tylerjroach Sep 22, 2023
106e6e5
sipmle selection set tests
tylerjroach Sep 25, 2023
c2beec2
When belongsTo orHas one, fetch single value rather than get first re…
tylerjroach Sep 25, 2023
0441d2d
change name of lazy deserializer file
tylerjroach Sep 26, 2023
bff9003
adding ll/css tests
tylerjroach Sep 26, 2023
2c6059b
Adding subscribe tests
tylerjroach Sep 26, 2023
f1d41c9
Adding more ll/css subscribe tests
tylerjroach Sep 27, 2023
b3b8ade
Fix case of null lazy reference
tylerjroach Sep 27, 2023
e7eb2bf
Add complex lazy cases
tylerjroach Sep 27, 2023
49ab9b5
Add ApiLazyModelReference unit tests
tylerjroach Sep 28, 2023
cc005b9
Add testability to ApiModelListTypes
tylerjroach Sep 28, 2023
4c00715
Added testing
tylerjroach Sep 28, 2023
c507e05
suppress datastore generated files
tylerjroach Sep 28, 2023
3e661ea
lint
tylerjroach Sep 28, 2023
322e88b
update ci config
tylerjroach Sep 28, 2023
9224baf
update tests
tylerjroach Sep 29, 2023
f6e4009
add schema / revert mistake
tylerjroach Sep 29, 2023
3547619
lint
tylerjroach Sep 29, 2023
7749026
fix tests
tylerjroach Sep 29, 2023
6c41c25
attempt to fix tests
tylerjroach Sep 29, 2023
a617cdf
gson model deserialization cleanup
tylerjroach Oct 2, 2023
d6959bd
add comments for clarity
tylerjroach Oct 2, 2023
381b09b
add comments for clarity
tylerjroach Oct 2, 2023
f23ccf9
PR Comments
tylerjroach Oct 3, 2023
481f6e7
semaphore to mutex
tylerjroach Oct 4, 2023
8dc06a4
update comment
tylerjroach Oct 4, 2023
24b8a4d
Pr comments
tylerjroach Oct 4, 2023
e262410
Additional tests
tylerjroach Oct 4, 2023
aa2f90e
Merge branch 'main' into tjroach/custom-selection-set
tylerjroach Oct 9, 2023
bc19309
Downgrade androidx orchestrator and runner
tylerjroach Oct 10, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ __pycache__/
**/amplifyconfiguration_v2.json
**/credentials.json
**/google_client_creds.json
**/amplifyconfiguration*.json

# IDE files
.idea/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,19 @@ public final class ApiGraphQLRequestOptions implements GraphQLRequestOptions {
private static final String ITEMS_KEY = "items";
private static final String NEXT_TOKEN_KEY = "nextToken";

private static final int DEFAULT_MAX_DEPTH = 2;

private int maxDepth = DEFAULT_MAX_DEPTH;

/**
* Public constructor to create ApiGraphQLRequestOptions.
*/
public ApiGraphQLRequestOptions() {}

ApiGraphQLRequestOptions(int maxDepth) {
this.maxDepth = maxDepth;
}

@NonNull
@Override
public List<String> paginationFields() {
Expand All @@ -47,7 +60,7 @@ public String listField() {

@Override
public int maxDepth() {
return 2;
return maxDepth;
}

@NonNull
Expand Down
ankpshah marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,12 @@
import com.amplifyframework.api.graphql.MutationType;
import com.amplifyframework.core.model.AuthRule;
import com.amplifyframework.core.model.AuthStrategy;
import com.amplifyframework.core.model.LoadedModelReference;
import com.amplifyframework.core.model.Model;
import com.amplifyframework.core.model.ModelAssociation;
import com.amplifyframework.core.model.ModelField;
import com.amplifyframework.core.model.ModelIdentifier;
import com.amplifyframework.core.model.ModelReference;
import com.amplifyframework.core.model.ModelSchema;
import com.amplifyframework.core.model.SerializedCustomType;
import com.amplifyframework.core.model.SerializedModel;
Expand Down Expand Up @@ -171,7 +173,7 @@ public static Map<String, Object> getDeleteMutationInputMap(
@NonNull ModelSchema schema, @NonNull Model instance) throws AmplifyException {
final Map<String, Object> input = new HashMap<>();
for (String fieldName : schema.getPrimaryIndexFields()) {
input.put(fieldName, extractFieldValue(fieldName, instance, schema));
input.put(fieldName, extractFieldValue(fieldName, instance, schema, true));
}
return input;
}
Expand Down Expand Up @@ -224,21 +226,30 @@ private static Map<String, Object> extractFieldLevelData(
continue;
}

Object fieldValue = extractFieldValue(modelField.getName(), instance, schema);
Object fieldValue = extractFieldValue(modelField.getName(), instance, schema, false);
ankpshah marked this conversation as resolved.
Show resolved Hide resolved
Object underlyingFieldValue = fieldValue;
if (modelField.isModelReference() && fieldValue != null) {
ModelReference<?> modelReference = (ModelReference<?>) fieldValue;
if (modelReference instanceof LoadedModelReference) {
underlyingFieldValue = ((LoadedModelReference) modelReference).getValue();
}
}

if (association == null) {
result.put(fieldName, fieldValue);
} else if (association.isOwner()) {
if (fieldValue == null && MutationType.CREATE.equals(type)) {
if ((fieldValue == null ||
(modelField.isModelReference() && underlyingFieldValue == null)) &&
MutationType.CREATE.equals(type)) {
// Do not set null values on associations for create mutations.
ankpshah marked this conversation as resolved.
Show resolved Hide resolved
} else if (schema.getVersion() >= 1 && association.getTargetNames() != null
ankpshah marked this conversation as resolved.
Show resolved Hide resolved
&& association.getTargetNames().length > 0) {
// When target name length is more than 0 there are two scenarios, one is when
// there is custom primary key and other is when we have composite primary key.
insertForeignKeyValues(result, modelField, fieldValue, association);
insertForeignKeyValues(result, modelField, fieldValue, underlyingFieldValue, association);
} else {
String targetName = association.getTargetName();
result.put(targetName, extractAssociateId(modelField, fieldValue));
result.put(targetName, extractAssociateId(modelField, fieldValue, underlyingFieldValue));
}
}
// Ignore if field is associated, but is not a "belongsTo" relationship
Expand All @@ -250,58 +261,94 @@ private static void insertForeignKeyValues(
Map<String, Object> result,
ModelField modelField,
Object fieldValue,
Object underlyingFieldValue,
ModelAssociation association) {
if (modelField.isModel() && fieldValue == null) {
// When there is no model field value, set null for removal of values or deassociation.
// When there is no model field value, set null for removal of values or association.
for (String key : association.getTargetNames()) {
result.put(key, null);
}
} else if (modelField.isModel() && fieldValue instanceof Model) {
if (((Model) fieldValue).resolveIdentifier() instanceof ModelIdentifier<?>) {
final ModelIdentifier<?> primaryKey = (ModelIdentifier<?>) ((Model) fieldValue).resolveIdentifier();
ListIterator<String> targetNames = Arrays.asList(association.getTargetNames()).listIterator();
Iterator<? extends Serializable> sortedKeys = primaryKey.sortedKeys().listIterator();
} else if ((modelField.isModel() || modelField.isModelReference()) && underlyingFieldValue instanceof Model) {
mattcreaser marked this conversation as resolved.
Show resolved Hide resolved
if (((Model) underlyingFieldValue).resolveIdentifier() instanceof ModelIdentifier<?>) {
// Here, we are unwrapping our ModelReference to grab our foreign keys.
// If we have a ModelIdentifier, we can pull all the key values, but we don't have
// the key names. We must grab those from the association target names
final ModelIdentifier<?> primaryKey =
(ModelIdentifier<?>) ((Model) underlyingFieldValue).resolveIdentifier();
ListIterator<String> targetNames =
Arrays.asList(association.getTargetNames()).listIterator();
Iterator<? extends Serializable> sortedKeys =
primaryKey.sortedKeys().listIterator();

result.put(targetNames.next(), primaryKey.key());

while (targetNames.hasNext()) {
result.put(targetNames.next(), sortedKeys.next());
}
} else if ((fieldValue instanceof SerializedModel)) {
SerializedModel serializedModel = ((SerializedModel) fieldValue);
} else if ((underlyingFieldValue instanceof SerializedModel)) {
SerializedModel serializedModel = ((SerializedModel) underlyingFieldValue);
ModelSchema serializedSchema = serializedModel.getModelSchema();
if (serializedSchema != null &&
serializedSchema.getPrimaryIndexFields().size() > 1) {

ListIterator<String> primaryKeyFieldsIterator = serializedSchema.getPrimaryIndexFields()
ListIterator<String> primaryKeyFieldsIterator =
serializedSchema.getPrimaryIndexFields()
.listIterator();
for (String targetName : association.getTargetNames()) {
result.put(targetName, serializedModel.getSerializedData()
.get(primaryKeyFieldsIterator.next()));
}
} else {
result.put(association.getTargetNames()[0], ((Model) fieldValue).resolveIdentifier().toString());
// our key was not a ModelIdentifier type, so it must be a singular primary key
result.put(
association.getTargetNames()[0],
((Model) underlyingFieldValue).resolveIdentifier().toString()
);
}
} else {
result.put(association.getTargetNames()[0], ((Model) fieldValue).resolveIdentifier().toString());
// our key was not a ModelIdentifier type, so it must be a singular primary key
result.put(
association.getTargetNames()[0],
((Model) underlyingFieldValue).resolveIdentifier().toString()
);
}
} else if (modelField.isModelReference() && fieldValue instanceof ModelReference) {
// Here we are unwrapping our ModelReference and inserting
Map<String, Object> identifiers = ((ModelReference<?>) fieldValue).getIdentifier();
if (identifiers.isEmpty()) {
for (String key : association.getTargetNames()) {
result.put(key, null);
}
}
}
}

private static Object extractAssociateId(ModelField modelField, Object fieldValue) {
if (modelField.isModel() && fieldValue instanceof Model) {
return ((Model) fieldValue).resolveIdentifier();
private static Object extractAssociateId(ModelField modelField, Object fieldValue, Object underlyingFieldValue) {
if ((modelField.isModel() || modelField.isModelReference()) && underlyingFieldValue instanceof Model) {
return ((Model) underlyingFieldValue).resolveIdentifier();
} else if (modelField.isModel() && fieldValue instanceof Map) {
return ((Map<?, ?>) fieldValue).get("id");
} else if (modelField.isModel() && fieldValue == null) {
// When there is no model field value, set null for removal of values or deassociation.
return null;
} else if (modelField.isModelReference() && fieldValue instanceof ModelReference) {
Map<String, Object> identifiers = ((ModelReference<?>) fieldValue).getIdentifier();
if (identifiers.isEmpty()) {
return null;
} else {
return identifiers.get("id");
}
} else {
throw new IllegalStateException("Associated data is not Model or Map.");
}
}

private static Object extractFieldValue(String fieldName, Model instance, ModelSchema schema)
private static Object extractFieldValue(
String fieldName,
Model instance,
ModelSchema schema,
Boolean extractLazyValue
)
throws AmplifyException {
if (instance instanceof SerializedModel) {
SerializedModel serializedModel = (SerializedModel) instance;
Expand All @@ -316,7 +363,13 @@ private static Object extractFieldValue(String fieldName, Model instance, ModelS
try {
Field privateField = instance.getClass().getDeclaredField(fieldName);
privateField.setAccessible(true);
return privateField.get(instance);
Object fieldInstance = privateField.get(instance);
// In some cases, we don't want to return a ModelReference value. If extractLazyValue
// is set, we unwrap the reference to grab to value underneath
if (extractLazyValue && fieldInstance != null && privateField.getType() == LoadedModelReference.class) {
return ((LoadedModelReference<?>) fieldInstance).getValue();
}
return fieldInstance;
} catch (Exception exception) {
throw new AmplifyException(
"An invalid field was provided. " + fieldName + " is not present in " + schema.getName(),
Expand Down
Loading
Loading