-
Notifications
You must be signed in to change notification settings - Fork 70
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPIKE] Discovery for epic text-field migration. #14995
Comments
This is the question that should be reviewed first. It can be looked at at the field storage level; field instances are derived from field storage config and the updates can/should happen together. The code here handles this though it should be reviewed: #14028 The field types that should be updated are Some percentage of the
This will only be known based on gathering the set of field storages that need updated; see above.
#14028 is an implementation. The backup tables are not dropped at the end of the process, which would allow for confirmation by comparing the tables. The table data itself should not change between the two versions of the tables, and so they can in theory be checked against (hashing table dumps might be the quickest/easiest). The implementation in the PR is not the only way, but it is a way.
Revision tables are handled by the PR code, in addition to the primary field table.
Shouldn't matter; a cloned node is a new node, unless I misunderstand.
Unknown; worth looking at. At the same time, a field is a field is a field. Centralized content shouldn't matter here. The only thing that is really changing here is a) the text limit is removed, and b) the field form widget is changed.
Part of the planning process; that said, the POC showed that the updates did not take particularly long. Depending on the number of fields that need updated, this might be staged over ~5-10 deploys.
To live in a shack by the river and eat mud for fun. In seriousness, we (being CMS Team) probably want to look at what kinds of fields we want to allow for use, and to set up governance over what field types get used for what purposes. The cases where we would use a field with a database-enforced character limit are exceedingly rare.
To live in a shack by the river and sculpt mud for fun.
I don't have useful knowledge about this. |
Text field: only one. https://prod.cms.va.gov/admin/reports/content-model/fields?field_name=&field_type=text&entity_type=&bundle=&label=&order=field_type&sort=asc String fields: many: https://prod.cms.va.gov/admin/reports/content-model/fields?field_name=&field_type=string&entity_type=&bundle=&label=&order=field_type&sort=asc String fields sorted by translatability; you only want to update |
Current plans for FY2024 for translation priorities: https://github.com/department-of-veterans-affairs/va.gov-team/blob/master/teams/vsa/teams/sitewide-content/translation-work/State%20Department%20translation%20work/FY2024%20translation%20planning.md |
The Content Model Fields view didn't really the work the way I wanted it to, and I wasn't sure I understood what was happening under the hood, so I wrote a Bash script to generate the information I needed from the raw config YAML. #!/bin/bash
declare -A translatable_fields_by_entity_type
declare -A non_translatable_fields_by_entity_type
config_dir="./config/sync"
for file in $(find "${config_dir}" -name 'field.storage.*.yml'); do
field_type=$(yq e '.type' "${file}")
if [ "${field_type}" == "text" ] || [ "${field_type}" == "string" ]; then
filename="$(basename -- "${file}" ".yml")"
entity_type="${filename#field.storage.}"
entity_type="${entity_type%%.*}"
field_name="${filename##*.}"
translatable="false"
for instance_file in $(find "${config_dir}" -name "field.field.${entity_type}.*.${field_name}.yml"); do
instance_translatable=$(yq e '.translatable' "${instance_file}")
if [ "$instance_translatable" == "true" ]; then
translatable="true"
break
fi
done
if [ "$translatable" == "true" ]; then
translatable_fields_by_entity_type[$entity_type]+="$field_name "
else
non_translatable_fields_by_entity_type[$entity_type]+="$field_name "
fi
fi
done
for entity_type in "${!translatable_fields_by_entity_type[@]}"; do
echo "Entity type: $entity_type"
echo "Translatable Fields:"
for field in ${translatable_fields_by_entity_type[$entity_type]}; do
echo " - $field"
done
echo ""
done
for entity_type in "${!non_translatable_fields_by_entity_type[@]}"; do
echo "Entity type: $entity_type"
echo "Non-Translatable Fields:"
for field in ${non_translatable_fields_by_entity_type[$entity_type]}; do
echo " - $field"
done
echo ""
done This yielded the following list:
As far as I can tell, this matches the data presented by the Content Model Fields view but in a form that I don't have to mess with as much. Next, I'll go through and manually prune the fields that I don't believe would be appropriate to migrate. |
The items I've removed, and the reasons why. The burden of proof is on the field to be obviously incapable of realistically exceeding the expressed limits.
So here are the fields that we should deal with:
|
So I wrote this script to tease out which content types use paragraphs whose fields will be affected: #!/bin/bash
target_fields=(
field_phone_label
field_alert_heading
field_text_expander
field_error_message
field_section_header
field_question
field_email_label
field_button_label
field_loading_message
field_short_phrase_with_a_number
field_title
field_link_summary
)
declare -A content_types_by_paragraph
config_dir="./config/sync"
for file in $(find "${config_dir}" -name 'field.field.node.*.yml'); do
field_type=$(yq e '.field_type' "${file}")
if [ "${field_type}" != "entity_reference_revisions" ]; then
continue
fi
filename="$(basename -- "${file}" ".yml")"
content_type="${filename#field.field.node.}"
content_type="${content_type%%.*}"
paragraph_types=$(yq e '.settings.handler_settings.target_bundles' "${file}")
for paragraph_type in $paragraph_types; do
for target_field in "${target_fields[@]}"; do
if [ -f "${config_dir}/field.field.paragraph.${paragraph_type}.${target_field}.yml" ]; then
content_types_by_paragraph[$content_type]=true
fi
done
done
done
echo "Content types using target fields in paragraphs:"
for content_type in "${!content_types_by_paragraph[@]}"; do
echo " - $content_type"
done For translatable fields, this works out to:
For non-translatable fields, this works out to:
So the content types that will be affected indirectly via modifications to fields on paragraphs should be:
|
To identify the nodes using targeted fields directly, I wrote this script: #!/bin/bash
config_dir="./config/sync"
target_fields=(
field_teaser_text
field_description
field_home_page_hub_label
)
declare -A content_types_by_field
for target_field in ${target_fields[@]}; do
for file in $(find "${config_dir}" -name "field.field.node.*.$target_field.yml"); do
filename="$(basename -- "${file}" ".yml")"
bundle="${filename#field.field.node.}"
bundle="${bundle%%.*}"
content_types_by_field[$bundle]=true
done
done
echo "Content types using target fields:"
for content_type in "${!content_types_by_field[@]}"; do
echo " - $content_type"
done This yielded the following content types for the translatable fields:
and the following content types for non-translatable fields:
So the content types that will be affected directly will be:
|
I also wanted to address the paragraphs that use #!/bin/bash
target_fields=(
field_alert_title
field_promo_headline
field_title
field_primary_cta_button_text
)
declare -A block_content_bundles
declare -A paragraph_bundles_by_block_content_bundle
declare -A paragraph_bundles_by_paragraph_bundle
declare -A content_types_by_block_content_bundle
declare -A content_types_by_paragraph_bundle
config_dir="./config/sync"
for target_field in ${target_fields[@]}; do
>&2 echo "Looking for field ${target_field}"
for file in $(find "${config_dir}" -name "field.field.block_content.*.${target_field}.yml"); do
filename="$(basename -- "${file}" ".yml")"
bundle="${filename#field.field.block_content.}"
bundle="${bundle%%.*}"
block_content_bundles[$bundle]=true
>&2 echo " Found block_content bundle ${bundle}"
done
done
>&2 echo "Looking for paragraph bundles that use block_content bundles"
for file in $(find "${config_dir}" -name "field.field.paragraph.*.yml"); do
field_type=$(yq e '.field_type' "${file}")
handler=$(yq e '.settings.handler' "${file}")
if [ "${field_type}" != "entity_reference" ] || [ "${handler}" != "default:block_content" ]; then
continue
fi
filename="$(basename -- "${file}" ".yml")"
bundle="${filename#field.field.paragraph.}"
bundle="${bundle%%.*}"
>&2 echo " Checking paragraph bundle ${bundle}"
for block_content_bundle in "${!block_content_bundles[@]}"; do
>&2 echo " Looking for block_content bundle ${block_content_bundle}"
paragraph_bundles_by_block_content_bundle[$bundle]=true
>&2 echo " Found paragraph bundle ${bundle}"
done
done
>&2 echo "Looking for paragraph bundles that use paragraph bundles that use block_content bundles"
for file in $(find "${config_dir}" -name "field.field.paragraph.*.yml"); do
field_type=$(yq e '.field_type' "${file}")
handler=$(yq e '.settings.handler' "${file}")
if [ "${field_type}" != "entity_reference_revisions" ] || [ "${handler}" != "default:paragraph" ]; then
continue
fi
filename="$(basename -- "${file}" ".yml")"
bundle="${filename#field.field.paragraph.}"
bundle="${bundle%%.*}"
paragraph_types=$(yq e '.settings.handler_settings.target_bundles' "${file}")
>&2 echo " Checking paragraph bundle ${bundle}"
for paragraph_bundle in "${!paragraph_bundles_by_block_content_bundle[@]}"; do
>&2 echo " Looking for paragraph bundle ${paragraph_bundle}"
for paragraph_type in $paragraph_types; do
>&2 echo " Checking ${paragraph_bundle} against ${paragraph_type}"
if [ "${paragraph_bundle}" == "${paragraph_type}" ]; then
paragraph_bundles_by_paragraph_bundle[$bundle]=true
>&2 echo " Found paragraph bundle ${bundle}"
continue 2
fi
done
done
done
>&2 echo "Looking for content types that use block_content bundles"
for file in $(find "${config_dir}" -name "field.field.node.*.yml"); do
field_type=$(yq e '.field_type' "${file}")
handler=$(yq e '.settings.handler' "${file}")
if [ "${field_type}" != "entity_reference" ] || [ "${handler}" != "default:block_content" ]; then
continue
fi
filename="$(basename -- "${file}" ".yml")"
bundle="${filename#field.field.node.}"
bundle="${bundle%%.*}"
>&2 echo " Checking content type ${bundle}"
for block_content_bundle in "${!block_content_bundles[@]}"; do
>&2 echo " Looking for block_content bundle ${block_content_bundle}"
content_types_by_block_content_bundle[$bundle]=true
>&2 echo " Found content type ${bundle}"
done
done
>&2 echo "Looking for content types that use paragraph bundles that use block_content bundles"
for file in $(find "${config_dir}" -name "field.field.node.*.yml"); do
field_type=$(yq e '.field_type' "${file}")
handler=$(yq e '.settings.handler' "${file}")
if [ "${field_type}" != "entity_reference_revisions" ] || [ "${handler}" != "default:paragraph" ]; then
continue
fi
filename="$(basename -- "${file}" ".yml")"
bundle="${filename#field.field.node.}"
bundle="${bundle%%.*}"
paragraph_types=$(yq e '.settings.handler_settings.target_bundles' "${file}")
>&2 echo " Checking content type ${bundle}"
for paragraph_bundle in "${!paragraph_bundles_by_paragraph_bundle[@]}"; do
>&2 echo " Looking for paragraph bundle ${paragraph_bundle}"
for paragraph_type in $paragraph_types; do
>&2 echo " Checking ${paragraph_bundle} against ${paragraph_type}"
if [ "${paragraph_bundle}" == "${paragraph_type}" ]; then
content_types_by_paragraph_bundle[$bundle]=true
>&2 echo " Found content type ${bundle}"
continue 2
fi
done
done
for paragraph_bundle in "${!paragraph_bundles_by_block_content_bundle[@]}"; do
>&2 echo " Looking for paragraph bundle ${paragraph_bundle}"
for paragraph_type in $paragraph_types; do
>&2 echo " Checking ${paragraph_bundle} against ${paragraph_type}"
if [ "${paragraph_bundle}" == "${paragraph_type}" ]; then
content_types_by_paragraph_bundle[$bundle]=true
>&2 echo " Found content type ${bundle}"
continue 2
fi
done
done
done
echo "Content types using block_content bundles via paragraphs:"
for content_type in "${!content_types_by_paragraph_bundle[@]}"; do
echo " - $content_type"
done
echo "Content types using block_content bundles directly:"
for content_type in "${!content_types_by_block_content_bundle[@]}"; do
echo " - $content_type"
done After a bit, it produced this output:
Or, combined:
|
Finally, I wanted to get the entity types affected via taxonomy terms: #!/bin/bash
target_fields=(
field_description
field_vba_com_conditions
field_vet_center_friendly_name
field_cms_option_label
field_vba_friendly_name
field_vet_center_com_conditions
field_va_benefit_plain_name
field_commonly_treated_condition
field_also_known_as
)
declare -A taxonomy_vocabularies_by_field
declare -A paragraph_bundles_by_vocabulary
declare -A content_types_by_vocabulary
declare -A content_types_by_paragraph_bundle
config_dir="./config/sync"
>&2 echo "Looking for taxonomy vocabularies that use target fields"
for target_field in ${target_fields[@]}; do
for file in $(find "${config_dir}" -name "field.field.taxonomy_term.*.${target_field}.yml"); do
filename="$(basename -- "${file}" ".yml")"
bundle="${filename#field.field.taxonomy_term.}"
bundle="${bundle%%.*}"
taxonomy_vocabularies_by_field[$bundle]=true
>&2 echo " Found taxonomy vocabulary ${bundle}"
done
done
>&2 echo "Looking for paragraph bundles that use taxonomy vocabularies"
for file in $(find "${config_dir}" -name "field.field.paragraph.*.yml"); do
field_type=$(yq e '.field_type' "${file}")
handler=$(yq e '.settings.handler' "${file}")
if [ "${field_type}" != "entity_reference" ] || [ "${handler}" != "default:taxonomy_term" ]; then
continue
fi
filename="$(basename -- "${file}" ".yml")"
bundle="${filename#field.field.paragraph.}"
field_name="${bundle##*.}"
bundle="${bundle%%.*}"
>&2 echo " Checking paragraph bundle ${bundle}, field ${field_name}"
for taxonomy_vocabulary in "${!taxonomy_vocabularies_by_field[@]}"; do
>&2 echo " Looking for taxonomy vocabulary ${taxonomy_vocabulary}"
paragraph_bundles_by_vocabulary[$bundle]=true
>&2 echo " Found paragraph bundle ${bundle}"
done
done
>&2 echo "Looking for paragraph bundles that use paragraph bundles that use taxonomy vocabularies"
for file in $(find "${config_dir}" -name "field.field.paragraph.*.yml"); do
field_type=$(yq e '.field_type' "${file}")
handler=$(yq e '.settings.handler' "${file}")
if [ "${field_type}" != "entity_reference_revisions" ] || [ "${handler}" != "default:paragraph" ]; then
continue
fi
filename="$(basename -- "${file}" ".yml")"
bundle="${filename#field.field.paragraph.}"
bundle="${bundle%%.*}"
paragraph_types=$(yq e '.settings.handler_settings.target_bundles[]' "${file}")
>&2 echo " Checking paragraph bundle ${bundle}"
for paragraph_bundle in "${!paragraph_bundles_by_vocabulary[@]}"; do
>&2 echo " Looking for paragraph bundle ${paragraph_bundle}"
for paragraph_type in $paragraph_types; do
>&2 echo " Checking ${paragraph_type} against ${paragraph_bundle}"
if [ "${paragraph_bundle}" == "${paragraph_type}" ]; then
paragraph_bundles_by_paragraph_bundle[$bundle]=true
>&2 echo " Found paragraph bundle ${bundle}"
continue 2
fi
done
done
done
# Determine which content types use the taxonomy vocabularies directly.
for file in $(find "${config_dir}" -name "field.field.node.*.yml"); do
field_type=$(yq e '.field_type' "${file}")
if [ "${field_type}" != "entity_reference" ]; then
continue
fi
filename="$(basename -- "${file}" ".yml")"
content_type="${filename#field.field.node.}"
content_type="${content_type%%.*}"
taxonomy_vocabularies=$(yq e '.settings.handler_settings.target_bundles[]' "${file}")
for taxonomy_vocabulary in $taxonomy_vocabularies; do
content_types_by_vocabulary[$content_type]=true
done
done
>&2 echo "Looking for content types that use paragraph bundles that use taxonomy vocabularies"
for file in $(find "${config_dir}" -name "field.field.node.*.yml"); do
field_type=$(yq e '.field_type' "${file}")
handler=$(yq e '.settings.handler' "${file}")
if [ "${field_type}" != "entity_reference_revisions" ] || [ "${handler}" != "default:paragraph" ]; then
continue
fi
filename="$(basename -- "${file}" ".yml")"
bundle="${filename#field.field.node.}"
bundle="${bundle%%.*}"
paragraph_types=$(yq e '.settings.handler_settings.target_bundles[]' "${file}")
>&2 echo " Checking content type ${bundle}"
for paragraph_bundle in "${!paragraph_bundles_by_paragraph_bundle[@]}"; do
>&2 echo " Looking for paragraph bundle ${paragraph_bundle}"
for paragraph_type in $paragraph_types; do
>&2 echo " Checking ${paragraph_bundle} against ${paragraph_type}"
if [ "${paragraph_bundle}" == "${paragraph_type}" ]; then
content_types_by_paragraph_bundle[$bundle]=true
>&2 echo " Found content type ${bundle}"
continue 2
fi
done
done
for paragraph_bundle in "${!paragraph_bundles_by_vocabulary[@]}"; do
>&2 echo " Looking for paragraph bundle ${paragraph_bundle}"
for paragraph_type in $paragraph_types; do
>&2 echo " Checking ${paragraph_bundle} against ${paragraph_type}"
if [ "${paragraph_bundle}" == "${paragraph_type}" ]; then
content_types_by_paragraph_bundle[$bundle]=true
>&2 echo " Found content type ${bundle}"
continue 2
fi
done
done
done
echo "Content types using taxonomy vocabularies via paragraphs:"
for content_type in "${!content_types_by_paragraph_bundle[@]}"; do
echo " - $content_type"
done
echo "Content types using taxonomy vocabularies directly:"
for content_type in "${!content_types_by_vocabulary[@]}"; do
echo " - $content_type"
done which yielded:
|
It's also possible that there are other things, e.g. taxonomy vocabularies referenced by block_content referenced by paragraphs referenced by paragraphs referenced by paragraphs referenced by nodes, but I think this is probably pretty accurate. Assuming I wrote these scripts correctly, which is by no means assured. Casual plan, which I haven't thought about too deeply, is to avoid performing this migration in update hooks and instead put the CMS into maintenance mode and perform all of the modifications live. The main reason for this is that I'm expecting to use Drupal services heavily to verify correctness, and I'm leery about the decreased consistency/coherence of the update phase. Also, I want to write tests for the code, and some commands to ease insight and development. Tomorrow I'll work on gathering the rest of the data I wanted to gather, and stretching and extruding it into some kind of useful report. |
Next, I wanted to get a sense of the programmatic uses of each of these fields. Where does x appear in code? Is it used in calculations in presave hooks? Is it modified or validated in some specific way? What entanglements can I expect to encounter? So I wrote this simple script... well, it started out simple and turned into a comparative deep dive (for me, at least) into #!/bin/bash
search_dir="docroot/modules/custom"
declare -a fields=(
field_teaser_text
field_description
field_home_page_hub_label
field_name_first
field_va_form_title
field_geographical_identifier
field_vamc_system_official_name
field_event_cost
field_non_va_official_name
field_last_name
field_official_name
field_applied_to
field_hero_blurb
field_clp_video_panel_header
field_clp_what_you_can_do_header
field_va_form_name
field_clp_resources_header
field_clp_spotlight_header
field_location_humanreadable
field_clp_events_header
field_suffix
field_clp_stories_header
field_phone_label
field_alert_heading
field_text_expander
field_error_message
field_section_header
field_question
field_email_label
field_button_label
field_loading_message
field_short_phrase_with_a_number
field_title
field_link_summary
field_additional_hours_info
field_cc_documentor_title
field_checklist_items
field_magichead_heading
field_clinic_name
field_building_name_number
field_header
field_alert_title
field_promo_headline
field_title
field_primary_cta_button_text
field_link_summary
field_label
field_description
field_vba_com_conditions
field_vet_center_friendly_name
field_cms_option_label
field_vba_friendly_name
field_vet_center_com_conditions
field_va_benefit_plain_name
field_commonly_treated_condition
field_also_known_as
)
for field in "${fields[@]}"; do
echo "------------------------------------------------------"
echo "Searching for field: $field"
echo "------------------------------------------------------"
last_file=""
grep -r -n -C 4 "$field" "$search_dir" | awk -F '[-:]' '
{
filename = $1;
line_num = $2;
separator = $3;
rest = substr($0, length($1) + length($2) + length($3) + 4);
if (last_file != filename) {
print filename;
last_file = filename;
}
printf "%d%s %s\n", line_num, separator, rest;
}'
done That yielded the following output (let's just ignore that it prunes out any dashes or semicolons in the matched text -- I can't be bothered to fix that for the sake of this):
|
I reviewed the above findings and added a table for the fields and their respective migration- and code-based entanglements. Then I created a raw list of content types affected, which... I think is all of them 😂 I created the following script to count the affected nodes and revisions for each content type: #!/bin/bash
total_nodes=0
total_revisions=0
content_types=(
"banner"
"basic_landing_page"
"campaign_landing_page"
"centralized_content"
"checklist"
"documentation_page"
"event"
"event_listing"
"faq_multiple_q_a"
"full_width_banner_alert"
"health_care_local_facility"
"health_care_local_health_service"
"health_care_region_detail_page"
"health_care_region_page"
"health_services_listing"
"landing_page"
"leadership_listing"
"locations_listing"
"media_list_images"
"media_list_videos"
"nca_facility"
"news_story"
"office"
"outreach_asset"
"page"
"person_profile"
"press_release"
"press_releases_listing"
"promo_banner"
"publication_listing"
"q_a"
"regional_health_care_service_des"
"service_region"
"step_by_step"
"story_listing"
"support_resources_detail_page"
"support_service"
"va_form"
"vamc_operating_status_and_alerts"
"vamc_system_billing_insurance"
"vamc_system_medical_records_offi"
"vamc_system_policies_page"
"vamc_system_register_for_care"
"vba_facility"
"vba_facility_service"
"vet_center"
"vet_center_cap"
"vet_center_facility_health_servi"
"vet_center_locations_list"
"vet_center_mobile_vet_center"
"vet_center_outstation"
"vha_facility_nonclinical_service"
)
for type in "${content_types[@]}"; do
num_nodes=$(drush sql:query "SELECT COUNT(*) FROM node WHERE type=\"$type\";")
num_revisions=$(drush sql:query "SELECT COUNT(*) FROM node_revision nr JOIN node_field_data nfd ON nr.nid = nfd.nid WHERE nfd.type=\"$type\";")
echo "Content type: $type, Nodes: $num_nodes, Revisions: $num_revisions"
total_nodes=$((total_nodes + num_nodes))
total_revisions=$((total_revisions + num_revisions))
done
echo "----------------------------------"
echo "Total Nodes: $total_nodes"
echo "Total Revisions: $total_revisions" This yielded the following output:
|
I slightly modified the script above to yield similar results for
and for
|
SummaryEvery field that stores text for translation and is potentially affected by this issue
Fields and Their Migration Entanglements
Fields and Their Code Entanglements
Affected Content Types, Nodes, and Revisions
Affected
|
Bundle | Entities | Revisions |
---|---|---|
alert | 98 | 766 |
benefit_promo | 1 | 1 |
cms_announcement | 1 | 1 |
cta_with_link | 1 | 2 |
news_promo | 1 | 3 |
promo | 144 | 424 |
Total | 246 | 1197 |
Affected taxonomy terms
Vocabulary | Terms | Revisions |
---|---|---|
administration | 570 | 1052 |
facility_supplemental_status | 3 | 33 |
health_care_service_taxonomy | 108 | 497 |
va_benefits_taxonomy | 0 | 0 |
Total | 681 | 1582 |
Relationship and potential impact of all changes
While this does seem to affect essentially everything directly or indirectly, I
didn't find any places in the CMS code that seemed intimately dependent upon
the structure and type of any given field. It's possible that these are
indirect, and will only pop up in actual use. There's not much I can do to
anticipate that.
The content-build
relationship with this work has an extremely broad cross-
section. I expect, though, that issues will be found by the content-build-gql
test or the full content build.
I don't see much reason to be concerned, but I can use the data gathered to
make some estimates on how much time the complete process will take. I still
think it is wise to make changes individually, allowing some time between each
batch. As my confidence grows in the approach, I might compound later batches.
Lessons Learned
This is ugly, and I don't like it. We shouldn't be in this predicament. Setting
a limit of 255 characters on a human text field in a database is just not very
forward-looking.
I'll place information about implementation in #13699.
Description
From Tim's issue:
Tim completed his proof-of-concept, but I am not Tim and therefore need to acquaint myself with this work. In addition, I have some questions that need to be answered:
Notes
Context
Acceptance Criteria
The text was updated successfully, but these errors were encountered: