Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite of batch.inc and added Drush command #39

Open
wants to merge 16 commits into
base: 7.x
Choose a base branch
from
Open
12 changes: 11 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Install as usual, see [this](https://drupal.org/documentation/install/modules-th

Set 'Last Modified Solr Field' and 'Maximum number of Islandora links to process at once' in Administration » Islandora » XML Sitemap Integration (admin/islandora/xmlsitemap).

![Configuration](https://camo.githubusercontent.com/407972e0a2c14bafd74924992c659021b800abb0/687474703a2f2f692e696d6775722e636f6d2f455a534f4b68372e706e67)
![Configuration](https://user-images.githubusercontent.com/2461961/35802085-ffafc758-0a6e-11e8-8a0c-e1f09e4d2a45.png)

### Notes

Expand All @@ -48,6 +48,16 @@ Please also note that objects marked as "inactive", whether manually or by using

Larger sites with greater than 100,000 objects may encounter issues during the sitemap building process with the default configuration, such as the process hanging around a specific number indefinitely or exiting the process entirely before completion. These users may want to try unchecking the "Prefetch URL aliases during sitemap generation" option found on the xmlsitemap admin configuration page (/admin/config/search/xmlsitemap/settings) and trying the process again.

## Bulk generation using drush
There is a Drush command available for the generation of sitemap links. This command allows you to optionally fetch a limited amount of objects (`limit`) similar to the hook_cron() and lets you define a custom amount to be fetched at once (`max_chunk_size`) from SOLR.

Command:
`drush islandora_xmlsitemap_generate [--max_chunk_size=100] [--limit=1000] [--regenerate]`

* The `max_chunk_size` defaults to 100
* If no `limit` is set all objects will be processed
* The `--regenerate` flag removes the "last_modified" value so will cause processing to start at the beginning. *Use with caution if you have lots of objects*.

## Documentation

This module's documentation is also available at [our wiki](https://wiki.duraspace.org/display/ISLANDORA/Islandora+XML+Sitemap).
Expand Down
25 changes: 19 additions & 6 deletions includes/admin.form.inc
Original file line number Diff line number Diff line change
Expand Up @@ -34,13 +34,22 @@ function islandora_xmlsitemap_admin_form($form, &$form_state) {
'#description' => t('Solr field in which we can perform sorting and range queries.'),
'#default_value' => variable_get('islandora_xmlsitemap_last_modified_field', 'fgs_lastModifiedDate_dt'),
);
$form['islandora_xmlsitemap_number_of_pids_to_process'] = array(
$form['islandora_xmlsitemap_generate_limit_cron'] = array(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@janjouketjalsma If you are going to change this variable name, you should add a hook_update_N() function to migrate the old setting from islandora_xmlsitemap_number_of_pids_to_process to islandora_xmlsitemap_generate_limit_cron and then unset the islandora_xmlsitemap_number_of_pids_to_process variable. Otherwise it will hang around forever.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once again, the old refrain, "Don't forget the uninstall hook!"
islandora_xmlsitemap_uninstall() - the variable name should be updated there too.

'#type' => 'textfield',
'#title' => 'Maximum Number of Islandora links to process at once',
'#title' => 'Maximum Number of Islandora links to process during hook_cron()',
'#size' => 10,
'#element_validate' => array('element_validate_integer_positive'),
'#default_value' => variable_get('islandora_xmlsitemap_number_of_pids_to_process', 1000),
'#description' => 'This is the number of Islandora/Fedora links we will process at once',
'#default_value' => variable_get('islandora_xmlsitemap_generate_limit_cron', 1000),
'#description' => 'This is the maximum number of Islandora/Fedora links we will automatically process when hook_cron() is called',
);

$form['islandora_xmlsitemap_generate_chunk_size'] = array(
'#type' => 'textfield',
'#title' => 'Amount of Islandora links to fetch and process at once',
'#size' => 10,
'#element_validate' => array('element_validate_integer_positive'),
'#default_value' => variable_get('islandora_xmlsitemap_generate_chunk_size', 100),
'#description' => 'This is the number of Islandora/Fedora links we will fetch and process at once (before updating the Last Modified value) when using the buttons below or when hook_cron() is called',
);

$form['actions'] = array(
Expand Down Expand Up @@ -74,15 +83,19 @@ function islandora_xmlsitemap_admin_form_submit(&$form, &$form_state) {

$to_set = array(
'islandora_xmlsitemap_last_modified_field',
'islandora_xmlsitemap_number_of_pids_to_process',
'islandora_xmlsitemap_generate_limit_cron',
'islandora_xmlsitemap_generate_chunk_size'
);
if ($button == 'generate' || $button == 'regenerate') {
if ($button == 'regenerate') {
variable_del('islandora_xmlsitemap_last_modified_value');
}

module_load_include('inc', 'islandora_xmlsitemap', 'includes/batch');
$batch = islandora_xmlsitemap_get_batch(100, -1);
$batch = islandora_xmlsitemap_get_batch(
variable_get('islandora_xmlsitemap_generate_chunk_size', 100),
NULL
);
batch_set($batch);
}
elseif ($button == 'submit') {
Expand Down
Loading