Merge pull request #885 from TOMToolkit/851-extending-target-model

851 extending target model

docs/api/tom_targets/models.rst

@@ -3,6 +3,10 @@ Models
 
 .. autosummary::
 
+.. automodule:: tom_targets.base_models
+    :members:
+    :special-members:
+
 .. automodule:: tom_targets.models
     :members:
     :special-members:

docs/brokers/create_broker.rst

@@ -47,7 +47,7 @@ a Django project directory of the form:
 Creating a Broker Module
 ************************
 
-In this example, we will create a broker named __MyBroker__.
+In this example, we will create a broker named **MyBroker**.
 
 Begin by creating a file ``my_broker.py``, and placing it in the inner ``mytom/`` directory
 of the project (in the directory with settings.py). ``my_broker.py`` will contain the classes that define our custom

docs/conf.py

@@ -52,7 +52,8 @@
     'sphinx.ext.autosectionlabel',
     'sphinx.ext.autosummary',
     'sphinx.ext.intersphinx',
-    'sphinx_panels'
+    'sphinx_panels',
+    'sphinx_copybutton',
 ]
 
 # Add any paths that contain templates here, relative to this directory.

docs/observing/observation_module.rst

@@ -92,7 +92,7 @@ class:
         'tom_observations.facilities.lco.LCOFacility',
         'tom_observations.facilities.gemini.GEMFacility',
         'tom_observations.facilities.soar.SOARFacility',
-        'tom_observations.facilities.lt.LTFacility'
+        'tom_observations.facilities.lt.LTFacility',
         'mytom.myfacility.MyObservationFacility'
    ]
 

docs/targets/index.rst

@@ -6,8 +6,10 @@ Targets
   :hidden:
 
   target_fields
+  target_matcher
   ../api/tom_targets/models
   ../api/tom_targets/views
+  ../api/tom_targets/groups
 
 
 The ``Target``, along with the associated ``TargetList``, ``TargetExtra``, and ``TargetName``, are the core models of the 
@@ -16,10 +18,10 @@ TOM Toolkit. The ``Target`` defines the concept of an astronomical target.
 :doc:`Adding Custom Target Fields <target_fields>` - Learn how to add custom fields to your TOM Targets if the
 defaults do not suffice.
 
-:doc:`Adding Custom Target Matcher <target_matcher>` - Learn how to replace or modify the TargetMatchManager if more
+:doc:`Customizing a Target Matcher <target_matcher>` - Learn how to replace or modify the TargetMatchManager if more
 options are needed.
 
-:doc:`Target API <../api/tom_targets/models>` - Take a look at the available properties for a ``Target`` and its associated models.
+:doc:`Target Models <../api/tom_targets/models>` - Take a look at the available properties for a ``Target`` and its associated models.
 
 :doc:`Target Views <../api/tom_targets/views>` - Familiarize yourself with the available Target Views.
 

docs/targets/target_fields.rst

@@ -10,6 +10,234 @@ the redshift of your targets. You could then do a search for targets
 with a redshift less than or greater than a particular value, or use the
 redshift value to make decisions in your science code.
 
+The TOM Toolkit currently supports two different methods for adding extra
+fields to targets: Extending Target Models and adding Extra Fields.
+
+Extending the Target Model
+==========================
+Users can extend the ``Target`` model by creating a custom target model in the app
+where they store their custom code. This method is more flexible and allows for
+more intuitive relationships between the new target fields and other code the user
+may create. This method requires database migrations and a greater understanding of
+Django models to implement.
+
+By default the TOM Toolkit will use the ``tom_targets.BaseTarget`` model as the target model,
+but users can create their own target model by subclassing ``tom_targets.BaseTarget`` and adding
+their own fields. The TOM Toolkit will then use the custom target model if it is defined
+in the ``TARGET_MODEL_CLASS`` setting of ``settings.py``. To implement this a user will first
+have to edit a ``models.py`` file in their custom code app and define a custom target model.
+
+Subclassing ``tom_targets.BaseTarget`` will give the user access to all the fields and methods
+of the ``BaseTarget`` model, but the user can also add their own fields and methods to the custom
+target model. Fields from the ``BaseTarget`` model will be stored in a separate table from the custom
+fields, and rely on separate migrations. See the
+`Django documentation on multi-table inheritance. <https://docs.djangoproject.com/en/5.0/topics/db/models/#multi-table-inheritance>`__
+
+Preparing your project for custom Target Models
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The first thing your project will need is a custom app. If you already have a custom app
+(usually called ``custom_code``) you can skip this section. You can read
+about custom apps in the Django tutorial
+`here <https://docs.djangoproject.com/en/dev/intro/tutorial01/>`__, but
+to quickly get started, the command to create a new app is as follows:
+
+.. code:: python
+
+   ./manage.py startapp custom_code
+
+Where ``custom_code`` is the name of your app. You will also need to
+ensure that ``custom_code`` is in your ``settings.py``. Append it to the
+end of ``INSTALLED_APPS``:
+
+.. code:: python
+
+   ...
+   INSTALLED_APPS = [
+       'django.contrib.admin',
+       'django.contrib.auth',
+       ...
+       'tom_dataproducts',
+       'custom_code',
+   ]
+   ...
+
+You should now have a directory within your TOM called ``custom_code``,
+which looks like this:
+
+::
+
+   ├── custom_code
+   |   ├── __init__.py
+   │   ├── admin.py
+   │   ├── apps.py
+   │   ├── models.py
+   │   ├── tests.py
+   │   └── views.py
+   ├── data
+   ├── db.sqlite3
+   ├── manage.py
+   ├── mytom
+   │   ├── __init__.py
+   │   ├── settings.py
+   │   ├── urls.py
+   │   └── wsgi.py
+   ├── static
+   ├── templates
+   └── tmp
+
+Editing ``models.py``
+~~~~~~~~~~~~~~~~~~~~~
+First you will need to create a custom target model in the ``models.py`` file of your custom app.
+The following is an example of a custom target model that adds a boolean field and a number field:
+
+.. code-block:: python
+    :caption: models.py
+    :linenos:
+
+    from django.db import models
+
+    from tom_targets.base_models import BaseTarget
+
+
+    class UserDefinedTarget(BaseTarget):
+        example_bool = models.BooleanField(default=False)
+        example_number = models.FloatField(default=0)
+
+        # Set Hidden Fields
+        example_bool.hidden = True
+
+        class Meta:
+            verbose_name = "target"
+            permissions = (
+                ('view_target', 'View Target'),
+                ('add_target', 'Add Target'),
+                ('change_target', 'Change Target'),
+                ('delete_target', 'Delete Target'),
+            )
+
+The model name, ``UserDefinedTarget`` in the example (line 6), can be replaced by whatever CamelCase name you want, but
+it must be a subclass of ``tom_targets.BaseTarget``. The permissions in the class Meta (lines 15-20) are required for the
+TOM Toolkit to work properly. The ``hidden`` attribute can be set to ``True`` to hide the field from the target
+detail page.
+
+Editing ``settings.py``
+~~~~~~~~~~~~~~~~~~~~~~~
+Next you will need to tell the TOM Toolkit to use your custom target model. In the ``settings.py`` file of your
+project, you will need to add the following line:
+
+.. code:: python
+
+    TARGET_MODEL_CLASS = 'custom_code.models.UserDefinedTarget'
+
+Changing ``custom_code`` to the name of your custom app and ``UserDefinedTarget`` to the name of your custom target model.
+
+Creating Migrations
+~~~~~~~~~~~~~~~~~~~~
+After you have created your custom target model, you will need to create a migration for it. To do this, run the
+following command:
+
+.. code:: python
+
+    ./manage.py makemigrations
+
+This will create a migration file in the ``migrations`` directory of your custom app. You can then apply the migration
+by running:
+
+.. code:: python
+
+    ./manage.py migrate
+
+This will build the appropriate tables in your database for your custom target model.
+
+Convert old targets to new model
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If you have existing targets in your database, you will need to convert them to the new model. This can be done by
+running a version of the following code. We incorporate this into a management command to make it easier to run.
+
+Create a new file in your custom app called ``management/commands/convert_targets.py`` and add the following code:
+
+.. code-block:: python
+    :caption: convert_targets.py
+    :linenos:
+
+    from django.core.management.base import BaseCommand
+
+    from tom_targets.base_models import BaseTarget
+    from tom_targets.models import Target
+
+
+    class Command(BaseCommand):
+        """
+        Core code based on information found at
+        https://code.djangoproject.com/ticket/7623
+        """
+
+        help = 'A helper command to convert existing BaseTargets to UserDefinedTargets.'
+
+        def handle(self, *args, **options):
+            # Make sure Target is a subclass of BaseTarget
+            if Target != BaseTarget and issubclass(Target, BaseTarget):
+                self.stdout.write(f'{Target} is a subclass of BaseTarget, updating existing Targets.')
+                base_targets = BaseTarget.objects.all()
+                targets = Target.objects.all()
+                for base_target in base_targets:
+                    # If the base_target is not already in the new target model, update it
+                    # Note: subclassed models share a PK with their parent
+                    if not targets.filter(pk=base_target.pk).exists():
+                        self.stdout.write(f'Updating {base_target}...')
+                        target = Target(basetarget_ptr_id=base_target.pk)  # Create a new target with the base_target PK
+                        target.__dict__.update(base_target.__dict__)  # add base_target fields to target dictionary
+                        target.save()
+                self.stdout.write(f'{Target.objects.count()} Targets updated.')
+
+            return
+
+Once this file is created, you can run the following command to convert your old targets to the new model:
+
+.. code:: python
+
+    ./manage.py convert_targets
+
+Once this command is run, all of your old targets will be converted to the new model, but will not have the new fields
+filled in. You will need to fill in these fields manually, but once you do any non-hidden fields will be displayed on
+the target detail page.
+
+Any fields added in this way are fully accessible in the TOM Toolkit as ``Target``, and can be used in the same way
+as the built-in fields from any custom code you write, the API, or from the admin interface.
+
+
+Transferring existing ``Extra Field`` Data to your ``Target`` Fields
+====================================================================
+
+If you have been using ``Extra Fields`` and have now created a custom target model, you may want to transfer the data
+from the ``Extra Fields`` to the new fields in your custom target model. This can be done by running a management
+command called ``converttargetextras``.  To use this command, be sure to have already created your custom target model.
+You can run the command without arguments for an interactive walkthrough.
+
+.. code:: python
+
+    ./manage.py converttargetextras
+
+Alternatively, you can run the command with the ``--target_extra`` and/or ``--model_field`` flags to specify one or
+more the of the ``Extra Field`` and ``Target Field`` names respectively.
+
+.. code:: python
+
+    ./manage.py converttargetextras --target_extra extra_bool extra_number --model_field example_bool example_number
+
+This command will go through each target and transfer the data from the ``Extra Field`` to the ``Target Field``. If the
+``Target Field`` is already populated with a value other than the default value, the data will not be transferred unless
+the ``--force`` flag is set. When finished, the ``Extra Field`` data will be
+deleted, and you will likely want to remove the ``EXTRA_FIELDS`` setting from your ``settings.py`` file.
+
+Adding ``Extra Fields``
+=======================
+If a user does not want to create a custom target model, they can use the ``EXTRA_FIELDS``
+setting to add extra fields to the ``Target`` model. This method is simpler and does not require
+any database migrations, but is less flexible than creating a custom target model.
+
 **Note**: There is a performance hit when using extra fields. Try to use
 the built in fields whenever possible.