Add documentation on using contexts.

src/codehelp/docs/contexts.md

@@ -0,0 +1,72 @@
+---
+title:  Using Contexts
+summary:  Guidance and suggestions on how to use contexts in your class.
+---
+
+# Using Contexts
+
+When responding to a query, a large language model (LLM) only has access to the information contained in that single query -- it will make assumptions about anything not included.
+It won't know anything about your course or the problems students are working on unless it is explicitly told.
+Furthermore, students do not always know what details they should provide to get the most helpful response.
+**Contexts** let you as the instructor provide one or more sets of information to be provided along with students' queries.
+As the name suggests, you can think of each one as a "context" within which a student is seeking help.
+
+A context contains:
+- A descriptive name
+- *[optional]* The languages, libraries, and/or frameworks that students are learning or using in the context.
+- *[optional]* Free-form written details for the context.
+- *[optional]* A set of keywords and concepts you want the responses to *not* include.  This is commonly used to prevent a response from suggesting a technique or tool that is more advanced than the course is expecting in this context.
+
+Contexts are very flexible, with many ways to configure them in a class.
+Here are some possibilities, some of which could be combined:
+
+1) **No context defined.**
+   - Not recommended.  The LLM will only know what is written by the student in each query.
+   - This may be acceptable for a demonstration of the tool or for a very general audience with a wide range of unpredictable needs.
+1) **A single, default context.**
+   - The minimum recommended.
+   - If the context in which students are working is consistent throughout the course, this can work.
+   - You could also use a single context that you update throughout the course term.
+1) **One context for each module** in the course.
+   - For example, you could define a separate context for each language used in a programming languages course.
+1) **Separate contexts for all assignments,** projects, or their separate sections.
+   - In the "details" section of each context, you can include all relevant specifications or instructions from the assignment.
+   - Without defined contexts, many students will copy and paste the complete assignment instructions into their query, and they often included irrelevant details from it as well.
+1) **A "conceptual question" context.**
+   - For students to ask questions that are not tied to any particular code or programming environment, it is best to define a separate context for that if all of the other contexts are oriented toward coding.
+
+## What to Include
+
+**Name** --- Context names are used by the students to select a context when there are multiple available, and the chosen context's name is included in the query sent to the LLM as well.  It should be short, clear, and descriptive.  Even with nothing else defined in a context, a name like "Python" will provide some information for the LLM to use.  If there is only one context for the class, however, it is recommended you name it "Default" and fill in other details.
+
+**Environment and Tools** --- List all languages, frameworks, libraries, and tools that could be relevant to a student's query in this context.  For example, in a web development class, a student could ask, "How do I make a route respond only to POST requests?"  If the context specifies the web framework being used in the course, the LLM's response will be much more helpful.  Whatever you enter here will be shown to the student for reference when they are writing a query.
+
+**Details** --- Here, you can describe the context in whatever level of detail makes sense.  For a general, course-wide context, you might state the level of the course, describe what background a student can be expected to have, or possibly just leave it blank.  For an assignment- or project-specific context, you can include the specifications and instructions that will be most relevant and unlikely to be assumed correctly by an LLM.  The details section of a context will also be shown to students when writing a query.
+
+**Keywords to Avoid** --- LLMs will often suggest whatever approach to solving a problem is widely considered a best practice, but that might not be what you are expecting your students to do, especially in a lower-level class.  This section can be used to tell the LLM to *not* use any of the listed keywords or concepts in its response.  Often, you won't know what to include here until you see a response use something more advanced than you expect, but then you can quickly edit the context at that point to apply to later queries.  The keywords here will *not* be shown to students.
+
+## Availability
+
+Every context has an availability setting.
+You can make it available "Now" (immediately), scheduled to become available on a future date, or "Hidden."
+This controls whether and when each context will be listed for students as an option when writing a query.
+If there is only one context available, it will be used automatically, and students will not have a choice.
+When multiple contexts are available, students will be able to choose from them when writing a query.  Their most recently-used context, if any, will be pre-selected for them, but they can choose another.
+
+Generally, hiding a context is meant to simplify the choices shown to a student but not to prevent them from using a context.
+When a context is "Hidden," it can still be used in a few cases:
+
+1) If you provide a link directly to a context (see the "Link to" action in the contexts table), students who use it will be taken to a help form with that context pre-selected.
+1) If a student's most recent query used that context, they can still use it in their next query.
+1) If a student chooses to "Retry" a query whose context is hidden, it can be used in the new query as well.
+
+One way you might use this is to have contexts defined for every assignment or exercise, keep them all hidden to avoid overloading the context choice dropdown in the help form, and then provide a link to the one relevant context directly from each assignment or exercise.
+
+## Suggestions
+
+- Set up contexts in advance of the course starting or before making CodeHelp available to your students.
+- Visit the help form page yourself to see how contexts can be selected and are displayed to your students.
+- Monitor the responses students are getting to look for opportunities to refine contexts and improve future responses.
+  - If you see a response that suggests something you don't want students to use, add it to the avoid set.
+  - If you see several responses that make incorrect assumptions or interpret queries incorrectly, consider whether additional context could help.
+- Don't overload a context.  Too much information can result in the LLM being less likely to find and use the most relevant details.  Additionally, everything included in a context adds to the length of the prompt sent to the LLM for every query, and LLM cost scales directly with prompt length.  Trying to include every possible detail that might be relevant in a context *could* make the responses worse and *will* increase the costs.

src/codehelp/docs/recommended_practices.md

@@ -7,8 +7,12 @@ summary:  Based on our experiences using CodeHelp with students.
 
 Based on our experiences, we have collected a few recommendations for integrating CodeHelp into a class effectively.
 
+## Using Contexts
 
-## Initial introduction
+Before you make CodeHelp available to your students, you should define one or more contexts.
+We have a separate page about <a href="contexts">Using Contexts</a>.
+
+## Initial Introduction
 
 When first introducing CodeHelp to students, motivate its use by sharing some of the benefits students will see, as relevant to your course.
 Explain carefully its strengths and limitations in the context of your course: what it will likely be able to help with, and where may it produce incorrect responses.
@@ -30,5 +34,5 @@ In this way, you can gain insight into where they are struggling at each point i
 Additionally, you might identify students whose usage is not effective (e.g., repeatedly submitting ineffective queries or demonstrating overreliance), and reach out to them directly to provide guidance or a nudge.
 
 Instructors and TAs should sample CodeHelp's responses in each section of the course to spot and mitigate issues.
-For example, if CodeHelp suggests a technique, function, or concept that does not fit the design of your course, you can add that to the avoid set in the instructor's class configuration interface to produce more appropriate responses to similar queries in the future.
+For example, if CodeHelp suggests a technique, function, or concept that does not fit the design of your course, you can add that to the avoid set in the the relevant <a href="contexts">context</a> to produce more appropriate responses to similar queries in the future.
 

src/gened/base.py

@@ -188,6 +188,11 @@ def create_app_base(import_name: str, app_config: dict[str, Any], instance_path:
     if docs_dir:
         app.register_blueprint(docs.bp)
 
+        # Inject docs pages list into template contexts
+        @app.context_processor
+        def inject_docs_list() -> dict[str, list[str]]:
+            return { 'docs_pages': docs.list_pages() }
+
     # Inject auth data into template contexts
     @app.context_processor
     def inject_auth_data() -> dict[str, Any]:

src/gened/docs.py

@@ -10,7 +10,8 @@
 from markdown_it import MarkdownIt
 
 # get a processor that does HTML parsing for our docs (trusted content, HTML parsing okay)
-_markdown_processor = MarkdownIt("commonmark")
+_markdown_processor = MarkdownIt("commonmark", {"typographer": True})
+_markdown_processor.enable(["replacements", "smartquotes"])
 
 bp = Blueprint('docs', __name__, url_prefix="/docs", template_folder='templates')
 
@@ -42,6 +43,14 @@ def _process_doc(docfile_path: Path) -> Document:
     )
 
 
+def list_pages() -> list[str]:
+    docs_dir = current_app.config.get('DOCS_DIR')
+    assert docs_dir  # base.py shouldn't load this blueprint if we have no documentation directory configured
+
+    pages_paths = docs_dir.glob('*.md')
+    return [path.stem for path in pages_paths]
+
+
 @bp.route('/')
 def main() -> str:
     ''' Show an index of documentation pages. '''

src/gened/templates/instructor_class_config.html

@@ -232,10 +232,14 @@ <h2 class="title is-size-4 mt-5">Language Model</h2>
 
       <div class="conf_col content">
         <h2 class="title is-size-4">Contexts</h2>
-        <p style="max-width: 50em">Contexts provide additional information to the LLM for each query a student makes.  You can have a single default context that is always used, or you can create separate contexts for individual assignments or modules.  If multiple contexts are available, students will be able to select from them when making queries.
+        <p style="max-width: 50em">Contexts provide additional information to the LLM for each query a student makes.  You can have a single default context that is always used, or you can create separate contexts for individual assignments or modules.  If multiple contexts are available, students will be able to select from them when making queries.</p>
         {% if contexts | length == 0 %}
           <p class="has-text-danger">While not strictly required, we recommend defining at least one context to specify the language(s), frameworks, and/or libraries in use in this class.</p>
         {% endif %}
+        {# Link to the 'contexts.md' docs page if it exists #}
+        {% if 'contexts' in docs_pages %}
+          <p>See the <a href="{{ url_for('docs.page', name='contexts') }}">contexts documentation</a> for more information and suggestions.</p>
+        {% endif %}
         <script type="text/javascript">
           document.addEventListener('alpine:init', () => {
             Alpine.data('reorderable', () => ({