Skip to content

Project Architecture Overview

Jump to bottom Edit New page
Douglas R. Miles edited this page Oct 1, 2024 · 11 revisions

Interaction Flow

To better understand how these components interact within the MeTTa system, here's a high-level flow of operations:

  1. Startup and Initialization

  2. User Interaction via REPL or Script Execution

    • If the user engages with MeTTa interactively, metta_repl.pl and repl.default.metta handle inputs and outputs.
    • For script execution, the interpreter processes MeTTa files directly.

  3. Parsing Source Code

    • Source code is fed into metta_parser.pl, which parses it into internal representations.
    • metta_convert.pl may further process these representations for compatibility with the evaluator or compiler.

  4. Compilation or Interpretation

  5. Runtime Execution

  6. Utilizing Core Libraries and Extensions

  7. Output and Feedback

  8. Persistence and State Management

  9. Server Operations

    • If running in server mode, metta_server.pl manages client connections and request handling.

System Architecture Diagram

Below is a textual representation of the MeTTa system architecture, showing the relationships between components:

[ User Interface ]
       |
       V
[ REPL (metta_repl.pl) ] -- [ REPL Script (repl.default.metta) ]
       |
       V
[ Parser (metta_parser.pl) ] -- [ Converter (metta_convert.pl) ]
       |
       V
[ Interpreter (metta_interp.pl) ] -- [ Compiler (metta_compiler.pl) ]
       |                               |
       V                               V
[ Evaluator (metta_eval.pl) ]        [ Compilation Templates (metta_comp_templates.pl) ]
       |
       V
[ Runtime Environment ]
       |
       |-- [ Execution Space (metta_space.pl) ]
       |-- [ Substitution (metta_subst.pl) ]
       |-- [ Type System (metta_types.pl) ]
       |-- [ Concurrency (metta_threads.pl) ]
       |
[ Core Libraries ]
       |
       |-- [ corelib.metta ]
       |-- [ metta_corelib.pl ]
       |
[ Extensions and Integrations ]
       |
       |-- [ Logic Programming ]
       |     |-- [ metta_ontology.pfc.pl ]
       |     |-- [ metta_pfc_base.pl ]
       |     |-- [ metta_pfc_support.pl ]
       |
       |-- [ Python Integration ]
             |-- [ metta_python.pl ]
             |-- [ metta_python_override.py ]
             |-- [ metta_python_patcher.py ]
             |-- [ metta_python_proxy.py ]

System Architecture Explanation for Rule-Based MeTTa

In this system, the architecture revolves around a rule-based pattern matching engine that leverages MeTTa's meta-programming capabilities. The system is designed to evaluate expressions dynamically by matching patterns against defined rules. The rules dictate how specific operations, like arithmetic, are handled within the system.

Here’s a breakdown of the architecture and flow of how rule-based arithmetic, such as (add $a $b), is processed and evaluated in the system:

Key Architectural Components

  1. Atomspace (Blackboard):

    • Role: Atomspace serves as the knowledge base and storage mechanism for all code modules (rules, operations, and logic). In this context, it holds the rules for pattern matching and transformations, like the rule (= (add $a $b) (+ $a $b)).
    • Storage: Rules and expressions are stored as atoms in Atomspace. These atoms represent the symbolic expressions and patterns used in the system. For example, the rule (= (add $a $b) (+ $a $b)) would be represented as atoms that link the patterns and their transformations.

  2. Rule-Based Pattern Matching Engine:

    • Role: This engine is the core processing unit that evaluates MeTTa expressions. It works by matching input expressions (e.g., (add 5 10)) against predefined rules stored in Atomspace.
    • Functionality: When an expression is submitted for evaluation, the engine:
      1. Searches Atomspace for matching patterns.
      2. Transforms the expression according to the matched rule (e.g., (add $a $b) becomes (+ $a $b)).
      3. Evaluates the transformed expression to produce a result.
    • Dynamic Behavior: This component is capable of dynamically creating and modifying rules during runtime, allowing for flexible and adaptive programming.

  3. Dynamic Evaluation (MeTTa Interpreter):

    • Role: The interpreter is responsible for executing the transformed expressions after the rule-based engine has processed them. It recursively evaluates expressions that have been transformed (e.g., (+ 5 10)), calculating their values.
    • Interaction with Atomspace: The interpreter may retrieve additional rules or definitions from Atomspace during the evaluation process if needed (e.g., recursive function calls or additional pattern matches).

  4. Module Manager:

    • Role: The module manager oversees the organization and management of code modules within Atomspace. These modules may contain arithmetic rules, logical rules, or even higher-level control structures. The manager is responsible for loading, linking, and organizing these modules in Atomspace.
    • Interaction with Subprocesses: In a more complex scenario, the module manager might also delegate certain tasks to subprocesses (e.g., Python or Scheme) when MeTTa modules define external dependencies or computations that need to be performed in other languages.

  5. Subprocess Manager (Optional):

    • Role: While not necessary for basic rule-based arithmetic, in more complex systems, the subprocess manager would handle the interaction with external subprocesses (e.g., calling a Python subprocess for specialized computations). In the context of rule-based arithmetic, the subprocess manager would be activated when a MeTTa rule specifies that external code needs to be executed.

  6. Data Conversion Layer:

    • Role: If subprocesses are used, the data conversion layer would handle translating the internal MeTTa expressions into forms that other subprocesses (like Python) can understand, and vice versa. For example, a rule might specify that a computation should be performed by an external Python subprocess, in which case the system would convert the MeTTa representation into Python code, execute it, and convert the results back into MeTTa atoms.

System Flow for Evaluating an Expression like (add 5 10)

Let’s go through the system flow when an expression like (add 5 10) is evaluated:

  1. Input Expression:

    • The user inputs the expression (add 5 10) for evaluation. This expression is represented internally as atoms in Atomspace.

  2. Pattern Matching:

    • The pattern matching engine searches Atomspace for a rule that matches the structure of the expression. It finds the rule (= (add $a $b) (+ $a $b)).

  3. Transformation:

    • The engine transforms the expression (add 5 10) by substituting 5 for $a and 10 for $b, resulting in the new expression (+ 5 10).

  4. Evaluation:

    • The transformed expression (+ 5 10) is passed to the dynamic evaluation component (MeTTa interpreter), which evaluates the expression and calculates the result: 15.

  5. Return Result:

    • The result 15 is returned as the output of the evaluation. This result may also be stored back into Atomspace for further use by other components or subsequent computations.

Diagram: High-Level System Architecture for Rule-Based Evaluation

+-----------------+        
|    Atomspace    |     <- Stores rules, expressions, and evaluation results
| (Blackboard for |     
| Code and Rules) |     
+-----------------+        
        |                         
        v                         
+-------------------------+     
|  Rule-Based Pattern     |     <- Matches and transforms input expressions
|  Matching Engine        |     
+-------------------------+     
        |                         
        v                         
+-------------------------+     
|   Dynamic Evaluation    |     <- Evaluates transformed expressions (e.g., (+ 5 10))
|  (MeTTa Interpreter)    |     
+-------------------------+     
        |                         
        v                         
+-------------------------+     
|   Execution Results     |     <- Returns the evaluated result (e.g., 15)
|                         |     
+-------------------------+

Detailed Breakdown of System Behavior

  1. Atomspace as Central Knowledge Base:

    • Atomspace holds not only code (e.g., arithmetic rules) but also the logic for how expressions are transformed and evaluated. This allows the system to be highly flexible, as the rules and logic can be updated at runtime without changing the underlying system architecture.

  2. Rule-Based Pattern Matching Engine:

    • The engine is responsible for transforming expressions based on the rules defined in Atomspace. For arithmetic, it matches expressions like (add $a $b) and replaces them with corresponding mathematical operations.

  3. Dynamic Evaluation:

    • Once the expression has been transformed, the MeTTa interpreter evaluates it recursively. The interpreter’s role is to carry out the final computation, such as resolving (+ 5 10) to 15.

  4. Module and Subprocess Management:

    • Although not needed for simple arithmetic, in a more complex system, the module manager would oversee the organization of different rule sets and subprocesses. When rules or code modules specify that external systems (like Python) need to be invoked, the subprocess manager would handle the orchestration.

  5. Data Flow:

    • The data flow follows a clear path: from the initial expression, through rule transformation, evaluation, and finally to the output result. Atomspace plays a crucial role by allowing different components of the system to access and modify shared information, ensuring consistency across the system.

Summary

In this architecture:

  • MeTTa enables meta-programming through dynamic pattern matching and rule-based transformations.
  • Atomspace serves as the central blackboard where rules and expressions are stored and accessed by various components.
  • The Rule-Based Pattern Matching Engine handles the transformation of input expressions based on predefined rules.
  • The Dynamic Evaluation Component (MeTTa Interpreter) evaluates the transformed expressions and computes the results.
  • The architecture is extensible to handle more complex interactions, such as integrating external subprocesses (like Python or Scheme) for specialized computations, managed by a module manager and subprocess manager.

Summary

The MeTTa language system is structured to facilitate ease of use, extensibility, and powerful features for logical reasoning and interoperability. Here's a brief recap:

  • Front-End: Parses and prepares code for execution.
  • Core Libraries: Provide essential functions and definitions.
  • Interpreter/Compiler: Execute or compile code for running programs.
  • Runtime Environment: Manages execution contexts and supports runtime features.
  • REPL: Offers an interactive environment for code execution.
  • Debugging/Testing: Tools for ensuring code correctness.
  • Extensions: Enhances MeTTa with logic programming and Python interoperability.
  • Server Support: Allows MeTTa to function as a networked service.
  • Utilities: Additional support functions and compatibility layers.
  • Initialization: Handles startup routines and environment setup.

Note: The descriptions and architecture are based on standard naming conventions and typical functionalities associated with similar files in programming language implementations. For precise implementation details, please refer to the actual source code and official documentation.

Other Directories and Files

The other directories and files in the project provide additional functionality and support. Each top-level directory is expandable, and the link is provided outside the spoiler.

src: Main Source Code of the Project

Click to expand the src directory contents.

src/ext: External Modules Interfacing with External Libraries

These modules provide integration with external libraries and tools, extending the capabilities of the Metta system.

  • bhv_binding.py: Implements behavior bindings, potentially for integrating AI behavior models.
  • dasgate.py: Gateway to the Distributed Annotation System (DAS), used for accessing and integrating biological data.
  • kwargsme.py: Experiments with PyTorch keyword arguments, potentially for machine learning applications.
  • metta_repl.py: Interactive REPL for Metta written in Python, providing an alternative interface.
  • neurospace.py: Defines neural spaces for simulations, aiding in neural network modeling.
  • numme.py: Numerical experiments using NumPy, supporting mathematical computations.
  • parse_torch_func_signatures.py: Parses PyTorch function signatures, facilitating integration with PyTorch.
  • parsing_exceptions.py: Handles parsing exceptions, improving error handling during code parsing.
  • r.py: Utility script for resolution tasks, possibly related to logic programming.
  • resolve.py: Implements resolution logic, essential for logical inference mechanisms.
  • sql_space.py: Provides SQL space integration, enabling database interactions.
  • tm_test.py: Test script for PyTorch models, useful for machine learning development.
  • torchme.py: Script interfacing with PyTorch, aiding in deep learning tasks.

src/main: Contains the Last Known Good Version of the Source Code

This directory holds a stable version of the source code, ensuring that a working copy is available for reference or rollback.

src/metta_jupyter_kernel.py: Implements a Jupyter Kernel for the Metta Language

Allows Metta code to be executed within Jupyter notebooks, facilitating interactive development and data analysis.

src/mettalog: Python Metta Logic Library for Logical Reasoning

This library provides logical reasoning capabilities within the Metta environment.

  • logic.py: Core logic functions that implement fundamental logical operations.
  • reasoning.py: Reasoning algorithms that enable inference and deduction.
  • utils.py: Utility functions supporting the logic library.

notebooks: Contains Jupyter Notebooks for Experimentation and Data Analysis

Contents

of data.

  • from_das: Demonstrations of data retrieval from the Distributed Annotation System (DAS).
    • das_example.ipynb: Using DAS data, demonstrating how to access and utilize biological data.

library: Reusable Code Modules and Libraries

Click to expand the library directory contents.

graphml: Scripts Related to the GraphML Format

  • ext_loader_graphml.pl: Parses GraphML files, enabling the import of graph data.
  • tests: Test cases for the GraphML loader, ensuring correct functionality.

genome: Prolog Scripts for Genomic Data Processing

Modules focused on processing genomic data, integrating with bioinformatics databases.

tests: Various Test Suites to Ensure Correctness and Stability

Test Suites
  • baseline_compat: Tests for baseline compatibility, ensuring fundamental functionalities work as expected.
  • compiler_baseline: Tests for compiler baseline functionality.
  • direct_comp: Tests for direct compilation processes without intermediate steps.
  • extended_compat: Tests for compatibility with extended features and modules.
  • features: Tests focused on specific features of the Metta language and system.
  • flybase: Tests related to the processing of FlyBase data.
  • more-anti-regression: Additional regression tests to catch unintended changes.
  • nars_interp: Tests for the NARS (Non-Axiomatic Reasoning System) interpreter integration.
  • nars_w_comp: Tests for NARS with compiler integrations, ensuring reasoning capabilities.
  • performance: Performance measurement tests to benchmark system efficiency.
  • python_compat: Tests for Python compatibility, verifying the integration between Metta and Python.
  • timing: Scripts and tests focused on measuring execution times and performance metrics.

Conclusion

This detailed overview provides an in-depth look into the architecture and workflow of the Metta language processing system. Starting from the REPL interface, which allows users to interact with the interpreter and load files, the system efficiently manages code addition, parsing, forward chaining for logical inference, compilation, and execution. The inclusion of utilities and support modules enhances the system's capabilities, offering integration with external languages like Python and providing tools for debugging and testing.

Understanding each component's role and how they interact enhances appreciation of the system's workflow and capabilities. This comprehensive guide serves as a valuable resource for both users and developers navigating and utilizing the Metta language processing system.

For further clarification, refer to the docs directory or contact the development team.

Add a custom footer
Add a custom sidebar
Clone this wiki locally